Design and Analysis of Computer Algorithms: Lecture Notes

Raul Lopes
raulh@inf.ufes.br

Chapter 1
Program Correctness

1.1  The Coffee Can problem

Given:

a coffee with white and black beans; inexhaustible supply of balck beans.

The only acceptable movement consists in taking exactly 2 beans from the can and then:

• if they have the same color, throw them away and put one black bean (from the supply) in the can;
• if they are of different colors, put the white one back and the black one away.

The only movement is repeated as far as it is possible and what we are asked:

1. Does the game ever terminate?
2. If it does, with how many beans in the can?
3. If there are beans left in the can, what can be said about their color?

1.1.1  Bound functions

InfDef 1 Bound Function\oversetdef =   a map from a set of a variables (of a process) to a well ordered set (one that can be put into an order that has no infinetely decreasing chain.)

Note 1 Bound functions will be used to prove program termination.

Theorem 1 All movements reduce the number of beans in the can. At any moment if there are n beans in the can, a movement takes 2 from the can it either puts one black int the can or it puts one white back, and that gives (n-2+1) beans at the end of the movement.

Thus, each movement reduces the number of beans in the can by one. As such, and assuming the number of beans in the can is finite, after a finite number of steps there will be just one bean left. At that point, no movement is possible because each movement takes 2 beans from the can.

1.1.2  Invariant properties

InfDef 2 Invariant \oversetdef =  a property that does not change whatever the movement.

Many invariants can be established with respect to game (or a program), some of them can be useful in determining the final state of the game. For example: There are zero or more black beans in the coffe is a property that can be easily shown to hold and yet that does not teach us much about the final state of the game.

In opposition, the number of black beans in coffe can is always odd is a property that does not always hold because the number of black beans is either decreased or increased by one at every movement.

The parity of the white beans is not affected by any movement: the number of white beans doesn't change when beans taken from the can are either of different colors or they are both black, and it is decreased by two white beans are taken out of the can.

Thus, of the final state of the can, it can be said that ot will zero beans if at the beginning the number of white beans in it is even; there will one white bean in the can otherwise.

1.2  Juggling with jugs

• Given: 2 jugs that can hold respectivelly 3 and 5 units of water.
• The only allowed movements are either:
• completely filling one jug, or
• completely emptying one jug.

• At the beginning, they are both empty and you are requested to get one of them with 4 units of water.

Definition 1 A finite state machine is a tuple , where:

• Q is a set of states, usually representing all variables involved in the computation which is under study.
• Q₀ is the set of initial states (Q₀ Ì Q).
• d Í Q×Q is a transition relation, defining the allowed tansitions between states.

In the two jugs game:

• Q = {(a,b): 0 £ a £ 3 and 0 £ b £ 5}
• Q = {(0,0)}

In a pair (a,b), the a could represent the state of the 3-units jug, while b would represent the other one. The transition relation d should reflect the fact that the only allowable movement are either completely filling or emptying one jug.

Definition 2 An execution of a finite state machine is a sequence of states q₀,q₁,q₂,¼,q_n, such that for any 0 £ i < n (q_i,q_i+1) Î d.

An execution that solves the puzzle is:

(0,0),(0,5),(3,2),(0,2),(2,0),(2,5),(3,4)

In a variation over the same game (also proposed by Professor Leighton), we are given 3 units and 6 units jugs, and this time demanded to produce a proof that it is not possible to end up one jug containing with 4 units of water.

This time Q = {(a,b): 0 £ a £ 3 and 0 £ b £ 6}. The transition relation d would allow the following movements: (a,b) ® (3,b)
(a,b) ® (a,6)
(a,b) ® (0,b)
(a,b) ® (a,0)
(a,b) ® (max(0,a+b-6),min(a+b,6))
(a,b) ® (min(a+b,3),max(0,a+b-3))

We can prove that it is not possible to produce a jug with 4 units of water by defining a property that is both an invariant of any execution of this machine and that excludes the possibility of any jug having 4 units of water. Our property will be a predicate P over Q:

P(a,b) \oversetdef =  a and b are multiples of 3

If we prove that P is true for any state of any execution of our machine (P is satisfied by any execution of the machine), then, because 4 is not multiple of 3, we'll have proved that it is not possible to obtain a jug with 4 units of water with this set of movements.

1.3  Proving Invariants

• The language: the language of our proofs will be essentially that of classical first-order logic with equality, and enriched with a set of natural numbers (represented by \mathbbN) and the Axiom of Induction. Eventually, this language will be enriched to allow propositions over real numbers and involving program variables. Deduction will be based on the rules of that logic.
• Axioms: an axiom will be anything that is assumed to be true.

Definition 3 Axiom of induction: Let P be a predicate over \mathbbN. If P(0) and for any n Î \mathbbN P(n) implies P(n+1) then P(n), for any n Î \mathbbN.

Indeed the Axiom of Induction can extended to any well-ordered set.

Definition 4 A partial order on a set S is a relation \prec over S that is transitive, i.e., if x,y,z S, x\prec y and y\prec z then \prec z.

An order on a set S is a partial order where, for every two element x,y Î S either x = y, x\prec y, or y\prec x. (set S is ordered if well-ordered set

A set S is well-ordered if it is ordered, and for any nonempty A Í S there is an element s Î A such that x\preceq y (x\prec y or x = y), for all y Î A.

Definition 5 Axiom of Induction (extended): If S is a well-ordered set with a least element x (one such that x\preceq y, for all y Î S) and P is a property over S, then if P(x) and for all n Î S P(n) implies P(n¢) (where n¢ is the successor of n in the ordering of S) then P(n) for all n Î S.

Induction will enable us to prove things about infinite sets.

Theorem 2 "n\mathbbN: 1+2+¼+n = n(n+1)/2.

Let P(n) be our conjecture that 1+2+...+n = n(n+1)/2. It can easily be checked P(1): 1 = 1(1+1)/2.

Assuming that P(n), for (n+1):

1+2+...+n+(n+1) = [n(n+1)/2]+(n+1)

that is easily reduced to (n+1)(n+2)/2, proving P(n+1).

Can induction be used to prove that all horses are of the same color (a problem from []?)

1.4  Induction to prove Invariants

Theorem 3 Invariant Theorem: Let M = \tupleQ,Q₀,d be a finite state machine and P a predicate on Q such that P(q₀) for all q₀ Î Q₀, and let P(q) imply P(q¢) for any pair (q,q¢) such that (q,q¢) Î d. Then P is an invariant for M. The proof is performed by induction on the length of executions of M. We assume that \mathfrakP(n) be the predicate that P(q) is true for any state q contained in an exectution of length n.

\mathfrakP(1) is true because any execution with length 1 contains just an initial state, and the hipothesis of the theorem states P(q₀) for all q₀ Î Q₀. Assuming that \mathfrakP(n), for an execution of length n+1 and one of its states q¢ we have: if q¢ is its initial state, the P(q¢); otherwise q¢ is preceded by another state q that is part of an execution of length n and as such P(q). But, also (by the definition of execution of a finite state machine) (q,q¢) Î d and, because P(q), by the hipothesis of the theorem, P(q¢). Thus, P(q) for any q in an execution of M.

The significance of this theorem is that we can prove invariants by induction. In general, given a property about a program, we will prove that the property holds after the initialization of the program's variables and that it is preserved by any transition in the program (any change the values of those variables.)

1.5  Back to the jug's problem...

This is true for initial state when they are both empty. It can easily be proved that the property is preserved for each of the allowed transitions. For example, for the transition

(a,b) ®(min(a+b,3),max(0,a+b-3))

if a and b are multiples of 3, so it is (a+b), and, as a result, also min(a+b,3). Also, (a+b-3) is a multiple of 3 and so is max(0,a+b-3). The invariance of the property can easily be checked for the other transitions.

1.6  Invariants and programs' properties

• partial correctness, which is related to the correctness of the results of a given program;
• complexity, which relates a program to the time it spents on a given input.

1.6.1  A naï ve multiplication algorithm

Algorithm 1 Naïve multiplication.

int¯
 naive(int a,int b){
 l₁: int x=a, y=b,z=0;

 loop: //Invariant: x*y+z = a*b
 l₂: if¯ (x==0)
 l₃: return z;
 l₄: z+=y;
 l₅: x-;
 l₆: goto loop;
}

Partial correctness. It is not difficult to check that the invariant holds after each cycle of the loop. It After the initial step (l₂), the invariant because z = 0, x = a,y = b. If, at any time, the program is executing l₂, and invariant holds, at the end of cycle (step l₆), we have: z'=z+y
x'=x-1
y'=y the invariant at l_6: x'*y'+z'=a*b
(x-1)*y+(z+y)=a*b
x*y+z=a*b In general, if x is variable, we will use x¢ to denote the value of x after some update.

Termination. It is easy to see that if a ³ 0 the algorithm terminates: x is initialized with a, and is decremented in every each of the loop. When it reaches 0 the loop terminates, and z = a*b, given our loop invariant that states:

x*y+z = a*b

Algorithm's complexity. The following invariant can be easily proved:

x+k = a

where k is counts the number of times around the loop. At the start, k = 0, and x = a, which trivially verifies our invariant. Every cycle of the loop decrements x and increments k, preserving the invariant. By the end of the loop, k will have count a cycles of the loop. Assuming that instruction l_i takes time t_i, that gives

æ è t1+n*(t2+t4+t5+t6)+t2+tt3 ö ø units of time

1.6.2  Another multiplication algorithm

Algorithm 2 Russian peasant. int¯
 russian(int a,int b){
 l₁: int x=a,y=b,z=0;
 loop: // Invariant: x*y+z = a*b
 l₂:if¯ (x = = 0)
 l₃: return z;
 l₄:if¯ (1 = = x%2)
 l₅: z+ = y;
 l₆:y << = 1;
 l₇:x >> = 1;
 l₈:goto loop;
 }

Partial correctness. The invariant is same as in the previous algorithm. For the same reasons, it holds step l₁, that initializes variables. The proof that invariant is mantained by the loop is has two cases. when x is odd, we have: x'=(x-1)/2
y'=2y
z'=z+y and
x'*y'+z'=x-1*2y+z+y=x*y+z If x is even: x'=x/2
y'=2y
z'=z
x'*y'+z'=x*2y+z=x*y+z

Program termination. It is possible to define a bound function mapping the variables of the program to the value of x. Every time around the loop, the value of x is shifted one bit to right. When it becomes 0, the loop is finished, and, given the invariant above is mantained, z = a*b.

Algorithm's complexity. We can prove the following invariant, where |n₂| represents the number of bits necessary to represent n, and k is the number of times around the loop:

|x2|+k = |a2|

The invariant is true after l₂, when x = a and k = 0. It is preserved by each step around the which decrements the number of bits in x by 1 and increments k by 1. By the end, k = |a₂|. But, how can we get the number of bits necessary to represent a? This number is given by the following relation, that in the end defines the number of times the loop is repeated: T(1) = 1 T(a) = 1+T(a>>1)

This is a recurrence relation. Recurrence relations have the nice property that they can be easily obtained from the code of our algorithms. They present an obstacle that is the fact that they don't give a closed formula: T(a) is defined in terms of T(a >> 1), which is in the end T(a/2). But, the intuition behind the relation above seems clear enough: we are counting the number of times we can divide a by 2, and that would be log₂^a, more exactly:

T(a) = 1+log2a

would be the solution for that recurrence. We can prove it by induction: T(1) = 1+log_2^1= 1 as in our relation
T(a)=1+_2^a is assumed for any a<n
T(a)=1+T(a) our relation
T(a)=1+(1+_2^a our inductive hipothesis
T(a)=1+1+_2^a-_2^2
T(a)=1+_2^a

Thus, the time for that algorithm is given by:

Trussian(a,b) < t1+log2c*(t2+t4+t5+t6+t7+t8)+t2+t3

1.6.3  Horner algorithm

Given:

a vector with the coeficients ai; the size of n; the value of x.

Algorithm 3 Horner's algorithm.

dou¯ble
 horner(const double a[],const int n,const double x){
l1:  double v=0;
l2:  int i=n;
l3:  // Invariant: -1 £ i £ n Ùv = å_{j = 0}^n-i-1a_n-j x^n-i-1-j
 loop:¯
l4:   if¯ (i < 0)
l5:    return v;
l6:   v = (v*x)+a[i];
l7:   i-;
l8:   goto loop;
 }

Partial correctness. It is not difficult to check that invariant defined at line l3 holds for each cycle of the loop. The invariant holds (trivially) after l2, when i = n and (å_{j = 0}^n-i-1a_n-j x^n-i-1-j) equals 0 (0 being the identity of the sum.) Let us assume that the property holds for some i = h, (0 £ h £ n). _j=0^n-h-1a_n-jx^n-h-1-j,                                     at l4:i=h      
v'=(_j=0^n-h-1a_n-j*x^n-h-1-j)*x+a_h,              at l7: v updated to v'
v'=_j=0^n-h-1a_n-j*x^n-h-j+a_h,              at l7: v updated to v'
v'=(a_n x^n-h+a_n-1x^n-h-1+...+a_h+1 x^1)+a_h
v'=_j=0^n-ha_n-jx^n-h-j
v'=_j=0^n-(h-1)-1a_n-jx^n-(h-1)-1-j But, the last one is the value of v given by our invariant, when i = h-1, i.e., after i is updated at the of the cycle that started with i = h. Thus, the invariant is preserved throughout the loop.

Loop termination. To understand why the loop terminates, we just have to notice that values of i for a well-ordered set, with least element always equal to -1, the value of i for which the loop finishes.

But, if the loop finishes with i = -1, then it finishes with

v = n-(-1)-1 å j = 0  an-j xn-(-1)-1-j = n å j = 0  an-jxn-j

the value we want to compute.

Footnotes:

¹Even though, this game has been played in brazilian bars for the last 50 years, Leighton detected that the game was presented in the film ``Die Hard 3'' as a hard puzzle for Bruce Willis to solve in order avoid a bomb to explode.