[1]á#1ñ

Design and Analysis of Computer Algorithms: Lecture Notes

Design and Analysis of Computer Algorithms: Lecture Notes

Raul Lopes
raulh@inf.ufes.br

Chapter 1
Program Correctness

1.1  The Coffee Can problem

A game from [].

Given:
  • a coffee with white and black beans;
  • inexhaustible supply of balck beans.


The only acceptable movement consists in taking exactly 2 beans from the can and then:

The only movement is repeated as far as it is possible and what we are asked:

  1. Does the game ever terminate?
  2. If it does, with how many beans in the can?
  3. If there are beans left in the can, what can be said about their color?

1.1.1  Bound functions

InfDef 1 Bound Function\oversetdef =   a map from a set of a variables (of a process) to a well ordered set (one that can be put into an order that has no infinetely decreasing chain.)

Note 1 Bound functions will be used to prove program termination.

Theorem 1 All movements reduce the number of beans in the can. At any moment if there are n beans in the can, a movement takes 2 from the can it either puts one black int the can or it puts one white back, and that gives (n-2+1) beans at the end of the movement.

Thus, each movement reduces the number of beans in the can by one. As such, and assuming the number of beans in the can is finite, after a finite number of steps there will be just one bean left. At that point, no movement is possible because each movement takes 2 beans from the can.

1.1.2  Invariant properties

InfDef 2 Invariant \oversetdef =  a property that does not change whatever the movement.

Many invariants can be established with respect to game (or a program), some of them can be useful in determining the final state of the game. For example: There are zero or more black beans in the coffe is a property that can be easily shown to hold and yet that does not teach us much about the final state of the game.

In opposition, the number of black beans in coffe can is always odd is a property that does not always hold because the number of black beans is either decreased or increased by one at every movement.

The parity of the white beans is not affected by any movement: the number of white beans doesn't change when beans taken from the can are either of different colors or they are both black, and it is decreased by two white beans are taken out of the can.

Thus, of the final state of the can, it can be said that ot will zero beans if at the beginning the number of white beans in it is even; there will one white bean in the can otherwise.

1.2  Juggling with jugs

A game taken from Professor Tom Leighton's Notes for ``Mathematics for Computer Science''.1

Solutions for games like this (and progrmas as well) can be represented using finite state machines.

Definition 1 A finite state machine is a tuple , where:

In the two jugs game:

In a pair (a,b), the a could represent the state of the 3-units jug, while b would represent the other one. The transition relation d should reflect the fact that the only allowable movement are either completely filling or emptying one jug.

Definition 2 An execution of a finite state machine is a sequence of states q0,q1,q2,¼,qn, such that for any 0 £ i < n (qi,qi+1) Î d.

An execution that solves the puzzle is:


(0,0),(0,5),(3,2),(0,2),(2,0),(2,5),(3,4)

In a variation over the same game (also proposed by Professor Leighton), we are given 3 units and 6 units jugs, and this time demanded to produce a proof that it is not possible to end up one jug containing with 4 units of water.

This time Q = {(a,b): 0 £ a £ 3 and 0 £ b £ 6}. The transition relation d would allow the following movements: (a,b) ® (3,b)
(a,b) ® (a,6)
(a,b) ® (0,b)
(a,b) ® (a,0)
(a,b) ® (max(0,a+b-6),min(a+b,6))
(a,b) ® (min(a+b,3),max(0,a+b-3))

We can prove that it is not possible to produce a jug with 4 units of water by defining a property that is both an invariant of any execution of this machine and that excludes the possibility of any jug having 4 units of water. Our property will be a predicate P over Q:
P(a,b) \oversetdef =  a and b are multiples of 3

If we prove that P is true for any state of any execution of our machine (P is satisfied by any execution of the machine), then, because 4 is not multiple of 3, we'll have proved that it is not possible to obtain a jug with 4 units of water with this set of movements.

1.3  Proving Invariants

How can proof be established? A proof will be a chain of logical deductions starting with a set of axioms and ending a statement of the proposition to be proved. In our chains of logical deductions, some elements will be of fundamental importance and they must be mentioned:

Definition 3 Axiom of induction: Let P be a predicate over \mathbbN. If P(0) and for any n Î \mathbbN P(n) implies P(n+1) then P(n), for any n Î \mathbbN.

Indeed the Axiom of Induction can extended to any well-ordered set.

Definition 4 A partial order on a set S is a relation \prec over S that is transitive, i.e., if x,y,z S, x\prec y and y\prec z then \prec z.

An order on a set S is a partial order where, for every two element x,y Î S either x = y, x\prec y, or y\prec x. (set S is ordered if well-ordered set

A set S is well-ordered if it is ordered, and for any nonempty A Í S there is an element s Î A such that x\preceq y (x\prec y or x = y), for all y Î A.

Definition 5 Axiom of Induction (extended): If S is a well-ordered set with a least element x (one such that x\preceq y, for all y Î S) and P is a property over S, then if P(x) and for all n Î S P(n) implies P(n¢) (where n¢ is the successor of n in the ordering of S) then P(n) for all n Î S.

Induction will enable us to prove things about infinite sets.

Theorem 2 "n\mathbbN: 1+2+¼+n = n(n+1)/2.

Let P(n) be our conjecture that 1+2+...+n = n(n+1)/2. It can easily be checked P(1): 1 = 1(1+1)/2.

Assuming that P(n), for (n+1):
1+2+...+n+(n+1) = [n(n+1)/2]+(n+1)
that is easily reduced to (n+1)(n+2)/2, proving P(n+1).

Can induction be used to prove that all horses are of the same color (a problem from []?)

1.4  Induction to prove Invariants

(Based on Professor Leighton's notes.)

Theorem 3 Invariant Theorem: Let M = \tupleQ,Q0,d be a finite state machine and P a predicate on Q such that P(q0) for all q0 Î Q0, and let P(q) imply P(q¢) for any pair (q,q¢) such that (q,q¢) Î d. Then P is an invariant for M. The proof is performed by induction on the length of executions of M. We assume that \mathfrakP(n) be the predicate that P(q) is true for any state q contained in an exectution of length n.

\mathfrakP(1) is true because any execution with length 1 contains just an initial state, and the hipothesis of the theorem states P(q0) for all q0 Î Q0. Assuming that \mathfrakP(n), for an execution of length n+1 and one of its states q¢ we have: if q¢ is its initial state, the P(q¢); otherwise q¢ is preceded by another state q that is part of an execution of length n and as such P(q). But, also (by the definition of execution of a finite state machine) (q,q¢) Î d and, because P(q), by the hipothesis of the theorem, P(q¢). Thus, P(q) for any q in an execution of M.

The significance of this theorem is that we can prove invariants by induction. In general, given a property about a program, we will prove that the property holds after the initialization of the program's variables and that it is preserved by any transition in the program (any change the values of those variables.)

1.5  Back to the jug's problem...

Returning to the problem of proving that given just 2 jugs (with respective capacities of 3 and 6 units), we can easily prove that both jugs will always contain a multiple of 3 units in water.

This is true for initial state when they are both empty. It can easily be proved that the property is preserved for each of the allowed transitions. For example, for the transition
(a,b) ®(min(a+b,3),max(0,a+b-3))
if a and b are multiples of 3, so it is (a+b), and, as a result, also min(a+b,3). Also, (a+b-3) is a multiple of 3 and so is max(0,a+b-3). The invariance of the property can easily be checked for the other transitions.

1.6  Invariants and programs' properties

Invariants can be used prove interisting properties of given programs. The most interesting are:

1.6.1  A naï ve multiplication algorithm

An algorithm for multiplying 2 natural numbers. Given two non-negative integers, it returns their product.

Algorithm 1 Naïve multiplication.

int¯
 naive(int a,int b){
 l1: int x=a, y=b,z=0;

 loop: //Invariant: x*y+z = a*b
 l2: if¯ (x==0)
 l3: return z;
 l4: z+=y;
 l5: x-;
 l6: goto loop;
}

Partial correctness. It is not difficult to check that the invariant holds after each cycle of the loop. It After the initial step (l2), the invariant because z = 0, x = a,y = b. If, at any time, the program is executing l2, and invariant holds, at the end of cycle (step l6), we have: z'=z+y
x'=x-1
y'=y the invariant at l_6: x'*y'+z'=a*b
(x-1)*y+(z+y)=a*b
x*y+z=a*b In general, if x is variable, we will use x¢ to denote the value of x after some update.

Termination. It is easy to see that if a ³ 0 the algorithm terminates: x is initialized with a, and is decremented in every each of the loop. When it reaches 0 the loop terminates, and z = a*b, given our loop invariant that states:
x*y+z = a*b

Algorithm's complexity. The following invariant can be easily proved:
x+k = a
where k is counts the number of times around the loop. At the start, k = 0, and x = a, which trivially verifies our invariant. Every cycle of the loop decrements x and increments k, preserving the invariant. By the end of the loop, k will have count a cycles of the loop. Assuming that instruction li takes time ti, that gives
æ
è
t1+n*(t2+t4+t5+t6)+t2+tt3 ö
ø
units of time

1.6.2  Another multiplication algorithm

Algorithm 2 Russian peasant. int¯
 russian(int a,int b){
 l1: int x=a,y=b,z=0;
 loop: // Invariant: x*y+z = a*b
 l2:if¯ (x = = 0)
 l3: return z;
 l4:if¯ (1 = = x%2)
 l5: z+ = y;
 l6:y << = 1;
 l7:x >> = 1;
 l8:goto loop;
 }

Partial correctness. The invariant is same as in the previous algorithm. For the same reasons, it holds step l1, that initializes variables. The proof that invariant is mantained by the loop is has two cases. when x is odd, we have: x'=(x-1)/2
y'=2y
z'=z+y and
x'*y'+z'=x-1*2y+z+y=x*y+z If x is even: x'=x/2
y'=2y
z'=z
x'*y'+z'=x*2y+z=x*y+z

Program termination. It is possible to define a bound function mapping the variables of the program to the value of x. Every time around the loop, the value of x is shifted one bit to right. When it becomes 0, the loop is finished, and, given the invariant above is mantained, z = a*b.

Algorithm's complexity. We can prove the following invariant, where |n2| represents the number of bits necessary to represent n, and k is the number of times around the loop:
|x2|+k = |a2|
The invariant is true after l2, when x = a and k = 0. It is preserved by each step around the which decrements the number of bits in x by 1 and increments k by 1. By the end, k = |a2|. But, how can we get the number of bits necessary to represent a? This number is given by the following relation, that in the end defines the number of times the loop is repeated: T(1) = 1 T(a) = 1+T(a>>1)

This is a recurrence relation. Recurrence relations have the nice property that they can be easily obtained from the code of our algorithms. They present an obstacle that is the fact that they don't give a closed formula: T(a) is defined in terms of T(a >> 1), which is in the end T(a/2). But, the intuition behind the relation above seems clear enough: we are counting the number of times we can divide a by 2, and that would be log2a, more exactly:
T(a) = 1+log2a
would be the solution for that recurrence. We can prove it by induction: T(1) = 1+log_2^1= 1 as in our relation
T(a)=1+_2^a is assumed for any a<n
T(a)=1+T(a) our relation
T(a)=1+(1+_2^a our inductive hipothesis
T(a)=1+1+_2^a-_2^2
T(a)=1+_2^a

Thus, the time for that algorithm is given by:


Trussian(a,b) < t1+log2c*(t2+t4+t5+t6+t7+t8)+t2+t3

1.6.3  Horner algorithm

Target: to write a program that computes the value of a polynomial
p(x) = an xn + an-1 xn-1 + ¼+ a0 x0
for a given value of the variable x.

Given:
  • a vector with the coeficients ai;
  • the size of n;
  • the value of x.


Algorithm 3 Horner's algorithm.

dou¯ble
 horner(const double a[],const int n,const double x){
l1:  double v=0;
l2:  int i=n;
l3:  // Invariant: -1 £ i £ n Ùv = åj = 0n-i-1an-j xn-i-1-j
 loop:¯
l4:   if¯ (i < 0)
l5:    return v;
l6:   v = (v*x)+a[i];
l7:   i-;
l8:   goto loop;
 }

Partial correctness. It is not difficult to check that invariant defined at line l3 holds for each cycle of the loop. The invariant holds (trivially) after l2, when i = n and (åj = 0n-i-1an-j xn-i-1-j) equals 0 (0 being the identity of the sum.) Let us assume that the property holds for some i = h, (0 £ h £ n). _j=0^n-h-1a_n-jx^n-h-1-j,                                     at l4:i=h      
v'=(_j=0^n-h-1a_n-j*x^n-h-1-j)*x+a_h,              at l7: v updated to v'
v'=_j=0^n-h-1a_n-j*x^n-h-j+a_h,              at l7: v updated to v'
v'=(a_n x^n-h+a_n-1x^n-h-1+...+a_h+1 x^1)+a_h
v'=_j=0^n-ha_n-jx^n-h-j
v'=_j=0^n-(h-1)-1a_n-jx^n-(h-1)-1-j But, the last one is the value of v given by our invariant, when i = h-1, i.e., after i is updated at the of the cycle that started with i = h. Thus, the invariant is preserved throughout the loop.

Loop termination. To understand why the loop terminates, we just have to notice that values of i for a well-ordered set, with least element always equal to -1, the value of i for which the loop finishes.

But, if the loop finishes with i = -1, then it finishes with
v = n-(-1)-1
å
j = 0 
an-j xn-(-1)-1-j = n
å
j = 0 
an-jxn-j
the value we want to compute.


Footnotes:

1Even though, this game has been played in brazilian bars for the last 50 years, Leighton detected that the game was presented in the film ``Die Hard 3'' as a hard puzzle for Bruce Willis to solve in order avoid a bomb to explode.


File translated from TEX by TTH, version 2.66.
On 5 Apr 2000, 17:20.