415 lecture note #6 

[Ch 3] Algorithms (4)

3.5 Complexity of Algorithms -- cont.

5. A few comments on deriving complexity

• Watch out for negative terms
  1. Show: 60n² + 5 = O(n²) 

     Proof:  

     60n² + 5 <= 60n² + 5n²  .. since n >= 1
                    = 65n²  

     So, we can take c = 65 and n0 = 1.

  2. Show: 60n² - 5 = O(n²) 

     Proof:  

     
     
     
     

      

      

• Some known sums
  1. Arithmetic Series:

     Complexity: O(____).

      

  2. Geometric Series:

     Example:

                                           2ⁿ⁺¹ - 1
     1 + 2¹ + 2² + .. + 2ⁿ =  ---------- = 2ⁿ⁺¹ - 1
                                             2 - 1

     Complexity: O(____).

     Proof:  

     
     
     
     
     
     
     

6. More (non-recursive) Example Algorithms

For each of the algorithm segments,

• Derive the exact number of times a particular line is executed; and
• State the complexity of the algorithm segment.

1. General iterative algorithms

1. 1. for i := 1 to 2n do
   2.   x := x + 1           // count this line

   • Number of times line 2 is executed:

   

   • The complexity is Q(n).

      

2. 1. for i := 1 to n do
   2.   for j := 1 to n do
   3.      x := x + 1           // count this line

   • Number of times line 3 is executed:

     

   • The complexity is Q(n²).

      

3. 1. for i := 1 to 2n do
   2.   for j := 1 to n do
   3.      x := x + 1           // count this line

   • Number of times line 3 is executed:

     
     
     
     
     
     

   • The complexity is ____________.

      

4. 1. for i := 1 to n do
   2.   for j := 1 to 2n do
   3.      x := x + 1           // count this line

   • Number of times line 3 is executed:

     
     
     
     
     
     

   • The complexity is ____________.

      

5. 1. for i := 1 to n do
   2.   for j := 1 to i do
   3.      x := x + 1           // count this line

   • Number of times line 3 is executed:

     

   • The complexity is Q(n²).

      

6. (Exercise #22, p. 150)

   1. for i := 1 to n do
   2.   for j := 1 to n do
   3.     for k := 1 to i do
   4.      x := x + 1                // count this line

   • Number of times line 4 is executed:

     
     
     
     
     
     
     

   • The complexity is ____________.

      

7. (Exercise #25, p. 150) -- once again

   1. i := n
   2. while i >= 1 do
   3.   begin
   4.   for j :=1 to n do
   5.     x := x + 1                // count this line
   6.   i := floor(i/2)
   7.   end

   • Number of times line 5 is executed

     
     
     
     
     
     
     
     
     
     

   • The complexity is ____________.

      

8. (Example 3.5.14, p. 147)

   1. j := n
   2. while j >= 1 do
   3.   begin
   4.   for i :=1 to j do
   5.     x := x + 1                // count this line
   6.   j := floor(j/2)
   7.   end

   • Number of times line 5 is executed

     
     
     
     
     
     
     
     
     
     
     

   • The complexity is ____________.

2. Search algorithms
   1. Linear search in an unsorted (or any) sequence

      Problem: finding an element in an array of size n
      Input: key, array A = [x₁, x₂,…, x_n], n
      Output: the index of key in A, (0 if key is not in the list)

      Program find_key( key, A, n )
      1. index := 1
      2. while index <= n
      3.   begin
      4.   if key = A[index] then   // count this comparison
      5.     return index
      6.   index := index + 1
      7.   end
      8. return 0
      end find_key

      • Complexity

        Worst-case	T(n) = _____ = Q(___)
        Best-case	T(n) = _____ = Q(___)
        Average-case	T(n) = _____ = Q(___)

2. Binary Search -- for sorted sequence (in ascending order)

   Example:  Search for 16

   

    

   • The algorithm:

   Problem: find an element in a sorted list (whose index is 0 through n-1)
   Input: key, L a list and n
   Output: The index of key in L, -1 otherwise

   procedure BinarySearch(key, L, n)
   1.  low := 0 
   2.  high := n-1
   3.  while (low <= high) do
   4.    begin
   5.    mid = floor((low + high)/2)
   6.    if (key = L[mid]) then        // count this comparison
   7.      return mid
   8.    else if (key < L[mid]) then 
   9.      high := mid - 1
   10.   else 
   11.     low := mid + 1
   12.   end
   13. return -1    // key not found (fall through)
   end BinarySearch

   • Complexity

     1. Worst -case

        The worse case occurs when key is not present is the list. How many runs through the while loop will we do? 

        • Initially, we have n element to consider. 
        • At the end of the first loop, we are down to n/2. 
        • After the second loop, we have n/4. 
        • Next, n/8, etc. 
        • In general, at the end of the kth loop, we have n/2^k elements left to consider.

        How far can we keep dividing (what is k?)? Again, according to the algorithm we exit the loop when low > high.  This means, in particular, that when we are down to 1 element and process it, we stop the loop.  So,

        

        So we get T(n) = lgn, thus Q(lgn).

         

     2. Best-case

        The best case happens when key is present at the middle of the array, namely L[n/2].   In this case, the loop executes only once, regardless of the size of array (i.e., n).  So, we get T(n) = 1, thus Q(1).

         

     3. Average-case