415 lecture note #9 

[Ch 5] Recurrence Relations (3)

5.3 Analysis of Algorithms

• Now we know how to analyze the complexity of various algorithms with loops and recursion.
• Lecture Note #6: Wrap-up and Review of non-recursive algorithms
  • Be sure to understand the S notation!!
  • Binary Search algorithm (Non-recursive version)

More Recursive Algorithms

• There are many computer algorithms which use recursion.  Now we can analyze their complexity!

A) Searching Algorithms

(1) Recursive Binary Search

• Binary Search can be written recursively as well.
• Note: 
  • The recursive version receives high and low indices instead of n (the size of a list).
  • The algorithm below is slightly different from the one in the textbook, mostly by the use of 0 as the start index (instead of 1).

  Problem: Find an element in a sorted list (whose index is 0 through n-1)  Input: key, L a list, and indices high and low Output: The index of key in L, -1 otherwise procedure RecBinSearch(L, key, high, low) 1. if (low > high) then // base case (1): key doesn't exist 2. return -1 3. mid = floor((low + high)/2) 4. if (key = L[mid]) then // base case (2): key found 5. return mid 6. else if (key < L[mid]) then 7. return RecBinSearch(L, key, mid-1, low) // recursive call to left half 8. else 9. return RecBinSearch(L, key, high, mid+1) // recursive call to right half end RecBinSearch

• Complexity analysis

  Notation:  A(n) denotes the number of times the function is called when there are n elements between  high and low (i.e., n = high - low + 1).

  Recurrence relation: A(n) = __________________________  for all n >= __________, with initial condition A(___) = _____.

We know how to solve this recurrence and derive complexity.

(will do in the class)

 

Closed form: A(n) = __________________________  for all n >= __________ Theta complexity: A(n) = Q(_______)

 Case analysis:

• Worst case  -- when key is not in L.  The algorithm must be invoked all the way down to when n = 1.  So, Q(_______).
• Best case -- when key is found in the first/original call to the algorithm (i.e., key is at L[mid]).  So the algorithm is invoked once, and Q(_______).
• Average case  -- when key is found in the middle of recursive invocations (1/2 of the worst case).  So, Q(_______).

(2) Linear Search in an Unsorted Sequence

• Recursive version of the find_key algorithm.

  (will do in the class)

B) Sorting Algorithms

(1) Merge Sort

• One of the most efficient sorting algorithms.  Uses a strategy called divide-and-conquer.
• How the algorithm works for an array L = [5, 2, 4, 6, 1, 3, 2, 6].

  

• The algorithm:

  Problem: Sort elements in a list L of length n (whose index is 0 through n-1) in an ascending order. Input: L (a list), and i, j (indices) Output: nothing.  Elements in L will be sorted in place. procedure MergeSort(L, i, j) 1. if i = j then // base case; 1-element sequence 2. return 3. m := floor((i+j)/2) 4. MergeSort(L, i, m) // recursive call to left half 5. MergeSort(L, m+1, j) // recursive call to right half 6. Merge(L, i, m, j) // merge 2 sequences L[i thru m] & L[m+1 thru j] end MergeSort

• To analyze this algorithm, we need to know how long it takes in the Merge procedure.

  -------------------------------------------------------------------
  The Merge Procedure

  Problem: Combine two lists into one list so that elements in the resulting list are sorted in ascending order.
  Input: L (a list), and i, m, j (indices) where i <= m <= j.  There are n elements between L[i] and L[j] (inclusive).
  Output: nothing.  Elements in a sublist in L will be sorted in place.

  Example: Merge two lists [1, 3] and [2, 6] in L = [5, 2, 4, 6, 1, 3, 2, 6] and i = 4, m = 5, j = 7

  
  

  • The algorithm for Merge:

    procedure Merge(L, i, m, j)
    1.  p := i           // index for the first list
    2.  q := m + 1       // index for the second list
    3.  C := array of size (j - i + 1)  // temporary list
    4.  r := 0           // index for C
    5.  while (p <= m) and (q <= j) do
    6.    begin
    7.    if (L[p] < L[q]) then  // L[p] is smaller
    8.      begin
    9.      C[r] := L[p]     // put it in C
    10.     p := p + 1
    11.     end
    12.   else                   // or L[q] is smaller
    13.     begin
    14.     C[r] := L[q]     // put it in C
    15.     q := q + 1
    16.     end
    17.   r := r + 1         // either case, increment r
    18.   end
    19. while (p <= m) do  // copy items left in the first half into C
    20.   begin
    21.   C[r] := L[p]
    22.   p := p + 1
    23.   r := r + 1
    24.   end
    25. while (q <= j) do  // copy items left in the second half into C
    26.   begin
    27.   C[r] := L[q]
    28.   q := q + 1
    29.   r := r + 1
    30.   end
    31. p := i             // reset p
    32. r := 0             // reset r
    33. while (p <= j) do  // copy items in C back into L
    34.   begin
    35.   L[p] := C[r]
    36.   p := p + 1
    37.   r := r + 1
    38.   end
    end Merge

  • Complexity of Merge procedure

    Count the total number of times elements in L are copied into C (and ignore the times needed to copy elements in C back into L in the last while-loop.  It is  (j - i + 1 ) = n times.

  -------------------------------------------------------------------

  back to MergeSort

• Complexity of MergeSort

  Notation:  A(n) denotes the number of times elements are moved/copied in Merge, when there are n elements between  i and j (i.e., n = i - j + 1). 
  Note the analysis here is slightly different from the textbook.

  Recurrence relation: A(n) = __________________________  for all n >= __________, with initial condition A(1) = 0.

  We know how to solve this recurrence and derive complexity.

  (will do in the class)

   

  Closed form: A(n) = __________________________  for all n >= __________ Theta complexity: A(n) = Q(_______)

   Case analysis:

  • Worst case  -- Q(_______).
  • Best case -- Q(_______).
  • Average case  -- Q(_______).