Algorithm Analysis (2)

0. Complexity Classes

• We often describe the complexity of an algorithm or method by giving which of 7 (or so) standard functions it is closest to.
• In addition to running time, we could also calculate the space usage as a function of the input size and talk about the space complexity.
• A set of "standard" functions of N that are used:

  F(N)	Complexity Class
  1	Constant
  log(N)	Logarithm
  N	Linear
  Nlog(N)	N-Log-N (or linearithmic)
  N2	Quadratic
  N3	Cubic
  2N	Exponential

1. Doubling Ratio Experiments -- What Happens if N is Doubled

• Table of Ratios of each Growth Function when the size is doubled.
   

  Complexity Class	F(N)	F(2N)	F(2N) / F(N) -- ratio
  Constant	1	1	1
  Linear	N	2N	2
  Quadratic	N2	4N2	4
  Cubic	N3	8N3	8
  Logarithm	log(N)	1 + log(N)	1 + 1/log(N)
  N-Log-N (or linearithmic)	N*log(N)	2N*log(N) + 2N	2 + 2/log(N)
  Exponential	2N	22N	2N

  For example, F(2N) = (2N)² = 4N²

• Example empirical results -- O(n) vs. O(n^2)
algorithms

Linear: n = 1000, time = 13859
Quadratic: n = 1000, time = 11861968

Linear: n = 2000, time = 24252
Quadratic: n = 2000, time = 47375115

Linear: n = 4000, time = 47734
Quadratic: n = 4000, time = 189609017

Linear: n = 8000, time = 96239
Quadratic: n = 8000, time = 757734300

Linear: n = 16000, time = 189397
Quadratic: n = 16000, time = 3030653875

2. Complexity of operations on Array-based lists and Linked lists

Java Collection	    ArrayList		LinkedList
======================================================

add(item) --------- constant/linear*    constant

add(idx, item) ---- linear (**)         linear

contains(item) ---- linear              linear

remove(item) ------ linear              linear

remove(idx, item)-- linear              linear

get(idx) ---------- constant            linear

set(idx,item) ----- constant            linear

iterator

    hasNext-------- constant            constant

    next ---------- constant            constant

    remove -------- linear              constant

*: Adding to the end of an arraylist is linear when the array is full and needs to be expanded

• Example: ArrayList: public void add(int index, E item) // (**)

  This method adds the new element x to the private array member at position index, but all the elements currently at position index, index+1, index+2, ... must be moved to the next higher position to make room for the new element x at position index.

  This method is not so efficient. In particular, adding a new element at the beginning of the array:

  	add(0, x)    

  will have to move all the current elements in the array and so the time for this method is not constant, but will depend on the number of elements currently in the array. Thus its complexity is O(N).

3. Timing Java Methods

• The System class has a static method:

        public static long currentTimeMillis()    

  which returns the number of milliseconds since the "epoch" (midnight, Jan 1, 1970).

  By calling System.currentTimeMillis() before and after a call to the method to be timed, then computing the difference, you get a reasonable measure of the execution time of the method in milliseconds.

• Disabling the JIT compiler

  • Timing with Eclipse

    • Eclipse interfes with timing. The problem is that Eclipse is using "just in time compiling" optimization during execution. Certain chunks of code will be "interpreted" one or more times, but then the jit compiler replaces the java byte code with native machine code. This means this chunk of code will run much faster if it is executed again during this run of the program. This obviously throws off any attempts to time execution.

    • For instructions on how to set the parameters in Eclipse, look at this page.

  • Executing from Command Line

    • Compile the code as usual.  Then when you run the code, you provide a switch "-Djava.compiler=NONE".  For example,

      java -cp . -Djava.compiler=NONE MyProg

4. Example problems

1. BinarySearch -- O(lgn)

2. "2-Sum problem"

   • Problem: Count the number of pairs of integers in an array of n integers which sum up to a given value.
   • Example: Array contains {1, 24, 45, 6, 10, -8}, and find how many pairs of integers which make up 16.
   • Algorithms: O(n^2), but can be done in O(nlgn) as well.
     
     

3. "3-Sum problem"

   • Problem: Count the number of triples of integers in an array of n integers which sum up to a given value.
   • Complexity: ??? (to be discussed in the class)