To Lecture Notes

IT 223 -- 2/23/11

Review Questions

1. What is the formula for the standard deviation ( σ_x ) for a Bernoulli random variable x?

   Ans: σ_x = √p(1 - p)

2. For a dataset consisting of n independent outcomes of a Bernoulli random variable, what does the sum represent? what does the sample mean represent?

   Ans: The sum represents the number of sucesses (because success = 1 and failure = 0). The average is the proportion of successes = S / n.

3. Suppose that E(x₁) = 3, E(x₂) = 7, and E(x₃) = 2. Also suppose that Var(x₁) = 1, Var(x₃) = 4, and Var(x₃) = 5. Evaluate:
   1. E(x₁ + x₂ + x₃)

   2. E(57 x₂)

   3. Var(x₁ + x₂ + x₂), if x₂, x₂, and x₂ are independent.

   Hint: See the Properties of Random Variables. Ans: a. 3+7+2 = 12;   b. 57*7 = 399;   c. 1+4+5 = 10.

4. Use SPSS to simulate the following situations. Use the random number generator Rv.Binomial(n, p). Compute the number of successes S and the proportion of successes p^ in each case.
   1. 1,000 outcomes of a Bernouilli random variable with p = 0.5.

   2. 10,000 outcomes of a Bernouilli random variable with p = 0.7.

   3. 100,000 outcomes of a Bernouilli random variable with p = 0.9.

   A binomial random variable outputs the sum of n outcomes of a Bernouilli random variable with probability p.

   Directions: To do Part c, use this SPSS Syntax script to generate line numbers from 1 to 100,000. Then use Transform >> Compute Variable with Target variable = x and expression Rv.Bernoulli(0.9) to generate the random variables.

5. Is it possible for two events to be both independent and mutually exclusive at the same time?

   Ans: If A and B are independent, P(A and B) = P(A) P(B). But if A and B are mutually exclusive, P(A and B) = 0. Thus P(A) P(B) = 0 and either P(A) = 0 or P(B) = 0.

Sums of Random Variables

• Define S = x₁ + ... + x_n, where each x_i has the same expected value and variance, and all of the x_i are independent.

• If the x_i are independent Bernoulli random variables, then S is a Binomial random variable.

• The expected value of the sum of n outcomes of the random variable x:
  E(S) = E(x₁ + ... + x_n) = E(x₁) + ... + E(x_n) = nE(x)

  E(S) = nE(x)

• The variance of the sum of n independent outcomes of the random variable x:
  Var(S) = Var(x₁ + ... + x_n) = Var(x₁) + ... + Var(x_n) = nVar(x)

  Var(S) = nVar(x)

  Recall that the variance of the sum of random variables equals the sum of the variances if the random variables are independent.

• The standard deviation of the sum of n independent outcomes of a random variable:
  σ_S = σ_{x1 + ... + xn} = √[Var(x₁ + ... + x_n)]

      = √[Var(x₁) + ... + Var(x_n)] = √nVar(x) = σ_x√n

  σS = σx√n

Practice Problems

5. Compute the expected value and standard deviation of a Bernoulli random variable for the sum of n trials.

   Ans: E(S) = nE(x) = np
   σ_S = σ_x √n = √p(1-p) √n = √np(1-p)

6. Suppose that my true percentage of making free throws is 70%. Out of 100 attempts, what is the expected value and standard deviation of the number of free throws made?

   Ans: For a binomial random variable, S is the number of freethrows, E(S) = np = 100*0.7 = 70, SE_S = sqrt(n*p*(1-p)) = sqrt(100*0.7*(1-0.7)) = 4.58.

The Law of Averages

• The official name of the Law of Averages is the Law of Large Numbers (LLN). It was first proved by Jakob Bernoulli from Basil, Switzerland in 1713.

• LLN states that if x₁, ... , x_n are independent outcomes of the same random variable x and n is large enough, then the average
  x = (x₁ + ... + x_n) / n

  is close to the expected value of the random variable E(x) with high probability.

• LLN works by swamping, not by compensation. What does this mean?

• Experiment: Use SPSS to simulate 100, 500, 1000, 5,000, 10,000, and 50,000 tosses of a fair coin. Are the proportions obtained consistant with LLN?

Factorials and Counting Combinations

• Recall that n! (read n factorial, n shriek, n bang) is defined as
  n! = n × (n-1) × (n-2) × ... × 3 × 2 × 1

• n! is the number of ways of arranging n distict objects.

• 3! = 6. For example, the ways of arranging A, B, and C is
  
  (ABC)   (ACB)   (BAC)   (BCA)   (CAB)   (CBA)

• 2! = 2. The ways of arranging A and B is
  
  (AB)   (BA)

• 1! = 1. The ways of arranging the single object A is
  
  (A)

• What is 0!?

• The number of ways of drawing k objects from an urn that contains n distinct objects is

  ( n k )

  =

  n! k!(n - k)!

• The binomial coefficient defined in the previous bullet point is also written as nCk and read "n choose k."

• The binomial coefficients can also be obtained from Pascal's Δ:

                      1
                   1     1
                1     2     1
              1    3     3     1
           1    4     6     4     1
        1     5    10    10    5     1
      1    6    15    20    15    6     1
    1   7    21    35    35    21    7     1
  1   8   28    56    70    56    28    8     1
  

• Each entry in Pascal's triangle can be obtained recursively by adding the two entries above it.

• nCk can be obtained from in Pascal's Δ as entry k in row n.

Practice Problems

8. What is the number of ways of choosing 2 out of 4 objects?

   Ans: 4C2 = 4! / (2!(4-2)!) = (4×3×2×1) / [(2×1)(2×1)] = (4×3) / (2×1) = 2×3 = 6

9. What is the number of ways of choosing 3 out of 5 objects?

   Ans: 5C3 = 5! / (3!(5-3)!) = (5×4×3×2×1) / [(3×2×1)(2×1)] = (5×4) / (2×1) = 5×2 = 10

10. What is the number of ways of choosing 6 out of 10 objects?

    Ans: 10C6 = 10! / (6!(10-4)!) = (10×9×8×7×6×5×4×3×2×1) / [(6×5×4×3×2×1) (4×3×2×1)] = (10×9×8×7) / (4×3×2×1) = 10×3×7, because 4 and 2 in the denominator cancel with 8 in the numerator, and 3 in the denominator cancels with 9 in the numerator, leaving 3. The answer is 210.

11. What is the number of ways of choosing 13 out of 20 objects?

    Ans: 20C13 = (20×19×18×17×16×15×14) / (7×6×5×4×3×2×1) = (19×17×16×15, because 5 and 4 in the denominator cancel with 10 in the numerator, 6 and 3 in the denominator cancel with 18 in the numerator, 7 and 2 the denominator cancels with 14. The answer is 77,520.

The Binomial Distribution

• The Binomial Formula is defined as

  P(k successes in n trials) =

  ( n k )

  pk (1 - p)n-k

• This formula is valid because
  1. The trials are independent so the multiplication rule can be used.

  2. P(success) = p, P(failure) = 1 - p

  3. There are n choose k ways to choose k successes out of n trials.

Practice Problems

12. When tossing a fair coin, what is the probability of obtaining
    1. 3 heads out of 5?

       Ans: 5C3 (0.5)^3 (1-0.5)^(5-3) = 10(0.125)(0.25) = 0.03125 = 3%.

    2. 7 heads out of 9?

       Ans: 9C7 (0.5)^7 (1-0.5)^(9-7) = 36 (0.001953125) = 0.0703125 = 7%.

    3. 10 heads out of 14?

       Ans: 14C10 (0.5)^10 (1-0.5)^(14-10) = 1001×0.0000061352 = 0.0611 = 6%.

    4. 10 or 11 heads out of 14?

       Ans: P(10 out of 14) + P(11 out of 14) = 0.0611 + 0.0222 = 0.0833

    5. 10 or more heads out of 14

       Ans: P(10 out of 14) + P(11 out of 14) + P(12 out of 14) + P(13 out of 14) + P(14 out of 14) = 0.0611 + 0.0222 + 0.00555 + 0.00085 + 0.0000610 = 0.089761.

13. If your true probability of shooting freethrows for practice is 70%, what is the probability that you shoot 16, 17, or 18 for 20?

    Ans: P(16 for 20) + P(17 for 20) + P(18 for 20) = 0.1304 + 0.0716 + 0.0278 = 0.2298 = 23%.

14. If your true probability of shooting freethrows in practice is 70%, what is your expected number of freethrows made out of 20? what is your standard deviation?

    Ans: E(S) = np = 10(0.7) = 7; σ_S = √np(1-p) = √20(0.7)(1-0.7) = 2.049

Project 4

• Look at the Project 4 Description.

• Simulating repeated trials from a Bernoulli random variable using SPSS.

• Use Rv. Binom to compute generate sums of Bernoulli random variables for Part C of Project 4.

The Central Limit Theorem

• The Central Limit Theorem (CLT) was first postulated by Abraham de Moivre in 1733 to estimate probabilities of the number of heads resulting from many tosses of a fair coin.

• In 1901, the Russian mathematician Alexandr Lyapunov stated and proved the CLT in its modern form:
  If x₁, ... , x_n are independent observations from the same random variable x of any distribution and n is large enough, then the sum
  S = x₁ + ... + x_n

  is approximately normally distributed with expected value E(S) = nE(x) and standard deviation SE_S = σ_x√n.

• For our purposes, "n is large enough" if n ≥ 30.