To Lecture Notes

ISP 121 -- 4/21/08

Descriptive Statistics

• A univariate dataset contains only one variable.

• Descriptive statistics allow us to get a sense of a dataset without looking at every observation in detail.

• Common descriptive statistics include the average, standard deviation, minimum, Q1, median, Q3 and maximum.

• Example: Bank Waiting Times in minutes (BWT)
  7.7  6.7  4.1  6.2  9.3  5.2  7.2  11.0  8.5  5.6  7.7

• The average the n observations x₁, ... , x_n is
  x = (x₁ + ... + x_n) / n

• The average of the BWT is 7.2.

• Here are the sorted BWT observations:
  4.1  5.2  5.6  6.2  6.7  7.2  7.7  7.7  8.5  9.3  11.0

• The median is the middle observation in the sorted list.

• If n is even, take the median to be the average of the two middle observations.

• Median of BWT is 7.2.

• The first quartile (Q1) is the median of the bottom half of the list.

• If n is odd, include the middle observation in the bottom half of the list.

• Q1 if BWT is 5.9.

• The third quartile is the median of the top half of the list.

• If n is odd, include the middle observation in the top half of the list.

• The second quartile is the same as the median.

• Q3 if BWT is 8.1.

Review Questions

1. Given the sorted list
   1   4   6   7   8   10   11   14   19

   compute the quartiles Q0, Q1, Q2, Q3, Q4.

   Note that Q0 = minimum, Q2 = median and Q4 = maximum.
   Ans: Q0 = 1, Q1 = 6, Q2 = 8, Q3 = 11, Q4 = 19.

2. Which Excel functions do you use to compute the quartiles of a sorted list?
   Ans: Use =QUARTILE(A1:A9, x) if A1:A9 is the list and x is the desired quartile.

Outliers

• The interquartile range IQR is defined as
  IQR = Q3 - Q1

• Compute the IQR of the observations in Review Question 1.

• An outlier is an observation that is 1.5 IQRs below Q1 or 1.5 IQRs above Q3.

• An extreme outlier is an observation that is 3 IQRs below Q1 or 3 IQRs above Q3.

• A mild outlier is an outlier that is not extreme.

• Identify the mild and extreme outliers in the observations in Review Question 1.

• Outliers in measurement data often indicate a problem with a measurement process.

• Look at the weighings of the 10 g prototype weight data in NBS-10.xls.

• The first three weighings in grams are
  9.999591   9.999600   9.999594

• The number of grams below 10 g is
  0.000409   0.000400   0.000406

• The Excel dataset records the number of micrograms below 10 g:
  409   400   406   ...

• Are there any outliers in the NBS 10 dataset?
  Q1 = 401    Q3 = 407    IQR = Q3 - Q1 = 407 - 401 = 6

  Q1 - 1.5 × IQR = 401 - 1.5 × 6 = 392    Q1 - 3 × IQR = 401 - 3 × 6 = 383

  Q1 + 1.5 × IQR = 407 + 1.5 × 6 = 416    Q1 + 3 × IQR = 401 + 3 × 6 = 425

• Look at the sorted NBS 10 data. These are the outliers:
  Extreme outliers less than 383: 375

  Mild outliers between 383 and 392: None

  Mild outliers between 416 and 425: 418 and 423

  Extreme outliers more than 425: 437

Histograms

• A histogram is a chart showing how the observations are distributed.

• Each of the observations is put into a bin.

• The base of the each rectangle is positioned over the bin.

• The height of each rectangle is equal to the number of observations in that bin.

• See the document Excel Histograms for creating a histogram in Excel.

• Here is a histogram of the NBS 10 dataset in NBS-10.xls:

  

• The average of the data in a histogram is the center of gravity of the histogram; the histogram will balance over the average.

• Many histograms of real data are bell shaped. Here is the standard bell-shaped curve:

  

• Notice that the bell-shaped curve is symmetric around its center.

• If we disregard the outliers 375 and 437, the histogram of the NBS-10 data is roughly bell-shaped.

• Here is the histogram of some times between eruptions of the Old Faithful Geyser in minutes:

  

• This histogram is not bell-shaped, so the average and SD are not a good summary of the data.

Standard Deviation

• Recall the definition of the average:
  x = (x₁ + ... + x_n) / n

• The standard deviation SD is defined as
  SD = square root (average of squared deviations from the mean)

       = square root [( sum of (x_i - x)²) / n]

• The SD describes about how far an observation is away from the mean, on the average.

• Example: What is the SD of these observations?
  20   10   15   15  
  1. Average = (20 + 10 + 15 + 15) / 4 = 60 / 4 = 15

  2. Deviations from Average = 5   -5   0   0

  3. Average of Squared Deviations = (5² + (-5)² + 0² + 5⁰) / 4 = 12.5

  4. SD = Square Root (Ave. of Squared Deviations) = sqrt(12.5) = 3.54

• Example: What is the SD of these observations?
  4   4   4   4   4   4

• Example: What is the SD of these observations?
  1   1   1   9   9   9

• If the histogram of a dataset is bell-shaped, it can be described with only two summary statistics, the average ( x ) and the standard deviation (SD).

• The average is the center of the bell-shaped curve.

• The SD is the distance from the center to the inflection point (point where the curve changes from curving down to curving up).

• If a histogram is not bell-shaped, then the five statistics Q0, Q1, Q2, Q3 and Q4 form a better summary.

• Consider the dataset consisting of incomes of people in the U.S. What is a better summary statistic, the average or the median?

Measurement Error

• Here is an Excel spreadsheet, NBS-10.xls, containing the data results of 100 weekly weighings of a
  10 gram prototype weight. The amount in micrograms below 10 grams is recorded.

• The RawData sheet shows just the observation number and the result of the weighing.

• The DataWithAnalysis sheet shows the data with descriptive statistics and histograms.

• The ideal measurement model:
  actual measurement = true measurement + random error

• A more realistic measurement model:
  actual measurement = true measurement + random error + bias

Activity 5

• The Activity A5 Description.