Programming Assignment 2

CSC 323 - Data Analysis and Statistical Software

Due: 5/24/2001

You have been assigned to a software development team that is conducting a series of usability tests on a new version of a browser based user interface. The interface has been designed with expert users in mind and so has features to accommodate such users (see the Novice vs. Expert Users essay from Jakob Nielsens useit.com site).

Your team must demonstrate that the proposed interface will allow expert users to complete tasks more efficiently than the existing interface. You have been asked to assist with a controlled experiment involving twenty nine expert users. The purpose of this experiment is to determine if the proposed interface allows expert users to complete a suite of tasks (i.e. a set of four basic tasks) more quickly than the existing interface. It is known that the mean time to complete the task suite with the existing interface is 215 seconds.
Note: See the Usability Metrics essay for additional details on usability as well as details of a controlled experiment conducted by Macromedia.

You have been presented with the data collected for this experiment. Each observation in the file consists of the following values:

• Task #1 (seconds); 1-3
• Task #2 (seconds); 4-6
• Task #3 (seconds); 7-9
• Task #4 (seconds); 10-12
• Error Count; 13-14
• User ID; 15-19
• Observer ID; 20-24
• Satisfaction Score; 25-26

Notice that the times provided are for the individual tasks in the suite. You are interested in total task time. Also, you will need to examine the error count. Several users did not complete the task suite without making errors and so the task times for these users should be ignored (if necessary, see "DATA step statements", point 8, SAS Review).

Conduct a thorough analysis of these data. You will need to conduct a test of hypotheses and submit a report summarizing your findings. See additional details below.

1. Write a SAS program to analyze this dataset. Your program should do the following:
1. Read your data from an external file.
2. Compute total task time.
3. Ignore users with more than zero errors.
4. Execute the PRINT procedure.
5. Use the appropriate SAS procedures to produce the statistics and p-values to test your hypotheses.

Note: For PROC PRINT, be sure to use labels for column headings rather than variable names. Use names for data sets and variables that are meaningful. You should generate an appropriate title for the output of these procedures.

2. Write a short analysis (no more than two pages) of the output of your SAS program. Your analysis should at least address the following:
1. State the primary hypotheses. That is, the NULL and ALTERNATIVE hypotheses for the experiment described above.
2. State and address any other hypotheses needed for your analysis.
3. Discuss the significance of the p-value obtained for the primary hypotheses.
4. Given your findings, briefly explain why you should (or should not) provide a point estimate for the parameter in question. If you argue that a point estimate for the parameter is appropriate you must provide it as well as a 95% confidence interval for the parameter (with an interpretation).