Programming Assignment 2

CSC 323 - Data Analysis and Statistical Software

Due: 10/24/2001

You have been assigned to a software development team that is conducting a series of usability tests on a new version of a browser based user interface. The interface has been designed with expert users in mind and so has features to accommodate such users (see the Novice vs. Expert Users essay from Jakob Nielsens useit.com site).

Your team must demonstrate that the proposed interface will allow users to complete tasks more efficiently than the existing interface. You have been asked to assist with a controlled experiment involving forty two users who were observed working on a suite of tasks (i.e. a set of four basic tasks). You have been asked to investigate the performance of the proposed interface for expert users only. That is, can expert users complete the task suite more quickly using this new interface than they could using the existing interface. It is known that the mean time for expert users to complete the task suite with the existing interface is 165 seconds.
Note: See the Usability Metrics essay for additional details on usability as well as details of a controlled experiment conducted by Macromedia.

You have been presented with the data collected for this experiment. Each observation in the file consists of the following values:

• Task #1 (seconds); 1-3
• Task #2 (seconds); 4-6
• Task #3 (seconds); 7-9
• Task #4 (seconds); 10-12
• Error Count; 13-14
• User ID; 15-19
• Observer ID; 20-24
• Satisfaction Score; 25-26
• User Type; 27 (i.e. E - Expert, N - Novice)

Notice that the times provided are for the individual tasks in the suite. You are interested in total task time. Remember that you are only interested in expert users (i.e. User Type E). Also, you will need to examine the error count. Several users did not complete the task suite without making errors and so the task times for these users should be ignored (if necessary, see "DATA step statements", point 8, SAS Review).

Note: Do not edit the data to remove observations. Unwanted obervations must be bypassed by using appropriate SAS statements.

Conduct a thorough analysis of these data. You will need to conduct a test of hypotheses and submit a report summarizing your findings. See additional details below.

1. Write a SAS program to analyze this dataset. Your program should do the following:
1. Read your data from an external file.
2. Compute total task time.
3. Ignore novice users.
4. Ignore users with more than zero errors.
5. Execute the PRINT procedure.
6. Use the appropriate SAS procedures to produce the statistics and p-values to test your hypotheses.

Note: For PROC PRINT, be sure to use labels for column headings rather than variable names. Use names for data sets and variables that are meaningful. You should generate an appropriate title for the output of these procedures.

2. Write a short analysis (no more than two pages) of the output of your SAS program. Your analysis should at least address the following:
1. State the primary hypotheses. That is, the NULL and ALTERNATIVE hypotheses for the experiment described above.
2. State and address any other hypotheses needed for your analysis.
3. Discuss the significance of the p-value obtained for the primary hypotheses.
4. Given your findings, briefly explain why you should (or should not) provide a point estimate for the parameter in question. If you argue that a point estimate for the parameter is appropriate you must provide it as well as a 90% confidence interval for the parameter (with an interpretation).