Purpose and Logic of Experimentation
I Experiments
Experiments have been used widely because they are very powerful tools for identifying causal relationships.
A Purpose of experiments
1. Test hypothesis
Hypothesis are statements that can be empirically tested. The following are all examples of hypothesis.
Playing laugh tracks during a TV sitcom makes watching TV shows more enjoyable.
Placing people into groups makes people try to maximize the difference between themselves and others.
2. Goal is to identify causal relationships.
Want to be able to say with confidence that a change in one variable actually causes or produces a change in another variable.
B Logic behind experiments (Key features)
At a very basic level, all experiments are the same. They all involve the same essential features.
1. Manipulation
All experiments involve the manipulation of a variable or variables. In other words, a researcher has control over changes in a variable.
If you were going to study the effects of using a laugh track with a sitcom or not, the researcher would manipulate whether or not a laugh track was played.
In short, manipulation simply means that a researcher has control over how the independent variable changes.
Now, when manipulating a variable or set of variables, it is important that these changes are real.
In particular, two things are very important.
a. Change has to be big enough that it will actually produce a difference.
Say for example, trying to determine whether or not similarity leads to attraction, well, I better be sure that when I manipulate similarity that the difference is big enough that it could actually produce an effect. In other words, you wouldn't want to make changes in a variable so minute that they do not have an impact.
b. Change in variable must be valid.
In other words, when manipulating a variable, I must make sure that I am actually changing what I think I am changing. That is, I must make sure that the manipulation actually worked.
For instance, think about the idea that similarity leads to attraction. If you are going to manipulate similarity, it is important to make sure that you are manipulating similarity and not some other variable.
To determining whether or not a manipulation was big enough and valid, researchers typically employ manipulation checks.
Manipulation Checks are tests done to evaluate whether or not the manipulation worked.
Basically two different ways that manipulation checks can be conducted.
1. Pre-testing.
Check the manipulation out with a different group of subjects before using them with the real subjects in the experiment.
So, say were trying to determine if watching violent films produces aggressive behavior.
In this experiment, you are manipulating violence. That is, some people would be exposed to a violent film and others would be exposed to a non-violent film and then you would measure their aggressive behavior.
However, before doing this study, you might want to check whether or not your two films really differ from each other in terms of violence.
So, you would have students watch the films and rate them for their violence, before actually using them in the real experiment.
2. Post-testing
Sometimes you can check whether your manipulation worked after experiment is over. Rather than testing the manipulation before the experiment takes place, you ask people to evaluate the manipulation after the experiment is over.
2. Control
All experiments also involve a great deal of control. That means that the researcher tries to hold all other variables constant. In other words, everything, expect the manipulated variable must be exactly the same for all of the people involved.
a. The procedures and setting used must be identical.
When conducting an experiment, the exact same thing must happen, that is the procedures followed must be identical.
Not only must the procedures be identical, but the setting should be identical as well.
For example, let's say you were going to test whether watching a TV show with a laugh track or no laugh track makes viewing a show more enjoyable.
In order to do this experiment, you would have to control everything except the variable you are manipulating (presence of a laugh track).
So, you would have to show the exact same show, in an identical room, with the exact same commercials, the temperature of the room should be the same, the lighting the same, the way you introduce the film should be the same.
In short, everything should be held constant except the variable you are manipulating.
Not only do you need to have identical procedures and setting, but you should also have identical groups of people watch the films.
b. People in each group should be the same to begin with.
Think about it, in order for you to do an experiment, not only do you have to use identical procedures and settings, but the people involved must be the same as well.
For example, you wouldn't want to do a study on TV violence, with one group of people that really dislike violence on TV and another group of people who really enjoy violence.
So, how do we make groups equal to begin with.
1. Random Assignment to groups
People should be randomly assigned to either the control of the experimental group.
That is, people should have an equal chance of being placed in either group.
So, if you were doing a study on TV violence. You would want to randomly assign people to either watch the Violent film or the non-violent film.
Random assignment assures that the groups are equal before the experiment begins.
That is, whatever differences exist among people before the experiment, these differences are equally spread throughout the study.
2. Matching
Matching is another method that is used to make sure that groups are equal to each other.
In matching, people are paired up with people like them. One person is assigned to one group, and the other person is assigned to the other group.
In other words, you simply pair people up based upon similarities, and then split the pair into two different groups.
This method is not used as often in communication research.
However, in some cases, when random assignment can't be used, matching is the best alternative.
For example, if I want to see whether or not men and women respond differently to a new Public Relations campaign that I am developing. Obviously I can't randomly assign people to be a woman or a man. Rather I can match each woman with a man that is very similar to her and then determine whether there are differences between men and women who are exposed to my campaign. However, If I do notice differences between men and women after experiment is over -- the possibility always exists that difference is not necessarily due to gender, some difference that existed between my sample to being with. (side note -- there are other ways to resolve this issue -- beyond the scope of this class)
So, matching is the next best alternative when random assignment can not be done.
C Why Experiments can Identify Causal Relationships
Remember, for a causal relationship to exist between two variables, three things must be identified.
1. Change one variable must occur before change in the other variable.
2. Must show that as the one variable changes, the other variable changes as well.
3. Must be certain that the change in one variable is really caused by the change in the other variable, and not other factors.
Well, the nice thing about experiments is that they allow us to examine each one of these conditions.
Let's go through each one.
First, in an experiment, the order is controlled by the researcher. That is, researcher determines when the manipulation in a variable will occur. The order is restricted by the experiment.
Second, if manipulation checks are done properly, we have proof that the two variables changed together. That is, has one variable changes, we can notice changes in another variable.
Finally, the fact that all other variables are controlled or are exactly the same, rules out the possibility that the change in one variable is due to a change in another variable. We can rule out all other variables as a possible source of the change noticed.
In short, if only one variable is being manipulated, and all other variables are being held constant, then a change noticed in the dependent variable must be due to one of two things.
1. the variable being manipulated (independent variable)
2. or chance
Through statistical analysis, we can estimate the odds of this change occurring due to chance or not.
So, if we randomly assign people two watch either a violent film or a non-violent film and then we notice a change in people's aggressive behavior. That is, people who watch the violent film are now more aggressive than those who watched the non-violent film, we can be pretty certain that this change in behavior is due to 1) the violence or 2) chance (and nothing else -- everything else was held constant).
In the next series of lectures, we'll cover how we determine whether or not we can say that the change we noticed was due to chance or due to the manipulation of the independent variable.
D Basic steps in experiments
Now, let's briefly go over the basic steps involved in doing an experiment.
1. randomly assign people to different groups.
2. manipulate or change a variable(s)
3. measure change in other variable(s)
4. decide whether differences in measured variables are due to chance or a change in manipulated variable.
Let's all work through an example:
Want to see whether or not writing encouraging comments on your exams will produce a change in your performance.
So, what is the first thing I would want to do?
Randomly assign everyone in the class to one of two groups (comment and no comment group).
Manipulate or change a variable.
Write comments on some exams (for people in the comment group)
Measure change in other variables -- See how the students scored on the second exam.
See whether or not the group that got the encouraging comments did better than the group that didn't get any comments.
If there are differences between these groups, then I must decide whether these differences are due to chance or my encouraging comments.