Psych Ch. 13

Psych Ch. 13

Quasi-experiment
differs from a true experiment in that the researchers do not have full experiments control
–still studying the effect of an IV on a DV
-often because people can not be assigned to specific levels of intended IV-gender, age (participant variables)
–why do people do them?
—convenience, ethics, external validity (at expense of internal)
-opportunity to observe-real world effect
Independent-Groups Quasi-experiment
different participants at each level of the independent variable
-nonequivalent control group design
-nonequivalent control group pretest/posttest
Nonequivalent control group design
a quasi experiment study that has at least one treatment group and one comparison group–but participants have no been randomly assigned to the two groups
–independent groups (between subjects)
-Head start study -match students based on school exams- one who qualifies financially for head start and one that doesn’t- test both at later date
Nonequivalent control group pretest/posttest design
participants not randomly assigned and were tested before and after the intervention
-plastic surgery example-tested on life satisfaction before and after surgery (tested twice)
–group only exposed to 1 level-people either get the surgery or don’t
-only measured twice
-you compare one group (non-surgery group) to the other (surgery group)
Repeated Measures Quasi Experiment
-within subjects
–same group of people exposed to different manipulations of the IV
-different from an experiment because researcher takes advantage of an already scheduled event/policy/chance occurrence to manipulate IV
–ex:interrupted time-series design, nonequivalent control-group interrupted time-series design
Interrupted time-series design
repeated measures quasi experiment (within subjects)
-a quasi experimental study that measured participants repeatedly on a DV before, during, and after an “interruption” caused by event
-measure the DV after each event (IV) (at least 3 times)
–ex:decision fatigue/judge granting parole: DV=proportion of criminals granted parole IV/event=snack/lunch
-television access and crime rate, crime rate measured before, during, and after introduction of television
-one continuous line on a graph-should show change over time (after introduction of IV)
-you compare measurement 1 of DV to measurement 2 of the DV-for the same group
Non-equivalent control group, interrupted time-series design
combines nonequivalent control group design (independent) with interrupted time series design (within-subjects)
-just like interrupted time series:measure DV before, during, and after introduction of IV (within subject) -measured crime rates before/during/after introduction of TV
-but also have a control group (nonequivalent control group) -also compared larceny rates in cities that had television with cities that did not
-two lines on a graph-experimental group should show change over time, and also show a greater change than the control group (to prevent against maturation threats)
Internal Validity problems in Quasi-experiments and how to rule out:
-selection effects, maturation, attrition, history, regression, testing/instrumentation threats
-design confounds
-observer bias, demand characteristics, and placebo effects
—good experimental designs, and pattern of results can help rule out internal validity threats
Selection Effects
threat to internal validity
-only relevant for independent groups design
-occurs when the groups at the various levels of the IV contain different types of participants
–>it’s not clear whether the IV or the different types of people caused the change in the DV
-ex: head start -experimental group had impoverished children
–solved with pretest/posttest, matched groups, or wait-list design
Wait-list design
-solution to selection effects (threat to internal validity) with between groups (independent-groups) design
-all participants plan to receive treatment, but are assigned to do so at different times
–helps het same kind of people in both groups
-cosmetic surgery example
–can still be unethical depending on what the treatment is
-random assignment? hopefully
Design confounds
-internal validity threat
–some outside variable accidentally and systematically varies with the levels of the targeted IV
-can lead to an alternate explanation
–inspect data carefully to rule out design confounds
Maturation threat
-internal validity threat
–quasi experiment with pretest posttest design, when a treatment group shows an improvement over time, but it’s unclear if the improvement was caused by the treatment of whether the group would have improved spontaneously without the treatment
–design (include a comparison group) + pattern of results (different between the two groups) help eliminate maturation threats
History Threat
-internal validity threat
-when an external/historical event happens for everyone in the study at the same time as the treatment variable
–can be especially relevant in quasi-experiments that rely on external factor to manipulate the key IV (ccc television broadcasting policies)
-recession possible history threat for TV introduction and crime rates
–comparison group–recession happened for all cities, but only cities with television had significant rises in larceny rates
–selection-history threat=only happens in one group
Regression to the Mean
-internal validity threat
–when an extreme finding is caused by a combination of random factors that are unlikely to happen in the same combination again
-head start-to create matched groups-paired high scoring head start kids with low scoring non-head start children–scores may have been caused by random factors–low scoring non-head start kids didn’t get a good night sleep
–wouldn’t happen again at the posttest–>regression to mean
-relevant for pretest/posttest-primatily when a group is selected for having extremely high or low scores (don’t choose high/low scores=solution)
–not usually a problem in true experiments because of random assignment
Attrition Threat
-internal validity threat
–pretest/posttest when people drop out over time
-only a problem when people drop out systematically
–ex:least satisfied people stop reporting to questionnaires in cosmetic surgery exp.
-just drop pretest scores of people you don’t get posttest scores from
Testing Threat
internal validity threat
-a problem when the DV is measured more than once (pretest/posttest or within [repeated measures])
–participant changes as a result of being tested before
-practice effect/fatigue effect
–comparison group helps
Instrumentation Threat
internal validity threat
-a problem when the DV is measured more than once
–when a measuring instrument changes over time
-comparison group helps
–instrumentation threat was the actual result in the parole hearing and lunch break exp.
Observer Bias
-internal validity threat
–also threatens construct validity (observer’s interpretations don’t reflect real levels of DV)
–expectations of the observer influence how they interpret the results
-especially relevant for behavioral studies
-double blind can help
-carefully train coders+manual+multiple codes
Demand Characteristics
internal validity threat
-participants guess what the study is about and change their behavior in the expected direction
–double blind study
Placebo effect
internal validity threat
-when participants experience actual change, but not as a result of the IV-as a result of their belief that they are receiving a legitimate treatment
-double blind+comparison groups
External Validity in quasi experiments
real world setting increase external validity
-still need to ask if experiments would generalize to other people/situations
–dont have to ask if it would generalize to real world situations
Ethics of Quasi Experiments
-isn’t ethical to withhold things like head start with random assignment-cant to a true experiment
-even wait list designs still questionable
Construct Validity in Quasi Experiments
usually have successful manipulation of the IV (how well was the IV manipulated)
–how well was the DV measured is more important
-can be questionable in behavioral DV-with observer bias
Statistical Validity in Quasi Experiments
-need to address how large the group differences were (effect size)
-were the results statistically significant
Quasi-Experiments vs Correlational Studies
-independent groups design are similar to correlational studies
–quasi=people not randomly assigned
–correlational=two measured variables
-have similar internal validity questions
–quasi=selection effects/design confounds
–correlational=lurking variables
-quasi-experiments attempt to have internal validity-matched groups, wait-list designs, or a natural comparison group
Small-N designs
-experiments that study only a few individuals
–external validity=how a sample is selected (more important that number of people in sample)
-instead of gathering a little information from a large sample, they obtain a lot of information from just a few cases
-researchers often study special cases, taking advantage of special medical cases (similar to quasi experiment)-ie split brain
–almost always repeated measures
Single-N design
sample is restricted to one animal/person
Differences between large/small-N groups
-Large N: participants are grouped and data is represented as a group average
-small N: each participant is treated as a separate experiment (almost always repeated measures) and individuals data is presented
Research on Split Brains
corpus callosum cut in severe seizure patients
–people with condition were studied to see how the different hemispheres of the brain act independently
-speech=left hemisphere/right eye
-left hemisphere responsible for speech, storytelling/cause and effect, and sense of self
–how much can you conclude from such a small N sample
—experimental control, strong manipulation, and replication help you conclude more
How much can you conclude from a small N?
experimental control, strong manipulation, and replication help you conclude more
Experimental control
-how much can you conclude from such a small N sample
control possible third variables?
-controlled the movement of participants eyes to control which hemisphere saw what
Strong manipulations
–how much can you conclude from such a small N sample
-making sure you manipulate the IV enough to cause a significant change in the DV (making your levels different enough from one another)
-tasks that would be easy for normal brain people and impossible for split brain people were used (showed the change in DV)
Replication
-how much can you conclude from such a small N sample
–are the results among your small number of participants similar?
-results were replicated among 5 split-brain patients
Disadvantages of Small-N designs
-a few people may not represent the whole population well (data was consistent among 5split-brain patients, but they were all severely epileptic=systematically different brains that the general population)
– it is often unethical to create the necessary comparison group
Triangulation
similarities in animal studies, brain imaging, and weight of evidence can support a claim and lead to a parsimonious theory (about brain organization)
-external validity
Behavior Change studies
small N designs that attempt to modify behavior
-alzheimer’s new memory strategy, special ed teacher discourages face touching,
–can have many internal validity questions
Stable-baseline design
Behavior Change studies (small n)
-a researcher observes behavior for an extended baseline period before beginning a treatment
–if baseline is stable-behavioral change during the treatment period was more likely a result of the IV
-better than single pretest (in pretest-posttest) extended pretest period lowers the chance that a single spontaneous event occurred at the same time as the pretest and changed the behavior
(rule our maturation/history/and regression threats)
Multiple baseline design
Behavior Change studies (small n)
-researchers stagger their introduction of an intervention across a variety of contexts, times, and situations
–special ed girl touched her face/objects/hair- introduced overcorrections for each behavior separately -each decreased after overcorrection was applied (overcorrection works)
-if a single event was responsible for the change, all of the behaviors would have changed at once instead of staggering with overcorrection
-baseline can also be different situations-correct a behavior at work/home/school
—can be different people, behavior modification for 3 children in the same class, one at a time
-provide automatic comparison condition
Reversal Designs
Behavior Change studies (small n)
-researcher observes subject with and without treatment but takes the treatment away for a while (reversal period) to see whether the problem returns (reverses)
–test internal validity-if treatment is causing change, problem should return when it is removed
-only really works when treatment doesn’t cause lasting change
–ex:doesn’t work for education-cant relearn the alphabet
-questionable ethics of removing a treatment that works (depression)
Validities in Small-N designs
-internal validity because of repeated measures/within group, replication, experimental control, strong manipulations, pretest/posttest, baseline designs and reversals
-external validity-triangulation + weight of evidence + specific population to generalize results to, and can still be useful without generalizing
-construct-multiple observers + interrater reliability for behavior
-statistical-graphs, effect size (by what margin did client’s behavior improve) not traditional statistics