CAREER Center
Volume 1, Number 1
May 2003

E-Research
Internet-based research poses new possibilities—and practical and ethical challenges—for graduate students.

A traditional mail-based survey with thousands of participants would require funds that are out of reach of most students. But given the appropriate technical expertise, the same survey could be done for almost nothing online, says Jeffrey M. Cohen, PhD, associate dean for research compliance at the Weill Medical College of Cornell University. "The Internet really opens up a lot for graduate students that they couldn't do before," he says.

Internet researchers can gather data from thousands of subjects at a low cost.

But doing effective online research continues to be challenging, for both practical and ethical reasons, says University of Virginia psychologist Brian Nosek, PhD. Nosek is one of the creators of the online version of the Implicit Associations Test (http://implicit.harvard.edu), which has collected data on more than two million visitors since 1998.

"I think that the mistake that has been made in thinking about Internet-based research is to think about it as a replacement for traditional laboratory methods," he says. "I don't think it can be a replacement, but it can be a wonderful complement."

POSSIBILITIES

One of the main advantages of Internet-based research, says Nosek, is that it enables researchers to conduct research that would be almost impossible using traditional laboratory methods—such as studies of widely dispersed populations, studies that require multiple assessments over long periods of time or studies in which many participants need to be assessed within a very short time.

Another advantage—perhaps the most important one for graduate students—is that it enables researchers to gather data from thousands of subjects at a low cost, says Nosek. That gives researchers the ability to explore the effects of minor procedural changes or to tease out interactions that might be missed in a smaller sample.

Also, despite concerns about Internet samples not representing the population as a whole, researchers say Internet-based samples are often more diverse—in age, ethnicity and other demographic variables—than samples recruited on university campuses. "We have been able to followup some of our laboratory research with similar Web-based data collections to show that the effects appear in a much more diverse population" than can be studied in the lab, says Nosek.

CHALLENGES

That doesn't mean the Internet is the right medium for every study, or that it doesn't raise some serious concerns, both practical and ethical, researchers say. The practical challenges include:

• Obtaining an unbiased sample. No uniform system for randomly selecting e-mail addresses exists, and not all segments of the population have equal access to the Internet.
• Verifying the identity of participants. E-mail addresses are hard to match to other forms of identification, single computers can be used by multiple people and minors can easily masquerade as adults.
• Controlling the experimental environment. Participants can access Internet-based experiments from settings as diverse as an office, a bedroom, a coffee shop, a library or, using a wireless device, almost anywhere.

As for ethical concerns, researchers say they are essentially the same online and off. The difference lies in how the concerns are manifested and what researchers can and should do to deal with them.

"The human subjects issues aren't different," notes Cohen, a former associate director at the federal Office of Human Research Protection. "You have the same issues of risk, confidentiality, participation by minors and so forth that any research with human subjects has to get into. The key point is that the technology creates new concerns about those same issues."

One example is informed consent. Until a system is in place for digital signatures, signing a consent form over the Internet isn't possible, so most researchers will need to apply for a waiver from their institutional review board (IRB), says Cohen. The only other option is to require participants to print out a form and mail it in before beginning the study—an option that undermines many of the advantages of online research.

Another issue is confidentiality. Just as with traditional laboratory experiments and surveys, protecting the privacy of participants is critical. But installing effective protections against hackers who might want to steal sensitive data requires a level of technical expertise most researchers don't have, says Cohen. So researchers either need to spend the time to acquire such expertise themselves or engage a consultant.

Online debriefing can also be difficult, researchers say, since users can shut down their browsers or walk away from computers at any time. One solution is to collect e-mail addresses at the beginning of the experiment and send debriefing messages after the experiment is over, but there's no way to ensure that participants actually read the messages they get.

Not all IRBs are fully aware of the issues around Internet-based research, says Cohen, but many are now trying to develop the necessary expertise. Researchers will increasingly be required to answer technical questions about how they plan to deal with informed consent, debriefing, confidentiality and other issues.

The same concerns apply whether one is developing the experiment oneself or using a commercial service. For the latter, says Nosek, "My biggest concern would be to know how the services ensured ethical delivery of the research materials." Secondary concerns include ensuring that the data the service produces is reliable, and that the experimental design features the service offers—such as random assignment of participants to different experimental conditions—are sufficient to meet the demands of the research.

—ETIENNE BENSON
gradPSYCH staff

On the Web
• Ethical and legal aspects of online research. www.aaas.org/spp/sfrl/projects/intres/report.pdf
• Social Psychology Network: www.socialpsychology.org/expts.htm
• PsychExperiments at the University of Mississippi: http://psychexps.olemiss.edu
• Psychology Research on the Net: http://psych.hanover.edu/research/exponnet.html

Further Reading
• Birnbaum, M.H. (2000). Psychological Experiments on the Internet. San Diego, CA: Academic Press.
• Nosek, B.A., Banaji, M.R., & Greenwald, A.G. (2002). E-research: Ethics, security, design, and control in psychological research on the Internet. Journal of Social Issues, 58(1), 161­176.

top