Which of the following studies would be least likely to be influenced by cultural differences?

Describing a study whose results have been reproduced when the study was repeated, or replicated. Also called reproducible. See also conceptual replication, direct replication, replication-plus-extension. (page 425)

A replication study in which researchers repeat the original study as closely as possible to see whether the original effect shows up in the newly collected data. See also conceptual replication, replication-plus-extension. (page 426)

A replication study in which researchers examine the same research question (the same conceptual variables) but use different procedures for operationalizing the variables. See also direct replication, replication-plus-extension. (page 427)

replication-plus-extension

A replication study in which researchers replicate their original study but add variables or conditions that test additional questions. See also conceptual replication, direct replication. (page 428)

A family of questionable data analysis techniques, such as adding participants after the results are initially analyzed, looking for outliers, or trying new analyses in order to obtain a p value of just under .05, which can lead to nonreplicable results. (page 432)

As part of a study's publication process, the practice of sharing one's data and materials freely so others can collaborate, use, and verify the results. (page 432)

The practice of posting a study's method, hypotheses, or statistical analyses publicly, in advance of data collection. (page 432)

A series of related studies, conducted by various researchers, that have tested similar variables. Also called literature. (page 433)

The extent to which the tasks and manipulations of a study are similar to real-world contexts; an aspect of external validity. Also called mundane realism. (page 439)

A researcher's intent for a study, testing association claims or causal claims to investigate support for a theory. See also generalization mode. (page 440)

The intent of researchers to generalize the findings from the samples and procedures in their study to other populations or contexts. See also theory-testing mode. (page 442)

A subdiscipline of psychology concerned with how cultural settings shape a person's thoughts, feelings, and behavior, and how these in turn shape cultural settings. (page 444)

A real-world setting for a research study. (page 449)

The extent to which a laboratory experiment is designed so that participants experience authentic emotions, motivations, and behaviors. (page 449)

A way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports. (page 433)

A problem relating to literature reviews and meta-analyses based only on published literature, which might overestimate the support for a theory because studies finding null effects are less likely to be published than studies finding significant results, and are thus less likely to be included in such reviews. (page 436)

Direct replication
•Conceptual replication
•Replication-plus-extension

The Replication Debate in Psychology

The “replication crisis”•Why might replication studies fail?•Improvements to scientific practice. 

Why Might Replication Studies Fail?

Contextually sensitive effects
•Number of replication attempts
•Problems with original study –Sample Size –Harking –P-Hacking

Improvements to Scientific Practice

Larger sample sizes •Report all analyses and variable •Open Science Collaboration •Preregistration

Strengths and Limitations of Meta-Analysis

File drawer problem •Null results and opposite results rarely published. •Example: antidepressants and depressive symptoms

Replicability, Importance, and Popular Media

Journalists do not always consider the importance of replicability.
•A responsible journalist will not only report on the latest and most exciting studies but will also summarize the body of literature on the topic.

Generalizing to Other Participants

It’s a population, not thepopulation. •External validity comes from how, not how many. •Just because a sample comes from a population doesn’t mean it generalizes to that population.

Generalizing to Other Settings

Ecological validity: an aspect of external validity in which the focus is on whether a laboratory study generalizes to real-world settings

Examples –The contact comfort theory–“Parent-as-grammar-coach” theory

Frequency claims: always in generalization mode
•Association and causal claims: sometimes in generalization mode

Cultural Psychology: A Special Case of Generalization Mode

Theory Testing Using WEIRD Participants

Most research in psychology has been conducted on North American college students.
•WEIRD = Western, educated industrialized, rich and democratic

Does a Study Have to Take Place in a Real-World Setting?

External validity and the real world •Generalization mode and the real world •Theory-testing mode and the real world

External Validity and the Real World

When a study takes place it the real world, it occurs in a field setting and has high external validity.
•Laboratory research can be just as realistic as research conducted in the real world, which is known as experimental realism.

Generalization Mode and the Real World

External validity is important when in generalization mode, so having a representative sample is paramount. •Ecological validity is also important to consider in terms of generalizing to nonlaboratory settings.

Theory-Testing Mode and the Real World

When in theory-testing mode, external validity and real-world applicability are not as important. 

Which of the following types of study support external validity?

NOT   A study that is a direct replication

In a previous correlational study, Dr. Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans. Which of the following would constitute a shift from theory-testing mode to generalization mode?

Conducting a study on dog ownership using a sample of elderly participants from a local retirement home

RESEARCH STUDY 14.1: Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand. Mark decides to conduct a variation of the original study in which he measures participants’ ability to solve verbal analogies as the cognitive task instead of measuring their ability to solve math problems. He finds that people who drink caffeine beforehand solve more verbal analogies than those who drink water beforehand, suggesting caffeine consumption causes improved cognitive performance. This study is known as a:

Which of the following is true of theory-testing mode?

It is related to the theory-data cycle.

Which of the following is true of p-hacking?

It can involve running a different analysis than originally proposed.

In open science, open data are more concerned with _________, while open materials are more interested in _________.

statistical validity; replication

Dr. Fern recently collected data for a study examining whether an intervention targeting coping strategies can reduce depression. After analyzing the results, she finds no effect of her intervention on depression symptoms. Which of the following would you recommend she do next?

Contact journal editors to see if they will publish null results.

When generalizing the results of a sample to a population, which of the following is the most important question to ask?

How were the participants sampled?

Which of the following studies is most likely to be conducted in generalization mode?

Determining how frequently people report fighting with their bosses

RESEARCH STUDY 14.2: Cindy decides to conduct a meta-analysis examining the relationship between daily stress and cardiovascular health for her senior psychology research project. Cindy’s advisor recommends that she contact several researchers in the field for articles that were not published and/or that found null effects. Doing this will address which of the following?

A study conducted in the “real world” is often said to be conducted in:

Which of the following studies would be least likely to be influenced by cultural differences?

A study examining motor responses of newborns

According to the textbook, which of the following is a component of WEIRD people?

Which of the following is another term for ecological validity?

Who determines the population to which a study’s findings generalize?

Some studies may not replicate because the original study used questionable scientific practices. Which of the following is a way open science practices attempt to deal with this problem?

Requiring scientists to make their data and materials available for review

According to the textbook, what is the problem with WEIRD samples?

They are not very representative of the world’s population.

What does it mean when a study cannot be replicated by an independent researcher?

The importance of the finding should be interpreted cautiously.

The text states that the first step in establishing a study’s importance is to establish its _________.

Which of the following types of replications investigate the same research question but use different procedures?

Which of the following is an advantage of studies that are conducted in real world settings?

Which of the following is an advantage of studies that are conducted in real-world settings? They have a high degree of internal validity.

Which of the following is a reason that a replication study might fail?

A replication study, then, is a study that aims at replication. Such replication may fail either because the original study turns out not to be replicable in the first place or because, even though it is replicable, a successful replication does not occur.

In what type of study do researchers replicate their original experiment and add variables to test additional questions?

REPLICATION-PLUS-EXTENTION: researchers replicate their original experiment and add variables to test additional questions. (You can introduce more conditions/levels, or present more independent variables).

Which of the following has been used as a defense of animal research by animal researchers?

Which of the following has been used as a defense of animal research by animal researchers? Animal research is less likely to result in physical harm to the subject compared to human research. Animal research requires less research funding than human research.