When the results of a study can be generalized to other populations in settings The study is said to have?

The goal of scientific research is to increase our understanding of the world around us. To do this, researchers study different groups of people or populations. These populations can be as small as a few individuals from one workplace or as large as thousands of people representing a cross-section of Canadian society. The results of this research often provide insights into how work and health interact in those groups. But how do we know if a study's results can be applied to another group or population?

To answer this question, we first need to understand the concept of generalizability.

In its simplest form, generalizability can be described as making predictions based on past observations.

In other words, if something has often happened in the past, it will likely occur in the future. In studies, once researchers have collected enough data to support a hypothesis, they can develop a premise to predict the outcome in similar circumstances with a certain degree of accuracy.

Two aspects of generalizability

Generalizing to a population. Sometimes when scientists talk about generalizability, they are applying results from a study sample to the larger population from which the sample was selected. For instance, consider the question, “What percentage of the Canadian population supports the Liberal party?” In this case, it would be important for researchers to survey people who represent the population at large. Therefore they must ensure that the survey respondents include relevant groups from the larger population in the correct proportions. Examples of relevant groups could be based on race, gender or age group.

Generalizing to a theory. More broadly, the concept of generalizability deals with moving from observations to scientific theories or hypotheses. This type of generalization amounts to taking time- and place-specific observations to create a universal hypothesis or theory. For instance, in the 1940s and 1950s, British researchers Richard Doll and Bradford Hill found that 647 out of 649 lung cancer patients in London hospitals were smokers. This led to many more research studies, with increasing sample sizes, with differing groups of people, with differing amounts of smoking and so on. When the results were found to be consistent across person, time and place, the observations were generalized into a theory: “cigarette smoking causes lung cancer.”

Requirements for generalizability

For generalizability we require a study sample that represents some population of interest — but we also need to understand the contexts in which the studies are done and how those might influence the results.

Suppose you read an article about a Swedish study of a new exercise program for male workers with back pain. The study was performed on male workers from fitness centres. Researchers compared two approaches. Half of the participants got a pamphlet on exercise from their therapist, and half were put on an exercise program led by a former Olympic athlete. The study findings showed that workers in the exercise group returned to work more quickly than workers who received the pamphlet.

Assuming the study was well conducted, with a strong design and rigorous reporting, we can trust the results. But to what populations could you generalize these results?

Some factors that need to be considered include: How important is it to have an Olympian delivering the exercise program? Would the exercise program work if delivered by an unknown therapist? Would the program work if delivered by the same Olympian but in a country where he or she is not well-known? Would the results apply to employees of other workplaces that differ from fitness centres? Would women respond the same way to the exercise program?

To increase our confidence in the generalizability of the study, it would have to be repeated with the same exercise program but with different providers in different settings (either worksites or countries) and yield the same results.

Source: At Work, Issue 45, Summer 2006: Institute for Work & Health, Toronto

Published on May 15, 2019 by Raimo Streefkerk. Revised on September 26, 2022.

When testing cause-and-effect relationships, validity can be split up into two types: internal and external validity.

Internal validity refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors or variables.

External validity refers to the extent to which results from a study can be applied (generalized) to other situations, groups or events.

The validity of a study is largely determined by the experimental design. To ensure the validity of the tools or tests you use, you also have to consider measurement validity.

Trade-off between internal and external validity

Better internal validity often comes at the expense of external validity (and vice versa). The type of study you choose reflects the priorities of your research.

Trade-off example
A causal relationship can be tested in an artificial lab setting or in the ‘real world’. A lab setting ensures higher internal validity because external influences can be minimized. However, the external validity diminishes because a lab environment is different than the ‘outside world’ (that does have external influencing factors).

A solution to this trade-off is to conduct the research first in a controlled (artificial) environment to establish the existence of a causal relationship, followed by a field experiment to analyze if the results hold in the real world.

Threats to internal validity

There are eight factors that can threaten the internal validity of your research. They are explained below using the following example:

Research example
The management of company X wants to know if flexible working hours will improve job satisfaction among employees. They set up an experiment with two groups: 1) control group of employees with fixed working hours 2) experiment group with employees with flexible working hours. The experiment will run for six months. All employees fill in a survey measuring their job satisfaction before the experiment (pre-test) and after the experiment (post-test).

Threats to internal validity
ThreatExplanationExample
HistoryUnanticipated events change the conditions of the study and influence the outcome. A new (better) manager starts during the study, which improves job satisfaction.
MaturationThe passage of time influences the dependent variable (job satisfaction). During the six-month experiment, employees become more experienced and better at their jobs. Therefore, job satisfaction may improve.
TestingThe pre-test (used to establish a baseline) affects the results of the post-test. Employees feel the need to be consistent in their answers in the pre-test and post-test.
Participant selectionParticipants in the control and experimental group differ substantially and can thus not be compared. Instead of a randomly assigning employees to one of two groups, employees can volunteer to participate in an experiment to improve job satisfaction. The experimental group now consists of more engaged (more satisfied) employees to begin with.
AttritionOver the course of a (longer) study, participants may drop out. If the drop out is caused by the experimental treatment (as opposed to coincidence) it can threaten the internal validity. Really dissatisfied employees quit their job during the study. The average job satisfaction will now improve, not because the “treatment” worked, but because the dissatisfied employees are not included in the post-test.
Regression towards meanExtreme scores tend to be closer to the average on a second measurement. Employees who score extremely low in the first job satisfaction survey probably show greater gain in job satisfaction than employees who scored average.
InstrumentationThere is a change in how the dependent variable is measured during the study. The questionnaire in the post test contains extra questions compared to the one used for the pre-test.
Social interactionInteraction between participants from different groups influences the outcome. The group of employees with fixed working hours are resentful of the group with flexible working hours, and their job satisfaction decreases as a result.

What can proofreading do for your paper?

Scribbr editors not only correct grammar and spelling mistakes, but also strengthen your writing by making sure your paper is free of vague language, redundant words and awkward phrasing.

When the results of a study can be generalized to other populations in settings The study is said to have?
When the results of a study can be generalized to other populations in settings The study is said to have?

See editing example

Threats to external validity

There are three main factors that might threaten the external validity of our study example.

Threats to external validity
ThreatExplanationExample
TestingParticipation in the pre-test influences the reaction to the ‘treatment’. The questionnaire about job satisfaction used in the pre-test triggers employees to start thinking more consciously about their job satisfaction.
Sampling biasParticipants of the study differ substantially from the population. Employees participating in the experiment are significantly younger than employees in other departments, so the results can’t be generalized.
Hawthorne effectParticipants change their behavior because they know they are being studied. The employees make an extra effort in their jobs and feel greater job satisfaction because they know they are participating in an experiment.

There are various other threats to external validity that can apply to different kinds of experiments.

Frequently asked questions about internal and external validity

What is experimental design?

Experimental design means planning a set of procedures to investigate a relationship between variables. To design a controlled experiment, you need:

  • A testable hypothesis
  • At least one independent variable that can be precisely manipulated
  • At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

  • How you will manipulate the variable(s)
  • How you will control for any potential confounding variables
  • How many subjects or samples will be included in the study
  • How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

Sources in this article

We strongly encourage students to use sources in their work. You can cite our article (APA Style) or take a deep dive into the articles below.

This Scribbr article

Streefkerk, R. (September 26, 2022). Internal vs. External Validity | Understanding Differences & Threats. Scribbr. Retrieved October 3, 2022, from https://www.scribbr.com/methodology/internal-vs-external-validity/

Is this article helpful?

You have already voted. Thanks :-) Your vote is saved :-) Processing your vote...

When the results of a study can be generalized to other subject populations The study is said to have validity?

Population validity and ecological validity are two types of external validity. Population validity refers to whether you can generalize the research outcomes to other populations or groups. Ecological validity refers to whether a study's findings can be generalized to additional situations or settings.

When can a study be generalized to the population?

If the results of a study are broadly applicable to many different types of people or situations, the study is said to have good generalizability. If the results can only be applied to a very narrow population or in a very specific situation, the results have poor generalizability.

What is the best way to ensure that results of a study is generalizable?

One way to ensure generalizability in research is to get an adequate, yet random sample size from the population. Another way would be to increase the participation of the persons within the study to help yield valid and complete results.
External validity is frequently associated with the term “generalizability,” often being used interchangeably with it. Generalization is the process of using particular data to in- fer a general statement that has applicability to other people, settings, or times.