Select the figures that depict a nonequivalent control group interrupted time-series design.

Chapter 13; Quasi-Experiments and Small-N Designs

Amazon's choice to use the wake word "Alexa" seems to have led to a decrease in that name's popularity for babies. Photo: vectorstockcompany/Depositphotos

Here's a report from the Washington Post, entitled "Amazon, can we have our name back?". The journalist interviewed several women named Alexa, reporting about their experiences related to having the same name as the popular digital assistant. For example, 

Alexa Morales wore her name proudly. But after Amazon launched its voice service, also called Alexa, in November 2014, people began speaking to Morales differently. She said they made jokes about her name, giving her commands or asking her questions in a robotic tone. “When I hear my name now, it’s not good thoughts, it’s like, tensing,” said Morales, 28, a pharmacy technician and student in Bridgeport, Conn.

Much of the article is dedicated to the negative experiences (including harassment) that people with the name Alexa have had. Several Alexas have chosen to adopt nicknames. One psychologist was quoted as saying,

“They’re sort of growing up with the communication that their name is associated with a servant role,” Christopher Kearney, a professor of psychology at the University of Nevada at Las Vegas, said in an interview. “If a teacher is saying something derogatory, even if it’s just as a joke, or it’s to change their name, or … allows other kids to participate in that, it can create a lot of psychological damage.”

Aside from these detailed experiences, one part of this article can be used as an example of an interrupted time-series design. 

Nearly 130,000 people in the United States have the name Alexa. It gained popularity after singer Billy Joel and model Christie Brinkley named their daughter Alexa in 1985. In 2015, more than 6,000 baby girls in the United States were named Alexa, according to a Washington Post analysis of Social Security Administration data.

After Amazon chose Alexa as the wake word of its voice service, the name’s popularity plummeted. In 2020, only about 1,300 babies were given the name. 

It's much easier to think about this result if you see the graph of it, so I urge you to visit the Washington Post story and scroll down to the line graph before reading on. 

a) In this interrupted time series design, what is the quasi-independent variable? What is the dependent variable?

b) What do you think--to what extent can the baby name data in the article support the causal claim that "Amazon's Echo devices caused a decrease in people naming their babies Alexa"?Specifically, the graph shows covariance and temporal precedence--Amazon's Alexa device is associated with a subsequent, dramatic drop in the number of babies named Alexa. But what about internal validity--are there alternative explanations we can consider for this drop? 

c) What are your thoughts after reading the article? Do you that Amazon should change the Echo's default "wake word?"

Credit: halfpoint/Deposit Photos

Summer is the time for music festivals like Burning Man, Burning Nest, Firefly, and Bonnaroo. If you are planning to attend one this year, this post might interest you.

The science journalism clearinghouse called Study Finds recently reported on a study conducted at Yale University. The journalists' report included the causal headline that Music festivals like Burning Man make people more generous, socially connectedto others". 

a) What are the variables in this causal headline, and what makes it a causal claim?

b) In general, researchers need to conduct an experiment in order to support a causal claim. Do you see any barriers to conducting an experiment on the causal effects of music festivals?

It will be interesting to read about whether the researchers' study can feasibly support the causal claim that the journalist attached to it. Here's how Study Finds described the study's method: 

The Yale researchers studied the behavior and experience of over 1,200 people attending festivals such as Burning Man, Burning Nest, Lightning in a Bottle, Dirty Bird, and Latitude. All of the festivals took place in the United States or the United Kingdom and emphasized art, music, and self-expression. The team set up booths at all the events, which they called “Play Games for Science.”

People ... self-reported their experiences at the event and whether they were willing to share their resources with friends and strangers. For example, week-long festivals such as Burning Man do not sell food, and people who find themselves short must rely on the generosity of others.

[...] Overall, 63.2 percent of people said they had a transformative experience and felt themselves changing. ... One of the takeaways from the transformative experience was feeling more connected to other humans, which made them more likely to share food and resources with other attendees.

c) So far, do the method and results of this study (as reported by the journalist) support the conclusion that attending the festivals caused a transformative experience? Why or why not? (Hint: Think about the need for a comparison group.)

d) Based on what you've read, was the variable of "transformative experience" operationalized as a physiological, observational, or self-report measure?

In this case, the journalist's coverage of the study doesn't do the original study justice, as I discovered when I read the original empirical article, which appeared in the journal Nature Communication (you can access it here).There are several important omissions in the journalist's report. One is that the researchers themselves did not use causal language in their work. Instead, their article is titled "Prosocial correlates of transformative experiences at secular multi-day mass gatherings" (my emphasis).

The Study Finds journalist also omitted key methods and variables. For example, one important variable in the original study was that the researchers graphed people's self-reports of having a transformative experience on each day of a 7-day festival (see Figure 2,  here). Using this method, the researchers found that the later the day (going from Day 1 to Day 7), the more likely respondents were to report having a transformative experience. 

e) Why is the above method and result important, in terms of supporting the claim that the music festivals can increase people's tendency to have a transformative experience? (Hint: think about temporal precedence.)

Another important detail omitted by the Study Finds story, is that in addition to asking people to self-report their transformative experiences, researchers also asked participants to imagine playing a "dictator game". In this game, participants are given 10 valuable tickets. Participants can then donate some number of their tickets to randomly selected stranger. The dictator game is an observational measure of generosity to strangers. The researchers found that festival participants chose to donate between 50 and 75% of their tickets to the stranger. This was much higher than the level of donations typically found, as reported in a meta-analysis of all previous studies using the dictator game (which found an average of 26%). 

f)  This dictator game result is a type of quasi-experiment--we might call it a non-equivalent control group, posttest-only design. What is the quasi-independent variable--that is, what are the two levels of the non-equivalent control group (and why are they "non-equivalent")? What is the dependent variable? 

g) The result suggests that attending the music festival might cause people to be more generous in the dictator game, but there might be a selection effect. Why? 

Humans who can see nature views from their windows tend to be happier. 
Photo: © Amla Sanghvi

The examples in this post come from an extended piece by journalist Alan Lightman. He writes about how, 

For more than 99 percent of our history as humans, we lived close to nature. We lived in the open. The first house with a roof appeared only 5,000 years ago...

Lightman argues that in our modern lives, most of us spend most of our time working on computers and interfacing through screens, and this shift has potential consequences for our well-being. 

One of the psychological concepts the journalist uses as evidence in this article is known as "connectedness to nature." Two social psychologists developed a connectedness to nature scale (CNS), which is a self-report questionnaire with multiple items on it, including

"I often feel a sense of oneness with the natural world around me"
"I feel as though I belong to the Earth as equally as it belongs to me"
"I think of the natural world as a community to which I belong"

People who score high on this scale are said to have more connectedness to nature. 

Here's how the journalist describes one line of research with the CNS:

In recent years, psychologists have undertaken a number of studies to investigate correlations between scores on the CNS test and well developed methods for measuring happiness and well-being. In 2014, the psychologist Colin Capaldi and his colleagues at the Public Health Agency in Canada combined 30 such studies, involving more than 8,500 participants. The psychologists found a significant association between nature connectedness and life satisfaction and happiness. 

a) The quote above describes a meta-analysis. Can you see why? 

b) If you were to sketch the association found in Capalidi's meta-analysis, how would you do so? Sketch a scatterplot, labeling the axes mindfully.

c) Is the research above considered correlational or experimental? 

Here are two quasi-experimental studies that the journalist reviewed: 

Hospital patients in rooms with foliage or windows looking out on gardens and trees do better after surgery. Workers in offices with windows that open up to pastoral-like views have less anxiety, more positive work attitudes, and more job satisfaction.

d) Pick one of the studies above and identify the quasi-independent variable and the dependent variable(s) in the study. Sketch a small bar graph of the results described.

e) What makes these studies quasi-experiments instead of true experiments? 

e) Given the studies described above, come up with your own nature-related hypothesis to test. For example, what else, besides well-being, might the CNS be correlated with? Or, what other variables, besides hospital views and office views, might be worth studying with a quasi-experiment?

The study evaluated both victims and defendants in criminal cases. Compared to white victims, the media were much less likely to depict photos of Black victims of crime with family or friends. How might this choice impact people's empathy for Black pain and suffering? Credit: Rawpixel.com/Shutterstock

The Equal Justice Initiative (EJI) is a human-rights organization whose mission is to challenge racial injustice and "end mass incarceration and excessive punishment in the United States." EJI was founded by attorney Bryan Stevenson, who wrote the book Just Mercy about his experiences advocating for people who were unjustly imprisoned. (The book is also a feature film).

The EJI recently worked with Global Strategy Group to document ways that the criminal cases of white and Black defendants have been portrayed by the media. The infographic created by their study provides a social-justice themed example of a correlational study or quasi-experiment.

In the study, the GSG:

conducted a comprehensive media analysis of national and local coverage around 10 criminal cases – 5 with a Black defendant and 5 with a white defendant. We collected and collated information on more than 20 different topics, including the use of imagery, language choices, framing of defendant and victim, and reporter background.

The main findings of the GSG/EJI study are depicted in this colorful infographic. Here are a few highlights.

First, the study found that 

Stark disparities exist between the types of images used for Black and white defendants

The study found that media outlets were four times more likely to run the mugshot of a Black defendant compared to a white defendant. How might this bias people's impressions of defendants? 
Credit: nimito/Shutterstock

Specifically, the study measured the types of photos that media outlets chose to depict white vs. Black defendants. 9% of the time, the media used mugshots for white defendants, but 45% of the time, the media used mugshots for Black defendants. In contrast, 13% of the time, the media showed white defendants in a suit and tie, but only 6% of the time for Black defendants.

Second, the media also differed in how the victims of crime were depicted. The infographic reports:

White victims [79%] were nearly 4x than Black victims [21%] to have a photo with friends or family included alongside coverage, reinforcing existing tendencies to dehumanize Black pain and suffering and, by the same token, put a face to white victims.

Third, the study also found that race was associated with whom the media quoted about the defendant. They write: 

Quotes from family and friends were nearly twice as likely to appear in articles about white defendants [50%] than articles about Black defendants [25%]. Black defendants were more likely to have judges or lawyers weigh in instead, which presents readers with a less humanizing account of Black defendants.

Questions

a) Select one of the three quoted findings above (or, find your own from the infographic). Identify the variables in the finding you chose. What are the levels of each variable?

b) All of the studies above can be described as correlational. Why? (refer to the terms "measured" and "manipulated" in your answer).

c) If you've studied Chapter 13, you might wish to describe these studies as quasi-experimental instead of correlational. (In studies like these, you could make the case either way). Why do both labels (correlational and quasi-experimental) make sense here? 

d) How might the GSG and EJI take this study further with an experimental method? What would they manipulate and what could they measure? 

Scientific reasoning questions: 

At the bottom of the infographic, the GSG included the following detail about their methodology:

Audit was conducted among ten criminal cases, 5 cases featuring a white defendant and 5 featuring a Black defendant. All cases had criminal proceedings that occurred in the past seven years. GSG analyzed 20 to 30 articles for each case, for a total of 257 articles, all of which were randomly selected and included a combination of national and local online coverage.

e) Why was it important that the media articles about each case were selected randomly? 

f) The GSG is not very clear on how they selected the 10 initial cases (the 5 with a white defendant and the 5 with a Black defendant). Why do we need to know more about how these specific 10 cases were selected? 

g) Scroll to the bottom of the infographic and consult the 10 criminal cases they selected. Do you notice any systematic differences in the cases selected for white vs. Black defendants? 

Did this Netflix show cause a bump in chess.com membership? FILMCRAFt/WONDERFUL FILMS/NETFLIX/ Album/Alamy Stock Photo

Did you watch the Netflix limited series Queen's Gambit, which came out in the fall of 2020? Many people did; it was one of Netflix's most popular series to date. Many folks have speculated that the release of this show influenced people to play chess for the first time, or to reignite an old chess interest they once had. 

Here is a blog post by a data scientistDavid Zhang, who was able to analyze traffic on the website, chess.com, over the past several months. He analyzed the number of new users who started new accounts on the site. Chess.com is the most popular chess community on the Internet.

Zhang graphed the number of games played by United Kingdom (UK) players by day from July 2020 (three months before the Queen's Gambit was released) to March 2021 (five months after). The Netflix show appeared in October, 2020. Please click on this story link and scroll to the first graph that you see, which appears under the heading, Number of new players.

Questions

a) This graph, by itself, should remind you of one of the four quasi-experimental designs in Chapter 13. Which one?

b) What is the quasi-independent variable in this study? What are its levels?

c) What is the dependent variable in this study? 

d) When we analyze how well a quasi experiment can support a causal claim (such as "Queens Gambit caused people to join chess.com"), we need to consider the design and the results. What do you think--do the design and results show here help us rule out any internal validity threats?

e) Many folks would nominate history threat as a likely issue for this situation. That is, they might suggest that some other event, other than Queen's Gambit, occurred in October that could have caused chess to become more popular. What do you think--is that a reasonable critique?  If so, what historical event would you nominate that would have occurred at the same time (October 2020), and would have affected most people in this UK sample?

(notice what the author of this post writes about a potential history threat:

....other chess events have occurred around the time frame analysed. For example, the first PogChamps tournament involving popular streamers was hosted in May 2020 by chess.com, which is likely to have spread awareness and garnered appeal for online chess.)

f) Other folks might say that maturation is a threat here. They might say that "perhaps people just naturally and spontaneously became more interested in chess over time." However, in my view, you can use the results to argue against this interpretation. You can discuss the stable baseline of chess interest before October 2020 as you prepare an answer to this critique. 

Now scroll down the blog post a little further, to where the author has depicted chess interest in four countries--the UK, Italy, Colombia, and China. These countries released the Queen's Gambit as well, but at later dates than the UK; China did not air this show at all because it restricts foreign media.

g)  This graph, by itself, should remind you of one of the four quasi-experimental designs in Chapter 13. Which one?

h) What are the quasi-independent variables (there are two) in this new study? What are their levels?

i) What is the dependent variable?

j) What do you think--how well do these results and design support the causal claim that "Queens Gambit caused people to join chess.com"?

You might be interested to read the author's analysis of the causality question, where he writes:

It is important to question the causal relationship between the two. It is likely that watching the show alone does not directly account for the entire increase in online chess popularity; many other factors may be at play. The current COVID situation has left many working from home, with some suggesting that online chess has profited as an outlet. Additionally, chess is by nature a multiplayer game and users lead to chess from The Queen’s Gambit may have propagated their newfound interest to others through their own social networks. Finally, other chess events have occurred around the time frame analysed. For example, the first PogChamps tournament involving popular streamers was hosted in May 2020 by chess.com, which is likely to have spread awareness and garnered appeal for online chess. 

After matching students based on their pre-college voting behaviors, the researchers found that students who had taken a political science class became more likely to register to vote. Photo credit: Lakshmiprasad/Depositphotos

In the online news outlet called The Conversation, researchers write summaries of their own research studies for a general audience. 

Here's an example from political scientists Frank Fernandez and Matthew Capaldi, both of the University of Florida. They introduce their study by writing,

...improving college student voter turnout is a national issue....According to data from the National Study of Learning, Voting, and Engagement, about 1 in 4 students – including at both two- and four-year colleges – were not registered to vote in the 2016 or 2018 elections.

The researchers conducted a study of 2,000 students at community colleges. All the students had completed an extensive survey about their civic behaviors both before and during college. Students also reported the classes they had taken in college.

The researchers used a "propensity score matching procedure", which is similar in some ways to multiple regression. Specifically, they matched up students who had the same demographic characteristics (e.g., race, parental education, family income, gender). They also matched students on whether, before going to college, they had been registered to vote or voted in elections. The researchers were mainly predicting whether students had newly registered to vote (since starting college) as well as whether they had actually voted since starting college. The authors write about their results: 

After taking students’ prior civic engagement and other college experiences into account, we found that students who took at least one political science course were 9% more likely to register to vote than those who did not.

Additionally, we found that students who took at least one political science class were 8% more likely to vote.

Questions

a) Name the variables in this study--there are at least seven variables mentioned in the summary provided above.

b) Is this a correlational study or an experiment? Explain your answer.

c) The authors imply that taking a political science course seems to cause students to be more likely to register to vote and to actually vote. Let's apply the three criteria for causation to this statement. First, do the results support covariance? 

d) Second, does the method of the study clearly establish temporal precedence? Why or why not? 

e) Finally, the authors used the matching technique to control for several alternative explanations such as race, parental education, family income, gender, as well as whether, before going to college, they had been registered to vote or had voted in elections. Can you think of any additional alternative explanations that they did not control for? 

f) In their summary in The Conversation, the authors discuss some of the downsides of their methodology. For example, they write, "We relied on self-reported data, so there is no practical way to confirm that they registered to vote or turned out to vote."  This critique is focused on construct validity--specifically, criterion validity. That is, they are admitting that they don't know whether or not a self-report item about voting actually predicts the behavioral criterion of voting. How could you run a pilot study to confirm that people who say they voted in a certain election have actually voted (and vice versa--people who say they did not vote have actually not voted)?

Here's a link to their study , which was published in Educational Researcher (sorry--paywalled).

Suggested answers:

a) The variables include race, parental education, family income, gender, being registered to vote before college, voting before college, and registering to vote during college and voting in college.

b) It's a correlational study--the survey method simply measured all of the variables, and a study where all the variables are measured is correlational. 

c) Yes, the results support covariance because students who took a poli sci course were 9% more likely to have registered to vote in college (and 8% more likely to actually vote). 

d) In my opinion, temporal precedence is not established here--students reported on whether they took a poli sci course at the same time they reported on whether they had registered to vote in college. It's not clear from this whether the course came first in time, before the voting behavior.

e) The authors did control for several likely explanations, including baseline interest in voting--and that is good. Another variable they might consider could be local political climate--perhaps some community colleges are more politically active than others. Students who attend more politically active CCs could be more likely to start to vote, and also be more likely to take a poli sci course. 

f) You could recruit a sample of people and ask them if they voted in the last election. Then, you can use publicly available voting records to confirm whether or not the actually voted. You should see a strong association:  People who say they voted, should have actually voted, and people who say they did not vote, should have actually not voted. 

Which type of quasi-experimental design does this study seem to fit best? The dark line on top represents respondents with children. The red line represents all respondents. The lighter blue line at the bottom represents respondents without children. Credit: Courtesy Patrick Cooney and H. Luke Shaefer. From Poverty Solutions, May 2021, University of Michigan,

Since the COVID-19 pandemic began, several governments, including the U.S. federal government, have distributed cash payments to citizens in an attempt to alleviate the  financial hardship caused by the global shutdown. 

As the New York Times reports here, policymakers wonder if such payments are useful. Now we have data to help address that research question. 

In offering most Americans two more rounds of stimulus checks in the past six months, totaling $2,000 a person, the federal government effectively conducted a huge experiment in safety net policy. Supporters said a quick, broad outpouring of cash would ease the economic hardships caused by the coronavirus pandemic. Skeptics called the policy wasteful and expensive.

[...]

A new analysis of Census Bureau surveys argues that the two latest rounds of aid significantly improved Americans’ ability to buy food and pay household bills and reduced anxiety and depression, with the largest benefits going to the poorest households and those with children. The analysis offers the fullest look at hardship reduction under the stimulus aid.

The full report is available here, and provides some excellent graphed examples of quasi-experimental data. We'll focus on Figure 1 from the report, reprinted above with permission from the authors.

The y-axis on the figure represents the percentage of Americans who "sometimes or often had not enough food to eat in the last seven days." The question is asked by U.S. Census' Household Pulse survey about once per month to a random sample of Americans. The dates of the two U.S. stimulus checks are indicated in the laddered, light-blue vertical lines. Payments occurred in December, 2020 and March, 2021.

Questions

a) The journalist writes that "the federal government effectively conducted a huge experiment in safety net policy".  What is the independent variable in this "experiment?"  What is the dependent variable (i.e., the one depicted in Figure 1)?

b) The independent variable here is not a true IV--it is a quasi-independent variable. What makes this variable quasi-independent?

c) The authors claim that the results from Figure 1 are consistent with the argument that the stimulus checks helped people. The authors wrote in their report: "Based on the speed with which we see hardship fall, we suspect much of this drop was the result of EIP checks, which the federal government was able to quickly deliver to bank accounts for most U.S. households following passage of both bills." Locate the areas on the figure that the authors are referring to. 

d) This quasi-experimental study is probably best described as an interrupted time-series design. What is the "interruption" here? What is the time series?

e) A history threat is one potential internal validity problem in an interrupted time-series design. Explain what a history threat is and also why a history threat might be a problem here. What do you think--to what extent can we be sure it was the stimulus checks, and not some other pair of events, that is responsible for the increased food security depicted here? 

You can see results for other variables in the report here, includingfigures that tracked people's feelings of anxiety and their feelings of depression. 

In one of the studies, researchers found that students whose classrooms were on the train-track side of the school made less progress in reading than those on the quieter side.  Credit: Alon Adika/Shutterstock

The weekly Freakonomics podcast recently put together an episode about the psychological impact of loud noises in our environment. You can listen to it (or read the transcript) here. 

The host and guest review several studies with a variety of methodologies, which include some excellent examples of quasi-experiments.

The introduction to the transcript reads, 

The modern world overwhelms us with sounds we didn’t ask for, like car alarms and cell-phone “halfalogues.” What does all this noise cost us in terms of productivity, health, and basic sanity?

Example I.

The first example comes from animal behavior and behavioral ecologist Peter Tyack, who studied whales in the Bay of Fundy. The Bay provided lots of food for the whales, but as a site of many shipping channels, the same place always had the background noise of ships going to and fro. 

Researchers like Tyack had never really thought much about this background noise. It was just there. But then all of a sudden it wasn’t. The change came with the 9/11 terrorist attacks.

TYACK: All of a sudden, the ships that were plying the ocean in that area stopped. 

This drop in ship traffic was only temporary. But it happened to coincide with some other whale research that was happening in the Bay of Fundy:

TYACK: Researchers from the New England Aquarium had been sampling feces from whales to look at stress hormones. It wasn’t part of a noise experiment at all. 

It was just your standard whale-feces research.

TYACK: But what they found was that if you compared the stress hormones in whales before 9/11 and after 9/11, their stress hormones actually went down after 9/11. 

a) In the quasi-experiment described here, what is the quasi-independent variable? What are its levels? What is the dependent variable?

b) Let's assume that the researchers collected and tested whale feces nearly every day, several days before 9/11 and several days afterwards, as well. What type of quasi-experiment is it (nonequivalent control groups, nonequivalent control group prettest-posttest, nonequivalent control groups interrupted time series, nonequivalent control groups interrupted time series)?  

c) Sketch a graph of the results of the whale study. 

Example 2

Here's another example: This one took place in a public school in New York City. The school was built right next to an elevated subway track, so kids in one of the school were exposed to the loud noise of subway trains approximately every four minutes. 

One side of the school building faced a nearby elevated subway; the other side faced away. Bronzaft matched second-, fourth-, and sixth-grade classrooms on the quiet side and on the noisy side, where a passing train would push the sound readings from 59 decibels to 89 decibels. Then she compared the average reading scores from the two sets of classrooms.

BRONZAFT: And the children exposed to the transit noise were nearly a year behind in reading by the sixth grade, and the teacher had difficulty teaching.

d) In the quasi-experiment described here, what is the quasi-independent variable? What are its levels? What is the dependent variable?

e)  What type of quasi-experiment is it (nonequivalent control groups, nonequivalent control group prettest-posttest, nonequivalent control groups interrupted time series, nonequivalent control groups interrupted time series)?  

f) Sketch a graph of the results of the reading study. 

g) In the podcast, they make two caveats:

It’s worth noting that Bronzaft’s subway research, as with similar studies at airports and elsewhere, have some limitations. For one thing, Bronzaft couldn’t randomly assign students to the noisy versus quieter classrooms. There were also relatively few classrooms to choose from, so there might have been some natural variation.

Which of the four big validities is the first caveat about? Which of the four big validities is the second caveat about?  How big of a problem would this be? 

Follow up: Bronzaft conducted and published additional studies, in which, after noise reduction efforts (such as adding acoustic tiles and treating the rails), children on the train-side of the building and children on quiet side of the building were reading at the same level.  

Would you like to read more about the effects of noise on child development and mental health? Visit this resource. There's also an educational module available about the effects of sound in NYC here.

These students might misattribute their early-morning fatigue to their interest in the subject. Photo credit: Gorodenkoff/Shutterstock

Podcasts have been gaining in popularity in the last few years, and several feature psychological research. I'm going to be devoting my next few posts to some podcasts. We'll start with this very short, 3-minute episode/ interview from the NPR podcast, Hidden Brain. The episode is called "How does the way you feel shape your life?" and it's about how students might be influenced by situational factors (such as how early a class is) when they select their majors. 

The podcast opens with a story:

...behavioral economist Kareem Haggag at Carnegie Mellon University... sometimes teaches early morning classes. And he's worried that when his students are tired or sleepy, they might draw the wrong conclusion about him and about the class.

The researcher himself states (around minute 1)

DR. KAREEM HAGGAG: Our hypothesis was that students who are assigned to an early morning section of the class, or to multiple back-to-back classes before a class, might mix up how tired they are in that class with how much they like the subject, thus leading them to be less likely to choose the subject as their major.

The host, Shankar Vedantam, reports:

This is a psychological phenomenon called misattribution. You know, you go to an amusement park on a sunny day and you think the park is great. You go when it's raining - you think the park is terrible. You're unconsciously confusing your feelings about the weather for the quality of the park.

The researchers used thousands of data points from students at the U.S. Military Academy at West Point to test their hypothesis. As Vedantam points out:

... students at West Point are randomly assigned classes that are scheduled at different times of the day, which means that some students are taking this introductory class first thing in the morning. Some students are taking the very same class later in the day. The researchers then measure the chance that students would later on choose that subject as their college major.

The researchers, led by Dr. Haggag, found that students were more likely to major in the topic of that introductory class if they'd taken it at a later time of the day: 

HAGGAG: We find that students who are randomly assigned to the first period, 7:30 a.m. section, are about 10% less likely to choose the corresponding major compared to a student who takes that class later in the day. 

Questions

a) What are the two main variables of interest in this study (the independent and dependent variables)? What are the levels of each variable?

b) Is this a true experiment or a quasi-experiment? If it's a true experiment, what kind is it (prettest-posttest, posttest-only, repeated measures, concurrent measures)?

If it's a quasi-experiment, which of the four types is it (nonequivalent control groups, nonequivalent control group prettest-posttest, nonequivalent control groups interrupted time series, nonequivalent control groups interrupted time series)?

c) Which is the IV and which is the DV here?

d) Sketch a small bar graph of the results they describe.

e) Can these researchers support the causal claim that "having an early morning introductory class makes you less likely to want to major in that discipline?" Apply the three rules of causation: covariance, temporal precedence, and internal validity in your response.

f) Ask a question about this study's internal, external, statistical, and construct validity.

Bonus content:

In the study above, the main IV was "time of day of the introductory class". As a bonus, here is another independent variable they studied--"number of breaks before the introductory class":

DR. KAREEM HAGGAG: We also compared two students who are sitting in the exact same classroom, but one of whom just had a free period as a break before, and the other came from one or more back-to-back classes. We find that each additional back-to-back class reduces the likelihood that that student enrolls in the major by about 12%.

g) Which is the IV and which is the DV here?

h) Sketch a small bar graph of the results they describe.

i) Can these researchers support the causal claim that "having breaks before your introductory class makes you more likely to want to major in that discipline?" Apply the three rules of causation: covariance, temporal precedence, and internal validity.

Which government-led social distancing measures are effective at stopping the spread of the novel Coronovirus?  One company has contributed data to answering this question. A private health technology company, Kinsa sells a "smart thermometer" which links to an app that guides people through what to do if they have a fever. When people use the app, they agree to have their temperature information (including fever and other symptoms they report) shared with Kinsa. 

Kinsa has used body temperature data points from all over the U.S. to create maps showing areas of "atypical illness." They define atypical illness on their website as "an unusual incidence of elevated flu-like illness levels."

It's really interesting to explore Kinsa's atypical illness maps of the U.S. here. 

More relevant for a research-methods course are the figures that Kinsa presents that map "flu-like illness" rates over time in certain high-population counties (Boston, New York, Los Angeles). I've pasted two of them here. Take a look at these figures, and then answer the questions below. 

Image courtesy of Kinsa, Inc.

Image courtesy of Kinsa, Inc

a) These figures should remind you of one of the quasi-experimental designs covered in Chapter 13. Which design is it?

b) The Kinsa company has used these charts to argue that social distancing measures work to reduce cases of COVID-19. Journalists (for example) have used the charts to specifically argue that limiting large gatherings and closing schools alone does not do much to stop the spread; it was only when bars and restaurants were shuttered that these counties saw reduced rates of atypical illness. Look carefully at the three counties above. What parts of the data are relevant to this argument? Do you think the data support the claim that closing bars and restaurants was effective? Can you think of alternative explanations for the pattern, other than "closing bars and restaurants"?

c)  Finally, consider the ethics and scientific openness principles. Do you think it's ethical for Kinsa to use and publicize the private health information of people who buy its products? Consider arguments both for and against the scientific ethics of this practice. Frame your arguments around the ethical principles of respect for persons, beneficence, and justice.

In your reflections, you might consider that users of Kinsa's products agree to data sharing via its terms and conditions. You might consider that the data from users is kept anonymous and is aggregated over more than a million users. You might consider the public health benefits of sharing the data. And you might also consider the extent to which Kinsa is following open science practices--that is, sharing all the data the way most university researchers now do (relevant Kinsa research information here).

d) If you're construct-validity-curious, you might be wondering if Kinsa's definition of flu-like illness actually tracks COVID-19 cases. Kinsa do not claim to be diagnosing actual COVID (you need a nasal swab for that). However, they present some maps showing how their temperature data has tracked with confirmed COVID cases in several counties. The data are in the middle of this webpage.  

By the way, here's another example of a company that collects our health data, using it to share national patterns. This one is Fitbit, sharing how sleep time has increased (in most age groups) since the COVID quarantine started.

What is the interruption in an interrupted time series design quizlet?

How is a nonequivalent control group design different from a true independent groups experiment?

When using a nonequivalent groups design the researcher will handle subject assignment to groups by?

Which of the following types of studies compares two nonequivalent groups one of which is exposed to the treatment and the two groups are compared?