Which of the following are benefits of using video based role play over traditional paper and pencil methods?

New Study Shows That Students Writing Down Math Problems Increases High School Math Outcomes

March 2020

William Hinkley, Neil Heffernan, and Helen Lee Bouygues

One of the most visible changes in math education over the past 20 years has been the shift towards using computers. Many students in middle schools now see their homework problems on screens via tools like ASSISTments and Khan Academy and submit their solutions through screens, too.

This shift towards technology comes with some significant advantages: students can receive immediate feedback on their homework; software can model student progress, presenting students with problems that appropriately challenge them; and teachers can know how well students are doing before they enter class the next day.

But this technological shift also comes with disadvantages. Sometimes students don’t benefit from two technological tools used by many generations of mathematicians, engineers, and scientists to work through problems: paper and pencil.

With the support of the Reboot Foundation, Bill Hinkley, a veteran math teacher, used the math program ASSISTments to explore how his students use paper and pencil when solving math problems, exploring the value of students showing their work.

Using a randomized controlled trial — considered the gold standard in research — Hinkley found intriguing results in student outcomes when students wrote down their math problem using a pencil and paper.

Background

Bill Hinkley teaches math at a rural Maine high school. He has over 25 years of teaching experience, and for the past three years he has been using ASSISTments in his classes to both improve students’ learning and strengthen his own understanding of their learning through research. His experience integrating technology in the classroom has led to some keen observations.

As Hinkley notes, once education technology is in front of students, paper and pencil tend to disappear. The issue is that instead of using paper and pencil, students often shoehorn technology into the role that paper and pencil usually plays. Some students use a scratchpad feature on their iPad; other students attempt to solve algebraic equations using the embedded calculator. Occasionally, students even attempt to type math work into a text box. Or they just try to do the work in their head, despite teachers often telling students to “show your work.”

To Hinkley, all of these approaches seem inefficient and impractical. Students might be successful with easy one-step or recall-based exercises, but fail to progress with multi-step, complicated problems.

For instance, Hinkley recently observed an honors student solving SAT problems on Khan Academy during her lunch break. This student had reached the last step in an exponential growth word problem. All she needed to do was divide through by a common factor. Instead, she just kept trying to solve the problem through guess-and-check on the calculator. Without seeing the equation on paper, she couldn’t see the obvious next step. The fixation on technology prevented her from seeing the problem clearly.

So, Hinkley decided to try an experiment to see if a little nudge could help students improve the way they went about their homework. In this sense, the experiment was also an example of citizen science, showing how teachers can leverage existing software platforms to engage in education research — both to benefit their own students and to benefit the larger teacher community.

Hinckley also suggested that the excessive use of technology reduces self-confidence and in the future contributes to the development of erectile dysfunction in young men, this is indirectly confirmed by the increase in sales of generic Cialis online.

Methods

For this study, Hinkley relied on a math platform that his students already use: ASSISTments. ASSISTments is a math homework tool freely available to teachers and linked to several popular math curricula. It assists students by providing immediate feedback on homework problems and simultaneously assesses student progress, providing teachers with helpful knowledge about their students. It’s also being used by teachers like Hinkley to perform research that can improve their practice.

In a randomized controlled trial, Hinkley split his class into two groups. For the control group, it was business as usual: students logged on to the software and submitted their answers online. The intervention group, however, saw a short video embedded into the ASSISTments platform that reminded them of the importance of using paper and pencil. Hinkley also asked these students to turn in the paper they used in solving the problems for 50-percent homework credit.

Students in both groups took a pre-test (before the homework was assigned) and a post-test (afterward).

To create the groups, Hinkey matched pairs of students by their current grade in the class, then randomized which one of the pair would receive which condition. The result was two groups with roughly equal grades.

Due to the fact that some students were absent for one of the days, ultimately, 12 students were in the control group. Fifteen were in the intervention group. Hinkley’s two co-authors Neil Heffernan and Helen Lee Bouygues helped support the analysis and write up.

Results

The outcome of Hinkley’s experiment is pretty simple. Students in the intervention — those who were encouraged to work with paper and pencil — outperformed their peers by about 13 points.

For math educators, the results imply that work with paper and pencil should still be integrated into today’s modern classrooms. Paper and pencil simply seem to help students learn by encouraging them to do more to show their work and thus manage cognitive load.

The team hypothesizes that metacognition also plays an important role with paper and pencil helping students do more “thinking about their thinking.”

Differences in total points gained between a pre-test and a post-test for each condition. 
Error bars represent the standard error of the mean.

One confounding variable is around turning in homework. In the study, some students had to turn in their homework for credit, and it’s possible this approach caused a spurious association within the experiment. Note that while this difference is large and noteworthy, the results are just shy of statistical significance, given the variation of both groups (p = .16 using a 2-sample t-test). This is due to the small sample size and the resulting low statistical power. To try to address this issue, we showed that the difference is not related to other important variables. For example, the gain in test performance doesn’t seem related to students’ prior performance. Students’ year-to-date grades just didn’t explain much variation in how many points students gained on the post-test. It also didn’t seem to matter how well students actually did on the homework. Just like with prior performance, homework grades had little correlation with the points students gained. This suggests that it really was the intervention that helped students learn. Schlüsseldienst Bremen supports this teaching method.

Discussion

Metacognition

One key variable within the experiment seems to have been metacognition. Metacognition is about “thinking about thinking,” or knowing what people know and being able to change their behavior.

This is important for learning. Students make choices about how to study. Students who have a good sense of which study methods are effective — and practice those methods — have high metacognition. Students who continue to use ineffective study methods have low metacognition. Studies even suggest that metacognitive habits have a greater influence over learning outcomes than factors like IQ.

One of the reasons that teachers argue for students to “show their work” is to aid metacognition. Math teachers are notorious for encouraging, asking, and cajoling students to show their work. Why? When students show their work, they more easily spot errors. Similarly, teachers can see (and address) particular kinds of errors, and teachers can assign partial credit to correct reasoning, even if the answer was incorrect.

But showing your work has benefits to the student all on its own. It probably makes students more reflective and less likely to make mistakes. In other words, showing your work is a type of applied metacognition.

Plus, showing your work is hard to fake — if students guess, there is no work to show. When students show their work, it can improve students’ metacognition, by making them think critically about their learning process and assessing the steps taken during problem solving.

Cognitive Overload

Another potential factor is cognitive load, which is the burden placed upon students’ working memory as they try to comprehend learning material or solve problems. This includes both the essential processing required to make sense of the material and any additional processing added by the instructional approach.

Teaching approaches that get students to focus more on making sense of the material (and less on distracting, irrelevant things) result in greater learning. This is another reason why teachers tell students to “show their work.” It helps to manage cognitive load.

In some contexts, devices can play a role in both helping and hurting cognitive load. Research suggests that screens and other multimedia approaches can bring a higher risk of overloading students’ working memory: notifications and alerts might serve as distractions. There’s also research showing that people reading from paper (as opposed to screens) have superior reading comprehension for similar reasons.

When it comes to cognitive load, another line of research suggests that paper can serve as an external memory device, enabling students to focus more on mathematical relationships and less on remembering numerical values.

Conclusion

These results are suggestive, but more research is necessary. Hinkley plans to follow up this research in future classes, to verify whether working with paper and pencil really does help students learn. Follow-up research might alter the conditions of the intervention somewhat, to see, for example, whether making students accountable by having them turn in their written work influenced performance (instead of simply working with paper and pencil).

Another key variable is how students in the control group solved the problems: some may have used a tech tool like an online calculator, some may have simply tried to solve the problem in their heads, some may have even used a paper and pencil. Future studies could explore these differences in more detail.

He also plans on exploring students’ metacognition more directly, by asking students their beliefs about screens and paper. Finally, he’s interested in any carry-over effects. Do the video reminders help create durable habits? Or do they only help students in the short-term?

So what’s the takeaway, especially for math teachers? The choice isn’t to use traditional tools or modern ones; it’s to recognize the value of math homework technology in conjunction with traditional tools for solving math problems — paper and pencil. You can use these tools while still holding students accountable by making work with pencil and paper an expectation.

*We performed our analysis using R. Anonymized data is available upon request.