2 Examples of Physics Task Items
2.1 Physics Tasks, Part 1: Quantitative Problem Solving
Students were provided with a general equation sheet for reference and asked to solve a single quantitative problem. They were instructed to use a blank sheet of paper to write out their work and final answer. The quantitative problems were developed for this study by Drs. Melanie Good and Eric Kuo (See Supplementary Figure 2.1).
2.2 Physics Tasks, Part 2: Problem Categorization
This part consisted of five items adapted from, and similar to, those used in Hardiman, Dufresne, and Mestre (1989). For each item, a model problem was presented along with two alternative problems. Students were instructed to select the alternative which is solved most like the model problem (deep structure match). Four out of five items contained surface feature distractors, meaning that there were surface features in the incorrect alternative which resembled the model problem (See Supplementary Figure 2.2).
Supplementary Figure 2.2: At first glance, the model problem and alternative 2 appear similar because they both involve projectiles, but this similarity is a surface-level distractor. The model problem and alternative 1 are asking for final velocity and can be solved using conservation of energy, while alternative 2 requires the use of kinematics to solve for time. Therefore, the correct response is alternative 1.
2.3 Physics Tasks, Part 3: Conceptual Questions (Qualitative Problem Solving)
This part consisted of three multiple-choice questions (See Supplementary Figure 2.3), followed by one open-ended problem (See Supplementary Figure 2.4). For each multiple-choice question, participants were also asked to provide a brief explanation for their choice. The open-ended problem was a word problem in which participants were asked to explain a solution.
Supplementary Figure 2.3: Qualitative multiple-choice item: The correct answer is option A because the frictional force is proportional to the normal force, which is equal to the weight force. Therefore, the only way to increase the frictional force is to increase the mass; Supplementary Figure 2.4: Qualitative open response item: Newton’s second law states that \(F_{net}=\frac{Δp}{Δt}\). Airbags reduce the force (outcome) of a collision by increasing the time it takes for the collision to occur (mechanism). Half credit was given to responses that mentioned either force or time correctly, and full credit was given to responses that mentioned both force and time.
2.4 Physics Tasks, Part 4: Preparation for Future Learning
The Preparation for Future Learning (PFL) task (Belenky and Nokes-Malach 2012; Schwartz and Martin 2004) is a 3-part learning activity that participants completed only at posttest. Physics problems used in the learning activity were taken from Weinlader et al. (2019). The first part consisted of solving a difficult multiple-choice problem which required comparing the trajectories of two projectiles and predicting which would hit their target first (see Supplementary Figure 2.5). In the second part, a learning resource explaining the first problem was provided. The resource contained an explanation of why the correct answer choice was valid, and why each incorrect answer was not. The third part consisted of a novel problem with similar surface features to the first problem but had a different correct response. Answering correctly required applying the underlying principles of the first problem to a different set of conditions. Each of the two problems required both a multiple-choice answer as well as a short-answer explanation for the selected choice.
