Math 180: Half-Year Implementation

Esparto ISD located in Esparto, CA, enrolls approximately 900 students in Grades K through 12. The majority of students in Esparto ISD are Hispanic (77%); 17% are White, 1% are African American, and 1% are two or more races. Eighty-two percent of enrolled students are eligible for free or reduced-price lunch.

During the 2019–2020 school year, Esparto, CA, school district began implementing Math 180^®with their Grade 7 and Grade 8 students and completed fall and winter testing. Due to massive disruptions in Spring 2020 due to COVID-19, spring testing did not take place. Most students were able to continue using the program remotely, but due to lack of spring assessment data, this study focuses on the efficacy of the Math 180 program during the first half-year of implementation.

Math 180 is a blended-learning intervention program designed to prepare students for Algebra Readiness. It is designed to build their confidence and competence in mathematics while providing teachers with comprehensive support. Built and informed by an extensive body of literature, Math 180 uses the best practices for serving older, striving math students. It combines personalized software and teacher-facilitated instruction so that students can accelerate swiftly and successfully toward grade-level curriculum.

The Esparto, CA, school administration focused on students with the greatest need, identifying Grade 7 and Grade 8 students scoring multiple grade levels below in math achievement. Identified students were randomly assigned to either the Math 180 or control group. Of these students (39 in total), 20 were enrolled in Math 180and utilized the traditional 50-minute rotational model for the program, while 19 were enrolled in the school’s business-as-usual intervention program.

For the Math 180 classrooms, one teacher taught both a Grade 7/Grade 8 Math 180 class and a Grade 7/Grade 8 control-group class. The school’s core mathematics curriculum was HMH’s GO Math! ^® The principal indicated that the school does not use a specific mathematics intervention other than Math 180 but has explored different programs. In addition to the Math 180 program, teachers were also provided an initial startup training, as well as monthly coaching visits provided by HMH Professional Services.

Students in the control group used Prodigy computer software paired with core curriculum worksheets from a lower grade level during the winter observation.

Math Inventory

Sites administered the Math Inventory^®in Fall 2019 and Winter 2020 to Math 180 and control-group students. The Math Inventory is a computer adaptive assessment that measures students’ readiness for mathematics instruction using the Quantile^® Framework for Mathematics, which utilizes the same scale for the difficulty of mathematical topics and concepts and a student’s knowledge of the prerequisite mathematics needed to be successful when learning these topics and concepts. The five strands of the Quantile Framework are based on the five content standards of the National Council of Teachers of Mathematics (NCTM) standards (i.e., Number and Operations, Geometry, Algebra/Patterns and Functions, Data Analysis and Probability, Measurement).

Student Survey

Participating teachers administered the Math 180student survey in the fall to students enrolled in Math 180 andcontrol-group classes. The student survey was designed to measure social and emotional learning in mathematics, including purpose and value, autonomy and agency, competence, and mindset. These social and emotional learning outcomes were prioritized in this study because they align with the Math 180 curriculum. Three scales in the Math 180 student survey (purpose and value, autonomy and agency, and competence) used a 5-point scale ranging from not at all true to totally true. The mindset survey scales used a 6-point scale ranging from disagree a lot to agree a lot, with scores closer to 1 being indicative of a fixed mindset and scores of 6 being indicative of a growth mindset. Survey questions were largely drawn from published survey scales. Negatively worded items were reverse coded. Items that compose survey scales were averaged to create scale scores for subsequent analyses.

At the beginning of the study, students in both groups completed a survey designed to measure key social and emotional learning (SEL) outcomes of the Math 180curriculum: purpose and value, autonomy and agency, competence beliefs, and mindset (both growth and fixed). Table 1 summarizes the mean ratings for each of these scales. Students in both groups rated themselves slightly above neutral across the first three dimensions. For growth mindset, students rated themselves as mixed, leaning slightly toward growth. Differences between the two groups at baseline were not statistically significant. Due to the disrupted school year, end-of-year survey data was not collected.

Teacher-Facilitated Instructional Progress

Researchers observed Math 180 and control classrooms in Fall 2019 and Winter 2020. All Math 180 teachers were implementing the curriculum during observations. During Fall 2019 and Winter 2020 observations, Math 180 classroom sizes ranged from 8 to 17 students (average 11 students), and class time averaged 44 minutes. Control math intervention classroom sizes ranged from 6 to 15 (average 10 students), and class time averaged 40 minutes.

During Fall 2019 observations, the teachers and students were working in Block 4 (fraction concepts) in late October. They were initially advanced to this block to better meet the students’ instructional needs. The HMH Professional Services coach for the site noted during her interview that the instructional pace was slow; even after the class advanced to Block 4, progress through the block was slower than expected.

During Winter 2020 classroom observations, teachers and students were in Block 5 (fraction relationships). During both fall and winter, teachers and students were doing a concept lesson during half of the observations. Students were observed to be fully engaged during teacher-led instruction, pairs or independent work, and computer software time. Overall, during observations in control classrooms, students were partially engaged during teacher-led instruction, independent work, and/or computer software time.

Software Progress

The Math 180 software tracks progress of each student. As of December 31, 2019, students had completed an average of 2.75 topics and had spent 52% of their time in the Learn Zone. Time spent in the other zones was 21% Brain Arcade, 11% Success Zone, and 8% Explore Zone.

In the fall and winter, students in both the Math 180 and control groups were assessed on the Math Inventory. Between fall and winter, average scores increased from 600 to 695 Quantile measures for the Math 180 group and from 640 to 666 Quantile measures for the control group (see Graph 1). The growth of 95 Quantile measures in half a year for the Math 180 students exceeded the average annual growth of 90 Quantile measures expected for those students, whereas at a growth of 26 Quantile measures, the control-group students did not exceed the average expected annual growth. The Math 180 students demonstrated greater fall-to-winter gains on the Math Inventory than control students; although the trend is positive, the difference in gains was only statistically significant at the p = .07 level. A larger sample would be required to increase the reliability of these findings.

Execution of the rigorous RCT study of Math 180 initiated in Fall 2019 when students in Grades 7 and 8 were randomly assigned to the Math 180 or control condition. Math intervention implementation began after Math 180 teachers were trained in the program. Consistent with a student-level random-assignment RCT, students in the Math 180 and control conditions were statistically similar on all demographic characteristics and on mathematics assessments, including Math Inventory, prior to beginning Math 180 implementation. In addition, across conditions almost all students’ Math Inventory scores were Below Basic. Students scoring in Below Basic range can be considered mathematically significantly below grade level. This performance is consistent with identification of students in need of mathematics intervention instruction. Finally, Math 180 and control-group students were statistically equivalent on all measured social and emotional learning outcomes.

After exposure to the Math 180 program for only one semester, students demonstrated over one year of achievement growth. The fall-to-winter gains on the Math Inventory at Esparto trended favorably toward Math 180 students over control students, but sample sizes were small, and differences were only statistically significant at the p = .07 level. Due to COVID‑19, schools discontinued in-person instruction in Spring 2020, resulting in no end-of-year data collection of student achievement or social and emotional learning outcomes.