READ 180 Universal/System 44: Lincoln Unified School District

Read 180 now incorporates the comprehensive foundational literacy skills scope and sequence from System 44.

Nationally, one-third of American middle school students fail to demonstrate competency over grade-level reading skills (NAEP, 2017). Students who do not meet or exceed benchmark scores on state and national tests are less likely to graduate from high school, are less likely to persist in or successfully complete future academic and workplace training endeavors, and are overall less likely to be on track for future academic and workplace success (CCRSC, 2013). The Lincoln Unified School District identified the need for an intervention to accelerate the district’s most struggling readers to grade-level reading proficiency.

READ 180 Universal is an intervention program designed by Houghton Mifflin Harcourt^® to build reading comprehension, academic vocabulary, and writing skills for struggling students in Grade 4 and up. Students progress through six workshops that include differentiated instruction, adaptive software, writing tasks, independent reading, and project-based learning assessments. This blended learning solution begins with whole-group learning that focuses on close-reading strategies, academic vocabulary exercises, writing practice, and discussions, then rotates between the following: a) student application with a personalized path of self-paced independent work; b) student-selected independent reading material to make reading a daily habit; and c) small groups with data-driven instruction, all of which is followed by a whole-group wrap-up. READ 180 Universal utilizes engaging and relevant content and research-based digital solutions to motivate students and increase reading fluency and comprehension.

System 44 is an intervention program designed to build foundational reading and decoding skills for the most challenged readers. This blended learning solution includes daily reading, writing, talking, and critical thinking exercises. Materials feature modeled and independent reading with age-appropriate books that target decoding skills and strategies to promote comprehension, build vocabulary, and increase content-area knowledge. Using adaptive technology, students complete a personalized learning progression through five strands that include The Code, Word Strategies, Sight Words, Success, and Writing. System 44 is designed to be used in conjunction with READ 180 Universal in integrated classrooms to serve as differentiated instruction during small group and the software rotation.

Students in Grades 6, 7, and 8 experienced the Double Period Instructional Model for 90 minutes approximately three days a week.

The Lincoln Unified School District (LUSD) in Stockton is one of 1,228 school districts in California. LUSD served 9,420 K–12^th-grade students in the 2017–2018 school year, of which, eight elementary schools served 5,630 PreK-8^th-grade students and one middle school served 658 7^th–8^th-grade students. LUSD serves a diverse population of students with a range of ethnic backgrounds: African American (12%), Asian (13%), Caucasian (22%), Hispanic (48%), Native American (2%), and students with multiple ethnic backgrounds (3%). In the 2017–2018 school year, 63% of students in the district were eligible to receive free or reduced-price lunch through the National School Lunch Program, 15% were classified as English learners (EL), and 11% were classified as Students with Disabilities (SWD).

All schools serving 6^th–8^th-grade students in LUSD utilized READ 180 Universal as a Tier 2 reading intervention and System 44 as a Tier 3 reading intervention during the 2017–2018 school year in integrated classrooms with differentiated instruction. All students who completed at least 15 READ 180 Universal sessions totaling at least 150 minutes and no System 44 sessions (n = 251) or who completed at least 10 sessions each in both System 44 and READ 180 Universal totaling at least 150 minutes (n = 38) were included in the analysis; 41 students with less use were excluded from this analysis. Students in this analysis attended Brookside School (n = 20), Claudia Landeen School (n = 27), Colonial Heights School (n = 26), Don Riggio School (n = 41), John R Williams School (n = 20), Lincoln Elementary School (n = 19), Mable Barron School (n = 26), Tully C Knoles School (n = 32), and Sierra Middle School (n = 78). Student ethnic backgrounds included African American (19%), Asian (8%), Caucasian (13%), Hispanic (55%), and Native American (2%), and students with multiple ethnic backgrounds (3%). Of these students, 60% were male and 40% were female, 84% were eligible to receive free or reduced-price lunch through the National School Lunch Program, 34% were classified as EL, and 31% were classified as students with disabilities.

LUSD students in Grades 6 to 8 were identified as struggling with reading comprehension based on a pattern of below grade level Measures of Academic Progress (MAP^®) scores, DIBELS^®, Smarter Balanced Assessment English Language Arts/Literacy scores and Reading Inventory Lexile^® scores. Students who scored less than 600 Lexile (L) measures on the Reading Inventory completed the Phonics Inventory; those students who scored in the “Beginning” or “Developing” range worked in System 44 until moving to READ 180 Universal after scoring in the “Advancing” range (called mixed-model implementation hereafter). Those students who scored in the recommended Lexile (L) range on the Reading Inventory (between 600-880L in Grade 6, 600-950L in Grade 7 or 600-990L in Grade 8 ) began READ 180 Universal instruction.

Students in Grades 6, 7, and 8 received 90-minute System 44 and/or READ 180 Universal instruction approximately three times a week. In this Double Period Instruction Model, students completed 20 minutes of whole-group instruction followed by 20 minutes of each of three station rotations (rotated between the personalized online student application in either System 44 or READ 180 Universal, differentiated small-group learning, and independent reading), and concluding with 10 minutes of whole-group wrap-up.

LUSD teachers completed a two-day “Getting Started” training to learn how to use the System 44 and READ 180 Universal program technology, differentiation features, and program assessments in the 2016–2017 school year (the first year of implementation). Teachers also had 29 coaching days in the 2017–2018 school year and 12 days of coaching in the 2017–2018 school year to learn about effective planning, progress monitoring, and to receive further instruction on using differentiation and assessments.

Student software usage data was collected as students used the online student application during System 44 and READ 180 Universal instruction. Software usage data included number of completed segments, number of completed sessions, average time spent in each session, and number of sessions averaged per week. 

The Smarter Balanced Assessment (SBA) was designed to measure end-of-year achievement in English Language Arts/Literacy (ELA) and to accurately capture growth in ELA proficiency from previous years. The SBA measures ELA achievement in reading, writing, listening, and research using a computer adaptive system to deliver between 42 and 48 questions. Assessment results include a scale score between 2,000 and 3,000 and an achievement level descriptor (ALD) indicating overall performance level of Standard Not Met, Standard Nearly Met, Standard Met (indicating grade-level ELA proficiency), and Standard Exceeded. LUSD students complete the SBA each spring in Grades 3–8 and also in Grade 11. SBA scores are a part of the California Assessment of Student Performance and Progress (CAASPP).

The Reading Inventory measures reading comprehension proficiency for students in Grades K–12. The Reading Inventory uses adaptive technology to determine a student’s reading comprehension level on the Lexile Framework for Reading; the higher the Lexile score, the more challenging reading material the student can comprehend. Test item difficulty ranges from items appropriate for developing readers to items requiring a reading proficiency indicating preparedness for college-level texts, allowing measurement of skill growth regardless of the students’ initial ability. Assessment results include a Lexile scale score that indicates reading ability at a level of text complexity and a performance level of below basic, basic, proficient, or advanced, indicating achieved reading comprehension compared to grade-level expectations. LUSD students receiving System 44 and READ 180 Universal instruction completed the Reading Inventory in the fall of 2017 (August through October) before beginning instruction, again in the winter (November through January), and again in the spring of 2018 (April or May) following instruction.

The HMH Phonics Inventory measures proficiency in the foundational reading skills of phonological decoding and sight word reading for students in Grades 3–12. The Phonics Inventory is used to identify whether students with low reading comprehension achievement also lack the skills needed to decode new words (leading to placement in System 44) or are best served by an intervention to develop reading comprehension strategies, text analysis skills, and background knowledge (leading to placement in READ 180 Universal). Assessment results include a fluency score and decoder status of pre-decoder, beginning, developing, or advancing. LUSD students with Reading Inventory scores at 400L or below completed the Phonics Inventory before beginning instruction in System 44, and every couple of months following instruction until promotion to READ 180 Universal.

LUSD students whose home language is not English and students who are classified as English learners complete an English language proficiency exam each spring. For the 2016–2017 school year, students completed The California English Language Development Test (CELDT). The CELDT was designed to identify students who need to improve their skills in listening, speaking, reading, and writing in English. Students are assessed on the five domains of Reading, Writing, Listening, Speaking, and Comprehension. Assessment results include a scale score for each domain as well as an overall scale score with a range from 248–741 in Grades 6–8. For the 2017–2018 school year, the summative English Language Proficiency Assessments for California (ELPAC) replaced the CELDT to measure an English learner's progress in learning English and to identify the student's English language proficiency level. Assessment results include a scale score for each category of Oral Language (Listening, Speaking) and Written Language (Reading, Writing), an overall scale score with a range from 1,150 to 1,900, and an overall performance level from 1 to 4 (beginning English to well-developed English skills). The ELPAC, based on new California English Development Standards, does not produce scores that are comparable to CELDT scores. 

As LUSD uses frequent assessment data to place students into the appropriate tier of reading intervention when needed and exit students after reaching the appropriate level of proficiency, LUSD students may participate in an intervention for only one or both semesters of a school year and then move into another intervention or to standard instruction. Students who received READ 180 Universal instruction for the entire school year completed an average of 6.9 segments (SD = 4.81) over an average of 77.1 total sessions (SD = 25.45), with 2.8 (SD = 0.58) sessions averaged a week in the READ 180 Universal online student application. Each session lasted an average of 15.08 (SD = 3.72) minutes. These metrics are consistent with the implementation model. Students who completed at least 10 sessions each of both System 44 and READ 180 Universal instruction during at least 150 session minutes (in the mixed model implementation) during the 2017-2018 school year completed an average of 4.3 READ 180 Universal segments (SD = 3.64) over an average of 42.5 total sessions (SD = 20.64), with 2.7 (SD = 0.52) sessions averaged a week and averaging 15.4 minutes (SD = 3.71) per session in the READ 180 Universal online student application. Students in this mixed model implementation also completed an average of 26.6 System 44 topics (SD = 15.51) over an average of 39.0 total sessions (SD = 25.74), averaging 11.1 minutes (SD = 2.27) per session in the System 44 online student application. 

An independent evaluator from Forge Research Group analyzed student academic achievement using test score data provided by the Lincoln Unified School District and program usage data provided by Houghton Mifflin Harcourt. READ 180 Universal and System 44 + READ 180 Universal mixed-model students’ ELA performance was examined pre- and post-implementation using multiple independent measures of reading. This analysis included scores on the statewide end-of-year summative assessment (SBA), the Reading Inventory, and the Phonics Inventory. READ 180 Universal students and the mixed model students demonstrated statistically significant gains in ELA proficiency during the 2017–2018 school year on each of these reading measures.

LUSD students took the SBA ELA summative assessment in spring 2017, prior to receiving READ 180 Universal, and again in spring 2018, after instruction. READ 180 Universal students demonstrated a statistically significant overall gain in SBA ELA scale score, averaging a 39-point increase from 2,426 in 2017 to 2,465 in 2018 with effect size Cohen's d=.74 (see Figure 1). Further, the increase in the average the SBA ELA scale score was statistically significant for Grade 6 (56-point gain), Grade 7 (41-point gain) and Grade 8 (21-point gain) students. Mixed-model students also demonstrated a statistically significant overall gain in SBA ELA scale score, averaging a 49-point increase from 2,413 in 2017 to 2,462 in 2018. Notably, mixed-model students decreased the scale score gap from 13 points to 3 points, ending the year at a similar SBA scale score as compared to the READ 180 Universal students.

Notably, students showed accelerated growth in SBA ELA percentile compared to their initially same-scoring peers – on average, students scored better than 18% of their peers in 2017 but better than 21% of their peers in 2018, a statistically significant increase. 

READ 180 Universal students who completed the Reading Inventory in both the fall of 2017 and the spring of 2018 demonstrated a statistically significant overall gain in Lexile scores, averaging a 119L increase from 613L in fall to 732L in spring with effect size Cohen's d=1.09 (see Figure 2). Further, the increase in average Reading Inventory scores was statistically significant for Grade 6 (137L gain), Grade 7 (110L gain), and Grade 8 (112L gain) students. Mixed model students also demonstrated a statistically significant overall gain in Lexile scores, averaging a 244L increase from 428L in fall to 672L in spring. Notably, mixed-model students started the school year at a lower grade-level ability of reading comprehension but decreased the Lexile score gap from 185L to 60L, ending the year in the same grade-level Lexile range as the READ 180 Universal students.

In addition to demonstrating statistically significant Lexile gains, READ 180 Universal and the mixed-model students demonstrated accelerated Lexile score gains compared to the average annual growth demonstrated by an initially same-scoring national sample. Notably, on average, students gained approximately 1.4 times as many Lexile scores as would be expected from beginning to exiting instruction with effect size Cohen's d=.31 (see Figure 3), demonstrating accelerated growth towards grade-level performance.

Summary Growth Metrics (see Figures 4 and 5) show the significant extent of reading comprehension growth during the 2017-2018 school year. With just one year of use, the majority of READ 180 Universal students achieved a Lexile gain on Reading Inventory (87%) and met end-of-year growth expectations (60%) based on Estimated Average Annual Growth⁴ adjusted for the number of days in the program. Additionally, 30% of READ 180 Universal students more than doubled end-of-year growth expectations, and 31% of students increased at least one level on Reading Inventory College & Career Performance Levels.

Demonstrating accelerated growth, nearly all mixed-model students achieved a Lexile gain on the Reading Inventory (97%) and a large majority met end-of-year growth expectations (74%) based on Estimated Average Annual Growth adjusted for the number of days in the program. Additionally, 53% of mixed-model students more than doubled end-of-year growth expectations, and 41% of students increased at least one level on the Reading Inventory College & Career performance levels.

Importantly, READ 180 Universal students increased in grade-level reading ability (based on the text complexity grade correlated with the Lexile range aligned to college and career readiness) during the 2017-2018 school year (see Figure 6). The percentage of Grade 6, Grade 7, and Grade 8 students who demonstrated at least grade-level reading comprehension increased six-fold from 1% to 6% and the percentage who demonstrated reading comprehension ability five or more grades below grade level decreased from 42% to 22%. With just one year of instruction, 27% of the students increased two or more grade levels in reading comprehension.

Likewise, mixed-model students increased in grade-level reading ability during the 2017-2018 school year (see Figure 7). The percentage of Grade 6, Grade 7, and Grade 8 students who demonstrated at least grade-level reading comprehension increased from 0% to 3% and the percentage who demonstrated reading comprehension ability five or more grades below grade level decreased from 60% to 21%. With just one year of instruction, 15% of the students increased two or more grade levels in reading comprehension. On average, mixed-model students began the year demonstrating reading comprehension skills five grades below grade-level and averaged reading comprehension skills three grades below grade-level by the end of the year. Of note, increasing a grade level at the lower elementary school reading comprehension level requires close to twice as many Lexile score gains as does an increase in grade level ability at the middle school level. As such, the increase of one grade level in reading comprehension represents more absolute growth for the students with the lowest initial grade-level reading ability than for students with the highest initial grade-level reading ability.

Notably, growth on the Reading Inventory from fall 2017 to spring 2018 was highest for the most struggling readers. READ 180 Universal Instruction students who initially scored in the Below Basic performance level on the Reading Inventory gained an average of 133L (large effect size of Cohen’s d = .86) during the 2017-2018 school year, demonstrating accelerated growth in approaching grade-level proficiency compared to students who initially scored in the Basic performance level, who gained an average of 57L (medium-large effect size of Cohen’s d = .75). All mixed-model students initially scored in the Below Basic performance level, but gained an average of 244L during the 2017-2018 school year (large effect size of Cohen’s d = 1.61).

Further, increased use of the READ 180 Universal online software was a statistically significant predictor of reading Lexile score growth, both before and after correcting for selection bias (see Appendix Table 6 for details). On average, READ 180 Universal students who completed more segments of the READ 180 Universal online software also achieved higher gains in the Reading Inventory Lexile score (see Figure 8). Students who completed 10 or more segments of the online software achieved an average of 1.5 times more Lexile score gain (143L) compared to students who completed 1 to 3 segments (98L), with effect size Cohen’s d=.87 for low implementation, effect size Cohen’s d=1.06 for moderate implementation, effect size Cohen’s d=1.06 for high implementation, and effect size Cohen’s d=1.17 for very high implementation (see Figure 8). Similarly, for mixed-model students, as the number of System 44 topics completed increased so did student Lexile score gain on the Reading Inventory; this positive correlation was also statistically significant.

Mixed-model students also demonstrated statistically significant gains in Phonics Inventory fluency scores (see Figure 9), increasing from an average of 12 on first assessment to 18 on the final assessment. The percent of students achieving an “Advanced” decoder status increased from 0% to 33%, while the percent of students scoring at a “Developing” decoder status decreased from 50% to 21% from 2017 to 2018 (see Figure 10). 

In addition to SBA scale score and Reading Inventory Lexile gains, READ 180 Universal students also achieved increased performance levels reading measures with effect size Cohen’s d=.49 for SBA ELA performance levels and effect size Cohen’s d=.66 for the Reading Inventory (see Figures 11 and 12). The percent of students achieving a “Standard Met” SBA ELA Achievement Level Descriptor (ALD) increased from 0% to 7%, while the percent of students scoring at a “Standard Not Met” ALD decreased from 79% to 55% from 2017 to 2018. The percent of students achieving at least a Proficient Reading Inventory performance level increased six-fold from 1% to 6% and the percent of students scoring at a Below Basic performance level decreased from 76% to 50% from 2017 to 2018. The increase in students’ performance levels on both the SBA ELA assessment and the Reading Inventory was statistically significant.

Mixed-model students also achieved increased performance levels on reading measures (see Figures 13 and 14). The percent of students achieving at a “Standard Nearly Met” SBA ELA Achievement Level Descriptor (ALD) increased from 3% to 55% while the percent of students scoring at a “Standard Not Met” ALD decreased from nearly everyone at 94% to less than half at 42% from 2017 to 2018. The percent of students achieving at least a Proficient Reading Inventory performance level increased from 0% to 3% and the percent of students scoring at a Below Basic performance level decreased from 100% to 59% from 2017 to 2018. The increase in students’ performance levels on both the SBA ELA assessment and the Reading Inventory was statistically significant.

After one year of READ 180 Universal instruction or mixed-model instruction, many students increased an achievement level descriptor on the state summative assessment (33% and 50%, respectively), and/or increased a performance level on the Reading Inventory (31% and 41%, respectively).

Notably, disaggregation of the data indicated that use of both READ 180 Universal alone or in a mixed-model implementation with System 44 was associated with significant gains in both SBA ELA scale scores and Reading Inventory Lexile gains for all categories of students (results are presented together here but see Appendix for a separation by instruction group).

When results were disaggregated by gender, both males and females achieved statistically significant SBA ELA scale score gains (see Figure 15) and Reading Inventory gains (see Figure 16) from pre- to post-instruction.

When results were disaggregated by education classification, students with disabilities achieved statistically significant SBA ELA scale score gains (see Figure 17) and statistically significant Reading Inventory gains (see Figure 18). In addition to gaining an average of 120L on the Reading Inventory, 51% of the SWD met yearly growth expectations.

When results were disaggregated by eligibility to receive free or reduced-price lunch (FRPL) through the National School Lunch Program, FRPL-eligible students achieved statistically significant SBA ELA scale score gains (see Figure 19) and statistically significant Reading Inventory gains (see Figure 20). In addition to gaining an average of 129L on the Reading Inventory, 62% of the FRPL-eligible students met yearly growth expectations.

When results were disaggregated by ethnicity, African American, Asian, Caucasian, and Hispanic students achieved statistically significant SBA ELA scale score gains (see Figure 21) and Reading Inventory gains (see Figure 22) during the 2017–2018 school year. Although the sample was too small to capture the true significance of gains statistically, Native American students gained an average of 80L on the Reading Inventory and 67% met yearly growth goals.

When results were disaggregated by EL classification, EL students achieved statistically significant SBA ELA scale score gains (see Figure 23) and significant Reading Inventory gains (see Figure 24). In addition to an average gain of 146L on the Reading Inventory, 63% of EL students met yearly growth goals.

LUSD students classified as ELs as well as some students who exited the EL program in the previous four years (n = 96) completed the CELDT in spring 2017 and the ELPAC in spring 2018 to measure English language proficiency. The CELDT assessment results include an overall scale score with a range from 248 to 741 while the ELPAC results include an overall scale score with a range from 1150 to 1900. To examine year-to-year changes in scores that are on different scales, the CELDT and ELPAC scores were placed on a common scale using the percent of maximum possible (POMP) calculation. READ 180 Universal and mixed-model EL students averaged a statistically significant 6-point decrease in relative magnitude of scores when comparing 2017 CELDT scores to 2018 ELPAC scores. As these assessments measure different skills and define proficiency in different ways, it is unclear whether a change in relative magnitude of scores from 2017 to 2018 reflects a true change in English language arts skills or is a consequence of a more difficult exam, a new exam format, or another unmeasured variable. 

Multiple independent measures support the idea that students who received System 44 and/or READ 180 Universal instruction in a mixed-model classroom made significant improvements in English Language Arts and Literacy achievement. After one year of instruction, Grade 6, 7, and 8 students in the Lincoln Unified School District demonstrated statistically significant increases in Smarter Balanced Assessment ELA scale scores, Reading Inventory Lexile scores, and Phonics Inventory fluency scores. In addition to statistically significant year-to-year gains, students achieved accelerated growth in SBA ELA scale scores compared to initially same-scoring peers and gained approximately 1.4 times as many Lexile scores as would be expected based on the average yearly growth of an initially same-scoring national sample. Overall, the percentage of students demonstrating grade-level or above reading comprehension increased from 1% to 6% after one year of instruction, and the percentage of students demonstrating reading comprehension five or more grades below grade-level decreased from 45% to 22%. 

Additionally, 27% of READ 180 Universal instruction students and 15% of mixed-model students increased at least two grade levels in reading comprehension. Students also increased in assessment performance levels: 33% of READ 180 Universal instruction students and 50% of mixed-model students moved up an Achievement Level Descriptor on the SBA and 31% of READ 180 Universal instruction students and 41% of mixed-model students moved up a performance level on the Reading Inventory. Notably, on average, Tier 3 intervention students who received a combination of System 44 and READ 180 Universal instruction began the school year with significantly lower SBA ELA scale scores, HMH Reading Inventory Lexile scores, and grade-level reading comprehension ability than the Tier 2 intervention classmates who received READ 180 Universal instruction, but demonstrated accelerated learning and averaged gains great enough to end the year at similar achievement levels. Disaggregation of the data by gender, SWD status, FRPL eligibility, ethnicity, and EL classification indicated that use of both READ 180 Universal alone or in a mixed model implementation with System 44 was associated with significant gains in both SBA ELA scale scores and Reading Inventory Lexile gains for all categories of students. Students worked through varying amounts of the self-paced, personalized online instruction components – increased completion of System 44 topics was significantly correlated with reading Lexile gains and increased completion of READ 180 Universal segments was a significant predictor of reading Lexile gains during the 2017–2018 school year. This study demonstrates that using System 44 and READ 180 Universal to provide differentiated instruction in a mixed-model classroom is an effective method of increasing literacy for students struggling to achieve grade-level ELA proficiency.

College and Career Readiness and Success Center (CCRSC). (2013). Predictors of Postsecondary Success. Washington, DC: American Institute for Research.

U.S. Department of Education. National Department of Educational Progress (NAEP). (2017). The Nation’s Report Card: 2017 Reading Assessment. Washington, DC: U.S. Department of Education.