Amira: Wawasee Community School Corporation (2020–21)

Wawasee Community School Corporation (WCSC), located in north-central Indiana, enrolls approximately 750 K–3 elementary students across three elementary schools. District student enrollments are 92% white and 6% Hispanic.

In this report, we address the following research question: What is the relationship between participation in Amira Learning practice sessions and student literacy development in Wawasee Community School Corporation during the 2020-21 school year?

Amira Learning^™ provides 1:1 reading tutoring, reading fluency assessment, and dyslexia risk screening—from anywhere. After students take their first Oral Reading Fluency Benchmark Assessment, they are automatically placed into 1:1 reading tutoring sessions that provide instructional micro-interventions rooted in the science of reading.

While the district intended for all K-3 grade WCSC students to have the opportunity to participate in Amira Learning throughout the 2020 - 2021 school year, there was significant variation in student usage. The district encouraged teachers to use Amira Learning in their classrooms; however, it was not mandated given the COVID-19 pandemic. 

Beginning in mid-September and continuing through May, schools and teachers were asked to encourage students to complete two to three practice sessions per week, equivalent to approximately 30 minutes a week for 30 weeks. Students initially engaged with Amira Learning in a remote setting due to the COVID-19 pandemic, but transitioned back to in-person, classroom settings mid-year.

Actual usage rates, however, were lower than expected with the modal student participating in a practice session for 11-12 weeks throughout the school year. Roughly two-thirds of students participated in 14 or fewer weeks of practice sessions, meaning the majority of students received less than half of the intended treatment. Ultimately, only one student participated for 20 weeks or more. Further, the average number of minutes read per week was nine, well below the recommended 30 minutes per week.

For purposes of the study, we only selected students who completed the fall, winter, and spring Oral Reading Fluency (ORF) benchmark assessments and engaged in at least one practice session, which produced an analytic sample of 88 K–3 students. See table 1 for distribution of study participants across grades.

LITERACY OUTCOMES

The Amira Learning suite provides two assessments: The Dyslexia Screener and the Oral Reading Fluency (ORF) assessment. The Dyslexia Screener was administered once during the year, and the ORF assessment was administered three times throughout the year (fall, winter, and spring).

As part of the Dyslexia Screener, students complete a variety of tasks including items such as: nonsense word fluency, sound and letter recognition, blending, reading comprehension, or other phonics or phonemic awareness tasks. They also complete a reading fluency passage from which additional data is extracted about WCPM, accuracy, vocabulary, and sight word knowledge. Schools have the ability to configure the assessments to include all or some of the tasks available through the Amira Learning assessment. WCSC configured their assessments to include all the available tasks. Typically, students take 8–12 minutes to complete the full assessment.

We examine the associations between Amira Learning usage and five literacy outcomes, including Oral Reading Fluency, Vocabulary Size, Sight Recognition, Phonological Awareness (PA), and Lexile score. Oral reading fluency is a student's ability to read text accurately, at an appropriate rate, and with expression. Oral reading fluency is measured through Words Correct Per Minute (WCPM), which captures the number of words a student can read accurately within the one-minute timeframe. We use the adjusted WCPM score, which accounts for differences in passage difficulty. Vocabulary Size estimates the number of words likely present in a student’s expressive vocabulary. Sight Recognition uses the Estimated Sight Recognition Inventory (ESRI) to estimate the percentage of sight words a student has mastered. Phonological Awareness, measured through Phoneme Segmentation Fluency, captures a student’s ability to accurately produce phonemes within words. Finally, Amira Learning produces a Lexile score based on the Oral Reading Fluency. Lexile is an outcome of reading ability with a higher Lexile score indicating that a student is capable of reading and understanding more challenging texts.

AMIRA LEARNING USAGE INDICATORS

To capture Amira Learning usage, we used continuous measures of the number of weeks students completed at least one Amira Learning practice session. We created separate indicators for the fall-to-winter, winter-to-spring, and fall-to-spring periods. We also created a categorical version of the measure that indicates very low-usage (1–4 weeks), low-usage (5–9 weeks), medium-usage (10–14 weeks), and high-usage (15 or more weeks) for fall-to-spring. Similarly, we explored Amira Learning usage with both a continuous measure of the number of practice sessions a student completed and a categorical version of the same variable (1–9 practice sessions, 10–19 practice sessions, and 20 or more practice sessions). A practice session is considered any day a student logs on to the platform and reads one or more practice stories. 

We begin by exploring background differences between students who engaged Amira Learning to different degrees. We find a relationship between student grade level and usage (p<.001). For example, first grade students are overrepresented in the very low-usage and low-usage categories, while third grade students are overrepresented in the high-usage categories.

We also find that high-usage (15+ weeks) students read more minutes (+6 minutes; p<0.01) and complete more practice sessions (+35 sessions; p<.001) as compared to their very low-usage peers (see Table 2). We also find that medium-usage (10–14 weeks) students read slightly more minutes (+4 minutes; p<0.10) and complete more practice sessions (+19 sessions; p<.001) compared to very low-usage students. These findings indicate that not only do our high-usage and medium-usage students participate in the platform more frequently by definition, but that when they are on the platform, they read more and complete more sessions than their very low-usage peers.

In addition, we see large gains during the fall-to-winter period across all grade levels; however, we find losses or limited gains during the winter-to-spring period.

The next focus of this study is the test score patterns among Amira Learning students with varying usage rates. We do not find a significant relationship between fall literacy outcomes and usage. However, by spring, we find meaningful results in three of the five Oral Reading Fluency measures. In Graph 1, we see high-usage students (p<.10) have higher adjusted spring WCPM compared to their very low-usage peers.

In Graph 2, we see high-usage students (p<.05) and medium-usage students (p<.05) have higher spring vocabulary size compared to their very low-usage peers.

In Graph 3, we see high-usage students (p<.05) and medium-usage students (p<.01) have higher spring Lexile scores compared to their very low-usage peers.

Next, we constructed an Ordinary Least Squares (OLS) regression model to explore the link between Amira Learning usage and student literacy development. We find a relationship between Amira Learning usage and literacy development. As indicated in Model 1 in Table 3, we find that students gained 0.054 SD in WCPM for each additional week of usage (p<.01). Rather than a continuous measure, Model 2 uses a categorical measure that compares low-, medium-, and high-usage to very low-usage students. In this model, we find that high-usage students gained 0.845 SD more in WCPM than their very low-usage peers (p<.05). We also find that medium-usage students gain 0.514 SD more in WCPM than their very low-usage peers (p<.10). In Models 3 and 4, which use number of practice sessions to capture usage, we find a similar relationship.

As indicated in Table 4, we consistently find a positive and significant relationship between the number of weeks on Amira Learning and literacy development across all outcomes and time periods.

This report explored the implementation of Amira Learning in Wawasee Community School Corporation during the 2020-2021 academic school year. Although limitations in sample size, differences in implementation, and disruptions on education by the COVID-19 pandemic limit our findings, our models do suggest a relationship between usage of Amira Learning and literacy development. Accounting for student’s grade level, school, and baseline literacy score, we find high-usage students experienced greater gains as compared to very low-usage students. These results provide promising evidence that frequent usage of Amira may increase literacy development.