Insights from 392,603 students across 48 districts help address the question: “Is this app having a positive impact on student learning?”
San Francisco, CA - November 8, 2018 - Education data management and analytics platform, BrightBytes, today published its second annual Technology and Learning Insights Report which is authored by Dr. Ryan S. Baker, Director of the Penn Center for Learning Analytics, and examines the cost, usage, and impact of education apps, leveraging data captured by the firm’s Learning Outcomes module.
"BrightBytes’ Learning Outcomes module provides our school with critical, real-time data that informs our decisions when allocating resources, including our most precious resource: time. While surveys or staff meetings provide perceptions which may have some value, Learning Outcomes presents the objective reality of how software is actually being used," says Marcus R.W. Mead, Director of Administrative and Instructional Technology at Glen Lake Community Schools in Michigan.
The analysis, which utilized a dataset from 48 school districts serving more than 390,000 students and 1.48 million hours of usage nationwide, measured digital apps across three domains: investment (subscription cost, number of licenses, active/inactive users), engagement (student usage, session duration, frequency, and quality, as well as user perception), and impact (relationship between standardized test scores and student usage).
Key findings from the report include:
Usage rates matter, sometimes. Many apps had no association with growth in student learning, regardless of student usage rates. However, several apps were positively associated with gains in student learning. In short, usage of high-quality apps matters to potential app impact on student achievement.
Most purchased licenses don’t get used. The degree to which licenses are used varies by app. The study found that a median of 30% of licenses are used by learners.
The number of licenses purchased does not predict the number of intensive users. 97.6% of licenses in this study were never used intensively (a learner who used the program for at least 10 hours between assessments), and the number of licenses purchased did not predict the number of intensive users. This suggests that some apps were more effective at converting licenses purchased into intense usage than others. Important to note that school implementation of apps can vary widely with different intended phasing-in of usage.
Many schools aren’t following the recommended “dosage” for apps to contribute to improved student outcomes. Many learners are not meeting the target number of instructional minutes recommended by ed tech app providers. The average usage was fairly low across schools and districts. It is likely that lower than expected usage may lead to less benefit for students than the apps are intended to produce.
There is a wide range in app cost per user. The study found there to be large variation in cost; the cost per user varies from under a cent per user to $393 per user (when omitting apps used by under 100 users), with a median cost per user of $6.45. This variance is not a surprise given the range of expected use and impact for each app.
The study uncovered some encouraging trends concerning the correlation between improved student math scores and 21 apps. The study found that these 21 apps, when used intensively, were positively associated with gains in math test performance. While this positive correlation does not mean that these apps were found to have caused the improvement in student math performance, this is an encouraging finding nonetheless.
“School and district leaders are faced with an ever-increasing selection of education apps, and the lack of easily accessible data makes it difficult to invest in the ones that demonstrate the greatest potential for ROL (return on learning) for students,” says Traci Burgess, CEO of BrightBytes. “Through the Learning Outcomes module, we are partnering with districts to provide on-demand access to this kind of analysis to surface insights that can inform strategic decision-making.”
To access additional highlights from the report, please visit the report’s interactive landing page here.
###
About BrightBytes: From complex integrations to actionable analysis, BrightBytes is the leading end-to-end data solution for education organizations. The BrightBytes platforms efficiently unify education data and combine top research and advanced analytics, delivered across easy-to-understand, intuitive dashboards, so educators can turn big data into big benefits for students.
Media Contact:
Tracy Kleine
P. 415.855.5000
Email: tracy@brightbytes.net
Follow BrightBytes
Twitter: https://twitter.com/BrightBytes
LinkedIn: https://www.linkedin.com/company/brightbytes
###
