A response to Olli Vallo’s article, ‘Is Efficacy Research Killing EdTech Innovation?’
GUEST COLUMN | by Karl Rectanus
Across the country, school districts are seeking solutions that modernize learning to drive student outcomes. Expectedly, district leaders require rigorous, practical evaluation that provides evidence at the speed of their decision-making when determining which solutions will work best. And in cities like Chicago and Los Angeles, officials are now requiring vendors, including education technology companies, to provide research showing their solution works.
In May—for the first time—Chicago Public Schools included a question in their standard request for information from vendors asking them to share their research showing their solution has an evidence base aligned to The Every Student Succeeds Act (ESSA), the federal law which includes requirements for districts and states to use federal funding and stimulus dollars for “evidence-based interventions.”
This spring, Los Angeles Unified School District requested its providers show their evidence, along with data on and analysis of their solutions’ impact across the district.
Clearly, the time has come for edtech companies to show that their solutions are effective.
Does Evidence Inhibit Innovation?
The recent piece, “Is Efficacy Research Killing EdTech Innovation?,” correctly addresses the previous barriers that made it difficult to incorporate evidence into scaling innovative edtech solutions. School constraints, the prohibitive cost of traditional research, and the previously slow and time-intensive nature of building an ongoing portfolio of evaluation to truly understand how well education technology solutions are working to improve outcomes for students—or when they work best, for whom, and under what conditions—was daunting.
But research in the modern age doesn’t have to be slow, expensive, and a barrier to innovation. The key is making research iterative, replicable, and ongoing, thus reducing the time and expense involved, and making it accessible to all solution providers and their partners, so they can innovate together.
‘…research in the modern age doesn’t have to be slow, expensive, and a barrier to innovation. The key is making research iterative, replicable, and ongoing…’
Innovation is always challenging. And high-stakes innovation – for example, when the expectation is a need to show effectiveness – is even more challenging, but that doesn’t make it impossible. By drastically lowering the time and cost restraints for securing quality research and focusing on the full spectrum of evidence referenced within ESSA, practical and rigorous evaluation can actually spur innovation and help scale the most effective solutions.
Practical, Rigorous Evidence Leads to Innovation
When a solution provider is continually evaluating what is working, being nimble and responsive, rapid-cycle evaluations can be incorporated into real-time findings to ensure customer expectations are being met. Quantitative and qualitative feedback from administrators, teachers and students can drive product roadmaps and ensure professional development offerings and customer support are impactful and driving innovation.
It’s Already Happening
As schools and districts modernize how they evaluate programs, so too have providers and their quest to build an evidence base for their products. Offerings, such as our own Evidence-as-a-Service, allow providers of all stages – early, growing and established – to align to ESSA-defined evidence tiers in a matter of weeks, at a fraction of the cost of traditional research. By creating a more integrated, seamless evaluation process that eliminates the time and cost barriers it’s easier for solution providers to work with districts and state partners to build evidence that continuously improves teaching and learning.
For example, global edtech innovator Age of Learning™, a company with a long track record of engaging in evaluation research, has conducted multiple randomized control trials on its newest solutions, My Math Academy ™, a patented, adaptive game-based math program for PreK- 2nd graders. This research was foundational to prove that the program accelerates learning and increases students’ confidence, interest, and motivation to learn math.
However, as states and districts now evaluate research against federal requirements, Age of Learning validated their existing research through Evidence-as-a-Service, as an additional proof point to certify that it met the highest level of evidence (Level I or “Strong Evidence”) under ESSA. Providing independent, third-party review that its research aligns with federal requirements makes it easier for educators to assess the quality of the company’s evidence. Age of Learning has multiple studies for its programs in flight now, and will earn additional ESSA level badges for each, ensuring educators that its programs are effective at producing student outcomes for a district’s unique context.
This approach also works for companies that are just starting their evidence-building journey as well. When Bamboo Learning began pilot testing its voice application programs in schools, they built in a logic model and research plan from the very beginning. Expanding their evidence base in just a few months proved influential in future contracts, one of which included a more rigorous study with a district which showed “Promising” results, thus meeting the requirements of ESSA Level III. Their research agenda continues with larger studies planned with more districts in the coming school year.
When solution providers conduct research and share results for the right reasons—to better support student learning and drive implementation and product improvement in partnership with their schools and districts–it’s a win-win for everyone.
How to Ensure Evidence Drives Innovation
Here are three best practices to ensure edtech effectiveness research drives innovation, regardless of your role, organization or size:
1. Think of evaluation as a continuous, iterative and ongoing process; not a one-time, check-the-box endeavor. Use one proof point as a starting point on a pathway to learning and improvement. All results are a guidepost to better results in the future.
2. Embrace rapid-cycle evaluation to build and grow evidence to inform decisions. Practical evidence produced over time will provide more value than a multi-million dollar decades-long longitudinal study. Think about the impact on students now, not just years down the road.
3. Normalize asking for and sharing evidence widely. Do not think of research as picking winners and losers, as a marketing proof point, or a judgment of success or failure. Ask for evidence. Ask to build more together. Discuss it with your partners and work together to drive implementation success and student growth.
Solution providers and their state and district partners can strike a balance between evidence and innovation to scale and share findings in a way that continuously improves teaching and learning, ultimately allowing all students to thrive. And the ones who do, will innovate more quickly, grow more quickly, and help more students and teachers succeed.
Karl Rectanus is an educator and high-growth serial entrepreneur committed to expanding equitable outcomes through systemic change. As the Co-founder and Chief Executive Officer of LearnPlatform, a for-benefit research organization, he leads a team of educators, researchers and technologists committed to expanding access for all students to the tools and teaching that work best for them.