To Make a Difference with Learning Technology, Focus on How It Will Be Used

Take-aways from a months-long conversation with learning engineers, developers, entreprenuers, and educators.

GUEST COLUMN | by Ryan S. Baker

This past year, I spent six months virtually meeting, speaking with, brainstorming alongside, and swapping notes with 30 of the leading thinkers in the field of Learning Engineering, and I learned something surprising. Right now, the field puts a lot of effort into engineering the content and learner experience, but nowhere near as much thought to designing how educators will use their learning systems in real-life situations. And that needs to change if the field is ever going to fully realize its potential to improve outcomes for students.

Currently, dozens of interactive learning systems enter the educational marketplace each year, from academia, from startups, and from established players in edtech. Kids sometimes use these systems in class, and sometimes use them for homework. A few of the systems make a positive impact; a lot of the systems fade out of sight. Disappointingly many of them scale to a large number of classrooms, but don’t make a positive impact on learning.

‘…by studying how teachers use our systems, re-designing them to support best practices, and upgrading teacher professional development to scale these best practices, we can improve the implementation and use of educational technology…’

That’s because developers all too often neglect to think about, or plan for, how teachers should use their learning systems in classrooms. Imagine if a new medicine was released into the market, and every doctor had to figure out for themselves what dosage to give, and what other medicines might cause side effects. When a learning system is simply released into schools, without research-based guidance on how to use it, that’s essentially what happens.

Answering Questions for New Learning Systems

Instead, research and engineering work is needed to answer questions for new learning systems. Questions like: How much should it be used? What should it be used for? How should teachers interact with students as they use it? What data should teachers be given, and in what form? How should teachers use the data in their practice?

Ideally, every system should collect data (with appropriate privacy safeguards!) both on what the student is doing (common) and what the teacher is doing (not so common). This data shouldn’t be used to monitor teachers; it should be used to find out what teaching practices work with the technology (in many cases better than what the system developers initially envisioned!), and what practices don’t work. Then developers and researchers can study why some practices are more effective, and they can re-engineer systems to make effective practices easier to implement and unproductive practices harder to use. They can also take what they’ve learned and upgrade teacher professional development.

Examples of Successful Work and Opportunities

There are examples of successful work to engineer implementation in K-12 learning systems–the use of student homework data to drive classroom conversations in the ASSISTments system, steps to understand teacher choices around curricular sequencing in the MATHia system, the use of proactive remediation strategies by teachers in Reasoning Mind classrooms, just to give a few examples. However, this work remains rare.

This idea that the field needs to put more effort into designing how its products are used was one of 10 high-leverage opportunities we took away from our months-long conversation with learning engineers, developers, entrepreneurs and educators. We have written up our findings in a new report, “High-Leverage Opportunities for Learning Engineering,” which you can read here. The opportunities we identified present themselves in areas like designing algorithms and learning systems for diversity and equity, and enhancing R&D infrastructure in widely-deployed learning systems so that researchers can more easily build on each others’ findings.

With Billions into Learning, Now Is the Time to Act

All 10 of the high-leverage opportunities we discuss in the report are worthy of pursuit. But it’s this idea of engineering learning systems for implementation in real schools with real teachers in mind that strikes us as the biggest missed opportunity. Now is the time to act.

In the coming months, millions of dollars–perhaps even billions–will go into learning systems because of concerns around learning loss during the COVID-19 pandemic and the American Rescue Plan Act of 2021. Unfortunately, based on the current research evidence, a lot of this funding will go into systems that aren’t working that well for students.

If even a small fraction of that funding could be diverted to improving the engineering of implementation, the field of learning engineering could make huge strides toward improving student learning outcomes.

Specifically, by studying how teachers use our systems, re-designing them to support best practices, and upgrading teacher professional development to scale these best practices, we can improve the implementation and use of educational technology, produce better learning for students now, and put students on a better trajectory towards success.

Ryan S. Baker is Associate Professor and Director of the University of Pennsylvania Center for Learning Analytics. Connect with Ryan here.


    Leave a Comment

    %d bloggers like this: