Why The Edtech Evidence Movement Needs an Open Data Repository

Bettering edtech and collective efforts toward effective data use.

GUEST COLUMN | by Natalia I. Kucirkova

LIZ ABLASHI

Recently, the biggest players in tech (including the key educational tech players Google and Microsoft) have admitted to be lacking in safety and data protection measures. Global technology data breaches have been known for some time: In the aftermath of the pandemic, external evaluations showed that many popular edtech breached Data Protection laws during online teaching. As the need for better edtech emerges as a clear lesson from 2022, our collective efforts need to focus not only on safe data use but also effective data use.

‘As the need for better edtech emerges as a clear lesson from 2022, our collective efforts need to focus not only on safe data use but also effective data use.’

Here is why it matters.

Fragmented, Uneven, Scattered

The current level of edtech’s data use is fragmented and uneven. Some companies use data for causal testing, others use teachers’ reports to improve their design. Others still use efficacy data to drive sales.

In many contexts, teachers act as researchers by conducting ongoing evaluations of edtech implementation. Their data are used by edtech to improve their products, but few transparently share the process of evaluation. Scattered data mean that teachers miss out on opportunities to improve their practice and designers their design. Ultimately, children miss out on learning gains.

Many edtech collaborate with teachers to produce usability studies and iteratively improve their design. However, without a systematic approach to testing, such studies produce low effect sizes with data difficult to interpret or reanalyze. A platform with free data to use and re-use would increase the transparency of research conducted by individual providers. Through aggregation and iteration, it would also increase the overall quality of research. That way, the edtech ecosystem could move to more transformative research.

Opening Up Edtech 

Researchers have been advocating for the need for more transformative edtech research for some time. National reports from England and Norway show that edtech research show that the EdTech space urgently needs more advanced data measures. With an Open Edtech Data Repository, researchers could run more robust studies and ask not just whether individual solutions work, but also in which contexts, under which conditions and for which children and teachers. This could improve the design of the many hundreds of apps classed as “educational” but lacking educational criteria.

With shared systematic and transparent measures, an Open Edtech Data Repository could spawn review systems to support edtech on their journey to various types of evidence. Some countries, notably USA, sets out standards of efficacy, with randomized controlled trials as the highest level of demonstrating evidence. In other countries, edtech’s evidence is understood as a combination of teachers’ reports and certifications. Some edtech work across countries while others test solutions only in local schools. If we are to truly understand what worked and what could work, edtech research teams need to be able to aggregate data. If all edtech openly shared all their data in one shared repository, they could learn from each other and optimize their design to local contexts and international trends.

The process to open data sharing should facilitate and not complicate edtech’s journey to evidence. University accelerator programs and edtech incubators could support companies who need to systematize their data collection. EU has developed and implemented best practice in open science and data sharing, which could be easily adapted for edtech purpose.

‘The process to open data sharing should facilitate and not complicate edtech’s journey to evidence.’

The European parliament has recently approved some key rules for the ways in which technologies, including, edtech, collect and use data. The next step is to agree rules for edtech to share their data in an International Open Data Repository. To bolster global evidence efforts, edtech companies need to adopt ways to optimize their data collection and reporting procedures for an open dataset. An international Open Data Repository could propel the edtech field with data-based evidence of positive impact on children’s learning.

—

Natalia Kucirkova is Professor of Early Childhood Education and Development at the University of Stavanger, Norway and Professor of Reading and Children’s Development at The Open University, UK. She is the founder of the university spin-out Wikit, AS, which integrates science with the children’s edtech industry. Connect with her on LinkedIn.