The past year has demonstrated edtech’s capacity for providing innovative solutions that have the potential to help children learn under adverse circumstances. Given the increased reliance on technology to support learning, now may be a good time for edtech companies to start thinking more seriously about the data they collect and how they use it.
When I was a teacher 10 years ago, there was little conversation about evidence-based practice or programmes. Recently though, organisations like the Education Endowment Foundation, ResearchED and the Edtech Evidence Group have helped drive a real change in educators’ conversations about evidence and research gathering.
Changing the conversation
Schools are increasingly asking such questions as: how good is the evidence? Did you use a randomised controlled trial? Where’s the data coming from and was it independently evaluated? The conversation is certainly changing, and schools, as buyers of edtech, are driving the research and evidence agenda forward. Yet, edtech organisations don’t seem to be meeting this demand.
This may not be intentional; sometimes, in the push to get a product into schools, evidence-generation becomes an afterthought. However, edtech companies are uniquely positioned to add to their own evidence base as they already collect data for their programmes (i.e. time stamps, programme completion rates, success metrics). This is very different from other types of programmes and practices that are not able to collect data as systematically. A programme that aims to improve reading by providing more books to schools, for example, does not have an easy way of collecting data on how many children read their books, or how often, or how children progress.
Generating evidence is one thing, but data can reveal much more than whether or not a programme or practice works, and if its use in or out of the classroom aligns with the original intention of the developers. It allows developers to adjust programmes so they become more aligned with school systems and routines. Developers can also use the information to find ways of making programmes or practices more effective by identifying weaker elements.
RAND Europe led a recent evaluation for the learning technology company, Sparx, that provides a perfect example of the ways in which routine data gathered from edtech companies can be used to build evidence and generate useful findings for developers. We not only measured progress using start and end points, we also collected the type of data to allow us to examine the actual ‘process’ of using the platform. This meant we were able to tell that students actively using the platform made more progress, also providing insight about how students used and interacted with the different elements in the platform. In addition, we were able to find a positive correlation between learners using the company’s maths platform and an improvement in predicted GCSE grades. This is a valuable finding for evidence-hungry schools.
Encouraging a culture of ‘openness’
A greater openness in sharing results of such research may be beneficial for all edtech companies. There’s an increasing culture among edtech companies to be part of openly published findings. Sparx committed to publishing the research regardless of the results, while Third Space Learning agreed to have the findings from a large-scale trial publicly available. It also appears that this openness is valued from outside the edtech space. Third Space Learning became one of the key partners of England’s Department for Education’s National Tutoring Programme, which formed part of their £1 billion programme to support student progress, hindered by school closures due to COVID-19.
Edtech companies collect useful data, but they seem to be unaware of the value of the routine data they collect. This is a lost opportunity as edtech companies are in a unique position to generate the evidence to demonstrate their effectiveness and improve their own development.
You might also like: Why edtech is key to empowering financial wellbeing
Advertisement / Google
Advertisement / Google
The importance of gathering evidence of edtech impact
Elena Rosa Brown
The past year has demonstrated edtech’s capacity for providing innovative solutions that have the potential to help children learn under adverse circumstances. Given the increased reliance on technology to support learning, now may be a good time for edtech companies to start thinking more seriously about the data they collect and how they use it.
When I was a teacher 10 years ago, there was little conversation about evidence-based practice or programmes. Recently though, organisations like the Education Endowment Foundation, ResearchED and the Edtech Evidence Group have helped drive a real change in educators’ conversations about evidence and research gathering.
Changing the conversation
Schools are increasingly asking such questions as: how good is the evidence? Did you use a randomised controlled trial? Where’s the data coming from and was it independently evaluated? The conversation is certainly changing, and schools, as buyers of edtech, are driving the research and evidence agenda forward. Yet, edtech organisations don’t seem to be meeting this demand.
This may not be intentional; sometimes, in the push to get a product into schools, evidence-generation becomes an afterthought. However, edtech companies are uniquely positioned to add to their own evidence base as they already collect data for their programmes (i.e. time stamps, programme completion rates, success metrics). This is very different from other types of programmes and practices that are not able to collect data as systematically. A programme that aims to improve reading by providing more books to schools, for example, does not have an easy way of collecting data on how many children read their books, or how often, or how children progress.
Generating evidence is one thing, but data can reveal much more than whether or not a programme or practice works, and if its use in or out of the classroom aligns with the original intention of the developers. It allows developers to adjust programmes so they become more aligned with school systems and routines. Developers can also use the information to find ways of making programmes or practices more effective by identifying weaker elements.
RAND Europe led a recent evaluation for the learning technology company, Sparx, that provides a perfect example of the ways in which routine data gathered from edtech companies can be used to build evidence and generate useful findings for developers. We not only measured progress using start and end points, we also collected the type of data to allow us to examine the actual ‘process’ of using the platform. This meant we were able to tell that students actively using the platform made more progress, also providing insight about how students used and interacted with the different elements in the platform. In addition, we were able to find a positive correlation between learners using the company’s maths platform and an improvement in predicted GCSE grades. This is a valuable finding for evidence-hungry schools.
Encouraging a culture of ‘openness’
A greater openness in sharing results of such research may be beneficial for all edtech companies. There’s an increasing culture among edtech companies to be part of openly published findings. Sparx committed to publishing the research regardless of the results, while Third Space Learning agreed to have the findings from a large-scale trial publicly available. It also appears that this openness is valued from outside the edtech space. Third Space Learning became one of the key partners of England’s Department for Education’s National Tutoring Programme, which formed part of their £1 billion programme to support student progress, hindered by school closures due to COVID-19.
Edtech companies collect useful data, but they seem to be unaware of the value of the routine data they collect. This is a lost opportunity as edtech companies are in a unique position to generate the evidence to demonstrate their effectiveness and improve their own development.
You might also like: Why edtech is key to empowering financial wellbeing
Advertisement / Google
Digital innovation security hub set to boost invention in Manchester
Lancaster and Manchester universities are partnering with a consortium of cyber security specialists to establish…
Calling female STEM innovators: Submissions open for Margaret Awards 2022
JFD is calling for all women and girls in Europe and Africa who are making…
Natterhub wins ‘transformational impact’ award
79% of children feel remote learning has disrupted their education
Cybersecurity essentials for higher education
How the metaverse is transforming the edtech market