UNIT 3
Introduction
For EdTech solutions to be implemented effectively in K-12 education, evidence of their impact is critical. Different regions around the world have varying standards for evaluating the efficacy of educational technologies. This unit will provide an overview of evidence standards in various regions to illustrate the variation as well as provide models for possible consideration in other regions.
Learning objectives
Understand the evidence standards for EdTech evaluation in various regions, including the U.S. (ESSA framework), Latin America, Africa, Asia, and Europe.
Analyze how different regions adapt evidence-based approaches to align with their unique educational, economic, and social contexts.
Explore tools and frameworks like EdTech Tulna and global initiatives that assist in evaluating and implementing high-quality EdTech solutions.
Recognize the importance of adhering to rigorous evidence standards to ensure EdTech solutions are effective, scalable, and capable of delivering measurable educational outcomes.
Evidence Standards in the USA
The Every Student Succeeds Act (ESSA), enacted in 2015, plays a pivotal role in U.S. education policy, mandating that federal funding for education programs be directed toward interventions backed by evidence. ESSA classifies evidence into four tiers, with each tier indicating the level of rigour and research supporting an educational tool or program.
ESSA Evidence Tiers:
-
Requires at least one well-designed and well-implemented randomized controlled trial (RCT) that demonstrates a significant positive effect on student outcomes.
-
Requires at least one well-designed and well-implemented quasi-experimental study (QED), which compares intervention groups without full randomization.
-
Requires at least one well-designed and well-implemented correlational study that shows a meaningful relationship between the intervention and student outcomes.
-
The intervention is based on sound research principles and logic models, and efforts are underway to gather more rigorous evidence.
Why ESSA Matters for EdTech
Schools and districts in the U.S. prioritize EdTech tools that meet these evidence standards, as ESSA compliance is tied to federal funding.
ESSA provides a roadmap for EdTech developers, encouraging them to engage in rigorous research that demonstrates their products' effectiveness in improving student outcomes.
Office for EdTech (Department for Education, USA):
The Office for EdTech provides guidelines and tools for schools, districts, and developers to ensure that EdTech products meet ESSA standards. It offers resources such as:
Best practices for implementing and scaling EdTech.
Tools for conducting RCTs, quasi-experiments, and correlational studies.
Guidance for incorporating ESSA-aligned evidence into product development.
Evidence Standards in LATAM, Africa, Asia, and Europe
While the U.S. has established clear evidence standards through ESSA, other regions are also developing frameworks to ensure that EdTech solutions are supported by research-driven evidence. Each region presents unique challenges and opportunities for EdTech evaluation, but there is growing alignment on the importance of evidence-based approaches.
LATAM (Latin America):
Growing focus on impact evaluation: In Latin American countries such as Chile, Brazil, and Mexico, there is increasing recognition of the need for robust evaluations of EdTech tools. Initiatives like the Inter-American Development Bank’s focus on education technology encourage rigorous assessments of digital learning tools.
Evidence-based EdTech programs: Programs funded by international organizations require evaluation using methodologies similar to those in the U.S., particularly quasi-experimental designs that measure educational outcomes in diverse socioeconomic settings.
Africa:
Emerging focus on data-driven EdTech: Africa's growing interest in digital education, coupled with limited resources, makes impact measurement essential. Initiatives such as the EdTech Hub and Africa Educational Trust emphasize the need for EdTech solutions backed by evidence of effectiveness.
Localized frameworks: Many African countries rely on partnerships with non-profits and international donors, which often require adherence to global evidence standards, such as RCTs and QEDs, to ensure program success and scalability.
Diverse standards across countries: In countries like China, India, and Singapore, EdTech evidence standards vary, but the emphasis on impact is consistent. Governments and educational agencies are increasingly requiring that digital learning tools undergo evidence-based evaluations, particularly as part of pilot programs in public schools.
Innovative funding models: Many Asian countries leverage public-private partnerships (PPPs) to promote EdTech solutions. Funders often demand evidence of impact based on structured research studies before scaling these solutions.
Asia:
Europe:
Rigorous impact evaluations: Europe has a strong tradition of impact evaluation through academic partnerships. The European Commission funds numerous educational initiatives that require stringent evidence standards, including RCTs and long-term studies. EdTech products must demonstrate efficacy through clear, research-backed outcomes before receiving public or private funding.
Innovative collaborations: European countries often foster collaborations between EdTech companies and academic institutions, encouraging continuous research and refinement of educational tools. The European EdTech Alliance and similar initiatives promote evidence-driven approaches to digital learning.
CASE STUDIES
EdTech Hub has developed several tools and frameworks to help decision makers generate and apply evidence. To begin with, the Hub encourages those considering the use of EdTech to ask five key questions to ensure they are making evidence-based decisions:
Read more here to drill into the five questions. EdTech Hub has also developed and supported partners to use its 6Ps framework. This is a tool for those who are designing and implementing EdTech interventions to explicitly plan for all elements of the education system: people, product, pedagogy, policy, place, and provision.
Here you will find the “6Ps Audit Tool” to help you assess how well you have planned for each of the Ps. Finally, you can understand the evidence specifically related to EdTech for teachers by using EdTech Hub’s guidance on tools for teacher professional development. EdTech Hub breaks down EdTech tools for teachers into three categories: Tools to develop teaching knowledge and skills, Tools for lesson planning, and Tools for communication and collaboration. Here you will find evidence-based good practices for each of these types of tools, as well as examples of each.
Edtech Tulna is an evidence-backed product evaluation index that establishes criteria for good quality EdTech. The Tulna standards evaluate EdTech products on the dimensions of Content Quality, Pedagogical Alignment and Technology and Design. It fills a critical gap in the ecosystem by creating a common definition for good quality EdTech by publishing standards and frameworks for different types of EdTech solutions / use cases (Eg. Personalised Adaptive Learning, Digital Classrooms etc). EdTech Tulna also has toolkits to enable decision makers to use Tulna frameworks to conduct product evaluations. Tulna also offers training to build capacity to conduct these evaluations with large scale procurement bodies such as state governments, large school chains etc. EdTech Tulna also conducts independent expert evaluations of existing EdTech products against Tulna’s frameworks, and publishes reports available as public goods.
Access the Tulna criteria here
Access the catalogue of product evaluations here
Conclusion
Understanding and adhering to evidence standards is key to the successful implementation and scaling of EdTech solutions globally. While ESSA provides a well-established framework in the U.S., other regions are developing their own standards to ensure that EdTech tools deliver measurable educational outcomes. Policy-makers must stay informed of these evolving standards and prioritize the adoption of research-driven, evidence-based solutions that can improve learning outcomes for K-12 students across the world.
