The Bottom Line
EMQN, a UKAS-accredited organisation providing external quality assessments for genetic testing laboratories worldwide, needed to transform their manual assessment marking processes. OpenKit's four-week discovery sprint delivered a comprehensive AI solution design with rigorous evidence-based validation.
Hours Saved Annually
Projected assessor time savings
AI Accuracy
On 14 of 17 marking criteria
Pages Delivered
Strategic and technical documentation
Project Summary
EMQN (European Molecular Quality Network) is a Manchester-based Community Interest Company that functions as an "exam board for laboratories." They provide External Quality Assessment schemes for laboratories conducting human genetic testing across the globe, processing approximately 20,000 laboratory reports annually across 50+ schemes in 6 languages.
The CEO, Simon Patton, approached OpenKit with a clear vision: use AI to assist their expert volunteer assessors, reducing the burden of repetitive clerical checking while improving consistency and reducing turnaround time. The solution needed to maintain 100% human oversight in this high-stakes healthcare environment.
The Challenge
EMQN's assessment process involves expert volunteer assessors reviewing laboratory diagnostic reports against standardised marking criteria. Each report is independently marked by at least two assessors, with discordant results discussed in moderation meetings before final results are published.
The clerical accuracy component of marking comprises 17 objective, rule-based criteria that assessors must manually verify for every report. Assessors explicitly described this as "the most boring and laborious part" of their work, despite it being the least scientifically complex component.
The Key Challenges
EMQN operates in a high-stakes environment where incorrect assessments can trigger regulatory investigations and prevent laboratories from offering tests. Their UKAS accreditation and ISO 27001 certification impose strict requirements on any technology solution, particularly around data sovereignty and audit trails.
Our Approach: Evidence-Based Discovery Sprint
OpenKit designed a comprehensive four-week discovery sprint to establish an evidence-based foundation for EMQN's AI investment decision. Rather than making assumptions about AI capabilities, we conducted rigorous testing with EMQN's actual data across multiple languages and criteria.
How We Delivered Value
Comprehensive AI Benchmarking
Evaluated 6 frontier AI models across 150 test scenarios in all 6 supported languages, identifying optimal model with 93-96% accuracy on 14 of 17 criteria.
EU-Sovereign Hosting Architecture
Analysed 7 cloud providers to identify solutions with full EU data sovereignty, ensuring compliance with UKAS and ISO requirements.
Human-in-the-Loop Design
Designed traffic light confidence system ensuring AI assists rather than replaces expert assessors, maintaining 100% human oversight.
Strategic Documentation
Delivered 88 pages of strategic and technical documentation enabling informed board investment decisions.
Integration Risk Mitigation
Created comprehensive API questionnaire and fallback approaches to de-risk third-party platform integration.
Discovery Sprint Methodology
Our discovery sprint followed a structured four-phase approach, ensuring comprehensive coverage of stakeholder needs, technical feasibility, and compliance requirements.
Stakeholder Engagement
Conducted 5 in-depth interviews with key personnel including CEO, Assessment Lead, Scientific Team, IT Head, and IT Project Manager to understand processes from multiple perspectives.
AI Model Benchmarking
Evaluated 6 frontier AI models across 150 test scenarios using 15 representative reports in all 6 supported languages, with 10 independent runs per model.
Hosting Analysis
Analysed 7 cloud providers for data sovereignty compliance, evaluating CLOUD Act exposure, GDPR compliance, and infrastructure requirements.
Documentation Delivery
Produced 88 pages across a 43-page Strategy Report and 45-page Technical Specification, plus UI mockups, API questionnaire, and development quote.
Key Findings
Our rigorous benchmarking and analysis revealed several critical insights that shaped the final recommendations:
Human-in-the-Loop Essential
While AI achieved 93-96% per-criterion accuracy, strict accuracy (all 17 criteria correct) reached only 45%. This validated the human oversight requirement for high-stakes healthcare decisions.
Licensing Restrictions Matter
Meta's Llama 4 models achieved marginally higher accuracy but cannot be deployed by EU-based organisations under current licensing terms, eliminating our top performer from consideration.
Multilingual Performance Strong
English and Spanish achieved 99-100% accuracy, with German, French, Italian, and Portuguese at 93-95%. Minor variations in compound technical terms and diacritical marks.
Continuous Verification Critical
EMQN's IT Head emphasised that one-time testing is insufficient for healthcare AI. The system must support ongoing verification against pre-assessed baselines.
Results and Impact
The discovery sprint delivered comprehensive deliverables enabling EMQN's board to make an informed investment decision:
Strategy Report
43 pages covering market analysis, process mapping, model evaluation, hosting assessment, success metrics, and implementation roadmap.
Technical Specification
45 pages detailing architecture, AI requirements, security framework, UI specifications, testing strategy, and deployment procedures.
Model Benchmarks
150 evaluations across 6 models with per-criterion accuracy analysis, multilingual performance data, and licensing assessment.
Hosting Analysis
7-provider comparison with sovereignty assessment, operational cost modelling, and self-hosted alternative analysis.
UI Mockups
Modern assessor dashboard design with traffic light confidence system, PDF viewer integration, and batch processing workflows.
API Questionnaire
Comprehensive integration requirements for third-party platform with fallback approaches to de-risk critical dependency.
Client Testimonial
"Working with OpenKit has been a genuinely positive experience. Their team quickly understood the unique challenges of our business and the problem we were trying to solve, and delivered a thorough, evidence-based strategy for our AI-assisted marking platform.
We were particularly impressed by their transparent approach, technical expertise, and commitment to long-term partnership and support. I would recommend OpenKit to any organisation seeking a reliable, innovative technology partner."
Simon Patton
CEO, EMQN CIC
Why This Matters for Your Business
EMQN's project demonstrates the value of rigorous, evidence-based AI discovery before committing to development. By investing in comprehensive stakeholder engagement, systematic model evaluation, and thorough compliance analysis, EMQN received a strategic foundation that enabled confident decision-making.
This approach is particularly valuable for regulated industries where AI implementation requires demonstrable accuracy, data sovereignty compliance, and human oversight. Rather than generic AI strategy, EMQN received specific benchmarks against their actual data, detailed technical architecture, and clear risk mitigation strategies.
Ready to explore AI opportunities for your organisation?
We work with organisations in regulated industries that need evidence-based AI strategy. If you need rigorous evaluation of AI capabilities against your specific requirements, let's explore what's possible for your business.
More Success Stories
Wakari: AI Strategy Clarity
How AI consulting provided strategic guidance for accessibility platform development
BAiSICS: 92% Faster Document Review
From AI strategy to full platform development and deployment
Rubrical: AI Assessment Tool
AI-powered geography assessment saving teachers hours per week
