Measuring the echo: how we score synthetic answers against real ones
What does 85–93% accuracy actually mean? How we compare synthetic and real interviews, and why we score against the way real people disagree.
Read the note
Science · Method · Validation
The science behind the echo.
How we build synthetic users that hold up against real ones. Methods, benchmarks, and the ongoing research that makes Echo trustworthy for high-stakes decisions.
Echo Accuracy Score
How we measure truth.
The Echo Accuracy Score compares synthetic interview responses to real human interview data, normalised against natural human-to-human variance to eliminate over-fitting bias.
85–93%
Echo Accuracy Score
How closely Echo matches real customer interviews at study level, calibrated against how much real people differ from each other.
Truth layer
70% weight
Are Echo's synthetic respondents saying the same things as your real customers, feeling the same way about them, and staying in character throughout? The Truth layer checks whether the conclusions you'd act on match the ones a real interview would surface.
Richness layer
30% weight
Real customer groups are messy: some answer in a sentence, some in a paragraph; some love a concept, some hate it. The Richness layer makes sure Echo keeps that natural variety instead of collapsing into polished, averaged-out answers.
HH-baseline
Normalisation
Two real people rarely give identical answers either. We calibrate the final score against how much real customers naturally disagree with each other, so the number reflects honest agreement, not artificial perfection.
Research notes
From the lab.
How Echo works, how we validate it, and what we're learning along the way. Written by our team for people who like to look under the hood.
Start here
Three pieces to understand Echo.
How accurate it is, how it learns your customers, and the science it stands on.
What data do you need to build accurate digital twins?
A one-page brief, your existing research or your customers' own interviews. What each level unlocks, and why less can be more.
Read the note
What the science says: the peer-reviewed research behind synthetic research
The peer-reviewed studies showing language models can simulate human attitudes and behaviour, where the limits are, and how Echo responds.
Read the noteThe library
Research at the speed of decisions, not the other way around
Traditional research takes weeks to set up and can't change once it lands. Why your customers shouldn't be this hard to reach.
Read the note
How we create accurate digital twins
Your data, public data and our research data, retrieved fresh for every twin. How grounding keeps answers accurate and diverse.
Read the note
Why not just prompt ChatGPT or Claude?
A chatbot gives you one confident answer you can't check. The dartboard problem, and four things a prompt can't give you.
Read the note
What synthetic research can't do (yet)
Six limits from the scientific literature, what we do about each one, and where synthetic research shines.
Read the note
Is 100% accuracy the goal?
Echo scores 85–93%. Why not 100%? And would you even want it? On the gap between AI and humans, and the ideas that live in it.
Read the note
Qual vs quant: why we started with the why
Most synthetic research produces numbers. We built Echo for the long, open answers of a real interview. Here's why.
Read the note
Get started
See what your customers would tell you this week.
30 minutes. Bring one decision you're facing. We'll show you how Echo would research it, and leave you with a scoped audience blueprint.
Explore my optionsNo commitment. Bring your scepticism, we'll bring the evidence.
Not ready for a call? Try Echo free on your own question →