Is it safe to use synthetic personas? What we learned running the same study twice: consumers vs. digital twins

AI models have gotten good enough to "pretend" to be people and answer surveys as if they were real consumers. That has lit up a strong promise for anyone working in market research: recruiting 1,000 respondents takes time and costs money; it's faster and "cheaper" to create agents that mimic humans and get answers within a day.

Well, that's the promise. What we did here at Okiar was to take a real study (on banking primacy and financial behavior) as a baseline and run the same survey testing different approaches with synthetic consumers/personas. Our questions were: will the data match? Will the averages be the same? And the distributions? Where is there more or less error? Are some approaches safer than others?

Okiar had already produced an extensive literature review on the subject and consolidated its own synthetic-persona framework (largely based on the replication of Maier et al., 2025 - LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings), and we know the pros and cons of different approaches such as DLR, FLR, and SSR. But the question now is a practical one: can you actually use this?

The short answer: they match. The not-so-short answer: some data matches more, some less, and there is always background noise in the measurements as the metrics relate more to individual experiences than to common sense (reinforcing a social status quo rather than surfacing what actually goes on in people's heads).

But let's look at this more closely.

First of all: what is a synthetic persona?

Think of a very well-prepared actress. You hand her a script and a character sheet ("woman, 34, lower-middle class, uses Nubank day to day but keeps her money at Itaú, mother of 3, lives in the Bom Retiro neighborhood") and ask that actress to answer a questionnaire as if she were that person. That is a synthetic persona: an AI/LLM takes on a role and answers as if it were a real consumer.

You can create around 8 characters representing slices of society and ask each of them to answer dozens of questionnaires, or you can create characters that each represent 1 synthetic consumer at a time. It's a trade-off between a synthetic persona mimicking a territory (an age range, an income range) or mimicking a single consumer unit (a specific age, a specific income). One will be more accurate and the other less; one costs more tokens, takes more work, and requires a denser architecture, the other less. (P.S.: we tested this too - is it worth investing 500x more tokens? What's the gain?)

Is a synthetic persona the same as a digital twin?

No. A synthetic persona represents a PERSONA: it's the "territory" side of the trade-off we described. The digital twin is the other side, representing an INDIVIDUALITY. One is a range, the other is a specificity.

The thing is, to build a good digital twin you need to ensure veracity (it needs a connection to reality), micro representativeness (you need to make sure different individualities are captured), and macro representativeness (the final distribution has to line up with real demographics), as well as other criteria such as reliability, parsimony, diversity, and a good F statistic (in more statistical terms) - which is to say that the difference between larger groups has to be greater than the sum of individual differences: whoever is inside a group must be more similar to others inside that group than to those outside it.

How the AI answers matters MUCH more than who it pretends to be

There are 2 independent decisions in any synthetic-research project. The first is (a) how you build your persona or digital twin; the second is (b) how you elicit an answer from it (quantitative or even qualitative), whether that's a response to a script or a Likert rating (as in the study by Maier and colleagues, 2025).

Our experiment: averages match, distributions not always

Brand-image attributes and intentions are replicated better than memory and individual-experience questions.

Every metric has 2 sides, like a coin: on one side the average (how much customers think the "app is easy," for example) and on the other the distribution (25% think the app is terrible, 75% love it).

Synthetic personas (when very well built - considering our best benchmark after empirically testing a good dozen combinations of techniques and approaches) hit the average with up to 81% similarity. But the distribution is another story: in one specific approach (and note: one of the most widely used) it captures only 1/3 of the data's variance, reaching up to 58% distribution similarity in the best (and honest) internal benchmark.

Synthetic data captures the "outline" of opinion well, but the crowd of clones makes the real variety disappear.

And note this isn't really a problem of "well, LLM models are getting better, so these assumptions will improve." Because the issue is how these models are built: grounded in common sense, in homo economicus by concept. It's as if, in real life, each human being - among billions - were their own LLM model trained on their personal, individual experiences (what the literature calls ontogenetic and cultural learning), whereas the LLMs we use are trained on an aggregate of human experiences, relying on cultural learning alone. Without feeling, emotion, and all those components that make you and me so special and human - and that deliver the variability of responses we see in a real survey rather than in an AI survey.

Maybe, if a survey with 1,000 digital twins gave each of them, as individual context, decades of transcribed history recorded in these AI agents' scripts (just as it is for us humans, who carry our context everywhere we go), then perhaps those enormous contexts would answer "close" to a human being, hitting not only the average but the distribution.

The question remains: would it be worth it? Sometimes the cheapest path is to talk to a real person.

In practice: synthetic personas mirror human reason well, but they still have a long journey to learn to replicate emotion and individual experience. And even if they could, the cost would be higher than running a good study with real humans.

Can you trust a study with synthetic personas?

It depends on the decision you're going to make with the data. If you want direction, ranking, macro trends, quick product-appeal tests, contexts where the average score is a good guide - yes, with those caveats.

Now, if your decision needs those individual differences and the distribution - such as reviewing CX planning, positioning and territory strategy, defining communication campaigns - using synthetic data can be risky. Risky mainly because the average score can give you false comfort.

And, as we noted at the start, there are some metrics you simply can't trust. Memory is one of them. In no benchmark did we get reliable brand-recall data: brands like BTG, which has ~40% awareness in our measurements, jumped to ~90% with synthetic consumers. Meanwhile Nubank, Itaú, Santander, Bradesco, and the like showed 100% (an extremely rare figure in brand studies, as anyone who researches this knows).

Another critical data point was the primacy measurement itself ("which is your main bank?"), where Nubank - already the leader in our study, with ~20% primacy - jumped to ~70% with synthetic personas.

But the best benchmark did far better on brand-image metrics, intent to open an account, and the like. In short: more stable dispositions tend to always be replicated with greater confidence.

How do I make sure my synthetic-persona project delivers the best possible insights?

Metrics like satisfaction and loyalty are hard to capture well with synthetic personas. Because when you answer about them, you're answering according to something that HAPPENED to you - and the synthetic persona never stood in line at a bank, never got a scare using the app at 11 p.m. while trying to send a Pix to their own grandmother and had the account temporarily blocked, and so on. When one of these personas answers, it isn't REMEMBERING anything; it's answering what someone would PROBABLY and TYPICALLY say. That's the crux of it.

And even though these conceptual limitations exist, we can also say that, more than ever, a good quantitative questionnaire with validated scales and a good flow, a good script (with laddering, content structure, etc.) have never been so important.

That's because, in research, what exist are SOURCES OF ERROR. And a good questionnaire and a good script reduce those sources rather than increase them - with a real consumer or a synthetic one.

At the end of the day: there are many ways to do synthetic research, few of them get it right

A synthetic consumer does not replace research with real people. It should be used sparingly, just as you shouldn't eat fast food every day.

It's a good compass, it orders some hypotheses well, and it belongs in the same place as qualitative research. It doesn't confirm what happens in the real world; it surfaces possible and probable scenarios (and not with the same richness as a good qual, of course).

As a ruler, as a measurement, it fails. There's no way around it. So if you're going to use it, do it with a reliable method for creating these personas, with an insights culture that knows how to read this data, with good research instruments attached, and with critical thinking (always welcome in any methodology).

Takeaways

• Pre-test with synthetic, but validate with humans whenever possible.
• Focus on having good questionnaires and scripts, regardless of whether the respondent is real or an AI.
• Use the averages with care; use the distributions with even more care. In this experiment we had the counter-proof - will you have that in your project?
• In brand research, a synthetic persona kills the very concept of a brand tracker: we don't recommend using it. The same goes for satisfaction research.
• The best use case in the literature is testing concepts and taking general measures of habits and attitudes - here it works relatively well.
• A synthetic persona performs almost as well as a digital twin (with a marginal gain in similarity but a substantial increase in cost).
• Never stop talking to your real consumer. Research isn't expensive; what's expensive is failing to listen to the people who pay your business's bills.