By July 2024, the question of where the next generation of language models would get its AI training data had stopped being theoretical. Public internet text was already saturating, and the projected supply of high-quality human-written text had been overtaken by demand from frontier labs. Synthetic AI training data, meaning text generated by other large language models, was being floated as the obvious answer.

A paper published that month on arXiv, “Regurgitative Training: The Value of Real Data in Training Large Language Models” by Zhang, Qiao, Yang and Wei, set out to test whether synthetic AI training data actually works. The findings are sobering for anyone betting their roadmap on synthetic shortcuts, and they have particular weight for an emerging market like Vietnam. (Related: see our take in Blitzscaling in a world of AI.)

The experiment, plainly

The authors ran two parallel tests of AI training data quality. They fine-tuned GPT-3.5 on a machine translation task using either real human translations or text generated by other LLMs. Then they trained transformer models from scratch under the same conditions. In both cases, models trained on machine-generated AI training data performed worse than those trained on human data.

That alone is not surprising. What matters is the size of the gap and what is driving it. The authors point to two mechanisms in synthetic AI training data. The first is straightforward: LLM-generated training data carries a higher error rate than the human equivalent. The second is more interesting: LLM output has lower lexical diversity. Machine-generated text repeats itself, in a sense, more than humans do. Train on that narrow distribution and you inherit it.

The team did not stop at diagnosis. They tested three remedies. They built a quality metric and fed high-quality machine data to the model first. They mixed outputs from several different LLMs to widen the lexical range. They trained a classifier to detect which synthetic samples looked most human, and ordered training accordingly. Each method helped. None of them fully closed the gap. The paper’s conclusion is blunt: real, human-generated AI training data “cannot be easily substituted by synthetic, LLM-generated data.”

Why this matters more for Vietnamese AI than for English

In English, the supply of authentic human text is enormous. A team building a frontier model can paper over a lot of synthetic AI training data weakness through sheer volume. In Vietnamese, that volume does not exist. Indexed Vietnamese web text is a small fraction of indexed English text, much of it duplicated, machine-translated, or auto-generated. The base distribution any Vietnamese-focused model is drawing from is already narrow. We touched on the broader Vietnam data scarcity question in From Raw Data to Strategic Advantage.

If you accept the result, the implication for Vietnamese AI work is uncomfortable. Synthetic-data shortcuts compound an existing diversity problem rather than solve it. Every additional generation of model output added to a training set narrows the distribution a little more. The regurgitation effect, mild in English, is sharper in a small-language setting.

The same logic applies to fine-tuning and alignment. RLHF preferences scraped from machine outputs reflect machine preferences. Hallucination corrections drafted by an LLM tend to repeat the LLM’s blind spots. Supervised fine-tuning examples auto-written by a model often look fluent and miss the cultural register a human annotator would catch immediately. The error compounds across each pass of low-quality AI training data.

There is a concrete texture to this for Vietnamese specifically. The language has six tones, regional dialects with measurably different lexicons across the North, Central and South, code-switching with English and Chinese loanwords in technical domains, and a writing system that is mostly diacritic-stripped on social platforms. Each of those features carries real information that a current LLM does not reliably reproduce. A synthetic training corpus written by a model that already smooths these features will train the next model to smooth them further.

What is actually being built in Vietnam’s AI training data sector

Domestic infrastructure for human-generated AI training data is beginning to appear. Crowdsourcing platforms, university partnerships, and small specialist agencies are starting to compete for a market that did not meaningfully exist five years ago. Why these communities form and how they sustain output is a question we explored separately in Open Innovation in the Age of Data.

One example is QuestLab, a Vietnam-built crowdsourcing platform that describes itself on its homepage as “Vietnam’s leading community data platform” and reports a network of more than 50,000 verified contributors. Its public catalogue covers the categories that map directly to the gaps Zhang et al. identify: RLHF preference data and reward modelling, hallucination audits with fact-checking against trusted sources, supervised fine-tuning instruction datasets, image and video annotation, OCR for handwritten and structured Vietnamese documents, multi-region Vietnamese speech recording, and content moderation. It also runs market research and field operations such as retail audits and mystery shopping, which is a useful reminder that human-data infrastructure rarely scales on AI demand alone.

What is worth flagging objectively is that platforms like this operate at scale only because they sit on top of distributed contributor networks. A 50,000-person contributor base is not a marketing number for an AI team. It is a practical constraint on how many parallel labelling lines a project can run, how many dialects a speech dataset can cover, and how quickly a hallucination audit can be turned around. The same is true of regional rivals in Southeast Asia.

QuestLab is one of several players. The point is not that any single vendor solves the AI training data problem the paper raises, but that the paper has changed the cost calculation for buyers. If a Vietnamese AI team budgets purely for compute and synthetic pipelines and treats human-data spend as discretionary, the evidence suggests that team is structurally disadvantaged against one that does not. Procurement teams running these contracts can borrow discipline from the supplier verification approach we outlined in Vietnamese supplier verification in 2026.

The harder question

The three mitigation strategies the authors test deserve a closer look, because they are the strategies most enterprise AI teams reach for when human-data budgets get cut. Order high-quality synthetic data first. Mix outputs from multiple model families. Filter by human-likeness. All three are reasonable. All three help. None of them are full substitutes. The most plausible reading of the paper is that synthetic AI training data is a useful complement to real data, never a replacement.

That framing is not new. It echoes findings from Shumailov et al. (Nature, 2024) on “model collapse” under recursive training, and earlier work on data diversity in machine translation. What Zhang et al. add is a clean, controlled experiment on a model class people actually deploy, and a clear statement that the published mitigations for synthetic AI training data are not yet enough.

Three proven costs Vietnamese AI teams should price in

Tying the paper back to procurement reality, three concrete costs follow from over-reliance on synthetic AI training data, and Vietnamese teams should price each of them in.

Cost one: accuracy decay on downstream metrics. Higher error rates in synthetic AI training data flow straight into model outputs. For applications where accuracy matters (legal, medical, financial), that decay is a direct cost in correction, refunds, and trust loss.

Cost two: lexical and cultural blandness. The diversity collapse the authors document means Vietnamese-language models trained on heavily synthetic AI training data will sound flatter, miss regional nuance, and underperform on long-tail vocabulary. The cost shows up in user retention and brand perception.

Cost three: compounding model debt. Each retraining pass on cheaper synthetic AI training data narrows the distribution further. The cost is invisible quarter to quarter but accumulates as a real ceiling on what the next model generation can do.

What to watch over the next twelve months

Three things are worth tracking. First, whether the published mitigations for synthetic AI training data improve. There is a plausible technical path to closing more of the gap, through better quality metrics, better detection classifiers, and ensembling at scale, and any progress in those directions lowers the premium on human data. Second, whether Vietnamese-language data buyers start contracting differently. A shift from per-task pricing to retainer-based contributor-network access would be one signal that the buying side is internalising the result. Third, whether regulators in Hanoi start treating training data provenance as a compliance question. They have not yet.

For now, the practical takeaway is the unromantic one. Real human AI training data is expensive for a reason, and the cost of skipping it is no longer hypothetical. For Vietnamese teams in particular, building on a base distribution that is small to begin with, the case for investing in domestic human-data infrastructure is harder to argue against than it was a year ago.

References

Zhang, J., Qiao, D., Yang, M., & Wei, Q. (2024). Regurgitative Training: The Value of Real Data in Training Large Language Models. arXiv:2407.12835. https://arxiv.org/abs/2407.12835

Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature.

QuestLab. (Accessed May 2026). QuestLab: Vietnam’s Leading Community Data Platform. https://questlab.vn