What Data Does AI Use to Generate Content?

The sentences generated by AI are not simply copied from a specific database, but are newly combined results based on patterns and statistical relationships extracted from previously learned texts. Let me explain this more systematically.

• The main sources include materials that are copyright-compliant or licensed in a de-identified form, such as books, newspapers, blogs, wikis, academic papers, and government reports.

• A wide range of topics is included, from science, technology, medicine, law, history, culture, to everyday conversations. This allows for responses to various questions, from "kimchi fermentation temperature" to "black hole information."

• Refinement process: Noise (typos, ads, duplicates, etc.) is filtered out, and personal information and copyright infringement elements are removed to meet quality and ethical standards.

How do the learning and reasoning methods differ?

• In the pre-training stage, the model learns language patterns by repeating the task of "predicting the next token (word piece)" billions of times.

• In the fine-tuning stage, additional data and human feedback are used to enhance accuracy, safety, and usefulness.

• In the generation stage, the model reads the input prompt and sequentially selects the most natural tokens based on probability distribution to create sentences. In other words, it synthesizes new narratives using "learned statistical rules" rather than simply reproducing phrases seen during training.

What does it mean that "the primary language is English"?

• Since English has the largest share of text on the internet, it is relatively more learned compared to other languages.

• Therefore, the expression of grammar, nuance, and technical terms is most finely tuned in English.

• However, recent models use a multilingual multi-link approach, allowing sufficient learning in dozens of languages, including Korean, Spanish, Portuguese, and Japanese. Thanks to the structure that can translate the same concept into multiple languages simultaneously, natural Korean responses are possible without translation.

• However, expressions with fewer usage examples, such as dialects, slang, and neologisms, may have more errors in responses in Korean than in English.

So how is accuracy ensured?

• As AI is a statistical language model, it still has limitations in "deep fact-checking ability" to determine authenticity.

• The latest models have learned rules such as citation and source presentation during the fine-tuning stage, but topics that change rapidly (law, medicine, current affairs, etc.) require verification of the latest materials.

• Users should consider AI responses as the first draft and it is safer to conduct additional verification for important decisions.

Summary

• AI learns from a wide variety of publicly available texts, such as books, web documents, and papers, acquiring patterns of language and knowledge in the process.

• Responses are not "copy-paste" but sentences newly generated in real-time based on learned statistics.

• While English proficiency is the highest due to the learning weight, thanks to multi-link and parallel corpora, it can naturally understand and generate several languages, including Korean.

• It is not a complete truth engine, so it is always recommended to cross-verify important information.