Can a language model tell where a fact from its parametric knowledge came from?

Not reliably. Parametric knowledge is spread across a language model’s parameters and is not linked to the texts it came from; a 2020 research paper called it an open problem to show where such knowledge comes from. When asked for sources without retrieving any content, language models can even invent references that do not exist, as studies of literature references have shown.

Does parametric knowledge disappear when the original web page is deleted or changed?

No. What a language model learned from a web page during training stays in that model, even if the page is later changed or deleted; only a further-trained or new model version whose training data includes the new state can reflect it. Answers for which an AI system retrieves web content, by contrast, can reflect the current state of a website once the changed version has been captured.

Does an AI system still use its parametric knowledge when it retrieves content from the web?

Yes: even when an AI system retrieves web content, a language model writes the answer and combines the retrieved content with its parametric knowledge. In experiments published in 2023, language models often went along with a single coherent, convincing source even when it contradicted what they had learned. When they were given conflicting sources at the same time, they tended to favor the one that matched their stored knowledge.

Parametric knowledge: what a language model knows

Parametric knowledge is what a language model knows by heart, so to speak. Ask an AI system such as ChatGPT, Gemini, or Claude what a cordless drill is or what a well-known car brand stands for, and it will often answer without looking anything up: the answer comes from what its language model learned in training. Other names include parametric memory and model knowledge; it is also referred to more simply as trained knowledge.

Where the knowledge resides

Parametric knowledge forms during training. A large language model learns from vast amounts of text, its training data, and adjusts its parameters until it can continue text as accurately as possible. Parameters, often also called weights, are the numerical values the model consists of. Along the way, the parameters also come to hold factual knowledge, such as where a company is based, what a product does, or which providers operate in an industry. A 2019 research paper titled “Language Models as Knowledge Bases?” showed that language models can reproduce some factual knowledge without looking it up.

Even so, parametric knowledge is not an archive the model could look things up in. It is spread across the parameters and only surfaces when the model writes an answer piece by piece. After training, this knowledge stays the same until the provider trains the model further or releases a new model version. The knowledge cutoff marks how far in time it reaches.

Parametric and retrieved knowledge

The counterpart to parametric knowledge is content that an AI system retrieves only at the moment a question is asked, such as web pages. Researchers call this knowledge non-parametric because it lies outside the parameters. The difference resembles the one between a closed-book and an open-book exam. It shows mainly in three areas:

Origin: Retrieved content comes from specific pages that an answer can cite as sources. Answers based on parametric knowledge alone, by contrast, usually name no sources.
Timeliness: Parametric knowledge stays at the state of training. Retrieved content can be current: when a web page changes, an AI system can retrieve the new version once it has been captured, for example in a web index; the model does not need to be retrained for this.
Reliability: A model reproduces facts about well-known entities, such as major brands or companies, from memory more reliably than facts about little-known ones; for the latter, retrieving relevant content helps especially. Where a model has learned little, it can fill gaps with plausible-sounding but false statements, known as hallucinations.

Many AI systems combine both sources of knowledge: according to developer documentation from Google and Anthropic, when web access is enabled, Gemini and Claude, for example, decide for each question whether to also retrieve current content from the web. The method of basing an answer on previously retrieved content is called retrieval-augmented generation (RAG).

What parametric knowledge means for GEO

In GEO, parametric knowledge is the basis of every answer for which an AI system retrieves nothing. When a language model names your company in such an answer, it is usually a mention without a source reference, not an AI citation. Whether and how accurately a model knows your company depends, among other things, on what had been published about you before its training and made it into its training data: on your own website, but also in the media, in directories, and in reviews.

False or outdated information in parametric knowledge cannot be corrected directly. Within an answer, it is mainly content the AI system retrieves for the question that can set the record straight—provided the system retrieves anything at all.

This gives you two levers that complement each other. First, current content on your website can flow into answers that draw on the web, provided it is crawlable—that is, reachable by the programs AI providers use to fetch web pages. Second, an accurate, consistent portrayal of your company on your own website and across many third-party sources can shape what future models know about you; how to achieve that is covered under brand fact consistency.

Parametric knowledge

Definition

Where the knowledge resides

Parametric and retrieved knowledge

What parametric knowledge means for GEO

Frequently asked questions