Foundation Model

Foundation model (FM) is an artificial intelligence model trained on broad data, usually at substantial scale, so that it can be adapted to many different downstream tasks instead of being built for just one application. A foundation model provides reusable learned representations or capabilities on which developers can build specialized models and AI-enabled systems. Adaptation may involve prompting, fine-tuning or other techniques, depending on the model and task. Researchers at Stanford University introduced the term in 2021 to emphasize both this capacity for reuse and the fact that a pretrained model is an incomplete foundation for a functioning application.

In practical terms, an organization may use one broadly trained model to support information extraction, document classification, question answering or content generation, subject to the model's actual capabilities and suitable adaptation. This differs from a traditional task-specific model designed and trained chiefly to solve a narrowly defined problem. Foundation models include language models, some vision models and models spanning several types of data; they are not restricted to text generators or chatbots.

A foundation model is the model, rather than the complete agent, application or business process that uses it. A deployed system also needs appropriate data connections, permissions, evaluation, business rules, interfaces, oversight and monitoring. A single foundation model may underpin numerous systems, so both its useful capabilities and its defects can be carried into multiple downstream applications.

Meaning and defining characteristics

The term describes a model development and reuse paradigm. Its defining features are broad pretraining and adaptability, rather than a specific vendor, neural-network size, architecture or style of output. The adjective foundation indicates that the learned model becomes a shared starting point for further work. It does not mean that the model is universally reliable, complete or suitable for every task.

Characteristic	Meaning	Important qualification
Broad pretraining	The model learns from substantial, varied data rather than being trained only for one final business task.	Broad data does not guarantee representative, lawful, accurate or high-quality coverage.
Reusable learned capabilities	Learned representations support more than one downstream task.	The useful range of tasks must be demonstrated, not assumed from a general description.
Adaptability	The same pretrained core can be prompted, extended or further trained for different purposes.	Each adaptation can require separate tests and controls.
Shared upstream dependency	Different products may depend on the same underlying model or model family.	Defects, model changes and outages can affect more than one downstream system.
Incomplete application layer	A model supplies technical capabilities but does not itself establish a safe business workflow.	Access permissions, current records, human accountability and performance monitoring remain system responsibilities.

Foundation models are usually built using deep learning, often with self-supervised learning, which creates training signals from the structure of the data itself. The Stanford definition deliberately allows variation in training approach and architecture; a fixed parameter count or the presence of a transformer alone is not the universal classification rule. Large and foundation therefore describe related but different properties.

Why foundation models matter

Historically, many machine-learning projects required researchers to assemble data and train a largely separate model for each task. Broad pretraining changes that arrangement: capabilities learned once can be reused, adapted and incorporated into multiple applications. This can reduce task-specific training requirements and make it possible to experiment with new applications without reproducing the full original training effort. Whether it reduces total cost or improves performance depends on the adaptation, infrastructure, procurement and control needs of the particular deployment.

The same reuse also concentrates dependency. A bank might rely on the same model family for internal knowledge search, correspondence drafting and document extraction. Each application has its own prompts, data and permissions, but all three inherit some characteristics of the shared model. A model update can change outputs across several departments at once. This is one reason model governance cannot be confined to the team that built the first application.

Stanford's original report calls attention to homogenization: multiple systems can inherit strengths, biases and weaknesses from a common foundation. The resulting scale of reuse creates value, but also increases the importance of transparency, independent testing and mechanisms to identify shared failures.

How foundation models are developed and used

A useful conceptual sequence is broad pretraining → reusable foundation model → task adaptation → operational AI system. The sequence describes relationships between work products, not a universal one-directional development method. A deployed application can be revised, evaluated and adapted repeatedly.

Training data and objectives

A development team chooses data sources and a learning objective suited to the intended capabilities. Text models may train by predicting hidden or subsequent tokens. Vision or multimodal models may learn relationships between images and captions, image regions or other signals. Self-supervised objectives can exploit information implicit in the data without requiring manual labels for every final task.

The training material may include text, software code, images, audio, video or specialized scientific representations. A model is not necessarily trained on all these forms: a language-only foundation model and an image segmentation model can both qualify. Data selection and filtering influence downstream usefulness, exposure to private or copyrighted material, language coverage and bias. Large or diverse datasets are not automatically complete or representative.

Model architecture and pretraining

Pretraining adjusts the parameters of a neural network to improve performance on the chosen objectives. Many language and multimodal foundation models use transformer-based architectures, while visual, audio and generative tasks also use other architectures and combinations of techniques. Training at this scale may require substantial computing capacity, specialized hardware, storage and engineering effort.

The resulting parameters encode learned statistical structure. They are not a continuously updated database of authoritative facts, nor do they independently guarantee correct reasoning or context-sensitive judgment. A model may perform well on tasks that resemble its training and evaluation conditions and still fail on unfamiliar or consequential cases.

Adaptation to particular tasks

Developers can adapt the shared model through one or more methods: changing instructions or examples in a prompt, fine-tuning some or all parameters, adding task-specific components, or connecting the model to controlled external information. Different methods change different parts of the resulting system. In particular, retrieving documents at inference time usually changes the information supplied to the model, not its stored parameters.

Inference and system integration

During inference, a trained model processes a new input and returns a prediction, representation, classification, ranking or generated output, depending on its design. The surrounding application determines what the output is allowed to affect. A model that extracts a contract clause does not automatically possess authority to approve a contract; a model that generates code does not automatically have permission to deploy it.

A useful enterprise distinction is therefore between model capability, adapted task performance and system-level authority. None can safely be inferred from the others. The operational system must implement relevant access control, validation, escalation, logging, error recovery and review.

Major families and data modalities

Foundation models differ along several axes: the kind of data they process, the outputs they produce, their underlying architecture and the tasks to which they can transfer. The groups below overlap; they are not a mutually exclusive taxonomy.

Family or modality	Typical foundation capability	Examples of downstream work
Language and code	Reusable language representations or the ability to produce sequences of text or code.	Search, classification, extraction, summarization, translation, code assistance and question answering.
Vision	Reusable representations of images and visual relationships.	Image classification, segmentation, object identification and inspection support through computer vision applications.
Audio and speech	Reusable representations for speech, sound or audio-language tasks.	Transcription, speech understanding, speaker-related analysis and speech generation, where supported.
Multimodal	Learns relationships between more than one data modality, such as text and images.	Image-text retrieval, visual question answering and cross-modal search or generation.
Scientific and domain-focused	Broad reusable representations within a scientific or operational domain, rather than across every possible human task.	Biological sequence analysis, specialized imagery and other related downstream tasks, subject to domain validation.

The term foundation model is not limited to large language models. Some foundational models process visual or other nonlinguistic inputs, and some return embeddings or labels rather than prose. Similarly, not every model that generates content is a foundation model; a narrowly trained generator may lack broad downstream adaptability.

Documented examples

Examples demonstrate that the foundation-model paradigm encompasses more than conversational AI. These are historical or research examples, not an endorsement of a current model version for enterprise procurement.

Model or project	Evidence of foundation-model characteristics	Boundary
BERT	Pretrained bidirectional language representations can be adapted with a relatively small task-specific layer for multiple language-understanding tasks.	A representation model is not inherently a conversational text generator.
CLIP	Learns aligned image and language representations from natural-language supervision, permitting transfer to visual recognition tasks without conventional class-specific training for every task.	Vision-language representations do not automatically imply reliable reasoning about all images.
Segment Anything Model (SAM)	A promptable image-segmentation model designed for transfer to new image distributions and segmentation tasks.	Strong segmentation results do not establish suitability for every specialized imaging use.
GPT-3	A broadly pretrained language model used in Stanford's original 2021 analysis to illustrate adaptation across tasks through natural-language prompts.	A historical example; it is not a statement about present model rankings, prices or availability.

The significant common feature is reusable pretraining followed by downstream use, not identical architecture, model size or output type. BERT is especially useful as a counterexample to the assumption that all foundation models are generative chat models; CLIP and SAM illustrate non-text-only foundations.

Ways foundation models are adapted

The word adaptation covers both changes to a model and changes to the application around it. Conflating those forms can cause errors in procurement, evaluation and model-version records.

Adaptation method	What changes	Typical consideration
Prompting or in-context examples	Changes the instructions and examples supplied with an input; model parameters ordinarily stay fixed.	Results may be prompt-sensitive; test realistic inputs and failure cases.
Task or instruction fine-tuning	Updates model parameters using additional examples or feedback to improve behavior on a defined class of tasks.	Requires suitable training data, version control and regression testing.
Parameter-efficient fine-tuning	Trains a limited subset of parameters or additional components while much of the base model remains unchanged. LoRA is one researched technique.	Lower adaptation cost does not itself establish model reliability or fit.
Continued pretraining	Further updates model weights using more data, often for a new domain or modality distribution.	Can affect existing capabilities and introduce new data-provenance issues.
Retrieval-augmented generation	Adds retrieved external material to a generative application's context; the base model typically remains unchanged.	Retrieval accuracy, permissions, freshness and citation checks belong to the overall system.
External tools and business rules	Adds connected systems, tool permissions and application controls around the model.	Controls the model's use and consequences; it does not by itself prove changes to the underlying model.

These methods can be combined. For example, a pretrained language model might first receive domain-specific fine-tuning, then be embedded in an application that retrieves approved records and validates each output against business rules. An application update and a model update should be traceable as distinct events.

Foundation model lifecycle and dependencies

Foundation models are components in a wider lifecycle involving the model developer, downstream integrators, buyers and users. One model may be updated by its provider while many separate organizations continue adapting or deploying earlier variants. That shared lineage complicates change control.

Activity	Output or evidence	Typical risk
Data sourcing and preparation	Data provenance records, scope and filtering decisions.	Undisclosed or unsuitable data, coverage gaps and rights uncertainty.
Pretraining and model creation	Versioned weights or hosted model interface, training specification and development records.	Compute and supply-chain dependence, limited reproducibility and inherited faults.
Capability evaluation	Task tests, representative challenge sets and limitations.	Benchmarks disconnected from operating conditions.
Adaptation and integration	Prompts, tuning datasets, adapters, application code, retrieval configuration and access rules.	Task-specific defects and uncontrolled privileges.
Deployment and monitoring	Version log, operational metrics, incident records and review decisions.	Drift, service changes, silent quality loss and overreliance.
Replacement or retirement	Migration evidence, rollback plan, record retention and dependency inventory.	Breaking downstream integrations or losing the ability to reproduce past decisions.

A foundation model's model card, technical documentation or evaluation report can inform selection and oversight, but the document is evidence about the model, not a substitute for testing a specific deployment. These records should identify the exact model, relevant version, intended uses, known limitations, testing scope, access conditions and update arrangements when available.

Applications and enterprise use

Foundation models may support repeated classes of work across organizations rather than one fixed application. Their economic value depends on the complete process and how the reusable capability changes its results; it cannot be inferred from model capacity alone.

Business context	Foundation-model contribution	Separate system responsibilities
Knowledge search and support	Language representations, extraction or task-specific answer generation.	Access-filtered retrieval, source freshness, answer verification and escalation.
Document-intensive operations	Classification, entity extraction, drafting or comparison across varied document types.	Document controls, evidentiary integrity, accuracy checks and approval.
Software engineering	Code understanding, drafting or test suggestions.	Repository permissions, code review, security tests and release authorization.
Image-based operations	Reusable vision representations for inspection or segmentation.	Image acquisition quality, site-specific validation and qualified human decisions.
Customer or employee service	Text or speech interpretation and response suggestions.	Case handling, identity checks, privacy, quality monitoring and exceptions.
Research and analytical work	Information retrieval, multimodal representation or candidate generation.	Ground-truth testing, expert review and controls on consequential use.

A central evaluation question is whether the selected foundation model actually improves a defined task after accounting for application integration, human review, error costs and vendor operation. A highly capable base model can perform worse in a specialized application than a smaller or task-specific alternative because of mismatch, cost, latency, missing domain data or operational constraints.

Foundation models compared with related concepts

The following distinctions prevent one technology term from absorbing several different kinds of objects and reader questions.

Related concept	Relationship or distinction
Artificial intelligence	The encompassing field of techniques and systems; foundation models are one influential model-development paradigm within it.
Machine Learning	The broader family of methods that learn from data; not all machine-learning models are pretrained for broad reuse.
Large language model	A model category associated with language tasks; many modern LLMs qualify as foundation models, while foundation models also encompass vision and other modalities.
Generative AI	An output capability for producing new content. Foundation models frequently power it, but not all foundation models or generative systems are equivalent.
Base model	Often refers to a pretrained checkpoint before instruction tuning or other downstream changes. Some uses treat base and foundation as near-synonyms, but their meanings depend on context.
Frontier model	A context-dependent label for models near the leading edge of capability or scale. A foundation model need not be a frontier model.
General-purpose AI model	A legal category under the EU AI Act that substantially overlaps broad reusable models but has its own regulatory definition, exclusions and obligations.
AI Agent	A goal-directed actor or system that may use a foundation model. The model alone does not determine available tools, permissions or organizational accountability.
Transformer	A model architecture widely used in foundation models. The architecture is a technical design, not a sufficient definition of broad-task reuse.
Transfer Learning	The wider practice of applying learning from one task or dataset to another; foundation-model adaptation is an important contemporary form of this idea.

The boundaries have practical consequences: a vendor's statement that it provides a foundation model does not establish that the purchaser receives model weights, that the model has been independently validated for a regulated task, or that the purchaser can lawfully modify or redistribute it. Those are separate evidence and contract questions.

Access models and licensing

Organizations may obtain access to foundation models through a hosted service, a managed cloud platform, a licensed deployment or downloadable model weights. The model's technical category does not determine its commercial availability or legal permissions.

Access arrangement	What the organization typically controls	Questions to resolve
Hosted application programming interface (API)	Prompts, application logic and user-facing integration within the offered service.	Data handling, retention, model changes, location, uptime, audit rights and exit options.
Managed model service	Some configuration, orchestration and connectivity through a hosting platform.	Provider responsibilities, isolation boundaries, available versions and portability.
Licensed or customer-hosted deployment	More of the runtime environment, depending on the agreement.	Infrastructure cost, updates, weights access, permitted adaptations and security obligations.
Open-weight distribution	Access to specified learned parameters under particular terms.	License restrictions, missing training details, security of artifacts and suitability for intended reuse.

Open-weight is not automatically the same as open-source AI. The Open Source Initiative's 2024 Open Source AI Definition 1.0 requires freedoms to use, study, modify and share, supported by access to the preferred form for making changes; a weights download on its own need not satisfy those conditions.

Model governance should therefore bind technical identity to rights and operating evidence: the exact weights or hosted endpoint, version, license, approved uses, supported tasks, vendor commitments, dependencies and relevant assessments. A model described as open may still have use restrictions or limited transparency about its training data.

Evaluating a foundation model

Evaluation should reflect the distinction between the reusable core and an application derived from it. Standard benchmarks offer useful comparative evidence, but they do not demonstrate that a model is accurate, safe or economical on every downstream task. Evaluation is also affected by benchmark contamination, prompt choices, population differences, test design, model versions and available adaptation resources.

Level of evidence	Example measures	What it does not establish
Pretrained-model capability	Task accuracy, calibration, error patterns, robustness, multilingual or multimodal coverage, computation and latency.	Performance after adaptation for a different domain or population.
Adapted task performance	Domain-specific correctness, false-positive and false-negative rates, representative examples, red-team findings and version sensitivity.	Business value or safety in a live process by itself.
Operational system performance	Quality-adjusted throughput, reliability, human correction, incident frequency, privacy and security outcomes, service cost and user experience.	That observed benefits are caused solely by the foundation model.

Stanford's HELM project illustrates why model evaluation benefits from multiple scenarios and metrics rather than a single number. Its public evaluation framework supports reproducible comparisons, but even a transparent leaderboard is not a substitute for task-specific acceptance testing.

When selecting a model, the decision should consider the cost of adaptation, inference latency, domain error consequences, maintenance effort, access restrictions and the effect of future upstream changes. The appropriate baseline might be a conventional software method, a task-specific model, a smaller foundation model or a human-only process. These comparisons are application decisions; they do not redefine the model category.

Limitations and systemic risks

Foundation models can generalize across tasks but do not remove the fundamental limitations of machine learning. Their breadth can hide uneven performance by language, population, environment or type of input. A fluent explanation may be wrong; a visual model may fail on unfamiliar conditions; and apparent generality on a benchmark may not survive operational change.

Important risks include:

• Unreliable or fabricated outputs: generative models can produce plausible but unsupported content; classification and representation models can make other kinds of errors.
• Bias and unequal performance: pretraining datasets and adaptation choices can reproduce gaps or discriminatory effects across groups or contexts.
• Privacy and data exposure: training, prompting, retrieval, logging and model access may involve sensitive information.
• Cybersecurity and manipulation: data poisoning, model theft, prompt injection in connected applications and uncontrolled tool use can affect confidentiality, integrity and behavior.
• Intellectual property and rights: rights to training material, model artifacts, derived output and downstream redistribution require their own analysis.
• Operational concentration: shared providers, model families and infrastructure can create correlated failure and vendor dependence.
• Cost and environmental impact: development, adaptation and inference use computing resources; cost or energy implications depend on the specific system and workload.
• Limited interpretability and transparency: some training data, optimization details, model changes and failure modes may be inaccessible to downstream users.

A particular foundation-model defect can propagate differently across applications. An inaccurate representation may affect classification in one service, while a generative tendency to invent facts may affect a document assistant in another. Domain validation and application controls can reduce exposure, but one generic safety assertion about the underlying model cannot certify every derived use. Stanford's original work specifically identified concentration and shared downstream defects as part of the foundation-model paradigm.

Transparency is itself a matter for evidence, not provider reputation alone. Stanford's Foundation Model Transparency Index examines disclosures about development, model properties and downstream use; its 2025 edition provides a dated assessment rather than a permanent score for any provider.

Governance, standards and regulation

AI governance addresses the responsibilities and controls for the use of models and AI-enabled systems. The governance need arises at two levels: upstream, where the foundation-model developer controls important training and release choices; and downstream, where an integrator or deploying organization selects contexts, data connections, permissions and users. Responsibilities cannot be assigned reliably by treating the model developer and final user as the same actor.

NIST's voluntary AI Risk Management Framework and its generative-AI profile provide a structured basis for considering risks across the AI lifecycle. They are useful guidance, not proof that a particular model is fit for a particular task and not substitutes for applicable legal duties.

Within the European Union, the AI Act uses the legal term general-purpose AI model (GPAI model). The category overlaps many commercially important foundation models but is defined and governed in law; the expression foundation model should not be substituted automatically into a legal applicability decision. The European Commission identifies technical-documentation, training-content-summary and copyright-policy obligations for GPAI-model providers, with additional duties for models presenting systemic risk. Obligations and enforcement dates depend on the provider role, when the model was placed on the market and the relevant rules.

An enterprise acquiring access to a model should determine which regulatory role it occupies and what use-case-specific law applies, rather than infer compliance from a model label. Organizational records commonly include a model inventory, approved-use scope, supplier and license documentation, evaluation results, risk decisions, access controls, incident procedures, version tracking and review dates. These are oversight artifacts, not universally mandated fields for every enterprise or jurisdiction.

Illustrative enterprise example

Consider a hypothetical insurer that receives thousands of customer documents in different formats. The organization wants to support staff who identify the type of each document, extract selected fields and draft a summary. It evaluates a foundation model with capabilities spanning text representation and language processing.

The team first measures document classification accuracy and field extraction quality on a representative sample of its own documents, including low-quality scans, unusual forms and records from different customer groups. It compares the model with simpler task-specific tools rather than assuming that a large pretrained model will be better. For summary generation, the team adds permission-controlled access to records and checks whether the draft accurately reflects the underlying source.

The foundation model is only one part of the service. Identity and access management controls which records a user can view; application logic blocks unauthorized changes; staff review ambiguous or consequential cases; and records of model versions, overrides and errors support ongoing assessment. The organization may be able to reuse the model for several document tasks while maintaining distinct evaluation and approval criteria for each one. This is an illustrative scenario, not a claim that a particular insurer achieved a quantified benefit.

Historical development

The underlying idea of reusing learned representations predates the term foundation model. Transfer learning, pretraining and deep learning established many of the technical principles. BERT's 2019 publication demonstrated the usefulness of broadly pretrained language representations for different natural-language tasks; CLIP's 2021 research illustrated transferable visual representations learned from language supervision.

Stanford researchers introduced the foundation-model term in their 2021 report to describe the growing importance of broad pretraining, adaptation and downstream dependence. The report also emphasized that foundation models are incomplete and that their reuse can magnify risks. Later research extended the approach across additional modalities and domains, including promptable visual segmentation as demonstrated in the 2023 Segment Anything project.

Because architectures, model names and access conditions change, the historical examples on this page should not be read as a current best-model ranking. The category is defined by model characteristics and reuse, not by a fixed list of products.

Frequently asked questions

What is a foundation model in simple terms?

It is an AI model trained broadly enough to be reused as a starting point for multiple tasks. Instead of building every model from scratch, developers adapt the pretrained foundation to a specific application. A complete application still needs its own data handling, testing and controls.

Are foundation models and large language models the same?

No. Many modern large language models are foundation models, but foundation models also include important models for vision and other data types. The categories overlap but are not equivalent; a language model's size alone does not prove broad downstream adaptability.

Is every foundation model a generative AI model?

No. Some foundation models primarily provide representations or perform recognition, classification or segmentation rather than generating open-ended text or images. BERT, CLIP and SAM demonstrate the broader category.

Does a foundation model learn automatically from every new prompt?

Not necessarily. Prompts and retrieved documents may change an application's output without updating the model's parameters. Persistent changes to the model generally require an explicitly designed training or adaptation procedure, although a system can maintain external memory or state.

Does using a foundation model eliminate the need for specialized data or testing?

No. A broadly pretrained model can lower some adaptation barriers, but it still requires evaluation against the intended task, representative operating data, relevant users and acceptable error thresholds. High general benchmark scores do not establish safe performance in a regulated or high-consequence setting.

Is an open-weight foundation model automatically open source?

No. Access to weights does not by itself establish access to the information, code, permissions and freedoms described in the Open Source AI Definition. The license and released materials must be assessed separately.

Is a foundation model the same as a general-purpose AI model under the EU AI Act?

Not automatically. The first term is a research and technical description; the second is a defined legal category. They overlap, but determining which obligations apply requires the regulation's definition, scope and provider-role analysis.

See also

• Artificial Intelligence (AI)
• Machine Learning
• Deep Learning
• Large Language Model (LLM)
• Generative AI
• Transformer
• Transfer Learning
• Retrieval-Augmented Generation (RAG)
• AI Evaluation
• AI Governance
• Model Risk Management
• General-Purpose AI Model