Large Language Model (LLM) — Governance and Regulatory Obligations

Definition

A Large Language Model (LLM) is a type of deep learning AI model trained on large-scale text corpora to understand, generate, and transform natural language. LLMs are built on transformer architectures and trained using self-supervised learning on datasets comprising billions to trillions of tokens. Their emergent capabilities — reasoning, summarization, code generation, translation, question answering — make them foundational components of a wide range of AI-powered applications.

LLMs are the primary technical realization of General Purpose AI (GPAI) models under the EU AI Act. Organizations that develop, fine-tune, or distribute LLMs are subject to GPAI model obligations under the regulation. Organizations that integrate LLMs into products or services operating in high-risk domains face deployer obligations under the EU AI Act, independently of whether the underlying LLM provider is compliant.

The risk and governance profile of LLMs differs from narrow AI models in several dimensions: output unpredictability across the full distribution of possible inputs, hallucination risk (generating plausible but false information), prompt injection vulnerabilities, potential for misuse at scale, and difficulty of formal verification.

Why it matters operationally

LLMs are now embedded across enterprise technology stacks — customer service, document processing, code generation, decision support, content moderation. The speed of adoption has significantly outpaced governance. Most organizations cannot answer basic risk questions about their LLM deployments: what model versions are in use, what safeguards are in place, how output quality is monitored, what happens when the model hallucinates in a high-stakes context, and who is accountable.

This governance gap creates compounding risk. Hallucinations in legal or medical contexts cause harm. Prompt injection in enterprise LLM applications creates security vulnerabilities. Biased outputs in HR or lending contexts create discrimination exposure. Scale amplifies each of these risks — an LLM can make thousands of consequential decisions before any single failure is detected without adequate monitoring.

Regulatory framework

Framework	Application to LLMs
EU AI Act — Chapter V	LLMs are GPAI models under the Regulation. Providers have documentation, transparency, and copyright compliance obligations. Models with systemic risk (>10^25 FLOPs) have additional obligations.
ISO/IEC 42001	The management system covers the LLM lifecycle when developed or deployed by the organization. Annex A controls on risk, oversight, and governance apply.
NIST AI RMF	Provides the risk management framework for LLMs in the US market context, especially relevant for high-stakes use cases.
GDPR	LLMs processing personal data or generating outputs about individuals create GDPR obligations for deployers.

How Zertia evaluates it

Zertia evaluates LLM deployments through the AI Model Audit, examining the governance controls specific to LLM risk: output monitoring for hallucination and bias, prompt injection safeguards, version control and change management, human oversight for high-stakes outputs, documentation adequacy, and post-deployment performance tracking. The Ethical AI Mark evaluates ethical conformity of LLM systems against international ethics standards, including transparency (ISO TR 24028) and bias and equity (ISO TR 24027).

[AI Model Audit] · Ethical AI Mark