model_fields class attribute
BaseModel has a model_fields class attribute that contains the model's field definitions. It is a dictionary-like object where keys are field names and values are FieldInfo instances.
model_definition
107 notes in this subject, read out of this brain and free to use. This is page 1 of 2.
model_validate_json for JSON string validation
The BaseModel.model_validate_json() method validates data from a JSON string. It parses the JSON and validates it against the model schema. If validation fails, it raises a ValidationError.
TypeAdapter for validating non-BaseModel types
TypeAdapter is a Pydantic construct used to validate data against a single type that is not a BaseModel. It is created by passing a type annotation (such as list[Person]) to the TypeAdapter constructor, and has a validate_json() method for validating JSON strings.
Validating JSON list with TypeAdapter example
from pydantic import BaseModel, EmailStr, PositiveInt, TypeAdapter class Person(BaseModel): name: str age: PositiveInt email: EmailStr person_list_adapter = TypeAdapter(list[Person]) json_string = pathlib.Path('people.json').read_text() people = person_list_adapter.validate_json(json_string)
Validating JSONL (JSON Lines) files
JSONL files contain a sequence of JSON objects separated by newlines. To validate JSONL data, read the file as text, split by newlines, and validate each line as a separate JSON string using model_validate_json().
Validating JSONL example
import pathlib from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr json_lines = pathlib.Path('people.jsonl').read_text().splitlines() people = [Person.model_validate_json(line) for line in json_lines]
Validating CSV files with csv.DictReader
To validate CSV data, use Python's csv.DictReader to read the file as dictionaries, then validate each row using model_validate(). CSV files are read as strings, so numeric fields are automatically coerced by Pydantic.
Validating XML files
To validate XML data, use Python's xml.etree.ElementTree to parse the XML file, convert the tree to a dictionary by extracting child tag-text pairs, then validate using model_validate().
Validating CSV example
import csv from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr with open('people.csv') as f: reader = csv.DictReader(f) people = [Person.model_validate(row) for row in reader]
Validating TOML files
To validate TOML data, use Python's tomllib.load() to read a TOML file (opened in binary mode), then validate the resulting dictionary using model_validate().
Validating TOML example
import tomllib from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr with open('person.toml', 'rb') as f: data = tomllib.load(f) person = Person.model_validate(data)
Validating YAML files with PyYAML
To validate YAML data, use PyYAML's yaml.safe_load() to read a YAML file, then validate the resulting dictionary using model_validate().
Validating YAML example
import yaml from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr with open('person.yaml') as f: data = yaml.safe_load(f) person = Person.model_validate(data)
Validating XML example
import xml.etree.ElementTree as ET from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr tree = ET.parse('person.xml').getroot() data = {child.tag: child.text for child in tree} person = Person.model_validate(data)
Validating INI files with configparser
To validate INI data, use Python's configparser.ConfigParser to read the INI file, access the desired section as a dictionary-like object, then validate using model_validate().
Validating INI example
import configparser from pydantic import BaseModel, EmailStr, PositiveInt class Person(BaseModel): name: str age: PositiveInt email: EmailStr config = configparser.ConfigParser() config.read('person.ini') person = Person.model_validate(config['PERSON'])
model_validate method for dict validation
The BaseModel.model_validate() method validates data from a dictionary or dictionary-like object. It is used for validating data from various file formats after they have been parsed into dictionaries.
pydantic-settings for configuration files
The pydantic-settings library provides built-in support for parsing configuration data from various file formats (JSON, TOML, YAML, etc.). It should be considered when using Pydantic to parse configuration or settings files.
Pydantic with SQLAlchemy example using aliases
This example shows how to use Pydantic models alongside SQLAlchemy models to avoid code duplication. The Pydantic model uses ConfigDict(from_attributes=True) to validate ORM objects. Field aliases are used to handle SQLAlchemy reserved field names. The Field alias 'metadata_' maps the Pydantic field to the SQLAlchemy column name 'metadata', which is a reserved SQLAlchemy field. The SQLAlchemy model instance is validated using MyModel.model_validate(sql_model). When dumped with model_dump(), the field name is used; when dumped with model_dump(by_alias=True), the alias is used. import sqlalchemy as sa from sqlalchemy.orm import declarative_base from pydantic import BaseModel, ConfigDict, Field class MyModel(BaseModel): model_config = ConfigDict(from_attributes=True) metadata: dict[str, str] = Field(alias='metadata_') Base = declarative_base() class MyTableModel(Base): __tablename__ = 'my_table' id = sa.Column('id', sa.Integer, primary_key=True) # 'metadata' is reserved by SQLAlchemy, hence the '_' metadata_ = sa.Column('metadata', sa.JSON) sql_model = MyTableModel(metadata_={'key': 'val'}, id=1) pydantic_model = MyModel.model_validate(sql_model) print(pydantic_model.model_dump()) #> {'metadata': {'key': 'val'}} print(pydantic_model.model_dump(by_alias=True)) #> {'metadata_': {'key': 'val'}}
Aliases have priority over field names for field population
When populating fields in Pydantic models, aliases have priority over field names. This means if a field has an alias defined, Pydantic will match the alias name first before attempting to match the field name.
from_attributes ConfigDict option enables ORM object validation
The ConfigDict(from_attributes=True) option allows Pydantic models to validate ORM objects by populating fields from object attributes. This is essential when working with SQLAlchemy or other ORM libraries.
ORM validation failures from database constraints and NULL values
When validating ORM objects, validation can fail due to rows written before a constraint was added, or columns that allow NULL where the model does not. These failures only occur when the offending row is actually read, potentially long after a deploy. Recording failed validations retains the input object alongside the error, which helps identify the offending row.
SQLModel library eliminates code duplication with Pydantic and SQLAlchemy
SQLModel is a library that integrates Pydantic with SQLAlchemy to eliminate much of the code duplication that occurs when using Pydantic models alongside SQLAlchemy models. It is available at https://sqlmodel.tiangolo.com/.
Pydantic AI LLM structured output example
This example shows using Pydantic AI with an Agent that validates LLM output against a City model. The Agent is initialized with output_type=list[City], and validation_context=['Japan', 'United States', 'Germany'] is passed to constrain validation. A field_validator on the country field checks that the country is in the valid_countries list from validation context. When run, the agent produces output matching the specified model schema.
Pydantic AI output_type schema validation
Pydantic AI is a Python agent framework by the Pydantic team that uses Pydantic validation for structured output schema generation and validation. By specifying an output_type parameter on an Agent, you can constrain the LLM to return data that matches your Pydantic model schema.
Using BaseModel to validate HTTP response data
Pydantic models can be used to validate HTTP response data. Use model_validate() method to create a model instance from JSON response data. Example: user = User.model_validate(response.json()) validates JSON data and returns a User instance.
Pydantic integration with web frameworks
Pydantic is instrumental in many web frameworks and libraries, including FastAPI, Django, Flask, and HTTPX. It is used to validate and serialize data for requests and responses.
Using TypeAdapter to validate lists from HTTP responses
TypeAdapter can be used to validate complex types like lists from HTTP responses. Create a TypeAdapter for the target type (e.g., TypeAdapter(list[User])) and call validate_python() with the response JSON data. This is useful when the response is a list rather than a single object.
BaseModel field access as attributes
Fields of a Pydantic BaseModel instance can be accessed as attributes of the model object.
Annotation-only field declaration makes field required
In Pydantic models, a field declared with only a type annotation (no default value) is required. For example, `id: int` makes the id field required.
Pydantic example: User model with validation
Example code showing Pydantic model definition and validation: ```python from datetime import datetime from pydantic import BaseModel, PositiveInt class User(BaseModel): id: int name: str = 'John Doe' signup_ts: datetime | None tastes: dict[str, PositiveInt] external_data = { 'id': 123, 'signup_ts': '2019-06-01 12:22', 'tastes': { 'wine': 9, b'cheese': 7, 'cabbage': '1', }, } user = User(**external_data) print(user.id) #> 123 print(user.model_dump()) ``` This example demonstrates model definition, type coercion, and model instantiation.
model_dump() converts model to dictionary
The model_dump() method converts a Pydantic BaseModel instance to a dictionary representation.
Pydantic supports strict and lax validation modes
Pydantic can run in either strict mode (where data is not converted) or lax mode (where Pydantic tries to coerce data to the correct type where appropriate).
Pydantic core features
Pydantic's core features include: schema validation and serialization controlled by type annotations, JSON Schema emission, support for dataclasses and TypedDicts, custom validators and serializers, and an ecosystem of around 8,000 packages on PyPI.
Default value makes field optional
A field with a default value is not required. For example, `name: str = 'John Doe'` makes the name field optional with a default of 'John Doe'.
Core schema as communication structure between pydantic and pydantic-core
Pydantic uses the concept of a core schema to communicate collected information from model definition to pydantic-core for validation and serialization. A core schema is a structured and serializable Python dictionary (represented using TypedDict definitions) describing specific validation and serialization logic. Every core schema has a required type key, and extra properties depending on this type.
Pydantic V2 architecture: pydantic and pydantic-core packages
Starting with Pydantic V2, part of the codebase is written in Rust in a separate package called pydantic-core. This was done to improve validation and serialization performance, with the trade-off of limited customization and extendibility of internal logic.
Model definition and validation separation in Pydantic
Usage of the Pydantic library is divided into two parts: model definition, done in the pydantic package, and model validation and serialization, done in the pydantic-core package.
BaseModel metaclass analysis during definition
When a Pydantic BaseModel is defined, the metaclass analyzes the body of the model to collect: defined annotations to build model fields (collected in the model_fields attribute), model configuration set with model_config, additional validators and serializers, and private attributes, class variables, and identification of generic parametrization.
GenerateSchema class generates core schemas
The generation of a core schema is handled by the GenerateSchema class, which generates core schemas whether for a Pydantic model or anything else.
Custom core schemas cannot be defined
It is not possible to define a custom core schema. A core schema needs to be understood by the pydantic-core package, so only a fixed number of core schema types are supported. Core schema definitions can be found in the pydantic_core.core_schema module.
BaseModel __pydantic_core_schema__ attribute
In the case of a Pydantic model, a core schema is constructed and set as the __pydantic_core_schema__ attribute on the BaseModel.
Serialization logic in core schemas
Serialization logic is defined in the core schema. Custom serialization functions can be defined using decorators like field_serializer, which adds a serialization key to the core schema containing validation and serialization information.
GenerateJsonSchema class for JSON Schema generation
JSON Schema generation is handled by the GenerateJsonSchema class. The generate method is the main entry point and is given the core schema of a model. For example, a bool field core schema generates the JSON Schema {"type": "boolean"}.
__get_pydantic_core_schema__ method for custom core schemas
Pydantic offers a way to customize core schemas through the __get_pydantic_core_schema__ method, which follows a wrapper pattern. This method can be used with Annotated metadata classes to modify the generated core schema by calling the handler to get the base schema and then modifying it before returning.
__get_pydantic_json_schema__ method for custom JSON schemas
Pydantic offers a way to customize JSON schemas through the __get_pydantic_json_schema__ method, similar to __get_pydantic_core_schema__ for core schemas.
GetCoreSchemaHandler wrapper pattern with Annotated
When using __get_pydantic_core_schema__ with Annotated, the GetCoreSchemaHandler is defined in a nested way. Multiple annotations can be applied in sequence, with each __get_pydantic_core_schema__ method receiving the schema modified by previous annotations. Calling the handler recursively calls other __get_pydantic_core_schema__ methods until reaching the base type.
SchemaValidator and SchemaSerializer in pydantic-core
pydantic-core exposes a SchemaValidator class and SchemaSerializer class to perform validation and serialization tasks. The SchemaValidator.validate_python method is used to validate data, and the SchemaSerializer.to_python method is used to serialize model instances.
Model validation process
When calling model_validate on a model instance, the provided data is sent to pydantic-core using SchemaValidator.validate_python. pydantic-core validates the data following the core schema of the model and populates the model's __dict__ attribute.
Model serialization process
When calling model_dump on a model instance, the model instance is sent to pydantic-core using SchemaSerializer.to_python. pydantic-core reads the instance's __dict__ attribute and builds the appropriate result following the core schema of the model.
Plugin hook layer around validation in Pydantic
Pydantic wraps the SchemaValidator in a plugin layer. When plugins are installed, each of the validate_python, validate_json, and validate_strings calls can be intercepted, allowing plugins to observe the input, result, and any error for every validation.
plugin_settings configuration for model plugins
Plugins are configured per model through the plugin_settings configuration value in ConfigDict.
Limitation: locals snapshot in function-defined classes
When a Pydantic model is defined inside a function, Pydantic keeps a copy of the locals of that frame, but only includes symbols defined when the model class was created. Symbols defined after the class definition (like InnerType2 defined after Model) won't be included and won't resolve if the model is rebuilt later. Additionally, weak references are used to the function's locals to avoid memory leaks, meaning some forward references might not resolve outside the function scope.
Backwards compatibility: annotation resolution during core schema generation
During core schema generation, when Pydantic encounters class-like field types (e.g., dataclasses), it evaluates their annotations. For backwards compatibility, the locals used during this evaluation include both the parent namespace (for function-defined classes) and {ClassName: class_object} with lowest priority, even though the class being created hasn't been assigned to the module's __dict__ yet. This allows annotations to reference types currently being defined without requiring model_rebuild().
MockCoreSchema when forward references fail during model definition
When a forward reference fails to evaluate, Pydantic silently stops core schema generation. The __pydantic_core_schema__ attribute will contain a MockCoreSchema object instead of the actual core schema dictionary, indicating the model needs to be rebuilt.
Annotation resolution example: multiple namespace sources
In a model `Model(BaseModel, Base)` defined inside function `inner()`, with a base class `Base` from another module, forward references resolve as follows: `f1: 'MyType'` from Base resolves to the MyType from Base's module; `f2: 'MyType'` from Model resolves to the MyType from Model's module (higher priority); `f3: 'InnerType'` resolves from the function's locals; `f4: 'LocalType'` resolves from Model's own class dict; `f5: 'UnknownType'` remains unresolved as a string.
Example: model_rebuild() with unresolved forward reference in function-defined class
When a model defined in a function has a forward reference that cannot resolve outside the function scope, calling model_rebuild() with the forward reference still unresolved will raise PydanticUndefinedAnnotation. Example: defining `class Model(BaseModel)` with `f: 'A | Forward'` inside a function where A is a local variable, then calling `Model.model_rebuild(_types_namespace={'Forward': str})` outside the function will fail because A is no longer accessible.
Pydantic runtime annotation resolution challenges
Pydantic relies on type hints at runtime to build schemas for validation and serialization. While Python's standard library provides tools like typing.get_type_hints() and inspect.get_annotations(), they have limitations. Pydantic re-implements this logic with improved support for edge cases. In v2.10, the internal logic was refactored to simplify annotation evaluation, though backwards compatibility posed challenges.
model_rebuild() namespace semantics
The model_rebuild() method uses a rebuild namespace with these semantics: if an explicit _types_namespace argument is provided, it is used as the rebuild namespace; if no namespace is provided, the namespace where model_rebuild() is called will be used as the rebuild namespace. This rebuild namespace is merged with the model's parent namespace (if defined in a function) and used to evaluate any forward references that previously failed.
Namespace resolution order for annotations at class definition
When resolving forward references during class definition, Pydantic processes each base class in reverse MRO order. For each annotation, it evaluates the string using a custom eval wrapper with two namespaces: globals (the current module's __dict__) and locals (tried in priority order): 1) A namespace containing the current class name ({cls.__name__: cls}) for recursive references, 2) The class's own __dict__ (including class-level assignments), 3) The parent namespace of the frame where the class is defined (if different from globals, e.g., when defined inside a function).