![]() |
Dakota
Version 7.0
Explore and Predict with Confidence
|
Pydantic Grammar Table of Contents
The Pydantic models in dakota/python/dakota/spec constitute the canonical specification of Dakota's input grammar. Implement grammar edits and additions in these Python models and, where required, provide matching C++ validation or computed-field implementations so that the shared grammar contract remains consistent between schema generation and runtime.
Most models inherit from DakotaBaseModel (dakota/python/dakota/spec/base.py). Parsed fields are declared with DakotaField and represent user-visible keywords and arguments. ValidationRule subclasses capture checks or mutations that depend on multiple parsed fields. ComputedFieldSpec entries paired with @computed_field properties export derived values in the schema or validated models. This page is self-contained: procedural steps reference the detailed authoring sections that follow; readers may consult any section on demand.
Follow these steps when adding a new keyword. Each step references the authoritative section on this page that documents the required authoring surface and implementation details.
Decide grammar role and one-of semantics. Choose whether the keyword is a scalar or collection argument, a pointer- style keyword that names another block, a Union branch, or a selection- wrapper (method or model family). If the keyword participates in a Union, ensure the Python annotation, default/ and default_factory,dakota union_pattern metadata express identical one-of semantics. See How Union fields are processed.
Declare the parsed field on the appropriate model class. Add a typed class attribute to a DakotaBaseModel-derived class and declare it with DakotaField. Provide Pydantic parameters (default, default_factory, description, bounds) as appropriate. Include dakota metadata keys only as required: aliases, argument, block_pointer, anchor, union_pattern, model_default, internal_only, and materialization. For a new method or model selection, follow the selection-wrapper pattern described in DakotaField authoring and Adding a new method or model.
Author materialization mapping to IR. Select the IR key or keys that the field should modify and author the materialization metadata describing the write instructions. Each item must include at minimum ir_key, storage_type, and ir_value_type. Some storage_type values require additional properties (stored_value, secondary_ir_key, secondary_ir_value_type, enum_scope, secondary_literal_value). Prefer reusing existing storage-type handlers for identical IR effects; for domain-specific shapes use an existing model as a template. See Materialization and IR writes.
Select the IR-default source. Choose one canonical source for any IR key's default: a grammar-level default on the Pydantic field, an implementation-level override in default_overrides_registry.json, or a type-policy default in default_policy_registry.json. The IR codegen merges these disjoint sources; generate_ir_tables.py treats collisions as errors. See IR default values.
Add validation rules or computed fields if required. If correctness depends on relationships among fields, author a ValidationRule subclass, attach it via _VALIDATION_RULES on the model, and provide the matching C++ implementation and registration. If a derived value must appear in schema or the validated model, add a ComputedFieldSpec to _COMPUTED_FIELDS and implement the corresponding @computed_field property with a matching C++ registration. Both features require exact string-name agreement between Python metadata and the C++ registry entries. See Validation rules and computed fields.
Implement and register required C++ logic and bindings. Any new ValidationRule or computed-field function requires a C++ declaration and implementation, registration in the appropriate registry constructor, and exposure via pybind so the Python layer can invoke it. Typical files to edit are under packages/dakota_parser/dakota_validation (include, src, and test). Ensure the registered C++ string names match the Python metadata. See Validation rules and computed fields.
Regenerate schema and run IR code generation. Generate JSON schema from the updated Pydantic models and confirm that x-materialization, x-model-validations, x-computed-fields, and x-model-default metadata appear as expected. Run dakota/src/ir_codegen/extract_defaults_from_schema.py and dakota/src/ir_codegen/generate_ir_tables.py and inspect the generated IR contracts and write-instruction tables. Confirm intended ir_keys, ir_value_types, and defaults are present and that no collisions exist. See Materialization and IR writes and IR default values.
ProblemDescDB reads. Update documentation and example inputs to demonstrate the new keyword. Verify CI and style checks succeed before submitting the change. Dakota recognizes several Union patterns to express one-of semantics. The DakotaBaseModel layer applies a model-level validator to enforce single-branch selection for Union fields represented as single-field wrapper models. The wrapper field name is the user-visible discriminator.
Example:
Accepted user input is a single-key dictionary matching a branch wrapper, for example {"linear": {...}} or {"none": true}.
Python shape: Union[OptionA,...] with a model default_factory.
Semantics: omission yields a meaningful default branch.
Python shape: Union[OptionA,...] | None with default=None.
Semantics: omission disables the feature.
Python shape: Union[OptionA,...,dict] | None with default_factory.
Semantics: omission => disabled; {} => enabled with defaults; branch object => enabled with explicit configuration.
Python shape: Union[OptionA,...].
Semantics: user must select a branch.
Python shape: Union[OptionA,...,dict] | None with default=None.
Semantics: omission => disabled; {} => implementation defaults; branch => explicit.
Python shape: Union[OptionA,...,dict].
Semantics: parent must exist; {} is a valid "no subselection" choice.
Union has multiple branches, exactly one branch must be selected. dict is a branch, empty-dictionary semantics depend on the pattern. None while {} instantiates from default_factory, preserving three states. union_pattern. model_default when the pattern specifies a default branch. default/default_factory, union_pattern, and model_default consistent. DakotaField extends pydantic.Field with a dakota dictionary that controls parser, schema, and materialization behavior. Standard Pydantic kwargs (default, default_factory, description, gt, ge, etc.) remain valid.
Common dakota keys:
aliases: list[str] or str for accepted alternative spellings. block_pointer: string naming the referenced block family/role. argument: child field treated as inline parameter. anchor: boolean True to attach branches to parent context. union_pattern: integer pattern id for one-of semantics. model_default: default branch name for patterns with defaults. internal_only: boolean to reject user input for the field. materialization: list[dict] describing IR writes and presence effects. Notes:
aliases only for backward compatibility. block_pointer when the keyword names another block. argument when a child field acts as the inline argument. anchor=True only when grammar requires attached discriminators. internal_only fields are rejected from user input by DakotaBaseModel. Example:
Materialization maps validated Pydantic JSON into Dakota's IR. The models describe valid input, dakota["materialization"] prescribes how input affects IR keys, and the IR codegen emits write-instruction tables.
Pipeline:
x-materialization metadata.dakota/src/ir_codegen read schema and produce IR contracts and write tables.ProblemDescDB.Key scripts:
dakota/src/ir_codegen/extract_defaults_from_schema.py dakota/src/ir_codegen/generate_ir_tables.py Materialization item shape:
ir_key: destination IR key.storage_type: write-handler identifier.ir_value_type: Dakota IR type (Real, size_t, StringArray, short, etc.).stored_value: optional literal for presence-driven writes.Common materialization cases: These are the storage_type values authors are most likely to use when adding or editing grammar.
storage_type |
expected fields | when to use |
|---|---|---|
DIRECT_VALUE |
ir_key, ir_value_type |
Write the field's validated value directly to the IR key. Use for 1:1 scalar or collection values. |
ADD_TO_VALUE |
ir_key, ir_value_type |
Add the field's value into an existing numeric aggregate instead of replacing it. Use for roll-up counters. |
PRESENCE_TRUE |
ir_key, ir_value_type |
Set the target IR key to boolean true when the field is present. Use for presence-driven boolean flags. |
PRESENCE_FALSE |
ir_key, ir_value_type |
Set the target IR key to boolean false when the field is present. Use for presence-driven negating flags. |
PRESENCE_LITERAL |
ir_key, ir_value_type, stored_value[, enum_scope] |
Write a literal stored_value when the field is present. Use when
presence selects a specific enumerated or literal runtime value. |
PRESENCE_ENUM |
ir_key, ir_value_type, stored_value[, enum_scope] |
Presence selects one of a set of enumerated IR values. Use when the field is a symbolic enumerant. |
AUGMENT_ENUM |
ir_key, ir_value_type, stored_value |
Augment an enum-like IR setting with an additional symbolic value. Use when presence should extend an enumerated configuration. |
TYPE_DATA_COMBINED |
ir_key, ir_value_type, stored_value, secondary_ir_key, secondary_ir_value_type |
Composite handler: primary literal selection plus secondary direct value write. Use when a symbolic choice and an argument are required. |
METHOD_PIECEWISE |
ir_key, ir_value_type, stored_value, secondary_ir_key, secondary_ir_value_type, secondary_literal_value |
Composite handler for piecewise method semantics; expands into primitive writes during codegen. Use for piecewise method configs. |
Specialized materialization handlers: These handlers are for established Dakota-specific data shapes. Reuse an existing model that already uses the handler as your template.
storage_type |
expected fields | when to use |
|---|---|---|
ANALYSIS_COMPONENTS |
ir_key, ir_value_type, handler-specific props |
TODO |
ADJACENCY_MATRIX |
ir_key, ir_value_type, handler-specific props |
TODO |
CATEGORICAL |
ir_key, ir_value_type, handler-specific props |
TODO |
CONTINUOUS_INTERVAL_UNCERTAIN |
ir_key, ir_value_type, handler-specific props |
TODO |
DISCRETE_INTERVAL_UNCERTAIN |
ir_key, ir_value_type, handler-specific props |
TODO |
DISCRETE_SET_VALUES |
ir_key, ir_value_type, handler-specific props |
TODO |
DISCRETE_UNCERTAIN_SET_VALUES_PROBS |
ir_key, ir_value_type, handler-specific props |
TODO |
HISTOGRAM_BIN_UNCERTAIN |
ir_key, ir_value_type, handler-specific props |
TODO |
HISTOGRAM_POINT_UNCERTAIN |
ir_key, ir_value_type, handler-specific props |
TODO |
ID_TO_INDEX_SET |
ir_key, ir_value_type, handler-specific props |
TODO |
INT_SET |
ir_key, ir_value_type, handler-specific props |
TODO |
RESPONSE_LEVELS_ARRAY |
ir_key, ir_value_type, handler-specific props |
TODO |
UNCERTAIN_CORRELATION_MATRIX |
ir_key, ir_value_type, handler-specific props |
TODO |
UNCERTAIN_INIT_POINT_FLAG |
ir_key, ir_value_type, handler-specific props |
TODO |
For domain-specific handlers marked TODO, consult models under dakota/python/dakota/spec that already use the handler. Those handlers typically require additional, handler-specific metadata beyond the generic ir_key / ir_value_type shape.
IR defaults ensure a complete runtime IR even when users omit inputs. The generator merges three disjoint sources:
dakota/src/schema_defaults_extracted.json;dakota/src/default_overrides_registry.json; anddakota/src/default_policy_registry.json.The generator reads these via load_schema_defaults(), load_overrides(), and load_policies() and merges them into per-block IR contracts. Place a default in the Pydantic model when it is a grammar-level default. Use default_overrides_registry.json when the default is implementation- specific or not representable in schema. Use the policy registry for ordinary type zero/false/empty defaults. Ensure only one source claims an IR key; collisions cause generate_ir_tables.py to error.
Follow the selection-wrapper pattern when adding a method or model family.
Steps:
dakota/python/dakota/spec/method or dakota/python/dakota/spec/model.DakotaField.MethodSelection or ModelSelection. The wrapper must contain exactly one field named with the user-visible keyword.init.py so it becomes part of the union assembly (MethodUnion or ModelUnion).Minimal example:
Validation rules and computed fields constitute part of the shared grammar contract and require coordinated Python and C++ changes. The Python side provides schema metadata and dispatch; the authoritative execution logic resides in the C++ dakota_validation module.
To add a ValidationRule:
ValidationRule subclass in dakota/python/dakota/spec/validation.super().__init__ with rule name, context, referenced fields, literals, and error message.call(self,\ instance) to dispatch to C++ via _call_cpp_validator.packages/dakota_parser/dakota_validation.ValidatorRegistry::ValidatorRegistry().bindings.cpp.packages/dakota_parser/dakota_validation/test._VALIDATION_RULES.Validation call contract:
instance: model serialized to dict.fields: paths from validationFields.literals: from validationLiterals.context: descriptive string.dict of mutations or empty dict for check-only rules.To add a computed field:
_COMPUTED_FIELDS:\ ClassVar[dict[str,\ ComputedFieldSpec]] on the model.@computed_field property that computes the value.computed_fields.hpp and source.ComputedFieldRegistry::ComputedFieldRegistry().Example:
DakotaBaseModel.__init_subclass__ detects _VALIDATION_RULES and _COMPUTED_FIELDS and exports them as x-model-validations and x-computed-fields in schema. Keep Python property names, ComputedFieldSpec function strings, and C++ registry entries in exact agreement.
Block pointers are string-valued keywords whose arguments name other blocks in a Dakota study. For example, a method block may contain a model_pointer keyword whose value is the id_model of a model block. In a freeform .in input file, these pointers define the study graph — they are how the user wires methods, models, variables, interfaces, and responses together.
Dakota's Pydantic models serve two consumers:
validate_json_input_to_json, read_json_input_file_to_json). In this mode, pointer keywords that the schema marks as required must be present because the study graph cannot be constructed without them. validate_*_block_json_to_json) let users construct individual components and assemble the study graph via dependency injection. Pointer keywords are meaningless in this context because the graph is defined programmatically, not by string references. Because we maintain a single set of Pydantic models for both consumers, validation must distinguish between the two modes so that required pointers are enforced in input-file mode but relaxed in API mode.
In Python, the default behavior is API mode. A model_validator on DakotaBaseModel (_fill_api_mode_pointer_sentinels) inspects every field that has an x-block-pointer annotation. When such a required field is missing from the user's input dict, the validator silently injects a sentinel value (POINTER_SENTINEL) so that Pydantic's normal required-field check passes. Optional pointer fields need no sentinel — Pydantic accepts their absence already.
To switch to input-file mode (where all required pointers are enforced), wrap the validation call in the input_file_mode context manager:
This is a Python contextvars-based context manager, so it is thread-safe and re-entrant.
The C++ JSON validation path threads a bool api_mode parameter through validate_json_document, walk_and_validate, and validate_definition. When api_mode is true, the required-field check in validate_definition skips any field that the generated metadata identifies as a block pointer.
The distinction is wired at the entry-point level:
validate_json_input_to_json, read_json_input_file_to_json, and all freeform read_*_to_json functions) use api_mode=false. All required fields including pointers must be present. validate_method_block_json_to_json, validate_model_block_json_to_json, etc.) use api_mode=true. Required pointer fields are silently skipped; all other required fields are still enforced. The api_mode parameter is threaded as a function argument (not a global variable) so the validation path is thread-safe and re-entrant.
A field is recognized as a block pointer when it carries x-block-pointer metadata in the exported JSON schema. In the Pydantic model, this is specified in the dakota dict of DakotaField:
The block_pointer key is mapped to x-block-pointer in the exported JSON schema. The code generator reads this annotation to build the set of pointer fields per definition, and the runtime required-field check consults it when api_mode is true.
Do not add child keywords (sub-keywords) to a pointer field.
In API mode, required pointer fields may be omitted entirely. If a pointer field carried child configuration keywords, that configuration would become inaccessible to API users who (correctly) omit the pointer. The result would be features that are configurable from a freeform input file but unreachable from the Python or C++ API.
If a pointer currently has associated sub-keywords, or a design calls for configuration that logically accompanies a pointer, factor the configuration into a sibling field or a separate model. The pointer should remain a plain str (or list[str]) with no nested structure.
Example of what NOT to do:
Instead, make the option a sibling field on the parent model:
Some required model-typed fields wrap a config model that contains required pointer fields plus optional configuration siblings. These are called pointer groups. A typical example is NestedConfig.sub_method_pointer, whose type SubMethodPointer holds a required pointer: str block-pointer field and optional scheduling fields such as iterator_servers.
When to use: A required field is a pointer group when its model contains required block_pointer fields (and optionally other fields with defaults) but the model as a whole must be omittable in API mode.
API-mode semantics: If the pointer-group field is absent, the _fill_api_mode_pointer_sentinels validator injects an empty dict {}. The child model's own before-validator then fills in the pointer sentinel. If the field is present (because the user wants to configure optional fields like iterator_servers), no injection occurs and validation proceeds normally.
Annotation: Add "pointer_group": True to the dakota dict of the DakotaField on the parent model's field:
Origin: In freeform input, these correspond to keywords that take a string argument (the pointer) and also have optional sub-keywords. The Pydantic model splits this into a synthetic pointer field (annotated block_pointer) and sibling config fields. The parent field on the containing model is annotated pointer_group so that API-mode injection can reach it.
Do not annotate a field pointer_group if its wrapped model has required non-pointer fields (e.g., a required type-selection bool or a required method-name string). Such models require real user input and cannot be meaningfully auto-filled with an empty dict.
Some required Union fields have at least one branch that is a pointer-only or pointer-group wrapper. These are called pointer unions. A typical example is EmbeddedConfig.sub_method : Union[GlobalMethodName, GlobalMethodPointer], where GlobalMethodPointer is a pure pointer branch and GlobalMethodName has a real method-name string.
When to use: A required Union field is a pointer union when at least one of its branches is all-pointer (i.e., every required field on the branch model is either a block_pointer, a pointer_group, or another pointer_union).
API-mode semantics: If the pointer-union field is absent, the validator introspects the Union branches to find the first all-pointer branch. It then builds a minimal injection dict for that branch (injecting the pointer sentinel into the branch's pointer fields) and injects it. Child validators fill in sentinels bottom-up. If the user provides a branch (e.g., a method-name branch), that is real input and validates normally.
Annotation: Add "pointer_union": True to the dakota dict of the DakotaField on the Union field:
Defensive check: If no all-pointer branch is found, a ValueError is raised during validation with a message like: "pointer_union field 'sub_method' on EmbeddedConfig has no all-pointer branch". This catches developer annotation mistakes at validation time rather than silently misbehaving.
Do not annotate SurrogateConfig.category or other real type-selection Unions as pointer_union. Those Unions require the user to always pick a branch; they are not pointer plumbing.
The full pointer-relaxation stack supports three levels of nesting:
pointer_union -> pointer_group -> block_pointer
Deeper than three levels is administratively prohibited. If a future design requires deeper nesting, the developer must find a flatter representation. Deeply nested option hierarchies were tolerable for freeform input files but create ergonomic problems for API users and make automated injection brittle.
Guidance for new grammar: Favor broad, flat specifications. Prefer sibling fields over deeply nested option sub-models. When a pointer and optional configuration must coexist, use the pointer-group pattern (pointer as a sibling of the configuration, not as a child of it).
Use DakotaFloat for fields that may carry non-finite values (NaN, inf, -inf). DakotaFloat is an Annotated float with a custom serializer:
This preserves compatibility between Python JSON and downstream C++ handling. Prefer DakotaFloat for fields that historically used sentinel floating values (e.g., DBL_MAX or -DBL_MAX).