Dakota  Version 7.0
Explore and Predict with Confidence
Adding a Python Binding for a Dakota Class

Python Binding Table of Contents

Overview

The dependency-injection Python API is implemented by the pybind11 module dakota.study._study. Its C++ sources are in dakota/src/python/study, its Python package is in dakota/python/dakota/study, and its unit tests are in dakota/src/unit.

Adding a class normally involves two distinct bindings:

  1. Register the C++ type so Python can hold it, recognize its inheritance, and call any intentionally exposed instance methods.
  2. If Python users should construct the class, add a factory method to Study::MethodFactory or Study::ModelFactory. The factory accepts already-constructed semantic dependencies plus a validated configuration fragment.

Do not expose a ProblemDescDB constructor through this API. New library construction uses a component's IRStore constructor and a shared StudyServices instance. Child models, iterators, interfaces, variables, and responses are passed as objects rather than recovered from pointer names.

The normal configuration flow is:

dict, keyword arguments, or Pydantic model
        |
        v
normalize_factory_config()       Python validation, exactly once
        |
        v
materialize_{method,model,...}()  validated JSON -> typed IRStore
        |
        v
Study factory / DI constructor    injected dependencies + shared services

C++ prerequisites

Before writing the Python binding, make sure the C++ library API is complete. The class should have an IR-based constructor whose shape follows the nearby component hierarchy. For example, an iterator that consumes a model normally accepts:

NewIterator(const IRStore& method_store,
std::shared_ptr<Model> model,
std::shared_ptr<StudyServices> services);

Some established hierarchies place StudyServices first. Preserve the class's C++ convention; the binding can adapt the ordering without exposing it to Python.

The constructor must:

  • read its own configuration from IRStore;
  • accept semantic child objects directly;
  • reject null required children;
  • validate shared services with detail::validate_services(); and
  • preserve the existing ProblemDescDB construction path.

Validated IR contains every declared key, including defaults. Read required keys with store.get<T>("key") directly. Do not guard reads with contains(), supply ad hoc fallback values, or infer missing services from a child. A missing IR key is a contract error and should remain visible.

Add C++ construction coverage in src/unit/dakota_di_construction/di_construction.cpp before or with the Python binding. Test successful construction, service identity, inconsistent services, and null required dependencies.

Register the class

Put study binding code in the file matching the component family:

  • BindIterator.cpp for iterators;
  • BindModel.cpp for models;
  • BindVariables.cpp, BindResponse.cpp, or BindInterface.cpp for the corresponding value or interface type; and
  • BindStudyConfig.cpp for study-level configuration.

Register base classes before derived classes. Components managed with std::shared_ptr in C++ must use the same holder in pybind11:

py::class_<NewIterator, Iterator, std::shared_ptr<NewIterator>>(
m, "NewIterator", py::module_local());

The base type must already be registered:

py::class_<Iterator, std::shared_ptr<Iterator>>(
m, "Iterator", py::module_local());

Use py::module_local() consistently with the existing _study bindings. Expose only useful public behavior. Prefer a small lambda when a direct member pointer is ambiguous, overloaded, or would expose an internal container:

.def("num_responses",
[](const NewIterator& iterator) {
return iterator.all_responses().size();
});

Do not bind raw pointers or return references whose owner may be destroyed. Use std::shared_ptr holders and pybind11 lifetime policies where ownership is not already carried by the returned object.

Expose construction through Study

Most methods and models should be constructed through a Study factory so all objects share the same StudyServices. Add the C++ factory first in Study.hpp and Study.cpp. Its IR overload should perform construction; the JSON overload should only validate/materialize and delegate.

For a method with one injected model, the Python signature should place the semantic dependency first and configuration last:

iterator = study.method.new_method(model, config)
iterator = study.method.new_method(model, samples=20)

Bind the factory on Study::MethodFactory:

factory.def("new_method",
[](const Study::MethodFactory& self,
std::shared_ptr<Model> model,
const py::object& config, py::kwargs kwargs) {
const auto fragment = normalize_factory_config(
config, kwargs, "MethodFactory.new_method");
const auto store = materialize_method(
nlohmann::json{{"new_method", fragment}});
return std::make_shared<NewIterator>(
store, std::move(model), self.services());
},
py::arg("model").none(false), py::arg("config") = py::none());

BindIterator.cpp provides bind_method() and construct<T>() helpers for the common one-model iterator case. Use those helpers when the class fits their contract. Write an explicit binding when dependencies or constructor semantics differ.

For models, follow the explicit examples in BindStudy.cpp. Keep required dependencies non-null with py::arg("name").none(false). Check elements of injected vectors when pybind11 cannot enforce non-null entries itself.

Do not resolve child components from pointer strings in the configuration. Pointer fields may still be materialized because the shared input grammar uses them, but injected C++ objects are the semantic dependencies in library mode.

Validate and materialize configuration

The Python API accepts one of three exclusive configuration forms:

factory(model, config={"samples": 20})
factory(model, config=SamplingConfig(samples=20))
factory(model, samples=20)

Do not accept both config and configuration keyword arguments. Route all three forms through normalize_factory_config() in python/dakota/study/_validation.py. This function validates dictionaries, trusts an instance of the expected Pydantic class, serializes to JSON form, and prevents accidental revalidation.

Add the factory-to-schema mapping to _FACTORY_CONFIG_CLASSES:

"MethodFactory.new_method": method.NewMethodConfig,

If the class is selected by the generated method grammar, confirm that the generic registration loop already discovers it before adding a manual entry. The schema models and their materialization metadata are authoritative; do not reimplement schema defaults or selector rules in the binding. See Pydantic Input Grammar: Complete Reference and Procedure for grammar and materialization changes.

After normalization, wrap the fragment with its selection keyword before materialization:

materialize_method(nlohmann::json{{"new_method", fragment}})

Use materialize_model(), materialize_variables(), materialize_interface(), or materialize_responses() for other block types. Materialization happens after Python validation; it is not a second user-input validation layer.

Literal selectors use Boolean true, whereas configurable selectors use an object. For example, a field declared as Literal[True] is represented by {"quadratic": true}, not {"quadratic": {}}. Consult the actual Pydantic annotation rather than copying a superficially similar selector.

Register and package the binding

Each binding file exposes one registration function declared in DakotaStudyPython.hpp and called from the PYBIND11_MODULE block in DakotaStudyPython.cpp. If adding a new binding file or registration group:

  1. Declare bind_new_classes(py::module_& m) in DakotaStudyPython.hpp.
  2. Implement it in src/python/study/BindNewClasses.cpp.
  3. Add that source file to dakota_study_python_src in src/CMakeLists.txt.
  4. Call bind_new_classes(m) from PYBIND11_MODULE(_study, m), after any base classes it depends on have been registered.

For a class added to an existing binding group, only the existing group function normally changes.

Export public names from python/dakota/study/__init__.py. Unconditional classes can appear in the main import list. Classes controlled by optional libraries must be imported conditionally or exported through the existing iterator manifest so importing dakota.study still works when the feature is disabled.

The _study target is placed directly in the build-tree dakota/study package and installed by src/CMakeLists.txt. Do not add a second extension module solely for one class.

Optional dependencies

Match the feature guard used by the C++ implementation in all relevant places:

  • the class header include;
  • the py::class_ registration;
  • the factory binding;
  • the public Python export; and
  • tests that assert availability.

For example, a class requiring DOT must remain inside HAVE_DOT guards. Tests should skip or test absence when the dependency is unavailable; they must not make importing the package fail.

Add unit tests

Place Python unit tests in src/unit and use the standard-library unittest framework. Register them in src/unit/CMakeLists.txt with the Unit and Python labels and the build-tree PYTHONPATH. Do not place Dakota unit tests under python/tests.

At minimum, test:

  • the class is exported and has the expected base type;
  • dict, Pydantic-model, keyword, and supported positional configuration forms construct the same component type;
  • configuration inputs are not mutated;
  • supplying both config and keyword configuration fails;
  • unknown fields and wrong config types fail during validation;
  • required semantic dependencies reject None and wrong Python types;
  • optional dependencies behave correctly when omitted; and
  • the constructed object can execute a small meaningful operation when the required driver is available.

When calling model_validate() inside a test, pass a deep copy if generated before-validators may transform nested dictionaries. Otherwise the test can mutate its own fixture before the factory is called and incorrectly attribute that mutation to production code.

Run the focused tests through CTest so they use the same extension module and environment as CI. Typical commands are:

ctest --test-dir build -R dakota_python_study --output-on-failure
ctest --test-dir build -R dakota_di_construction --output-on-failure

Also run the relevant configuration-normalization tests directly with python -m unittest when the build-tree package is available.

Review checklist

Before submitting a new binding, verify:

  • The C++ class has a tested IR/DI constructor.
  • The binding uses the correct base class and std::shared_ptr holder.
  • Base classes are registered before derived classes.
  • Construction goes through the owning Study factory and shared services.
  • Semantic dependencies are injected as objects, not reconstructed from pointer names.
  • Configuration is validated once, then materialized to the correct block type.
  • Required IR keys are read directly without contains() fallbacks.
  • Python argument order is dependencies first and configuration last.
  • Optional build guards are consistent across C++, Python exports, and tests.
  • Tests live in src/unit, use unittest, and are registered with CTest.
  • Both Python and C++ DI construction tests pass.