.. _model-surrogate:

"""""""""
surrogate
"""""""""


An empirical model that is created from data or the results of a submodel



.. toctree::
   :hidden:
   :maxdepth: 1

   model-surrogate-id_surrogates
   model-surrogate-global
   model-surrogate-multipoint
   model-surrogate-local
   model-surrogate-ensemble


**Specification**

- *Alias:* None

- *Arguments:* None


**Child Keywords:**

+-------------------------+--------------------+--------------------+-----------------------------------------------+
| Required/Optional       | Description of     | Dakota Keyword     | Dakota Keyword Description                    |
|                         | Group              |                    |                                               |
+=========================+====================+====================+===============================================+
| Optional                                     | `id_surrogates`__  | Identifies the subset of the response         |
|                                              |                    | functions by number that are to be            |
|                                              |                    | approximated (the default is all functions).  |
+-------------------------+--------------------+--------------------+-----------------------------------------------+
| Required (Choose One)   | Surrogate Category | `global`__         | Select a surrogate model with global support  |
|                         |                    +--------------------+-----------------------------------------------+
|                         |                    | `multipoint`__     | Construct a surrogate from multiple existing  |
|                         |                    |                    | training points                               |
|                         |                    +--------------------+-----------------------------------------------+
|                         |                    | `local`__          | Build a locally accurate surrogate from data  |
|                         |                    |                    | at a single point                             |
|                         |                    +--------------------+-----------------------------------------------+
|                         |                    | `ensemble`__       | Ensemble surrogates employ a collection of    |
|                         |                    |                    | lower-fidelity models to approximate a truth  |
|                         |                    |                    | reference model at reduced cost.              |
+-------------------------+--------------------+--------------------+-----------------------------------------------+

.. __: model-surrogate-id_surrogates.html
__ model-surrogate-global.html
__ model-surrogate-multipoint.html
__ model-surrogate-local.html
__ model-surrogate-ensemble.html



**Description**


Surrogate models are inexpensive approximate models that are intended
to capture the salient features of an expensive high-fidelity model.
They can be used to explore the variations in response quantities over
regions of the parameter space, or they can serve as inexpensive
stand-ins for optimization or uncertainty quantification studies (see,
for example, the surrogate-based optimization methods,
:dakkw:`method-surrogate_based_global` and :dakkw:`method-surrogate_based_local`).

Surrogate models supported in Dakota are
categorized as Data Fitting or Ensemble, as shown below.
Each of these surrogate types provides an approximate representation
of a "truth" model which is used to perform the parameter to response
mappings. This approximation is built and updated using results from the
truth model, called the "training data".



- Data fits: *Known Issue: When using discrete variables, there have been sometimes significant differences in surrogate behavior observed across computing platforms in some cases.  The cause has not yet been fully diagnosed and is currently under investigation.  In addition, guidance on appropriate construction and use of surrogates with discrete variables is under development.  In the meantime, users should therefore be aware that there is a risk of inaccurate results when using surrogates with discrete variables.* Data fitting methods involve construction of an approximation or surrogate model using data (response values, gradients, and Hessians) generated from the original truth model. Data fit methods can be further categorized as local, multipoint, and global approximation techniques, based on the number of points used in generating the data fit.


  1. Local: built from response data from a single point in parameter space


    - Taylor series expansion: :dakkw:`model-surrogate-local-taylor_series` Training data consists of a single point, plus gradient and Hessian information.


  2. Multipoint:  built from two or more points in parameter space, often involving the current and previous iterates of a minimization algorithm.


    - TANA-3: :dakkw:`model-surrogate-multipoint-tana` Training Data comes from a few previously evaluated points


  3. Global full space response surface methods:


    - Polynomial regression: :dakkw:`model-surrogate-global-polynomial`

    - Gaussian process (Kriging): :dakkw:`model-surrogate-global-gaussian_process`

    - Artifical neutral network: :dakkw:`model-surrogate-global-neural_network`

    - MARS: :dakkw:`model-surrogate-global-mars`

    - Radial Basis Functions: :dakkw:`model-surrogate-global-radial_basis`

    - Orthogonal polynomials (only supported in PCE/SC for now): :dakkw:`method-polynomial_chaos` and :dakkw:`method-stoch_collocation` Training data is generated using either a design of experiments method applied to the truth model ( specified by :dakkw:`model-surrogate-global-dace_method_pointer`), or from saved data (specified by :dakkw:`model-surrogate-global-reuse_points` ) in a restart database, or an import file.


- Multifidelity/ensemble: Multifidelity modeling involves the use of a low-fidelity physics-based model as a surrogate for the original high-fidelity model. The low-fidelity model typically involves a coarser mesh, looser convergence tolerances, reduced element order, or omitted physics. See :dakkw:`model-surrogate-ensemble`.

The global and ensemble surrogates have a correction feature
in order to improve the local accuracy of the surrogate models.
The correction factors force the surrogate models to match the true
function values and possibly true function derivatives at the center
point of each trust region.

Details can be found on global :dakkw:`model-surrogate-global-correction`
or ensemble :dakkw:`model-surrogate-ensemble-ordered_model_fidelities-correction`.



**Theory**


Surrogate models are used extensively in the
surrogate-based optimization and least squares methods, in
which the goals are to reduce expense by minimizing the number of
truth function evaluations and to smooth out noisy data with a global
data fit. However, the use of surrogate models is not restricted to
optimization techniques; uncertainty quantification and optimization
under uncertainty methods are other primary users.

*Data Fit Surrogate Models*

A surrogate of the {\em data fit} type is a non-physics-based
approximation typically involving interpolation or regression of a set
of data generated from the original model. Data fit surrogates can be
further characterized by the number of data points used in the fit,
where a local approximation (e.g., first or second-order Taylor
series) uses data from a single point, a multipoint approximation
(e.g., two-point exponential approximations (TPEA) or two-point
adaptive nonlinearity approximations (TANA)) uses a small number of
data points often drawn from the previous iterates of a particular
algorithm, and a global approximation (e.g., polynomial response
surfaces, kriging/gaussian_process, neural networks, radial basis
functions, splines)
uses a set of data points distributed over the domain of interest,
often generated using a design of computer experiments.

Dakota contains several types of surface fitting methods that can be
used with optimization and uncertainty quantification methods and
strategies such as surrogate-based optimization and optimization under
uncertainty. These are: polynomial models (linear, quadratic, and
cubic), first-order Taylor series expansion, kriging spatial
interpolation, artificial neural networks, multivariate adaptive
regression splines, radial basis functions, and moving least squares.
With the exception of Taylor series methods, all of the above methods
listed in the previous sentence are accessed in Dakota through the
Surfpack library. All of these surface fitting methods can be
applied to problems having an arbitrary number of design parameters.
However, surface fitting methods usually are practical only for
problems where there are a small number of parameters (e.g., a maximum
of somewhere in the range of 30-50 design parameters). The
mathematical models created by surface fitting methods have a variety
of names in the engineering community. These include surrogate models,
meta-models, approximation models, and response surfaces. For this
manual, the terms surface fit model and surrogate model are used.

The data fitting methods in Dakota include software developed by
Sandia researchers and by various researchers in the academic
community.



*Multifidelity Surrogate Models*

A second type of surrogate is the {\em model ensemble} type (also
called multifidelity, variable fidelity, variable complexity, etc.).
In this case, one or more models that are still physics-based but are
of lower fidelity (e.g., coarser discretization, reduced element
order, looser convergence tolerances, omitted physics) are used as
surrogates in place of the high-fidelity model. For example, an
inviscid, incompressible Euler CFD model on a coarse discretization
could be used as a low-fidelity surrogate for a high-fidelity
large-eddy simulation model on a fine discretization.

*Surrogate Model Selection*

This section offers some guidance on choosing from among the available
surrogate model types.

 - For Surrogate Based Local Optimization, using the :dakkw:`method-surrogate_based_local` method with a trust region: using the keywords: :dakkw:`model-surrogate` :dakkw:`model-surrogate-local` :dakkw:`model-surrogate-local-taylor_series` or :dakkw:`model-surrogate` :dakkw:`model-surrogate-multipoint` :dakkw:`model-surrogate-multipoint-tana` will probably work best. If for some reason you wish or need to use a global surrogate (not recommended) then the best of these options is likely to be either: :dakkw:`model-surrogate` :dakkw:`model-surrogate-global` :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` or :dakkw:`model-surrogate` :dakkw:`model-surrogate-global` :dakkw:`model-surrogate-global-moving_least_squares`.
 - For Efficient Global Optimization (EGO), the :dakkw:`method-efficient_global` method: the default surrogate is: :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` which is likely to find a more optimal value and/or require fewer true function evaluations than the alternative, :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-dakota`. However, the :dakkw:`model-surrogate-global-gaussian_process-surfpack` will likely take more time to build than the ``dakota`` version. Note that currently the :dakkw:`method-efficient_global-use_derivatives` keyword is not recommended for use with EGO based methods.
 - For EGO based global interval estimation, the :dakkw:`method-global_interval_est` :dakkw:`method-global_interval_est-ego` method: the default :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` will likely work better than the alternative :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-dakota`.
 - For Efficient Global Reliability Analysis (EGRA), the :dakkw:`method-global_reliability` method: the :dakkw:`model-surrogate-global-gaussian_process-surfpack` and ``dakota`` versions of the gaussian process tend to give similar answers with the ``dakota`` version tending to use fewer true function evaluations. Since this is based on EGO, it is likely that the default :dakkw:`model-surrogate-global-gaussian_process-surfpack` is more accurate, although this has not been rigorously demonstrated.
 - For EGO based Dempster-Shafer Theory of Evidence, i.e. the :dakkw:`method-global_evidence` :dakkw:`method-global_evidence-ego` method, the default :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` often use significantly fewer true function evaluations than the alternative :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-dakota`.
 - When using a global surrogate to extrapolate, any of the surrogates:

   - :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack`
   - :dakkw:`model-surrogate-global-polynomial` :dakkw:`model-surrogate-global-polynomial-quadratic`
   - :dakkw:`model-surrogate-global-polynomial` :dakkw:`model-surrogate-global-polynomial-cubic` are recommended.

 - When there is over roughly two or three thousand data points and you wish to interpolate (or approximately interpolate) then a Taylor series, Radial Basis Function Network, or Moving Least Squares fit is recommended. The only reason that the :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` is not recommended is that it can take a considerable amount of time to construct when the number of data points is very large. Use of the third party MARS package included in Dakota is generally discouraged.
 - In other situations that call for a global surrogate, the :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` is generally recommended. The ``use_derivatives`` keyword will only be useful if accurate and inexpensive derivatives are available. Finite difference derivatives are disqualified on both counts. However, derivatives generated by analytical, automatic differentiation, or continuous adjoint techniques can be appropriate. Currently, first order derivatives, i.e. gradients, are the highest order derivatives that can be used to construct the :dakkw:`model-surrogate-global-gaussian_process` :dakkw:`model-surrogate-global-gaussian_process-surfpack` model; Hessians will not be used even if they are available.