.. _input:
Inputs to Dakota
================
.. _`input:overview`:
Overview of Inputs
------------------
Dakota supports a number of command-line arguments, as described in
:ref:`command-line-main`. Among these are specifications
for the Dakota input file and, optionally, a restart file. Dakota
input file syntax is described in the adjoining sections in
:ref:`inputfile-main`, with detailed keyword syntax in
:ref:`keyword-reference-area`. The restart file is described in
:ref:`dakota_restart`.
A Dakota input file may be prepared with a text editor such as Emacs,
Vi, or WordPad, or with the Dakota graphical user interface. The
Dakota GUI is built on the Java-based Eclipse Framework
:cite:p:`Eclipse` and presents the Dakota input specification options
in either a text editor view or a graphical view. It further includes
templates and wizards for helping create Dakota studies and can invoke
Dakota to run an analysis. Dakota GUI downloads for Linux, Windows,
and Mac, are available from the Dakota website
http://dakota.sandia.gov/, along with licensing information and
installation tips. See :ref:`dakota-gui-main` for more documentation.
.. _`input:tabularformat`:
Tabular Data Formats
~~~~~~~~~~~~~~~~~~~~
The Dakota input file and/or command line may identify additional text
files for tabular data import in contexts described in
:ref:`input:import`. Examples include data from which to
build a surrogate, points at which to run a list parameter study,
post-run input data, and least squares and Bayesian calibration data.
Dakota writes and reads tabular data with C++ stream
operators/conversions, so most integer and floating point formats are
acceptable for imported numeric data. Dakota supports the following
tabular formats:
- **Annotated:** In most contexts, Dakota tabular data defaults to
"annotated" tabular format. An annotated tabular file is a
whitespace-separated text file with one leading header row of
comments/column labels. In most imports/exports, each subsequent row
contains an evaluation ID and interface ID, followed by data for
variables, or variables followed by responses, depending on context.
This example shows 5 variables, followed by the 1 text_book
response:
::
%eval_id interface TF1ln TF1ln hpu_r1 hpu_r2 ModelForm text_book
1 I1 0.97399 1.0476 12 4.133 3 14753
2 I1 0.94468 1.0636 4.133 12 3 14753
3 I1 1.0279 1.0035 12 4.133 3 14753
Another example is shown in :numref:`output:tabcont`.
.. note::
Dakota 6.1 and newer include a column for the interface ID. See the
discussion of custom-annotated format below for importing/exporting
Dakota 6.0 format files.
For scalar experiment data files, each subsequent row contains an
experiment ID, followed by data for configuration variables,
observations, and/or observation errors, depending on context. This
example shows 3 data points for each of two experiments.
::
%experiment d1 d2 d3
1 82 15.5 2.02
2 82.2 15.45 2
- **Free-form:** When optionally specifying ``freeform`` for a given
tabular import, the data file must be provided in a free-form
format, omitting the leading header row and ID column(s). The raw
num_rows x num_cols numeric data entries may appear separated with
any whitespace including arbitrary spaces, tabs, and newlines. In
this format, vectors may therefore appear as a single row or single
column (or mixture; entries will populate the vector in order). This
example shows the free-form version of the annotated data above:
::
0.97399 1.0476 12 4.133 3 14753
0.94468 1.0636 4.133 12 3 14753
1.0279 1.0035 12 4.133 3 14753
- **Custom-annotated:** In Dakota 6.2 and newer, a custom-annotated
format is supported, to allow backward-compatibility with Dakota 6.0
tabular formats, which had a header and evaluation ID, but no
interface ID. This can be specified, for example, with
::
method
list_parameter_study
import_points_file = 'dakota_pstudy.3.dat'
custom_annotated header eval_id
The ``custom_annotated`` keyword has options to control ``header``
row, ``eval_id`` column, and ``interface_id`` column.
In tabular files, variables appear in input specification order as
documented in the reference manual. As of Dakota 6.1, tabular I/O has
columns for all of the variables (active and inactive), not only the
active variables as in previous versions. To import data corresponding
only to the active variables, use the keyword ``active_only`` when specifying the import file.
.. note::
Prior to October 2011, samples, calibration, and surrogate data
files were free-form format. They now default to annotated format,
though there are ``freeform`` and ``custom_annotated`` options. For
both formats, a warning will be generated if a specific number of
data are expected, but extra is found and an error generated when
there is insufficient data. Some TPLs like SCOLIB and JEGA manage
their own file I/O and only support the free-form option.
.. _`input:import`:
Data Imports
------------
The Dakota input file and/or command line may identify additional files
used to import data into Dakota.
AMPL algebraic mappings
~~~~~~~~~~~~~~~~~~~~~~~
As described in :ref:`advint:algebraic`, an AMPL specification of
algebraic input-to-output relationships may be imported into Dakota
and used to define or augment the mappings of a particular
interface. The files :file:`stub.nl`, :file:`stub.row`, and
:file:`stub.col` define the mapping.
Genetic algorithm population import
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Genetic algorithms (GAs) from the JEGA and SCOLIB packages support a
population import feature using the keywords ``initialization_type
flat_file = STRING``. This is useful for warm starting GAs from
available data or previous runs. Refer to the ``flat_file`` keywords
in the :dakkw:`method` reference. The flat file must be in free-form
format.
.. _`input:calib_data`:
Calibration data import
~~~~~~~~~~~~~~~~~~~~~~~
Calibration methods (deterministic least squares and Bayesian) require
residuals, or differences between model predictions
:math:`\mathbf{q}(\mathbf{\theta})` and data :math:`\mathbf{d}`:
.. math::
\mathbf{r}(\mathbf{\theta}) =
\mathbf{q}(\mathbf{\theta}) - \mathbf{d},
By default, if a Dakota input file specifies
:ref:`responses-calibration_terms <responses-calibration_terms>`, the
simulation interface is required to return a vector of residuals
:math:`\mathbf{r}` to Dakota. If in addition the input file includes
:dakkw:`responses-calibration_terms-calibration_data` or
:dakkw:`responses-calibration_terms-calibration_data_file`, Dakota
assumes the interface will return the model predictions
:math:`\mathbf{q}(\mathbf{\theta})` themselves and Dakota will compute
residuals by differencing with the provided data.
There are two calibration data import mechanisms:
#. Scalar responses only with
:dakkw:`responses-calibration_terms-calibration_data_file`: This
uses a single tabular text file to import data values and
(optionally) experiment numbers, configurations, and observation
variances. Each row of the data file expresses this information for
a single experiment.
#. Field and/or scalar responses with
:dakkw:`responses-calibration_terms-calibration_data`: In order to
accommodate the richer structure of field-valued responses, this
specification requires separate data files per response group
(descriptor) ``DESC``, per experiment ``NUM``. The files are named
``DESC.NUM.*`` and must each be in a tabular text format.
The tabular data files may be specified to be ``annotated`` (default),
``custom_annotated``, or ``freeform`` format.
Calibration data imports include the following information:
- **Configuration variables (optional):** state variable values
indicating the configuration at which this experiment was conducted;
length must agree with the number of state variables active in the
study.
.. warning::
In versions of Dakota prior to 6.14, string-valued configuration
variables were specified in data files with 0-based indices into
the admissible values. As of Dakota 6.14, strings must be
specified by value. For example a string-valued configuration
variable for an experimental condition might appear in the file
as ``low_pressure`` vs. ``high_pressure``.
- **Experimental observations (required):** experimental data values to
difference with model responses; length equal to the total response
length (number of scalars + sum(field lengths)).
- **Experimental variances (optional):** measurement errors
(variances/covariances) associated with the experimental observations
For more on specifying calibration data imports, see :ref:`the
nonlinear least squares examples <nls:examples>` and the reference
documentation for :dakkw:`responses-calibration_terms`.
**Note on variance:** Field responses may optionally have scalar,
diagonal, or matrix-valued error covariance information. As an
example, :numref:`fig:input:obs_err_cov` shows an observation vector
with 5 responses; 2 scalar + 3 field (each field of length >1). The
corresponding covariance matrix has scalar variances
:math:`\sigma_1^2, \sigma_2^2` for each of the scalars :math:`s1, s2`,
diagonal covariance :math:`D_3` for field :math:`f3`, scalar
covariance :math:`\sigma_4^2` for field :math:`f4`, and full matrix
covariance :math:`C_5` for field :math:`f5`. In total, Dakota supports
block diagonal covariance :math:`\Sigma` across the responses, with
blocks :math:`\Sigma_i`, which could be fully dense within a given
field response group. Covariance across the highest-level responses
(off-diagonal blocks) is not supported, nor is covariance between
experiments.
.. figure:: img/ObsErrorCovariance.png
:alt: An example of scalar and field response data, with associated
block-diagonal observation error covariance.
:name: fig:input:obs_err_cov
An example of scalar and field response data, with associated
block-diagonal observation error covariance.
PCE coefficient import
~~~~~~~~~~~~~~~~~~~~~~
Polynomial chaos expansion (PCE) methods compute coefficients for
response expansions which employ a basis of multivariate orthogonal
polynomials. Normally, the :dakkw:`method-polynomial_chaos` method
calculates these coefficients based either on a spectral projection or
a linear regression (see :ref:`uq:expansion`). However, Dakota also
supports the option of importing a set of response PCE coefficients
from a file specified with ``import_expansion_file = STRING``. Each
row of the free-form formatted file must be comprised of a coefficient
followed by its associated multi-index (the same format used for
output described in :ref:`other:output:pce`). This file import can be
used to evaluate moments analytically or compute probabilities
numerically from a known response expansion. Refer to
:dakkw:`method-polynomial_chaos-import_expansion_file` for additional
information on this specification.
Surrogate Model Imports
~~~~~~~~~~~~~~~~~~~~~~~
Global data fit surrogates, including some stochastic expansions, may
be constructed from a variety of data sources. One of these sources is
an auxiliary data file, as specified by the keyword :dakkw:`model-surrogate-global-import_build_points_file`. The file may be
in annotated (default), custom-annotated, or free-form format with
columns corresponding to variables and responses. For global
surrogates specified directly via keywords ``model surrogate global``,
the keyword ``use_variable_labels`` will trigger validation and potential reordering of
imported variable columns based on labels provided in the tabular
header. Surfpack global surrogate models may also be evaluated at a
user-provided file containing challenge (test) points. Refer to the
:ref:`model-surrogate-global` keywords for additional information on
these specifications.
Previously exported surfpack and experimental global surrogate models
can be re-imported when used directly in the global surrogate model
context. Importing from binary or text archive instead of building
from data can sometimes result in significant time savings with models
such as Gaussian processes. See the ``export_model`` and
``import_model`` keywords in :ref:`keyword-reference-area` for
important caveats on its use.
Variables/responses import to post-run
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The post-run mode (supported only for sampling, parameter study, and
DACE methods) requires specification of a file containing parameter and
response data. Annotated is the default format (see :ref:`input:tabularformat`), where leading columns for evaluation
and interface IDs are followed by columns for variables (active and
inactive by default), then those for responses, with an ignored header
row of labels and then one row per evaluation. Typically this file would
be generated by executing
::
dakota -i dakota.in -pre_run ::variables.dat
and then separate from daktoa adding columns of response data to
:file:`variables.dat` to make :file:`varsresponses.dat`. The file is
specified at the command line with:
::
dakota -i dakota.in -post_run varsresponses.dat::
To import post-run data in other formats, specify
:dakkw:`environment-post_run` in the input file instead of on the
command-line, and provide a format option.