.. _gui-hdf-main:

""""""""""""""""""
HDF5 in Dakota GUI
""""""""""""""""""

.. _gui-hdf-introduction:

============
Introduction
============

In 2018, with the release of :ref:`version 6.9.0 <releasenotes-69>`, Dakota began providing HDF5 database files as a supported output format for its data.  Historically, all of
Dakota's output data was written to plain text formats - tabular data files, console output stream, etc.  This meant that parsing Dakota's output data was a manual,
sometimes difficult process.  On the other hand, HDF5 is a standard for scientific programming data storage, and there are many tools readily available for reading
data out of an HDF5 file.

==============================
Recommended Reading about HDF5
==============================

- `Visit the HDF Group website for further information on the HDF5 format. <https://www.hdfgroup.org/wp-content/uploads/2017/12/HDF512-17.pdf>`__
- :ref:`Learn about the way Dakota organizes its output data into HDF5 format. <hdf5_output>`
- `Learn about HDFView, the HDF Group's tool for graphically browsing HDF5 files. <https://support.hdfgroup.org/products/java/hdfview/>`__
- `Learn about h5py, the Pythonic interface to HDF5 format. <https://www.h5py.org/>`__

==========================================
Generating an HDF5 Output File from Dakota
==========================================

Configuring Dakota to generate HDF5 output is easy.  In the :ref:`environment <environment>` block of your study, add
the keywords :ref:`results_output <environment-results_output>` and :ref:`hdf5 <environment-results_output-hdf5>`.

.. image:: img/HDF_Generation_1.png 
   :alt: Example environment block

Dakota GUI also provides a :ref:`block recipe <gui-inputfiles-blockrecipes>` to do this for you:

.. image:: img/HDF_Generation_2.png
   :alt: Environment with HDF5 output

Now, the next time you run Dakota, an h5 file that contains your Dakota data in HDF5 format will be added to your project directory.

=================================
Browsing HDF5 Files in Dakota GUI
=================================

All HDF5 files (not just those generated by Dakota) are browseable in the Dakota GUI, thanks to the
incorporation of the `HDF Group's HDFView <https://support.hdfgroup.org/products/java/hdfview/>`__ tool.

Clicking on the small arrow next to our h5 file will expand it to the first level of data contained within. 

.. image:: img/HDF_Browsing_1.png 
   :alt: A collapsed h5 file

The data contained in HDF5 files are primarily composed of **groups** and **datasets** which roughly correspond to the idea of folders and files in an operating system:

.. image:: img/HDF_Browsing_2.png
   :alt: An expanded h5 file

This is what an average Dakota HDF5 file looks like after expanding a few of the folders.  Groups are designated by folder icons, whereas datasets are designated by page icons.

To view the data in an individual dataset, simply double-click it to display the table of data in your editor area.

.. image:: img/HDF_Browsing_3.png
   :alt: Data from a centered parameter study

Attributes about a group or dataset can be viewed using the Settings Editor view (Window > Show View > Settings).

.. image:: img/HDF_Browsing_4.png
   :alt: continuous_variables attributes

.. note::
   The Dakota GUI provides an **HDF File Viewing** perspective that docks the Settings Editor view in the center of your perspective.  To activate
   this perspective, go to "Window > Perspective > Open Perspective > Other" and then select "HDF File Viewing."

.. note::
   Dakota GUI only supports viewing HDF5 files in read-only mode.  You cannot use the Dakota GUI to write to an h5 file after it has been created.

=======================================
Using HDF5 Files for Graphical Plotting
=======================================

HDF5 files are recognized by :ref:`Chartreuse <chartreuse-main>` (Dakota GUI's plotting library) as a source of plottable data.

- :ref:`Learn about how to extract plotting data from plain HDF5 files. <chartreuse-getting-data-from-hdf5>`
- :ref:`Learn about how to extract plotting data from Dakota-generated HDF5 files. <chartreuse-getting-data-from-dakota-hdf5>`

=====================================
Using HDF5 Files in Next-Gen Workflow
=====================================

HDF5 files can also be used when building workflows in :ref:`Next-Gen Workflow. <ngw-main>`

- :ref:`Learn about the simple HDF5 plotting nodes that Next-Gen Workflow provides. <chartreuse-ngw-hdf-tutorial>`
- :ref:`Learn about the advanced "megaplotters" that combine knowledge of Dakota, HDF5, and Chartreuse to produce extremely complex plots. <chartreuse-ngw-composite-plotters>`