Ecl home. Last accessed: July 6, 2009.

Vanderplaats Research and Development, Inc.95

DOT Users Manual, Version 4.20. Vanderplaats Research and Development, Inc., Colorado Springs, CO, 1995.


M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York, 1965.


M.A. Abramson, C. Audet, G. Couture, J.E. Dennis, Jr., S. Le Digabel, and C. Tribes. The NOMAD project. Software available at URL:


B. M. Adams, W. J. Bohnhoff, R. A. Canfield, K. R. Dalbey, M. S. Ebeida, J. P. Eddy, M. S. Eldred, G. Geraci, R. W. Hooper, P. D. Hough, K. T. Hu, J. D. Jakeman, K. Carson, M. Khalil, K. A. Maupin, J. A. Monschke, E. E. Prudencio, P. Ridgway, Robbe, E. M., A. A. Rushdi, D. T. Seidl, J. A. Stephens, L. P. Swiler, A. Tran, D. M. Vigil, G. J. von Winckel, T. M. Wildey, J. G. Winokur, and (with Menhorn, F. and Zeng, X.). Dakota, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.19 Developers Manual. Sandia National Laboratories, Albuquerque, NM, November 2023. Available online from


B. M. Adams, R. W. Hooper, A. Lewis, J. A. McMahan, R. C. Smith, L. P. Swiler, and B. J. Williams. User guidelines and best practices for casl vuq analysis using dakota. Technical Report CASL-U-2014-0038-000, Consortium for the Advanced Simulation of LWRs, 2014.


N. Agarwal and N.R. Aluru. A domain adaptive stochastic collocation approach for analysis of MEMS under uncertainties. Journal of Computational Physics, 228:7662–7688, 2009.


N. M. Alexandrov, R. M. Lewis, C. R. Gumbert, L. L. Green, and P. A. Newman. Optimization with variable-fidelity models applied to wing design. In Proceedings of the 38th Aerospace Sciences Meeting and Exhibit. Reno, NV, 2000. AIAA Paper 2000-0841.


M. Allen and K. Maute. Reliability-based design optimization of aeroelastic structures. Struct. Multidiscip. O., 27:228–242, 2004.


G. Anderson and P. Anderson. The UNIX C Shell Field Guide. Prentice-Hall, Englewood Cliffs, NJ, 1986.


Ilya A Antonov and VM Saleev. An economic method of computing lpτ-sequences. USSR Computational Mathematics and Mathematical Physics, 19(1):252–256, 1979.


J. S. Arora. Introduction to Optimum Design. McGraw-Hill, New York, 1989.


R. Askey and J. Wilson. Some basic hypergeometric polynomials that generalize jacobi polynomials. In Mem. Amer. Math. Soc. 319. Providence, RI, 1985. AMS.


C. Audet, S. Le Digabel, and C. Tribes. NOMAD user guide. Technical Report G-2009-37, Les cahiers du GERAD, 2009. URL:


J. M. Aughenbaugh and C. J. J. Paredis. Probability bounds analysis as a general approach to sensitivity analysis in decision making under uncertainty. In SAE World Congress and Exposition, number SAE-2007-01-1480. Detroit, MI, 2007. SAE.


F. Augustin and Y. M. Marzouk. NOWPAC: a provably convergent derivative-free nonlinear optimizer with path-augmented constraints. arXiv, math.OC:1403.1931, 2014. URL:


R.G. Baraniuk, V. Cevher, M.F. Duarte, and C. Hegde. Model-based compressive sensing. Information Theory, IEEE Transactions on, 56(4):1982–2001, 2010.


V. Barthelmann, E. Novak, and K. Ritter. High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math., 12(4):273–288, 2000. Multivariate polynomial interpolation.


C. Berg, J. Mateu, and E. Porcu. The dagum family of isotropic correlation functions. Bernoulli, 14(4):1134–1149, 2008.


J.-P. Berrut and L. N. Trefethen. Barycentric lagrange interpolation. SIAM Review, 46(3):501–517, 2004.


B. J. Bichon, M. S. Eldred, L. P. Swiler, S. Mahadevan, and J. M. McFarland. Multimodal reliability assessment for complex engineering applications using efficient global optimization. In Proceedings of the 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (9th AIAA Non-Deterministic Approaches Conference), number AIAA-2007-1946. Honolulu, HI, April 2007.


B. J. Bichon, M. S. Eldred, L. P. Swiler, S. Mahadevan, and J. M. McFarland. Efficient global reliability analysis for nonlinear implicit performance functions. AIAA Journal, 46(10):2459–2468, 2008.


Géraud Blatman and Bruno Sudret. Adaptive sparse polynomial chaos expansion based on least angle regression. Journal of Computational Physics, 230(6):2345 – 2367, 2011. URL:, doi:10.1016/


B. Blinn. Portable Shell Programming: An Extensive Collection of Bourne Shell Examples. Prentice Hall PTR, New Jersey, 1996.


G.F. Bomarito, P.E. Leser, J.E. Warner, and W.P. Leser. On the optimization of approximate control variates with parametrically defined estimators. Journal of Computational Physics, 451:110882, 2022. doi:


G. E. P. Box and D. R. Cox. An analysis of transformations. J. Royal Stat. Soc., Series B, 26:211–252, 1964.


S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, March 2004. ISBN 0521833787. URL:


Paul Bratley and Bennett L Fox. Algorithm 659: Implementing Sobol's quasirandom sequence generator. ACM Transactions on Mathematical Software (TOMS), 14(1):88–100, 1988.


K. Breitung. Asymptotic approximation for multinormal integrals. J. Eng. Mech., ASCE, 110(3):357–366, 1984.


J. Burkardt. Computing the hermite interpolation polynomial. Technical Report, Florida State University, Gainesville, FL, 2009. jburkardt/presentations/hermite_interpolant.pdf.


J. Burkardt. The “combining coefficient” for anisotropic sparse grids. Technical Report, Virginia Tech, Blacksburg, VA, 2009. jburkardt/presentations/sgmga_coefficient.pdf.


T. Butler, J. D. Jakeman, and T. M. Wildey. A consistent bayesian formulation for stochastic inverse problems based on push-forward measures. arXiv, math.NA:1704.00680, 2017. URL:


R. H. Byrd, R. B. Schnabel, and G. A. Schultz. Parallel quasi-newton methods for unconstrained optimization. Mathematical Programming, 42:273–306, 1988.


Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 16(5):1190–1208, 1995.


M. R. Celis, J. .E. Dennis, and R. .A Tapia. A trust region strategy for nonlinear equality constrained optimization. In P. .T. Boggs, R. H. Byrd, and R. B. Schnabel, editors, Numerical Optimization 1984, pages 71–82. SIAM, Philadelphia, USA, 1985.


K. J. Chang, R. T. Haftka, G. L. Giles, and P.-J. Kao. Sensitivity-based scaling for approximating structural response. J. Aircraft, 30:283–288, 1993.


S. Chen, D. Donoho, and M. Saunders. Atomic decomposition by basis pursuit. SIAM Rev., 43(1):129–159, January 2001. URL:, doi:10.1137/S003614450037906X.


X. Chen and N.C. Lind. Fast probability integration by three-parameter normal tail approximation. Struct. Saf., 1:269–276, 1983.


S.-C. T. Choi, F. J. Hickernell, M. McCourt, and A. Sorokin. QMCPy: a quasi-Monte Carlo Python library. 2020. URL:


C. A. Coello, D. A. Van Veldhuizen, and G. B. Lamont. Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic/Plenum Publishers, New York, 2002.


A. R. Conn, N. I. M. Gould, and P. L. Toint. Trust-Region Methods. MPS-SIAM Series on Optimization, SIAM-MPS, Philadelphia, 2000.


P. G. Constantine. Active subspaces: Emerging ideas for dimension reduction in parameter studies. Volume 2. SIAM, 2015.


P. G. Constantine, E. Dow, and Q. Wang. Active subspace methods in theory and practice: applications to kriging surfaces. SIAM Journal on Scientific Computing, 36(4):A1500–A1524, 2014.


P. G. Constantine, M. S. Eldred, and E. T. Phipps. Sparse pseudospectral approximation method. Computer Methods in Applied Mechanics and Engineering, 229–232:1–12, July 2012.


P. G. Constantine and D. F. Gleich. Computing active subspaces. arXiv, math.NA:1408.0545, 2014. URL:


P. G. Constantine, D. F. Gleich, and G. Iaccarino. Spectral methods for parameterized matrix equations. SIAM Journal on Matrix Analysis and Applications, 31(5):2681–2699, 2010.


Emile Contal, David Buffoni, Alexandre Robicquet, and Nicolas Vayatis. Parallel Gaussian process optimization with upper confidence bound and pure exploration. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 225–240. Springer, 2013.


Ronald Cools, Frances Y Kuo, and Dirk Nuyens. Constructing embedded lattice rules for multivariate integration. SIAM Journal on Scientific Computing, 28(6):2162–2188, 2006.


T.M. Cover and J.A. Thomas. Elements of Information Theory. A Wiley-Interscience publication. Wiley, 2006.


N. Cressie. Statistics of Spatial Data. John Wiley and Sons, New York, 1991.


K. Dalbey and L. P. Swiler. Gaussian Process Adaptive Importance Sampling. International Journal for Uncertainty Quantification, 4(2):133–149, 2014.


Christian Daniel, Malte Viering, Jan Metz, Oliver Kroemer, and Jan Peters. Active reward learning. In Robotics: Science and Systems. 2014.


G. Davis, S. Mallat, and M. Avellaneda. Adaptive greedy approximations. Constructive Approximation, 13:57–98, 1997. 10.1007/BF02678430. URL:


G.J. Davis and M.D. Morris. Six factors which affect the condition number of matrices associated with kriging. Mathematical geology, 29(5):669–683, 1997.


J. E. Dennis, D. M. Gay, and R. E. Welsch. ALGORITHM 573: NL2SOL–an adaptive nonlinear least-squares algorithm. ACM Trans. Math. Software, 7:369–383, 1981.


J. E. Dennis and R. M. Lewis. Problem formulations and other optimization issues in multidisciplinary optimization. In Proc. AIAA Symposium on Fluid Dynamics, number AIAA-94-2196. Colorado Springs, Colordao, June 1994.


J. E. Dennis and V. J. Torczon. Derivative-free pattern search methods for multidisciplinary design problems. In Proc. 5th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-94-4349, 922–932. Panama City, FL, September 7–9 1994.


A. Der Kiureghian and P. L. Liu. Structural reliability under incomplete information. J. Eng. Mech., ASCE, 112(EM-1):85–104, 1986.


Thomas Desautels, Andreas Krause, and Joel W Burdick. Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. The Journal of Machine Learning Research, 15(1):3873–3923, 2014.


A. Dey and S. Mahadevan. Ductile structural system reliability analysis using adaptive importance sampling. Structural Safety, 20:137–154, 1998.


Josef Dick, Frances Y Kuo, and Ian H Sloan. High-dimensional integration: the quasi-Monte Carlo way. Acta Numerica, 22:133–288, 2013.


Josef Dick and Friedrich Pillichshammer. Digital nets and sequences: discrepancy theory and quasi–Monte Carlo integration. Cambridge University Press, 2010.


D.L. Donoho and Y. Tsaig. Fast solution of $\ell _1$-norm minimization problems when the solution may be sparse. Information Theory, IEEE Transactions on, 54(11):4789 –4812, nov. 2008. doi:10.1109/TIT.2008.929958.


A . Doostan and H. Owhadi. A non-adapted sparse approximation of PDEs with stochastic inputs. Journal of Computational Physics, 230(8):3015 – 3034, 2011. URL:, doi:10.1016/


Q. Du, V. Faber, and M. Gunzburger. Centroidal voronoi tessellations: applications and algorithms. SIAM Review, 41:637–676, 1999.


X. Du and W. Chen. Sequential optimization and reliability assessment method for efficient probabilistic design. J. Mech. Design, 126:225–233, 2004.


Marco F. Duarte, Michael B. Wakin, and Richard G. Baraniuk. Fast reconstruction of piecewise smooth signals from random projections. In Online Proceedings of the Workshop on Signal Processing with Adaptative Sparse Structured Representations (SPARS). Rennes, France, 2005.


R.P. Dwight and Z. Han. Efficient uncertainty quantification using gradient-enhanced kriging. In 11th AIAA Non-Deterministic Approaches Conference. Reston, VA, USA, May 2009. American Institute of Aeronautics and Astronautics.


Mohamed S Ebeida, Scott A Mitchell, Laura P Swiler, Vicente J Romero, and Ahmad A Rushdi. Pof-darts: geometric adaptive sampling for probability of failure. Reliability Engineering & System Safety, 155:64–77, 2016.


J. Eckstein, C. A. Phillips, and W. E. Hart. PEBBL 1.0 user guide. Technical Report, N/A, October 2009.


J. E. Eddy and K. Lewis. Effective generation of pareto sets using genetic programming. In Proceedings of ASME Design Engineering Technical Conference. 2001.


B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. Annals of Statistics, 32:407–499, 2004. doi:10.1214/009053604000000067.


A. S. El-Bakry, R. A. Tapia, T. Tsuchiya, and Y. Zhang. On the formulation and theory of the newton interior-point method for nonlinear programming. Journal of Optimization Theory and Applications, 89:507–541, 1996.


M. S Eldred and L. P. Swiler. Efficient algorithms for mixed aleatory-epistemic uncertainty quantification with application to radiation-hardened electronics. part 1: algorithms and benchmark results. Technical Report SAND2009-5805, Sandia National Laboratories, Albuquerque, NM, 2009.


M. S. Eldred. Optimization strategies for complex engineering applications. Technical Report SAND98-0340, Sandia National Laboratories, Albuquerque, NM, 1998.


M. S. Eldred, H. Agarwal, V. M. Perez, S. F. Wojtkiewicz, Jr., and J. E. Renaud. Investigation of reliability method formulations in DAKOTA/UQ. Structure & Infrastructure Engineering: Maintenance, Management, Life-Cycle Design & Performance, 3(3):199–213, 2007.


M. S. Eldred and B. J. Bichon. Second-order reliability formulations in DAKOTA/UQ. In Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, number AIAA-2006-1828. Newport, RI, May 1–4 2006.


M. S. Eldred and J. Burkardt. Comparison of non-intrusive polynomial chaos and stochastic collocation methods for uncertainty quantification. In Proceedings of the 47th AIAA Aerospace Sciences Meeting and Exhibit, number AIAA-2009-0976. Orlando, FL, January 5–8, 2009.


M. S. Eldred and D. M. Dunlavy. Formulations for surrogate-based optimization with data fit, multifidelity, and reduced-order models. In Proceedings of the 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, number AIAA-2006-7117. Portsmouth, VA, September 6–8 2006.


M. S. Eldred, A. A. Giunta, and S. S. Collis. Second-order corrections for surrogate-based optimization with model hierarchies. In Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. Albany, NY,, Aug. 30–Sept. 1, 2004. AIAA Paper 2004-4457.


M. S. Eldred, A. A. Giunta, S. F. Wojtkiewicz, Jr., and T. G. Trucano. Formulations for surrogate-based optimization under uncertainty. In Proc. 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-2002-5585. Atlanta, GA, September 4–6, 2002.


M. S. Eldred and W. E. Hart. Design and implementation of multilevel parallel optimization on the Intel TeraFLOPS. In Proc. 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-98-4707, 44–54. St. Louis, MO, September 2–4 1998.


M. S. Eldred, W. E. Hart, W. J. Bohnhoff, V. J. Romero, S. A. Hutchinson, and A. G. Salinger. Utilizing object-oriented design to build advanced optimization strategies with generic implementation. In Proc. 6th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-96-4164, 1568–1582. Bellevue, WA, September 4–6 1996.


M. S. Eldred, W. E. Hart, B. D. Schimel, and B. G. van Bloemen Waanders. Multilevel parallelism for optimization on MP computers: theory and experiment. In Proc. 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-2000-4818. Long Beach, CA, 2000.


M. S. Eldred, D. E. Outka, W. J. Bohnhoff, W. R. Witkowski, V. J. Romero, E. R. Ponslet, and K. S. Chen. Optimization of complex mechanics simulations with object-oriented software design. Computer Modeling and Simulation in Engineering, August 1996.


M. S. Eldred and B. D. Schimel. Extended parallelism models for optimization on massively parallel computers. In Proc. 3rd World Congress of Structural and Multidisciplinary Optimization (WCSMO-3), number 16-POM-2. Amherst, NY, May 17–21 1999.


M. S. Eldred, C. G. Webster, and P. Constantine. Evaluation of non-intrusive approaches for wiener-askey generalized polynomial chaos. In Proceedings of the 10th AIAA Non-Deterministic Approaches Conference, number AIAA-2008-1892. Schaumburg, IL, April 7–10 2008.


M.S. Eldred, B.M. Adams, D.M. Gay, L.P. Swiler, K. Haskell, W.J. Bohnhoff, J.P. Eddy, W.E. Hart, J.P Watson, J.D. Griffin, P.D. Hough, T.G. Kolda, P.J. Williams, and M.L. Martinez-Canales. Dakota version 4.1 users manual. Sandia Technical Report SAND2006-6337, Sandia National Laboratories, Albuquerque, NM, 2007. URL:


G. M. Fadel, M. F. Riley, and J.-F. M. Barthelemy. Two point exponential approximation method for structural optimization. Structural Optimization, 2(2):117–124, 1990.


H.R. Fairbanks, A. Doostan, C. Ketelsen, and G. Iaccarino. A low-rank control variate for multilevel monte carlo simulation of high-dimensional uncertain systems. Journal of Computational Physics, 341:121–139, 2017.


S. Ferson and W. T. Tucker. Sensitivity analysis using probability bounding. Reliability Engineering and System Safety, 91:1435–1442, 2006.


D. Flaggs. JPrePost user's manual. In preparation.


J. M. Flegal and G. L. Jones. Batch means and spectral variance estimators in Markov chain Monte Carlo. The Annals of Statistics, 38(2):1034–1070, 2010.


R. Fletcher, S. Leyffer, and P. L. Toint. On the global convergence of a filter-SQP algorithm. SIAM J. Optim., 13(1):44–59, 2002.


R. Fourer, D. M. Gay, and B. W. Kernighan. AMPL: A Modeling Language for Mathematical Programming, 2nd ed. Duxbury Press/Brooks/Cole Publishing Co., Pacific Grove, CA, 2003. For small examples, e.g., at most 300 variables, a student version of AMPL suffices; see \texttt


P. Frauenfelder, C. Schwab, and R. A. Todor. Finite elements for elliptic problems with stochastic coefficients. Comput. Methods Appl. Mech. Engrg., 194(2-5):205–228, 2005.


J. H. Friedman. Multivariate adaptive regression splines. Annals of Statistics, 19(1):1–141, March 1991.


J. Gablonsky. Direct version 2.0 userguide technical report. Technical Report CRSC-TR01-08, North Carolina State University, Center for Research in Scientific Computation, Raleigh, NC, 2001.


S. Gao, G. Ver Steeg, and A Galstyan. Efficient estimation of mutual information for strongly dependent variables. CoRR, 2014. URL:, arXiv:1411.2003.


W. Gautschi. Orthogonal Polynomials: Computation and Approximation. Oxford University Press, New York, 2004.


D. M. Gay. Hooking your solver to AMPL. Technical Report Technical Report 97-4-06, Bell Laboratories, Murray Hill, NJ, 1997. Available online as and and


G. Geraci, G. Iaccarino, and Michael S. Eldred. A multi fidelity control variate approach for the multilevel monte carlo technique. CTR Annual Research Briefs 2015, pages 169–181, 2015.


Gianluca Geraci, Michael S. Eldred, and Gianluca Iaccarino. A multifidelity multilevel Monte Carlo method for uncertainty propagation in aerospace applications. In 19th AIAA Non-Deterministic Approaches Conference. AIAA, January 2017. URL: (visited on 2019-10-04), doi:10.2514/6.2017-1951.


T. Gerstner and M. Griebel. Numerical integration using sparse grids. Numer. Algorithms, 18(3-4):209–232, 1998.


T. Gerstner and M. Griebel. Dimension-adaptive tensor-product quadrature. Computing, 71(1):65–87, 2003.


R. Ghanem and J. R. Red-Horse. Propagation of probabilistic uncertainty in complex physical systems using a stochastic finite element technique. Physica D, 133:137–144, 1999.


R. G. Ghanem and P. D. Spanos. Stochastic Finite Elements: A Spectral Approach. Springer-Verlag, New York, 1991.


Michael B. Giles. Multilevel Monte Carlo Path Simulation. Operations Research, 56(3):607–617, June 2008. URL:, doi:10.1287/opre.1070.0496.


Michael B. Giles. Multilevel monte carlo methods. Acta Numerica, 24:259–328, 2015. doi:10.1017/S096249291500001X.


W.R. Gilks, S. Richardson, and D.J. Spiegelhalter. Markov chain Monte Carlo in practice. Chapman & Hall, 1998.


P. E. Gill, W. Murray, M. A. Saunders, and M. H. Wright. User's guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming. Technical Report TR SOL-86-2, System Optimization Laboratory, Stanford University, Stanford, CA, 1986.


P. E. Gill, W. Murray, and M. H. Wright. Practical Optimization. Academic Press, San Diego, CA, 1981.


D. Gilly. UNIX in a Nutshell. O'Reilly and Associates, Inc., Sebastopol, CA, 1992.


David Ginsbourger, Rodolphe Riche, and Laurent Carraro. Kriging is well-suited to parallelize optimization. In Yoel Tenne and Chi-Keong Goh, editors, Computational Intelligence in Expensive Optimization Problems, volume 2 of Adaptation, Learning, and Optimization, pages 131–162. Springer Berlin Heidelberg, 2010. URL:, doi:10.1007/978-3-642-10701-6_6.


A. A. Giunta. Use of data sampling, surrogate models, and numerical optimization in engineering design. In Proc. 40th AIAA Aerospace Science Meeting and Exhibit, number AIAA-2002-0538. Reno, NV, January 2002.


A. A. Giunta and M. S. Eldred. Implementation of a trust region model management strategy in the DAKOTA optimization toolkit. In Proc. 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-2000-4935. Long Beach, CA, September 6–8, 2000.


A. A. Giunta, J. M. McFarland, L. P. Swiler, and M. S. Eldred. The promise and peril of uncertainty quantification using respone surface approximations. Structure and Infrastructure Engineering, 2(3-4):175–189, September-December 2006.


A. A. Giunta, L. P. Swiler, S. L Brown, M. S. Eldred, M. D. Richards, and E. C. Cyr. The surfpack software library for surrogate modeling of sparse, irregularly spaced multidimensional data. In Proceedings of the 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, number AIAA-2006-7049. Portsmouth, VA, 2006.


A. A. Giunta and L. T. Watson. A comparison of approximation modeling techniques: polynomial versus interpolating models. In Proc. 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-98-4758, 392–404. St. Louis, MO, 1998.


T. Gneiting, M.G. Genton, and P. Guttorp. Geostatistical space-time models, stationarity, separability and full symmetry. In B. Finkenstadt, L. Held, and V. Isham, editors, Statistical Methods for Spatio-Temporal Systems, chapter 4, pages 151–172. Boca Raton: Chapman & Hall/CRC, 2007.


D. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wessley Publishing Co., Inc., Reading, MA, 1989.


G. H.. Golub and J. H. Welsch. Caclulation of gauss quadrature rules. Mathematics of Computation, 23(106):221–230, 1969.


I. R. Goodman and H. T. Nguyen. Probability updating using second order probabilities and conditional event algebra. Information Sciences, 121:295–347, 1999.


Alex A. Gorodetsky, Gianluca Geraci, Michael S. Eldred, and John D. Jakeman. A generalized approximate control variate framework for multifidelity uncertainty quantification. Journal of Computational Physics, 408:109257, 2020. URL:, doi:


G. A. Gray and T. G. Kolda. Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization. ACM Transactions on Mathematical Software, 32(3):485–507, September 2006.


J. D Griffin and T. G. Kolda. Nonlinearly-constrained optimization using asynchronous parallel generating set search. Technical Report SAND2007-3257, Sandia National Laboratories, Livermore, CA, 2007.


W. Gropp and E. Lusk. User's guide for mpich, a portable implementation of MPI. Technical Report ANL/MCS-TM-ANL-96/6, Argonne National Laboratory, Mathematics and Computer Science Division, 1996.


W. Gropp, E. Lusk, and A. Skjellum. Using MPI, Portable Parallel Programing with the Message-Passing Interface. The MIT Press, Cambridge, MA, 1994.


M. Gunburger and J. Burkardt. Uniformity measures for point samples in hypercubes. 2004. Available on John Burkardt's web site: burkardt/.


Heikki Haario, Marko Laine, Antonietta Mira, and Eero Saksman. DRAM: efficient adaptive MCMC. Statistics and Computing, 16:339–354, 2006. URL:


R. T. Haftka. Combining global and local approximations. AIAA Journal, 29(9):1523–1525, 1991.


R. T. Haftka and Z. Gurdal. Elements of Structural Optimization. Kluwer, Boston, 1992.


A. Haldar and S. Mahadevan. Probability, Reliability, and Statistical Methods in Engineering Design. Wiley, New York, 2000.


J. H. Halton. On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numerische Mathematik, 2:84–90, 1960.


J. H. Halton and G. B. Smith. Algorithm 247: radical-inverse quasi-random point sequence. Communications of the ACM, 7:701–702, 1964.


John Hammersley. Monte carlo methods. Springer Science & Business Media, 2013.


W. E. Hart. The coliny project. Web site, 2007.


W. E. Hart, A. A. Giunta, A. G. Salinger, and B. G. van Bloemen Waanders. An overview of the adaptive pattern search algorithm and its application to engineering optimization problems. In Proceedings of the McMaster Optimization Conference: Theory and Applications. Hamilton, Ontario, Canada, 2001.


T. Hastie, R. Tibshirani, and J.H. Friedman. The elements of statistical learning: data mining, inference, and prediction: with 200 full-color illustrations. New York: Springer-Verlag, 2001.


A. S. Hedayat, N. J. A. Sloane, and J. Stufken. Orthogonal Arrays: Theory and Applications. Springer-Verlag, New York, 1999.


J. C. Helton and F. J. Davis. Sampling-based methods for uncertainty and sensitivity analysis. Technical Report SAND99-2240, Sandia National Laboratories, Albuquerque, NM, 2000.


J. C. Helton, J. D. Johnson, W. L. Oberkampf, and C. B Storlie. A sampling-based computational strategy for the representation of epistemic uncertainty in model predictions with evidence theory. Comp. Methods in Applied Mechanics and Engineering, 196:3980–3998, 2007.


J. C. Helton and W. L. Oberkampf. Special Issue of Reliability Engineering and System Safety: Issue on Alternative Representations of Epistemic Uncertainty. Volume 85. TODO, Jul-Sep 2004.


D. Higdon, J. Gattiker, B. Williams, and M. Rightley. Computer model calibration using high-dimensional output. Journal of the American Statistical Association, 103(482):570–583, 2008.


N. J. Higham. The numerical stability of barycentric lagrange interpolation. IMA Journal of Numerical Analysis, 24(4):547–556, 2004.


Edmund Hlawka. Zur angenäherten berechnung mehrfacher integrale. Monatshefte für Mathematik, 66(2):140–151, 1962.


M. Hohenbichler and R. Rackwitz. Sensitivity and importance measures in structural reliability. Civil Eng. Syst., 3:203–209, 1986.


M. Hohenbichler and R. Rackwitz. Improvement of second-order reliability estimates by importance sampling. J. Eng. Mech., ASCE, 114(12):2195–2199, 1988.


H.P. Hong. Simple approximations for improving second-order reliability estimates. J. Eng. Mech., ASCE, 125(5):592–595, 1999.


S. Hosder, R. W. Walters, and M. Balch. Efficient sampling for non-intrusive polynomial chaos applications with multiple uncertain input variables. In Proceedings of the 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, number AIAA-2007-1939. Honolulu, HI, April 23–26, 2007.


D. Huang, T. T. Allen, W. I. Notz, and N. Zeng. Global optimization of stochastic black-box systems via sequential kriging meta-models. Journal of Global Optimization, 34:441–466, 2006.


R. L. Iman and W. J. Conover. A distribution-free approach to inducing rank correlation among input variables. Communications in Statistics: Simulation and Computation, B11(3):311–334, 1982.


R. L. Iman and M. J Shortencarier. A Fortran 77 program and user's guide for the generation of latin hypercube samples for use with computer models. Technical Report NUREG/CR-3624, SAND83-2365, Sandia National Laboratories, Albuquerque, NM, 1984.


Wenzel Jakob, Jason Rhinelander, and Dean Moldovan. Pybind11 — seamless operability between c++11 and python. 2016.


E. T. Jaynes and G. Larry. Bretthorst. Probability theory : the logic of science. Cambridge University Press, Cambridge, UK; New York, NY, 2003.


Charles F. Jekel and Vicente J. Romero. Conservative estimation of tail probabilities from limited sample data. Technical Report SAND2020-2828, Sandia National Laboratories, Albuquerque, NM, March 2020.


Stephen Joe and Frances Y Kuo. Remark on algorithm 659: Implementing Sobol's quasirandom sequence generator. ACM Transactions on Mathematical Software (TOMS), 29(1):49–57, 2003.


Stephen Joe and Frances Y Kuo. Constructing Sobol sequences with better two-dimensional projections. SIAM Journal on Scientific Computing, 30(5):2635–2654, 2008.


D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13:455–492, 1998.


A. Karamchandani and C. A. Cornell. Sensitivity estimation within first and second order reliability methods. Struct. Saf., 11:95–107, 1992.


D. R. Karanki, H. S. Kishwaha, A. K. Verma, and S. Ajit. Uncertainty analysis based on probability bounds (p-box) approach in probabilistic safety assessment. Risk Analysis, 29:662–675, 2009.


M. C. Kennedy and A. O'Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society, 63:425–464, 2001.


B. W. Kernighan and D. M. Ritchie. The C Programming Language. Prentice Hall PTR, Englewood Cliffs, NJ, 2nd edition, 1988.


A. Klimke and B. Wohlmuth. Algorithm 847: spinterp: piecewise multilinear hierarchical sparse grid interpolation in matlab. ACM Transactions on Mathematical Software, 31(4):561–579, 2005.


W. A. Klimke. Uncertainty Modeling using Fuzzy Arithmetic and Sparse Grids. PhD thesis, Universität Stuttgart, Stuttgart, Germany, 2005.


L. Kocis and Whiten W. Computational investigations of low-discrepancy sequences. ACM Transactions on Mathematical Software, 23(2):266–294, 1997.


J. R. Koehler and A. B. Owen. Computer experiments. In S. Ghosh and C. R. Rao, editors, Handbook of Statistics, volume 13. Elsevier Science, New York, 1996.


AN Korobov. The approximate computation of multiple integrals. In Dokl. Akad. Nauk SSSR, volume 124, 1207–1210. 1959.


Nikolai Mikhailovich Korobov. Number-theoretic methods in approximate analysis. 1963.


D. P. Kouri, D. Ridzal, B. G. van Bloeman Waanders, and G. von Winckel. Rapid optimization library. Technical Report SAND2014-19572, Sandia National Laboratories, Albuquerque, NM, 2014.


Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical review E, 69(6):066138, 2004.


Frances Y Kuo. Lattice rule generating vectors. URL:


Frances Y Kuo. Sobol sequence generator. URL:


Frances Y Kuo. Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted korobov and sobolev spaces. Journal of Complexity, 19(3):301–320, 2003.


N. Kuschel and R. Rackwitz. Two basic problems in reliability-based structural optimization. Math. Method Oper. Res., 46:309–333, 1997.


Harold J Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1):97–106, 1964.


C. La and M.N. Do. Tree-based orthogonal matching pursuit algorithm for signal reconstruction. In Image Processing, 2006 IEEE International Conference on, 1277–1280. 2006.


L. D. Lathauwer, B. D. Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000.


C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. Prentice–Hall, 1974.


H. Lee, R. Gramacy, C. Linkletter, and G. Gray. Optimization Subject to Hidden Constraints via Statistical Emulation. Pacific Journal of Optimization, 7(3):467–478, 2011.


Allison Lewis, Ralph Smith, Brian Williams, and Victor Figueroa. An information theoretic approach to use high-fidelity codes to calibrate low-fidelity codes. Journal of Computational Physics, 324:24 – 43, 2016.


R. M. Lewis and S. N. Nash. A multigrid approach to the optimization of systems governed by differential equations. In Proceedings of the 8th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, number AIAA-2000-4890. Long Beach, CA, Sep 2000.


Chenzhao Li and Sankaran Mahadevan. An efficient modularized sample-based method to estimate the first-order Sobol׳ index. Reliability Engineering & System Safety, 153:110–121, September 2016. URL:, doi:10.1016/j.ress.2016.04.012.


C. Lucas. Lapack-style codes for level 2 and 3 pivoted cholesky factorizations. Numerical Analysis Reports 442, Manchester Centre for Computational Mathematics, Manchester, England, February 2004. LAPACK Working Note 161.


W. Luo and B. Li. Combining eigenvalues and variation of eigenvectors for order determination. TODO, 2015.


David J. C. MacKay. Information-based objective functions for active data selection. Neural Computation, 4:590–604, 1992.


Daniel P. Maljovec, Bei Wang, Ana Kupresanin, Gardar Johannesson, Valerio Pascucci, and Peer-Timo Bremer. Adaptive sampling with topological scores. International Journal for Uncertainty Quantification, 2012.


A. Martelli. Python in a Nutshell. O'Reilly and Associates, Cambridge, MA, 2003.


J.D. Martin. Computational improvements to estimating kriging metamodel parameters. Journal of Mechanical Design, 131:084501, 2009.


G. Matheron. The theory of regionalized variables and its applications. Les Cahiers du Centre de morphologie mathématique de Fontainebleau. École national supérieure des mines, 1971. URL:


Jiřı́ Matoušek. On the L2-discrepancy for anchored boxes. Journal of Complexity, 14(4):527–556, 1998.


J. M. McFarland. Uncertainty Analysis for Computer Simulations through Validation and Calibration. PhD thesis, Vanderbilt University, Nashville, Tennesssee, 2008. available for download at


M. D. McKay, R. J. Beckman, and W. J. Conover. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2):239–245, 1979.


J. C. Meza, R. A. Oliva, P. D. Hough, and P. J. Williams. OPT++: an object oriented toolkit for nonlinear optimization. ACM Transactions on Mathematical Software, 2007.


J. More and D. Thuente. Line search algorithms with guaranteed sufficient decrease. ACM Transactions on Mathematical Software, 20(3):286–307, 1994.


M. D. Morris. Factorial sampling plans for preliminary computational experiments. Technometrics, 33(2):161–174, 1991.


R. H. Myers and D. C. Montgomery. Response Surface Methodology: Process and Product Optimization Using Designed Experiments. John Wiley & Sons, Inc., New York, 1995.


A. Narayan and D. Xiu. Stochastic collocation methods on unstructured grids in high dimensions via interpolation. SIAM J. Scientific Computing, 2012.


A. Nealen. A short-as-possible introduction to the least squares, weighted least squares, and moving least squares methods for scattered data approximation and interpolation. Technical Report, Discrete Geometric Modeling Group, Technishe Universitaet, Berlin, Germany, 2004.


J. Neter, W. Wasserman, and M. H. Kutner. Applied Linear Statistical Models. Irwin Professional Publishing, Burr Ridge, IL, 2nd edition, 1985.


L. W. T. Ng and M. S. Eldred. Multifidelity uncertainty quantification using nonintrusive polynomial chaos and stochastic collocation. In Proceedings of the 14th AIAA Non-Deterministic Approaches Conference, number AIAA-2012-1852. Honolulu, HI, April 23-26, 2012.


L. W. T. Ng. and K. E. Willcox. Multifidelity approaches for optimization under uncertainty. International Journal for numerical methods in Engineering, 100(10):746–772, 2014.


Harald Niederreiter. Point sets and sequences with small discrepancy. Monatshefte für Mathematik, 104:273–337, 1987.


Harald Niederreiter. Random number generation and quasi-Monte Carlo methods. SIAM, 1992.


F. Nobile, R. Tempone, and C. G. Webster. A sparse grid stochastic collocation method for partial differential equations with random input data. Technical Report Technical report TRITA-NA 2007:7, Royal Institute of Technology, Stockholm, Sweden, 2007.


F. Nobile, R. Tempone, and C. G. Webster. An anisotropic sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J. on Num. Anal., 46(5):2411–2442, 2008.


Fabio Nobile and Francesco Tesei. A multi level monte carlo method with control variate for elliptic pdes with log-normal coefficients. Stochastic Partial Differential Equations: Analysis and Computations, 3(3):398–444, Sep 2015.


J. Nocedal and Wright S. J. Numerical Optimization. Springer Series in Operations Research. Springer, New York, 1999.


W.T. Nutt and G.B. Wallis. Evaluation of nuclear safety from the outputs of computer codes in the presence of uncertainties. Reliability Engineering and System Safety, 83:57–77, 2004.


Dirk Nuyens. The “magic point shop“” of QMC point generators and generating vectors. URL:


Dirk Nuyens and Ronald Cools. Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces. Mathematics of Computation, 75(254):903–920, 2006.


W. L. Oberkampf and J. C. Helton. Evidence theory for engineering applications. Technical Report SAND2003-3559P, Sandia National Laboratories, Albuquerque, NM, 2003.


E. O. Omojokun. Trust Region Algorithms for Optimization with Nonlinear Equality and Inequality Constraints. PhD thesis, University of Colorado, Boulder, Colorado, 1989.


M. J. L. Orr. Introduction to radial basis function networks. Technical Report, University of Edinburgh, Edinburgh, Scotland, 1996.


M.R. Osborne, B. Presnell, and B.A. Turlach. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 20(3):389–403, 2000. URL:, doi:10.1093/imanum/20.3.389.


A. B. Owen. Orthogonal arrays for computer experiments, integration, and visualization. Statistica Sinica, 2(2):439–452, 1992.


Art B Owen. Scrambling Sobol'and Niederreiter–Xing points. Journal of complexity, 14(4):466–489, 1998.


Art B Owen. On dropping the first Sobol point. In International conference on Monte Carlo and quasi-Monte Carlo methods in scientific computing, 71–86. Springer, 2020.


Art B. Owen. Monte Carlo theory, methods and examples. Art Owen, 2013. URL:


V. M. Pérez, M. S. Eldred, and J. E. and Renaud. Solving the infeasible trust-region problem using approximations. In Proceedings of the 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. Albany, NY, Aug. 30–Sept. 1, 2004. AIAA Paper 2004-4312.


V. M. Pérez, J. E. Renaud, and L. T. Watson. An interior-point sequential approximation optimization methodology. Structural and Multidisciplinary Optimization, 27(5):360–370, July 2004.


Spassimir Paskov and Joseph F Traub. Faster valuation of financial derivatives. Journal of Portfolio Management, 1996.


R. Pasupathy, M. Taaffe, B. W. Schmeiser, and W. Wang. Control-variate estimation using estimated control means. IIE Transactions, 44(5):381–385, 2014.


Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Optimal model management for multifidelity monte carlo estimation. SIAM Journal on Scientific Computing, 38(5):A3163–A3194, 2016.


Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization. SIAM Review, 60(3):550–591, January 2018. URL: (visited on 2019-10-04), doi:10.1137/16M1082469.


N. Petra, J. Martin, G. Stadler, and O. Ghattas. A computational framework for infinite-dimensional bayesian inverse problems; part II: stochastic newton mcmc with application to ice sheet flow inverse problems. SIAM J. Sci. Comput., 36(4):A1525–A1555, 2014.


Michele Pisaroni, Sebastian Krumscheid, and Fabio Nobile. Mathicse technical report : quantifying uncertain system outputs via the multilevel monte carlo method - part 1: central moment estimation. TODO, 2017. MATHICSE Technical Report Nr. 23.2017 October 2017. URL:, doi:10.5075/epfl-MATHICSE-263564.


T. D. Plantenga. HOPSPACK 2.0 user manual. Technical Report SAND2009-6265, Sandia National Laboratories, 2009.


E. Prudencio and S. H. Cheung. Parallel adaptive multilevel sampling algorithms for the bayesian analysis of mathematical models. International Journal for Uncertainty Quantification, 2:215–237, 2012.


Fernando Pérez-Cruz. Kullback-leibler divergence estimation of continuous distributions. In 2008 IEEE international symposium on information theory, 1666–1670. IEEE, 2008.


R. Rackwitz. Optimization and risk acceptability based on the Life Quality Index. Struct. Saf, 24:297–331, 2002.


R. Rackwitz and B. Fiessler. Structural reliability under combined random load sequences. Comput. Struct., 9:489–494, 1978.


P. Ranjan, Bingham D., and Michailidis G. Sequential experiment design for contour estimation from complex computer codes. Technometrics, 50(4):527–541, 2008.


M. T. Reagan, H. N. Najm, P. P. Pebay, O. M. Knio, and R. G. Ghanem. Quantifying uncertainty in chemical systems modeling. Int. J. Chem. Kinet., 37:368–382, 2005.


G. Rennen. Subset selection from large datasets for kriging modeling. Structural and Multidisciplinary Optimization, 38(6):545–569, 2009.


N. L. Robertson, M. Khalil, and B. M. Adams. Comparing MCMC diagnostics and stopping rules. In A. Cangi and M. L. Parks, editors, Center for Computing Research Summer Proceedings 2018, 132–144. Sandia National Laboratories, 2018. Technical Report SAND2019-5093R.


D. G. Robinson and C. Atcitty. Comparison of quasi- and pseudo-monte carlo sampling for reliability and uncertainty analysis. In Proceedings of the AIAA Probabilistic Methods Conference, number AIAA99-1589. St. Louis, MO, 1999.


T. D. Robinson, M. S. Eldred, K. E. Willcox, and R. Haimes. Strategies for multifidelity optimization with variable dimensional hierarchical models. In Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (2nd AIAA Multidisciplinary Design Optimization Specialist Conference). Newport, RI, May 1–4, 2006. AIAA Paper 2006-1819.


T. D. Robinson, K. E. Willcox, M. S. Eldred, and R. Haimes. Multifidelity optimization for variable-complexity design. In Proceedings of the 11th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. Portsmouth, VA, September 6–8, 2006. AIAA Paper 2006-7114.


J. F. Rodriguez, J. E. Renaud, and L. T. Watson. Convergence of trust region augmented lagrangian methods using variable fidelity approximation data. Structural Optimization, 15:1–7, 1998.


M. Rosenblatt. Remarks on a multivariate transformation. Annals of Mathematical Statistics, 23(3):470–472, 1952.


J. Sacks, S. B. Schiller, and W. Welch. Design for computer experiments. Technometrics, 31:41–47, 1989.


A. Saltelli, S. Tarantola, F. Campolongo, and M. Ratto. Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. John Wiley & Sons, 2004.


K. Schittkowski. More Test Examples for Nonlinear Programming, Lecture Notes in Economics and Mathematical Systems, Vol. 282. Springer-Verlag, Berlin, 1987.


K. Schittkowski. NLPQLP: a fortran implementation of a sequential quadratic programming algorithm with distributed and non-monotone line search – user's guide. Technical Report, Department of Mathematics, University of Bayreuth, Bayreuth, Germany, 2004.


P. R. Schunk, P. A. Sackinger, R. R. Rao, K. S. Chen, and R. A. Cairncross. GOMA – a full-newton finite element program for free and moving boundary problems with coupled fluid/solid momentum, energy, mass, and chemical species transport: user's guide. Technical Report SAND95-2937, Sandia National Laboratories, Albuquerque, NM, 1995.


G. A. F. Seber and C. J. Wild. Nonlinear Regression. Wiley, New Jersey, 2003.


Sambu Seo, Marko Wallat, Thore Graepel, and Klaus Obermayer. Gaussian process regression: active data selection and test point rejection. In In Proceedings of the International Joint Conference on Neural Networks (IJCNN), 241–246. IEEE, 2000.


C.E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27(3):379–423, 1948.


I.C. Simpson. Numerical integration over a semi-infinite interval, using the lognormal distibution. Numerische Mathematik, 31:71–76, 1978.


G. D. Sjaardema. APREPRO: an algebraic preprocessor for parameterizing finite element analyses. Technical Report SAND92-2291, Sandia National Laboratories, Albuquerque, NM, 1992.


Ian H Sloan and Stephen Joe. Lattice methods for multiple integration. Oxford University Press, 1994.


Ian H Sloan and Henryk Woźniakowski. When are quasi-monte carlo algorithms efficient for high dimensional integrals? Journal of Complexity, 14(1):1–33, 1998.


R.C. Smith. Uncertainty Quantification: Theory, Implementation, and Applications. Computational Science and Engineering. SIAM, 2013.


S.A. Smolyak. Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR, 4:240–243, 1963.


M. Snir, S. Otto, S. Huss-Lederman, D. Walker, and J. Dongarra. MPI: The Complete Reference. MIT Press, Cambridge, MA, 1996.


Il'ya Meerovich Sobol'. On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, 7(4):784–802, 1967.


Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.


Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias W Seeger. Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory, 58(5):3250–3265, 2012.


R. Srinivasan. Importance Sampling. Springer-Verlag, 2002.


C. B. Storlie, L. P. Swiler, J. C. Helton, and C. J. Sallaberry. Implementation and evaluation of nonparametric regression procedures for s ensitivity analysis of computationally demanding models. Reliability Engineering and System Safety, 94:1735–1763, 2009.


A. Stroud. Approximate Calculation of Multiple Integrals. Prentice Hall, 1971.


L. P Swiler, R. Slepoy, and A. A. Giunta. Evaluation of sampling methods in constructing response surface approximations. In Proc. 47th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, number AIAA-2006-1827. Newport, RI, May 1-4 2006.


L. P. Swiler and N. J. West. Importance sampling: promises and limitations. In Proceedings of the 12th AIAA Non-Deterministic Approaches Conference, number AIAA-2010-2850. 2010.


L. P. Swiler and G. D. Wyss. A user's guide to Sandia's latin hypercube sampling software: LHS UNIX library and standalone version. Technical Report SAND04-2439, Sandia National Laboratories, Albuquerque, NM, July 2004.


G. Tang, G. Iaccarino, and M. S Eldred. Global sensitivity analysis for stochastic collocation expansion. In Proceedings of the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (12th AIAA Non-Deterministic Approaches conference). Orlando, FL, April 12-15, 2010. AIAA Paper 2010-XXXX.


G. Tang, L. P. Swiler, and M. S Eldred. Using stochastic expansion methods in evidence theory for mixed aleatory-epistemic uncertainty quantification. In Proceedings of the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (12th AIAA Non-Deterministic Approaches conference). Orlando, FL, April 12-15, 2010. AIAA Paper 2010-XXXX.


R. A. Tapia and M. Argaez. Global convergence of a primal-dual interior-point newton method for nonlinear programming using a modified augmented lagrangian function. (In Preparation).


M.A. Tatang. Direct incorporation of uncertainty in chemical and environmental engineering systems. PhD thesis, MIT, 1995.


Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267–288, 1996.


Ramakrishna Tipireddy and Roger Ghanem. Basis adaptation in homogeneous chaos spaces. Journal of Computational Physics, 259:304 – 317, 2014. URL:, doi:


C. H. Tong. The PSUADE software library. Web site, 2005.


C. H. Tong and J. C. Meza. DDACE: a distributed object-oriented software with multiple samplings for the design and analysis of computer experiments. Draft as yet unpublished, see also


Anh Tran, Mike Eldred, Tim Wildey, Scott McCann, Jing Sun, and Robert J Visintainer. aphBO-2GP-3B: a budgeted asynchronous parallel multi-acquisition functions for constrained Bayesian optimization on high-performing computing architecture. Structural and Multidisciplinary Optimization, 65(4):1–45, 2022.


Anh Tran, Jing Sun, John M Furlan, Krishnan V Pagalthivarthi, Robert J Visintainer, and Yan Wang. pBO-2GP-3B: A batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Computer Methods in Applied Mechanics and Engineering, 347:827–852, 2019.


J. Tu, K. K. Choi, and Y. H. Park. A new study on reliability-based design optimization. J. Mech. Design, 121:557–564, 1999.


L. Tvedt. Distribution of quadratic forms in normal space – applications to structural reliability. J. Eng. Mech., ASCE, 116(6):1183–1197, 1990.


A. van der Sluis. Condition numbers and equilibration of matrices. Numerische Mathematik, 14(1):14–23, 1969.


R. J. Vanderbei and D. F. Shanno. An interior-point algorithm for nonconvex nonlinear programming. Computational Optimization and Applications, 13:231–259, 1999.


G. N. Vanderplaats. CONMIN – a FORTRAN program for constrained function minimization. Technical Report TM X-62282, NASA, 1973. See also Addendum to Technical Memorandum, 1978.


G. N. Vanderplaats. Numerical Optimization Techniques for Engineering Design: With Applications. McGraw-Hill, New York, 1984.


J. A. Vrugt, C. J. F. ter Braak, C. G. H. Diks, B. A. Robinson, J. M. Hyman, and D. Higdon. Accelerating Markov chain Monte Carlo simulation by self-adaptive differential evolution with randomized subspace sampling. International Journal of Nonlinear Scientific Numerical Simulation, 2009.


K. W. Vugrin, L. P. Swiler, R. M. Roberts, N. J. Stuckey-Mack, and S. P Sullivan. Confidence region estimation techniques for nonlinear regression in groundwater flow: three case studies. Water Resources Research, 2007.


L. Wall, T. Christiansen, and R. L. Schwartz. Programming Perl. O'Reilly & Associates, Cambridge, 2nd edition, 1996.


S. N. Walsh, T. M. Wildey, and J. D. Jakeman. Optimal experimental design using a consistent bayesian approach. arXiv, stat.CO:1705.09395, 2017. URL:


R. W. Walters. Towards stochastic fluid mechanics via polynomial chaos. In Proceedings of the 41st AIAA Aerospace Sciences Meeting and Exhibit, number AIAA-2003-0413. Reno, NV, January 6–9, 2003.


B. Walton. BPREPRO preprocessor documentation. Online document


G. W. Wasilkowski and H. Woźniakowski. Explicit cost bounds of algorithms for multivariate tensor product problems. Journal of Complexity, 11:1–56, 1995.


Larry Wasserman. Bayesian model selection and model averaging. Journal of mathematical psychology, 44(1):92–107, 2000.


G. Weickum, M. S. Eldred, and K. Maute. Multi-point extended reduced order modeling for design optimization and uncertainty analysis. In Proceedings of the 47th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (2nd AIAA Multidisciplinary Design Optimization Specialist Conference). Newport, RI, May 1–4, 2006. AIAA Paper 2006-2145.


V. G. Weirs, J. R. Kamm, L. P. Swiler, M. Ratto, S. Tarantola, B. M. Adams, W. J. Rider, and M. S Eldred. Sensitivity analysis techniques applied to a system of hyperbolic conservation laws. Reliability Engineering and System Safety, 107:157–170, 2012.


N. Wiener. The homogoeneous chaos. Amer. J. Math., 60:897–936, 1938.


S. S. Wilks. Determination of sample sizes for setting tolerance limits. Ann. Math. Stat., 12(1):91–96, 1941.


J. A. S. Witteveen and H. Bijl. Modeling arbitrary uncertainties using gram-schmidt polynomial chaos. In Proceedings of the 44th AIAA Aerospace Sciences Meeting and Exhibit, number AIAA-2006-0896. Reno, NV, January 9–12 2006.


S. J. Wright. Primal-Dual Interior-Point Methods. SIAM, 1997.


Y.-T. Wu, Y. Shin, R. Sues, and M. Cesare. Safety-factor based approach for probability-based design optimization. In Proc. 42nd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, number AIAA-2001-1522. Seattle, WA, April 16–19 2001.


Y.-T. Wu and P.H. Wirsching. A new algorithm for structural reliability estimation. J. Eng. Mech., ASCE, 113:1319–1336, 1987.


B. A. Wujek and J. E. Renaud. New adaptive move-limit management strategy for approximate optimization, part 1. AIAA Journal, 36(10):1911–1921, 1998.


B. A. Wujek and J. E. Renaud. New adaptive move-limit management strategy for approximate optimization, part 2. AIAA Journal, 36(10):1922–1934, 1998.


G. D. Wyss and K. H. Jorgensen. A user's guide to LHS: Sandia's Latin hypercube sampling software. Technical Report SAND98-0210, Sandia National Laboratories, Albuquerque, NM, 1998.


D. Xiu. Numerical integration formulas of degree two. Applied Numerical Mathematics, 58:1515–1520, 2008.


D. Xiu and J.S. Hesthaven. High-order collocation methods for differential equations with random inputs. SIAM J. Sci. Comput., 27(3):1118–1139 (electronic), 2005.


D. Xiu and G. M. Karniadakis. The wiener-askey polynomial chaos for stochastic differential equations. SIAM J. Sci. Comput., 24(2):619–644, 2002.


S. Xu and R. V. Grandhi. Effective two-point function approximation for design optimization. AIAA J., 36(12):2269–2275, 1998.


D. C. Zimmerman. Genetic algorithms for navigating expensive and complex design spaces. September 1996. Final Report for Sandia National Laboratories contract AO-7736 CA 02.


Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2):301–320, 2005. URL:, doi:10.1111/j.1467-9868.2005.00503.x.


T. Zou, S. Mahadevan, and R. Rebba. Computational efficiency in reliability-based optimization. In Proceedings of the 9th ASCE Specialty Conference on Probabilistic Mechanics and Structural Reliability. Albuquerque, NM, July 26–28, 2004.


T. Zou, Z. Mourelatos, S. Mahadevan, and P. Meernik. Reliability analysis of automotive body-door subsystem. Rel. Eng. and Sys. Safety, 78:315–324, 2002.


S. Le Digabel. Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Transactions on Mathematical Software, 37(4):1–15, 2011.


National Research Council of the National Academies. Evaluation of Quantification of Margins and Uncertainties Methodology for Assessing and Certifying the Reliability of the Nuclear Stockpile. National Academy Press, Washington D.C., 2008.