optim

Methods for the optim module in hetgpy

crit_EI(x, model, cst=None, preds=None)

Expected Improvement (EI) criteria

Parameters:

• x (nd_arraylike) – model designs, one point per row

• model (hetgpy.hetGP) – hetGP or homGP model

• cst (float) – optional plugin value of the mean

• preds (Dict) – model predictions (optional)

References

Mockus, J.; Tiesis, V. & Zilinskas, A. (1978). The application of Bayesian methods for seeking the extremum Towards Global Optimization, Amsterdam: Elsevier, 2, 2. Vazquez E, Villemonteix J, Sidorkiewicz M, Walter E (2008). Global Optimization Based on Noisy Evaluations: An Empirical Study of Two Statistical Approaches, Journal of Physics: Conference Series, 135, IOP Publishing.

Examples

>>> from hetgpy.test_functions import f1d
>>> from hetgpy.homGP import homGP
>>> from hetgpy.optim import crit_EI
>>> import numpy as np
>>> ftest = f1d
>>> n_init = 5 # number of unique designs
>>> X = np.linspace(0, 1, n_init).reshape(-1,1)
>>> Z = ftest(X)
>>> xgrid = np.linspace(0,1,51).reshape(-1,1)
>>> model = homGP()
>>> model.mle(X = X, Z = Z, lower = np.array([0.01]), upper = np.array([1]), known = dict(g = 2e-8))
>>> EI = crit_EI(xgrid, model, cst = model.Z0.min())

crit_logEI(x, model, cst=None, preds=None)

Logarithm of Expected Improvement (EI) criteria

Parameters:

• x (nd_arraylike) – model designs, one point per row

• model (hetgpy.hetGP) – hetGP or homGP model

• cst (float) – optional plugin value of the mean

• preds (Dict) – model predictions (optional)

References

Ament, S., Daulton, S., Eriksson, D., Balandat, M., & Bakshy, E. (2024). Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36.

Examples

>>> from hetgpy.test_functions import f1d
>>> from hetgpy.homGP import homGP
>>> from hetgpy.optim import crit_logEI
>>> import numpy as np
>>> ftest = f1d
>>> n_init = 5 # number of unique designs
>>> X = np.linspace(0, 1, n_init).reshape(-1,1)
>>> Z = ftest(X)
>>> xgrid = np.linspace(0,1,51).reshape(-1,1)
>>> model = homGP()
>>> model.mle(X = X, Z = Z, lower = np.array([0.01]), upper = np.array([1]), known = dict(g = 2e-8))
>>> logEI = crit_logEI(xgrid, model, cst = model.Z0.min())

crit_optim()

crit_qEI(x, model, cst=None, preds=None)

Fast approximated batch-Expected Improvement criterion (for minimization) Parallel Expected improvement

Parameters:

• x (nd_array) – matrix of new designs representing the batch of q points, one point per row (q x d)

• model (homGP/hetGP) – model object including inverve matrices

• cst (nd_array) – optional optional plugin value used in the EI, see details

• preds (dict) – optional predictions at x to avoid recomputing if already done (must include the predictive covariance, i.e., the cov slot)

Returns:

• qEI_cpp

• Details

• ——-

• cst is classically the observed minimum in the deterministic case. In the noisy case, the min of the predictive mean works fine. This is a beta version at this point. It may work for for TP models as well.

References

M. Binois (2015), Uncertainty quantification on Pareto fronts and high-dimensional strategies in Bayesian optimization, with applications in multi-objective automotive design. Ecole Nationale Superieure des Mines de Saint-Etienne, PhD thesis.

Examples

>>> from hetgpy.test_functions import f1d
>>> from hetgpy.homGP import homGP
>>> from hetgpy.optim import crit_qEI
>>> import numpy as np
>>> ftest = f1d
>>> n_init = 5 # number of unique designs
>>> X = np.linspace(0, 1, n_init).reshape(-1,1)
>>> Z = ftest(X)
>>> xgrid = np.linspace(0,1,51).reshape(-1,1)
>>> model = homGP()
>>> model.mleHomGP(X = X, Z = Z, lower = np.array([0.01]), upper = np.array([1]), known = dict(g = 2e-8))
>>> xbatch = np.array((0.37, 0.17, 0.7)).reshape(3, 1)
>>> fqEI = crit_qEI(xbatch, model, cst = model.Z0.min())

deriv_crit_EI(x, model, cst=None, preds=None)

Derivative for crit_EI, used in crit_optim

Parameters:

• x (nd_arraylike) – model designs, one point per row

• model (hetgpy.hetGP) – hetGP or homGP model

• cst (float) – optional plugin value of the mean

• preds (Dict) – model predictions (optional)

References

Ginsbourger, D. Multiples metamodeles pour l’approximation et l’optimisation de fonctions numeriques multivariables Ecole Nationale Superieure des Mines de Saint-Etienne, Ecole Nationale Superieure des Mines de Saint-Etienne, 2009 Roustant, O., Ginsbourger, D., DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization, Journal of Statistical Software, 2012

deriv_crit_logEI(x, model, cst=None, preds=None)

Derivative of the logarithm of Expected Improvement (EI) criteria

Parameters:

• x (nd_arraylike) – model designs, one point per row

• model (hetgpy.hetGP) – hetGP or homGP model

• cst (float) – optional plugin value of the mean

• preds (Dict) – model predictions (optional)

References

Ament, S., Daulton, S., Eriksson, D., Balandat, M., & Bakshy, E. (2024). Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36.

Examples

dlambda(z, a)

Derivative of the student-t pdf

Parameters:

• z (float) – input location

• a (float) – degree of freedom parameter

Return type:

derivative of student-t pdf

Examples

>>> dlambda(0.55, 3.6) # -0.2005827

dlog_h(z, eps=np.float64(2.220446049250313e-16))

References

Ament, S., Daulton, S., Eriksson, D., Balandat, M., & Bakshy, E. (2024). Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36.

log1mexp(x)

References

Maechler, Martin (2012). Accurately Computing log(1-exp(-|a|)). Assessed from the Rmpfr package.

log_h(z, eps=np.float64(2.220446049250313e-16))

References

Ament, S., Daulton, S., Eriksson, D., Balandat, M., & Bakshy, E. (2024). Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems, 36.

predict_gr(model, x)

Gradient of the prediction given a model