AdaptiveKriging

The AdaptiveKriging class generates samples adaptively using a specified Kriging-based learning function in a general Adaptive Kriging-Monte Carlo Sampling (AKMCS) framework. Based on the specified learning function, different objectives can be achieved. In particular, the AdaptiveKriging class has learning functions for reliability analysis (probability of failure estimation), global optimization, best global fit surrogate models, and can also accept user-defined learning functions for these and other objectives. Note that the term Adaptive Kriging is adopted from [22] although the procedure is referred to by different names depending on the specific learning function employed. For example, when applied for optimization the algorithm leverages the expected improvement function and is known under the name Efficient Global Optimization (EGO) [23].

AdaptiveKriging Class

The AdaptiveKriging class is imported using the following command:

>>> from UQpy.sampling.AdaptiveKriging import AdaptiveKriging

Methods

class AdaptiveKriging(distributions, runmodel_object, surrogate, learning_function, samples=None, nsamples=None, learning_nsamples=None, qoi_name=None, n_add=1, random_state=None)[source]

Adaptively sample for construction of a kriging surrogate for different objectives including reliability, optimization, and global fit.

Parameters:

distributions (Union[Distribution, list[Distribution]]) – List of Distribution objects corresponding to each random variable.
runmodel_object (RunModel) – A RunModel object, which is used to evaluate the model.
surrogate (Union[Surrogate, GaussianProcessRegressor, object]) – A kriging surrogate model, this object must have fit() and predict() methods. May be an object of the UQpy() Kriging class or an object of the scikit-learn GaussianProcessRegressor
learning_function (LearningFunction) – Learning function used as the selection criteria to identify new samples.
samples (Optional[ndarray]) – The initial samples at which to evaluate the model. Either samples or nstart must be provided.
nsamples (Optional[int]) – Total number of samples to be drawn (including the initial samples). If nsamples and samples are provided when instantiating the class, the run() method will automatically be called. If either nsamples or samples is not provided, AdaptiveKriging can be executed by invoking the run() method and passing nsamples.
learning_nsamples (Optional[int]) – Number of samples generated for evaluation of the learning function. Samples for the learning set are drawn using LatinHypercubeSampling.
qoi_name (Optional[str]) – Name of the quantity of interest. If the quantity of interest is a dictionary, this is used to convert it to a list
n_add (int) – Number of samples to be added per iteration.
random_state (Union[None, int, RandomState]) – Random seed used to initialize the pseudo-random number generator. Default is None. If an int is provided, this sets the seed for an object of numpy.random.RandomState. Otherwise, the object itself can be passed directly.

run(nsamples, samples=None, append_samples=True, initial_nsamples=None)[source]

Execute the AdaptiveKriging learning iterations.

The run() method is the function that performs iterations in the AdaptiveKriging class. If nsamples is provided when defining the AdaptiveKriging object, the run() method is automatically called. The user may also call the run() method directly to generate samples. The run() method of the AdaptiveKriging class can be invoked many times.

The run() method has no returns, although it creates and/or appends the samples attribute of the AdaptiveKriging class.

Parameters:

nsamples (int) – Total number of samples to be drawn (including the initial samples).
samples (Optional[ndarray]) – Samples at which to evaluate the model.
append_samples (bool) – Append new samples and model evaluations to the existing samples and model evaluations. If append_samples = False, all previous samples and the corresponding quantities of interest from their model evaluations are deleted. If append_samples = True, samples and their resulting quantities of interest are appended to the existing ones.
initial_nsamples (Optional[int]) – Number of initial samples, randomly generated using LatinHypercubeSampling class.

Attributes

AdaptiveKriging.samples: ndarray: contains the samples at which the model is evaluated.

Examples

Adaptive Kriging Examples

Learning Functions

AdaptiveKriging provides a number of built-in learning functions as well as allowing the user to provide a custom learning function. These learning functions are described below.

U-Function

The U-function is a learning function adopted for Kriging-based reliability analysis adopted from [22]. Given a Kriging model \(\hat{y}(\mathbf{x})\), point estimator of its standard devaition \(\sigma_{\hat{y}}(\mathbf{x})\), and a set of learning points \(S\), the U-function seeks out the point \(\mathbf{x}\in S\) that minimizes the function:

\[U(\mathbf{x}) = \dfrac{|\hat{y}(\mathbf{x})|}{\sigma_{\hat{y}}(\mathbf{x})}\]

This point can be interpreted as the point in \(S\) where the Kriging model has the highest probabability of incorrectly identifying the sign of the performance function (i.e. incorrectly predicting the safe/fail state of the system).

The AdaptiveKriging then adds the corresponding point to the training set, re-fits the Kriging model and repeats the procedure until the following stopping criterion in met:

\[\min(U(\mathbf{x})) > \epsilon_u\]

where \(\epsilon_u\) is a user-defined error threshold (typically set to 2).

The UFunction class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.UFunction import UFunction

class UFunction(u_stop=2)[source]

U-function for reliability analysis. See [22] for a detailed explanation.

Parameters:: u_stop (int) – U-Function stopping parameter

Weighted U-Function

The probability weighted U-function is a learning function for reliability analysis adapted from the U-function in [24]. It modifies the U-function as follows:

\[W(\mathbf{x}) = \dfrac{\max_x[p(\mathbf{x})] - p(\mathbf{x})}{\max_x[p(\mathbf{x})]} U(\mathbf{x})\]

where \(p(\mathbf{x})\) is the probability density function of \(\mathbf{x}\). This has the effect of decreasing the learning function for points that have higher probability of occurrence. Thus, given two points with identical values of \(U(x)\), the weighted learning function will select the point with higher probability of occurrence.

As with the standard U-function, AdaptiveKriging with the weighted U-function iterates until \(\min(U(\mathbf{x})) > \epsilon_u\) (the same stopping criterion as the U-function).

The WeightedUFunction class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.WeightedUFunction import WeightedUFunction

class WeightedUFunction(weighted_u_stop)[source]

Probability Weighted U-function for reliability analysis. See [24] for a detailed explanation.

Parameters:: weighted_u_stop (int) – Stopping parameter required for the WeightedU learning function

Expected Feasibility Function

The Expected Feasibility Function (EFF) is a learning function for reliability analysis introduced as part of the Efficient Global Reliability Analysis (EGRA) method [25]. The EFF provides assesses how well the true value of the peformance function, \(y(\mathbf{x})\), is expected to satisfy the constraint \(y(\mathbf{x}) = a\) over a region \(a-\epsilon \le y(\mathbf{x}) \le a+\epsilon\). It is given by:

\[\begin{split}\begin{align} EFF(\mathbf{x}) &= (\hat{y}(\mathbf{x})-a)\bigg[2\Phi\bigg(\dfrac{a-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) - \Phi\bigg(\dfrac{(a-\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) - \Phi\bigg(\dfrac{(a+\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) \bigg] \\ &-\sigma_{\hat{y}}(\mathbf{x})\bigg[2\phi\bigg(\dfrac{a-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) - \phi\bigg(\dfrac{(a-\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) - \phi\bigg(\dfrac{(a+\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) \bigg] \\ &+ \bigg[ \Phi\bigg(\dfrac{(a+\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) - \Phi\bigg(\dfrac{(a-\epsilon)-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) \bigg] \end{align}\end{split}\]

where \(\Phi(\cdot)\) and \(\phi(\cdot)\) are the standard normal cdf and pdf, respectively. For reliabilty, \(a=0\), and it is suggest to use \(\epsilon=2\sigma_{\hat{y}}^2\).

At each iteration, the new point that is selected is the point that maximizes the EFF and iterations continue until

\[\max_x(EFF(\mathbf{x})) < \epsilon_{eff}\]

The Expectedfeasibility class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.Expectedfeasibility import ExpectedFeasibility

class ExpectedFeasibility(eff_a=0, eff_epsilon=2, eff_stop=0.001)[source]

Expected Feasibility Function (EFF) for reliability analysis, see [25] for a detailed explanation.

Parameters:

eff_a (Union[float, int]) – Reliability threshold.
eff_epsilon (Union[float, int]) – EGRA method epsilon
eff_stop (Union[float, int]) – Stopping threshold

Expected Improvement Function

The Expected Improvement Function (EIF) is a Kriging-based learning function for global optimization introduced as part of the Efficient Global Optimization (EGO) method in [23]. The EIF seeks to find the global minimum of a function. It searches the space by placing samples at locations that maximize the expected improvement, where the improvement is defined as \(I(\mathbf{x})=\max(y_{min}-y(\mathbf{x}), 0)\), where the model response \(y(\mathbf{x})\) is assumed to be a Gaussian random variable and \(y_{min}\) is the current minimum model response. The EIF is then expressed as:

\[EIF(\mathbf{x}) = E[I(\mathbf{x})] = (y_{min}-\hat{y}(\mathbf{x})) \Phi \bigg(\dfrac{y_{min}-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg) + \sigma_{\hat{y}}(\mathbf{x})\phi \bigg(\dfrac{y_{min}-\hat{y}(\mathbf{x})}{\sigma_{\hat{y}}(\mathbf{x})} \bigg)\]

where \(\Phi(\cdot)\) and \(\phi(\cdot)\) are the standard normal cdf and pdf, respectively.

At each iteration, the EGO algorithm selects the point in the learning set that maximizes the EIF. The algorithm continues until the maximum number of iterations or until:

\[\dfrac{EIF(\mathbf{x})}{|y_{min}|} < \epsilon_{eif}.\]

Typically a value of \(0.01\) is used for \(\epsilon_{eif}\).

The ExpectedImprovement class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.ExpectedImprovement import ExpectedImprovement

class ExpectedImprovement(eif_stop=0.01)[source]

Expected Improvement Function (EIF) for Efficient Global Optimization (EFO). See [23] for a detailed explanation.

Parameters:: eif_stop (Union[float, int]) – Stopping threshold

Expected Improvement for Global Fit

The Expected Improvement for Global Fit (EIGF) learning function aims to build the surrogate model that is the best global representation of model. It was introduced in [26]. It aims to balance between even space-filling design and sampling in regions of high variation and is given by:

\[EIGF(\mathbf{x}) = (\hat{y}(\mathbf{x}) - y(\mathbf{x}_*))^2 + \sigma_{\hat{y}}(\mathbf{x})^2\]

where \(\mathbf{x}_*\) is the point in the training set closest in distance to the point \(\mathbf{x}\) and \(y(\mathbf{x}_*)\) is the model response at that point.

No stopping criterion is suggested by the authors of [26], thus its implementation in AdaptiveKriging uses a fixed number of iterations.

The ExpectedImprovementGlobalFit class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.ExpectedImprovementGlobalFit import ExpectedImprovementGlobalFit

class ExpectedImprovementGlobalFit[source]: Expected Improvement for Global Fit (EIGF) learning function. See [26] for a detailed explanation.

User-Defined Learning Functions

The AdaptiveKriging class also allows new, user-defined learning functions to be specified in a straightforward way, by generating child classes of the LearningFunction abstract class.

The Learning class is imported using the following command:

>>> from UQpy.sampling.adaptive_kriging_functions.baseclass.LearningFunction import LearningFunction

class LearningFunction[source]

abstract evaluate_function(distributions, n_add, surrogate, population, qoi=None, samples=None)[source]: Abstract method that needs to be overriden by the user to create new Adaptive Kriging Learning functions.

The user only needs to implement the evaluate_function() method. This method takes as input the surrogate model, the randomly generated learning points, the number of points to be added in each iteration, any requisite parameters including a stopping criterion, existing samples, model evaluate at samples and distribution object. It returns a set of samples that are selected according to the user’s desired learning function and the corresponding learning function values. The outputs of this function should be (1) a numpy array of samples to be added; (2) the learning function values at the new sample points, and (3) a boolean stopping criterion indicating whether the iterations should continue (False) or stop (True). The numpy array of samples should be a two-dimensional array with the first dimension being the number of samples and the second dimension being the number of variables. An example user-defined learning function is given below:

>>> class UserLearningFunction(LearningFunction):
>>>
>>>    def __init__(self, u_stop: int = 2):
>>>        self.u_stop = u_stop
>>>
>>>    def evaluate_function(self, distributions, n_add, surrogate, population, qoi=None, samples=None):
>>>        # AKMS class use these inputs to compute the learning function
>>>
>>>        g, sig = surrogate.predict(population,)
>>>
>>>        # Remove the inconsistency in the shape of 'g' and 'sig' array
>>>        g = g.reshape([population.shape[0], 1])
>>>        sig = sig.reshape([population.shape[0], 1])
>>>
>>>        u = abs(g) / sig
>>>        rows = u[:, 0].argsort()[:n_add]
>>>
>>>        indicator = False
>>>        if min(u[:, 0]) >= self.u_stop:
>>>            indicator = True
>>>
>>>        return population[rows, :], u[rows, 0], indicator