.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/surrogates/gpr/gpr_constraints.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_surrogates_gpr_gpr_constraints.py: Gaussian Process with Noise and Constraints ====================================================================== .. GENERATED FROM PYTHON SOURCE LINES 9-11 This jupyter script shows the performance of GaussianProcessRegressor class in the UQpy. A training data is generated using a function (:math:`f(x)`, as defined below), which is used to train a surrogate model. .. GENERATED FROM PYTHON SOURCE LINES 16-18 Import the necessary modules to run the example script. Notice that FminCobyla is used here, to solve the MLE optimization problem with constraints. .. GENERATED FROM PYTHON SOURCE LINES 21-34 .. code-block:: default import numpy as np import matplotlib.pyplot as plt import warnings from UQpy.surrogates.gaussian_process.regression_models.QuadraticRegression import QuadraticRegression warnings.filterwarnings('ignore') from UQpy.utilities.MinimizeOptimizer import MinimizeOptimizer from UQpy.utilities.FminCobyla import FminCobyla from UQpy.surrogates import GaussianProcessRegression, NonNegative, RBF .. GENERATED FROM PYTHON SOURCE LINES 35-38 Consider the following function :math:`f(x)`. .. math:: f(x) = \frac{1}{100} + \frac{5}{8}(2x-1)^4[(2x-1)^2 + 4\sin{(5 \pi x)^2}], \quad \quad x \in [0,1] .. GENERATED FROM PYTHON SOURCE LINES 41-47 .. code-block:: default def funct(x): y = (1 / 100) + (5 / 8) * ((2 * x - 1) ** 4) * (((2 * x - 1) ** 2) + 4 * np.sin(5 * np.pi * x) ** 2) return y .. GENERATED FROM PYTHON SOURCE LINES 48-49 Define the training data set. The following 13 points have been used to fit the GP. .. GENERATED FROM PYTHON SOURCE LINES 52-56 .. code-block:: default X_train = np.array([0, 0.06, 0.08, 0.26, 0.27, 0.4, 0.52, 0.6, 0.68, 0.81, 0.9, 0.925, 1]).reshape(-1, 1) y_train = funct(X_train) .. GENERATED FROM PYTHON SOURCE LINES 57-58 Define the test data. .. GENERATED FROM PYTHON SOURCE LINES 61-66 .. code-block:: default X_test = np.linspace(0, 1, 100).reshape(-1, 1) y_test = funct(X_test) .. GENERATED FROM PYTHON SOURCE LINES 67-74 Train GPR ~~~~~~~~~~~~~ - Noise - Constraints Here, 30 equidistant point are selected over the domain of :math:`x`, lets call them constraint points. The idea is to train the surrogate model such that the probability of positive surrogates prediction is very high at these points. .. GENERATED FROM PYTHON SOURCE LINES 77-81 .. code-block:: default X_c = np.linspace(0, 1, 31).reshape(-1,1) y_c = funct(X_c) .. GENERATED FROM PYTHON SOURCE LINES 82-91 In this approach, MLE problem is solved with the following constraints: .. math:: \hat{y}(x_c)-Z \sigma_{\hat{y}}(x_c) > 0 \quad \quad Z = 2 .. math:: |\hat{y}(x_t) - y(x_t)| < \epsilon \quad \quad \epsilon = 0.3 where, :math:`x_c` and :math:`x_t` are the constraint and training sample points, respectively. Define kernel used to define the covariance matrix. Here, the application of Radial Basis Function (RBF) kernel is demonstrated. .. GENERATED FROM PYTHON SOURCE LINES 94-97 .. code-block:: default kernel3 = RBF() .. GENERATED FROM PYTHON SOURCE LINES 98-99 Define the optimizer used to identify the maximum likelihood estimate. .. GENERATED FROM PYTHON SOURCE LINES 102-106 .. code-block:: default bounds_3 = [[10**(-6), 10**(-1)], [10**(-5), 10**(-1)], [10**(-13), 10**(-5)]] optimizer3 = FminCobyla() .. GENERATED FROM PYTHON SOURCE LINES 107-108 Define constraints for the Cobyla optimizer using UQpy's Nonnegatice class. .. GENERATED FROM PYTHON SOURCE LINES 111-114 .. code-block:: default cons = NonNegative(constraint_points=X_c, observed_error=0.03, z_value=2) .. GENERATED FROM PYTHON SOURCE LINES 115-117 Define the 'GaussianProcessRegressor' class object, the input attributes defined here are kernel, optimizer, initial estimates of hyperparameters and number of times MLE is identified using random starting point. .. GENERATED FROM PYTHON SOURCE LINES 120-125 .. code-block:: default gpr3 = GaussianProcessRegression(kernel=kernel3, hyperparameters=[10**(-3), 10**(-2), 10**(-10)], optimizer=optimizer3, optimizations_number=10, optimize_constraints=cons, bounds=bounds_3, noise=True, regression_model=QuadraticRegression()) .. GENERATED FROM PYTHON SOURCE LINES 126-127 Call the 'fit' method to train the surrogate model (GPR). .. GENERATED FROM PYTHON SOURCE LINES 130-133 .. code-block:: default gpr3.fit(X_train, y_train) .. GENERATED FROM PYTHON SOURCE LINES 134-135 The maximum likelihood estimates of the hyperparameters are as follows: .. GENERATED FROM PYTHON SOURCE LINES 138-146 .. code-block:: default print(gpr3.hyperparameters) print('Length Scale: ', gpr3.hyperparameters[0]) print('Process Variance: ', gpr3.hyperparameters[1]) print('Noise Variance: ', gpr3.hyperparameters[2]) .. GENERATED FROM PYTHON SOURCE LINES 147-149 Use 'predict' method to compute surrogate prediction at the test samples. The attribute 'return_std' is a boolean indicator. If 'True', 'predict' method also returns the standard error at the test samples. .. GENERATED FROM PYTHON SOURCE LINES 152-155 .. code-block:: default y_pred3, y_std3 = gpr3.predict(X_test, return_std=True) .. GENERATED FROM PYTHON SOURCE LINES 156-158 The plot shows the test function in dashed red line and 13 training points are represented by blue dots. Also, blue curve shows the GPR prediction for $x \in (0, 1)$ and yellow shaded region represents 95% confidence interval. .. GENERATED FROM PYTHON SOURCE LINES 161-178 .. code-block:: default fig, ax = plt.subplots(figsize=(8.5,7)) ax.plot(X_test,y_test,'r--',linewidth=2,label='Test Function') ax.plot(X_train,y_train,'bo',markerfacecolor='b', markersize=10, label='Training Data') ax.plot(X_test,y_pred3,'b-', lw=2, label='GP Prediction') ax.plot(X_test, np.zeros((X_test.shape[0],1))) ax.fill_between(X_test.flatten(), y_pred3-1.96*y_std3, y_pred3+1.96*y_std3, facecolor='yellow',label='95% Credibility Interval') ax.tick_params(axis='both', which='major', labelsize=12) ax.set_xlabel('x', fontsize=15) ax.set_ylabel('f(x)', fontsize=15) ax.set_ylim([-0.3,1.8]) ax.legend(loc="upper right",prop={'size': 12}); plt.grid() .. GENERATED FROM PYTHON SOURCE LINES 179-181 Verify the constraints for the trained surrogate model. Notice that all values are positive, thus constraints are satisfied for the constraint points. .. GENERATED FROM PYTHON SOURCE LINES 184-188 .. code-block:: default y_, ys_ = gpr3.predict(X_c, return_std=True) y_ - 2*ys_ .. GENERATED FROM PYTHON SOURCE LINES 189-190 Notice that all values are negative, thus constraints are satisfied for the training points. .. GENERATED FROM PYTHON SOURCE LINES 193-196 .. code-block:: default y_ = gpr3.predict(X_train, return_std=False) np.abs(y_train[:, 0]-y_) - 0.03 .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.000 seconds) .. _sphx_glr_download_auto_examples_surrogates_gpr_gpr_constraints.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/SURGroup/UQpy/master?urlpath=lab/tree/notebooks/auto_examples/surrogates/gpr/gpr_constraints.ipynb :alt: Launch binder :width: 150 px .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: gpr_constraints.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: gpr_constraints.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_