BayesParameterEstimation

Given some data \(\mathcal{D}\), a parameterized model for the data, and a prior probability density for the model parameters \(p(\theta)\), the BayesParameterEstimation class is leveraged to draw samples from the posterior pdf of the model parameters using Markov Chain Monte Carlo or Importance Sampling. Via Bayes theorem, the posterior pdf is defined as follows:

\[p(\theta \vert \mathcal{D}) = \frac{p(\mathcal{D} \vert \theta)p(\theta)}{p(\mathcal{D})}\]

Note that if no prior is defined in the model, the prior pdf is chosen as uninformative, i.e., \(p(\theta) = 1\) (cautionary note, this is an improper prior).

The BayesParameterEstimation leverages the MCMC or ImportanceSampling classes of the sampling module of UQpy. When creating a BayesParameterEstimation object, an object of class MCMC or ImportanceSampling is created and saved as an attribute sampler. The run() method of the BayesParameterEstimation class then calls the run() method of that sampler, thus the user can add samples as they wish by calling the run() method several times.

BayesParameterEstimation Class

The BayesParameterEstimation class is imported using the following command:

>>> from UQpy.inference.BayesParameterEstimation import BayesParameterEstimation

Methods

class BayesParameterEstimation(inference_model, data, sampling_class=None, nsamples=None)[source]

Estimate the parameter posterior density given some data.

This class generates samples from the parameter posterior distribution using Markov Chain Monte Carlo or Importance Sampling. It leverages the MCMC and ImportanceSampling classes from the sampling module.

Parameters:
run(nsamples=None)[source]

Run the Bayesian inference procedure, i.e., sample from the parameter posterior distribution.

This function calls the run() method of the sampler attribute to generate samples from the parameter posterior distribution.

Parameters:

nsamples (Optional[int]) – Number of samples used in MCMC/ImportanceSampling. In case an ImportanceSampling method is used to perform the parameter estimation, then nsamples equal to the total number of samples. In case an MCMC sampler is used, and the given nsamples is not a multiple of n_chains, then nsamples is set to the next largest integer that is a multiple of nchains.

Attributes

BayesParameterEstimation.sampler: Union[MCMC, ImportanceSampling]

Sampling method object, contains e.g. the posterior samples.

This must be created along side the BayesParameterEstimation object, and its run method is called whenever the run() method of the BayesParameterEstimation is called.

Examples

Below, an example of the BayesParameterEstimation usage is provided. The goal is to learn the parameters of a Normal distribution.

  • Initially, a two distributions are created based on the prior belief on each one of the unknown parameters and are subsequently merged into joint distribution.

  • The second step is to define the model whose parameters we want to infer. In this case as already mentioned the goal is to learn two parameters of the Normal distribution, so a Distribution model is defined, where the number of parameters n_parameters, as well as the prior distributions is provided.

  • Before initializing the final BayesParameterEstimation object, the user must provide a method to sample the posterior distribution of the candidate model. Here a child class of MCMC is chosen and specifically, MetropolisHastings. Apart from the various parameters required by the algorithm such as jump or burn_length, the user must specify the args_target and log_pdf_target parameters as follows:

    • args_target = (data, )

    • log_pdf_target = candidate_model.evaluate_log_posterior

  • Finally, the BayesParameterEstimation object is created, with input, the sampling, the candidate model whose parameters we want to learn, the data, as well as the number of samples to be drawn from the posterior distribution. Since nsamples is provided at the object initialization, the BayesParameterEstimation will be automatically performed. Alternatively, the user must call the run() method.

>>> p0 = Uniform(loc=0., scale=15)
>>> p1 = Lognormal(s=1., loc=0., scale=1.)
>>> prior = JointIndependent(marginals=[p0, p1])
>>>
>>> # create an instance of class Model
>>> candidate_model = DistributionModel(distributions=Normal(loc=None, scale=None),
>>>                                     n_parameters=2, prior=prior)
>>>
>>> sampling = MetropolisHastings(jump=10, burn_length=10, seed=[1.0, 0.2], random_state=1,
>>>                               args_target=(data, ),
>>>                               log_pdf_target=candidate_model.evaluate_log_posterior)
>>>
>>> bayes_estimator = BayesParameterEstimation(sampling_class=sampling,
>>>                                            inference_model=candidate_model,
>>>                                            data=data,
>>>                                            nsamples=5)