BayesModelSelection

In the Bayesian approach to model selection, the posterior probability of each model is computed as

\[P(m_{i} \vert \mathcal{D}) = \frac{p(\mathcal{D} \vert m_{i})P(m_{i})}{\sum_{j} p(\mathcal{D} \vert m_{j})P(m_{j})}\]

where the evidence (also called marginal likelihood) \(p(\mathcal{D} \vert m_{i})\) involves an integration over the parameter space:

\[p(\mathcal{D} \vert m_{i}) = \int_{\Theta} p(\mathcal{D} \vert m_{i}, \theta) p(\theta \vert m_{i}) d\theta\]

Currently, calculation of the evidence is performed using the method of the harmonic mean [12]:

\[p(\mathcal{D} \vert m_{i}) = \left[ \frac{1}{B} \sum_{b=1}^{B} \frac{1}{p(\mathcal{D} \vert m_{i}, \theta_{b})} \right]^{-1}\]

where \(\theta_{1,\cdots,B}\) are samples from the posterior pdf of \(\theta\). In UQpy, these samples are obtained via the BayesParameterEstimation class. However, note that this method is known to yield evidence estimates with large variance. The Harmonic mean method for calculating the evidence is provided to the initializer of the BayesModelSelection class. No input parameters are required for its initialization.

The HarmonicMean class is imported using the following command:

>>> from UQpy.inference.evidence_methods.HarmonicMean import HarmonicMean

class HarmonicMean

Class used for the computation of model evidence using the harmonic mean method.

Future releases of UQpy will include more robust methods for computation of model evidences. Also, it is known that results of such Bayesian model selection procedure usually highly depends on the choice of prior for the parameters of the competing models, thus the user should carefully define such priors when creating instances of the InferenceModel class. The user can create custom methods for calculating evidences, by extending the EvidenceMethod abstract baseclass. To achieve that, a custom implementation of the estimate_evidence() must be provided.

EvidenceMethod Class

The EvidenceMethod class is imported using the following command:

>>> from UQpy.inference.evidence_methods.baseclass.EvidenceMethod import EvidenceMethod

class EvidenceMethod

abstract estimate_evidence(inference_model, posterior_samples, log_posterior_values)

Parameters:

• inference_model (InferenceModel) – Probabilistic model used for inference.

• posterior_samples (ndarray) – Samples drawn from the posterior distribution of the parameters using a BayesParameterEstimation object.

• log_posterior_values (ndarray) – Values of the log_pdf function generated during the sampling of the BayesParameterEstimation object.

Return type:

float

Returns:

The evidence of the inference specific model.

BayesModelSelection Class

The BayesModelSelection class is imported using the following command:

>>> from UQpy.inference.BayesModelSelection import BayesModelSelection

Methods

class BayesModelSelection(parameter_estimators, prior_probabilities=None, evidence_method=<UQpy.inference.evidence_methods.HarmonicMean.HarmonicMean object>, nsamples=None)

Perform model selection via Bayesian inference, i.e., compute model posterior probabilities given data.

This class leverages the BayesParameterEstimation class to get samples from the parameter posterior densities. These samples are then used to compute the model evidence p(data|model) for all models and the model posterior probabilities.

Parameters:

• data – Available data

• parameter_estimators (list[BayesParameterEstimation]) – Parameter estimators used during the model selection algorithm.

• prior_probabilities – Prior probabilities of each model, default is [1/nmodels, ] * nmodels

• evidence_method (EvidenceMethod) – as of v3, only the harmonic mean method is supported

• nsamples (Optional[list[int]]) – Number of samples used in MCMC/ImportanceSampling, for each model

run(nsamples)

Run the Bayesian model selection procedure, i.e., compute model posterior probabilities.

This function calls the run_estimation() method of the BayesParameterEstimation object for each model to sample from the parameter posterior probability, then computes the model evidence and model posterior probability. This function updates attributes bayes_estimators, evidences and probabilities. If nsamples are given when creating the object, this method is called directly when the object is created. It can also be called separately.

Parameters:

nsamples (Optional[list[int]]) – Number of samples used in MCMC/ImportanceSampling`, for each model

sort_models()

Sort models in descending order of model probability (increasing order of criterion value).

This function sorts - in place - the attribute lists candidate_models, probabilities and evidences so that they are sorted from most probable to the least probable model. It is a stand-alone function that is provided to help the user to easily visualize which model is the best.

No inputs/outputs.

Attributes

BayesModelSelection.candidate_models: list[InferenceModel]

Probabilistic models used during the model selection process.

BayesModelSelection.bayes_estimators: list[BayesParameterEstimation]

Results of the Bayesian parameter estimation.

BayesModelSelection.evidences: list

Value of the evidence for all models.

BayesModelSelection.probabilities: list

Posterior probability for all models

Examples

• Bayes Model Selection Examples