MLE

The MLE class evaluates the maximum likelihood estimate \(\hat{\theta}\) of the model parameters, i.e.

\[\hat{\theta} = \text{argmax}_{\Theta} \quad p(\mathcal{D} \vert \theta)\]

Note: for a Gaussian-error model of the form \(\mathcal{D}=h(\theta)+\epsilon\), \(\epsilon \sim N(0, \sigma)\) with fixed \(\sigma\) and independent measurements \(\mathcal{D}_{i}\), maximizing the likelihood is mathematically equivalent to minimizing the sum of squared residuals \(\sum_{i} \left( \mathcal{D}_{i}-h(\theta) \right)^{2}\).

A numerical optimization procedure is performed to compute the MLE. By default, the minimize() function of the scipy.optimize module is used, however other optimizers can be leveraged via the optimizer input of the MLE class.

MLE Class

The MLE class is imported using the following command:

>>> from UQpy.inference.MLE import MLE

Methods

class MLE(inference_model, data, n_optimizations=1, initial_parameters=None, optimizer=<UQpy.utilities.MinimizeOptimizer.MinimizeOptimizer object>, random_state=None)[source]

Estimate the maximum likelihood parameters of a model given some data.

Parameters:

  • inference_model (InferenceModel) – The inference model that defines the likelihood function.

  • data (Union[list, ndarray]) – Available data, numpy.ndarray of shape consistent with log likelihood function in InferenceModel. If this parameter is provided at InformationModelSelection object initialization, the maximum likelihood estimation algorithm will be automatically performed. Alternatively, the user must execute the run() method.

  • n_optimizations (Optional[int]) – Number of iterations that the optimization is run, starting at random initial guesses. It is only used if initial_parameters is not provided. Default is \(1\). The random initial guesses are sampled uniformly between \(0\) and \(1\), or uniformly between user-defined bounds if an input bounds is provided as a keyword argument to the optimizer input parameter.

  • initial_parameters (Union[list, ndarray, None]) – Initial guess(es) for optimization, numpy.ndarray of shape (nstarts, n_parameters) or (n_parameters, ), where nstarts is the number of times the optimizer will be called. Alternatively, the user can provide input n_optimizations to randomly sample initial guess(es). The identified MLE is the one that yields the maximum log likelihood over all calls of the optimizer.

  • optimizer – This parameter takes as input an object that implements the Optimizer class. Default is the Minimize which utilizes the scipy.optimize.minimize method.

  • random_state (Union[None, int, RandomState]) – Random seed used to initialize the pseudo-random number generator. Default is None.

run(n_optimizations=1, initial_parameters=None)[source]

Run the maximum likelihood estimation procedure.

This function runs the optimization and updates the mle and max_log_like attributes of the class. When learning the parameters of a distribution, if distributions possesses an mle() method this method is used. If data are given when creating the MLE object, this method is called automatically when the object is created.

Parameters:

  • n_optimizations (Optional[int]) – Number of iterations that the optimization is run, starting at random initial guesses. It is only used if initial_parameters is not provided. Default is \(1\). The random initial guesses are sampled uniformly between \(0\) and \(1\), or uniformly between user-defined bounds if an input bounds is provided as a keyword argument to the optimizer input parameter.

  • initial_parameters (Union[list, ndarray, None]) – Initial guess(es) for optimization, numpy.ndarray of shape (nstarts, n_parameters) or (n_parameters, ), where nstarts is the number of times the optimizer will be called. Alternatively, the user can provide input n_optimizations to randomly sample initial guess(es). The identified MLE is the one that yields the maximum log likelihood over all calls of the optimizer.

Attributes

MLE.mle: ndarray

Value of parameter vector that maximizes the likelihood function.

MLE.max_log_like: ndarray

Value of the likelihood function at the MLE.

Examples