likelihood

Likelihood expressions, negative log-likelihoods, and stochastic likelihoods. These are currently exposed from laddu.extensions in Python.

class laddu.extensions.ControlFlow

An enum used by a terminator to continue or stop an algorithm.

Break = ControlFlow.Break

Continue = ControlFlow.Continue

class laddu.extensions.EnsembleStatus

The status of an ensemble-method algorithm.

chain(*, burn=None, thin=None, flat=False)

Get the retained chain after optional burn-in and thinning.

Parameters:

burn (int | None, optional) – Number of retained steps discarded from the front of each walker chain.
thin (int | None, optional) – Retain every thin-th sample after burn-in.
flat (bool, default=False) – If False, return the chain with shape (n_walkers, n_steps, n_dim). If True, flatten the walker and step dimensions and return shape (n_samples, n_dim).

Return type:

numpy.ndarray

to_dict()

Export the EnsembleStatus as a plain Python dictionary.

Return type:: dict

dimension

Current dimension of the ensemble.

Returns:: Dimensions in (n_walkers, n_steps, n_variables) order.
Return type:: tuple[int, int, int]

message

The status message emitted by the algorithm at the current iteration.

Return type:: StatusMessage

message_text

The text content of the status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

n_g_evals

Number of gradient evaluations.

Return type:: int

status_type

The type of status.

Return type:: str

success

An indicator of the convergence/success state of an algorithm.

Return type:: bool

class laddu.extensions.GradientFreeStatus

The status of a gradient-free minimization algorithm.

to_dict()

Export the GradientFreeStatus as a plain Python dictionary.

Return type:: dict

cov

Current covariance matrix.

Return type:: numpy.ndarray or None

err

Current parameter uncertainties.

Return type:: numpy.ndarray or None

fx

Current objective value.

Return type:: float

hess

Current Hessian.

Return type:: numpy.ndarray or None

message

The status message emitted by the algorithm at the current iteration.

Return type:: StatusMessage

message_text

The text content of the status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

status_type

The type of status.

Return type:: str

success

An indicator of the convergence/success state of an algorithm.

Return type:: bool

x

Current parameter vector.

Return type:: numpy.ndarray

class laddu.extensions.GradientStatus

The status of a minimization algorithm which uses gradient information.

to_dict()

Export the GradientStatus as a plain Python dictionary.

Return type:: dict

cov

Current covariance matrix.

Return type:: numpy.ndarray or None

err

Current parameter uncertainties.

Return type:: numpy.ndarray or None

fx

Current objective value.

Return type:: float

hess

Current Hessian.

Return type:: numpy.ndarray or None

message

The status message emitted by the algorithm at the current iteration.

Return type:: StatusMessage

message_text

The text content of the status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

n_g_evals

Number of gradient evaluations.

Return type:: int

n_h_evals

Number of Hessian evaluations.

Return type:: int

status_type

The type of status.

Return type:: str

success

An indicator of the convergence/success state of an algorithm.

Return type:: bool

x

Current parameter vector.

Return type:: numpy.ndarray

class laddu.extensions.LikelihoodExpression

Python wrapper for [LikelihoodExpression].

evaluate(parameters, *, threads=None)

Evaluate the sum of all terms in the expression.

Parameters:

parameters (list of float) – Parameter values for the free parameters (length n_free).
threads (int, optional) – The number of threads to use (setting this to None or 0 uses the current global or context-managed default; any positive value overrides that default for this call only)

Returns:

result – The total negative log-likelihood summed over all terms

Return type:

float

Raises:

Exception – If there was an error building the thread pool

evaluate_gradient(parameters, *, threads=None)

Evaluate the gradient of the sum of all terms in the expression.

Parameters:

parameters (list of float) – Parameter values for the free parameters (length n_free).
threads (int, optional) – The number of threads to use (setting this to None or 0 uses the current global or context-managed default; any positive value overrides that default for this call only)

Returns:

result – A numpy array representing the gradient of the sum of all terms in the evaluator with length n_free.

Return type:

array_like

Raises:

Exception – If there was an error building the thread pool or problem creating the resulting numpy array

fix_parameter(name, value)

Fix a parameter to a constant value.

Parameters:

name (str) – Name of the parameter.
value (float) – Value used during evaluation.

free_parameter(name)

Free a parameter that was previously fixed.

Parameters:: name (str) – Name of the parameter.

mcmc(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

Run an MCMC algorithm on the free parameters of the LikelihoodTerm’s model

This method can be used to sample the underlying likelihood term given an initial position for each walker p0.

Parameters:

p0 (array_like or ganesh.AIESInit or ganesh.ESSInit) – Initial sampler state. Use a 2D walker matrix with shape (n_walkers, n_parameters) for the common case, or pass an explicit Ganesh init object for method-specific initialization.
method ({'aies', 'ess'}) – The MCMC algorithm to use
config (ganesh config object, optional) – Method-specific Ganesh configuration, such as ganesh.AIESConfig or ganesh.ESSConfig.
options (ganesh options object, optional) – Method-specific Ganesh options object controlling step limits, built-in observers, and built-in terminators.
observers (MCMCObserver or list of MCMCObserver, optional) – User-defined observers which are called at each step
terminators (MCMCTerminator or list of MCMCTerminator, optional) – User-defined terminators which are called at each step
threads (int, default=0) – The number of threads to use (setting this to 0 uses the current global or context-managed default; any positive value overrides that default for this call only)

Returns:

The status of the MCMC algorithm at termination

Return type:

MCMCSummary

Raises:

Exception – If there was an error building the thread pool

See also

NLL.mcmc, StochasticNLL.mcmc

Examples

>>> from laddu import LikelihoodScalar, likelihood_sum
>>> import ganesh
>>> expression = likelihood_sum([LikelihoodScalar('alpha')])
>>> summary = expression.mcmc(
...     [[0.0], [0.4]],
...     method='aies',
...     options=ganesh.AIESOptions(max_steps=4),
... )
>>> summary.dimension[2]
1
>>> summary.chain(flat=True).shape[1]
1

Notes

config and options use Ganesh’s Python API directly. For example, custom move mixes belong in ganesh.AIESConfig or ganesh.ESSConfig, while ganesh.AIESOptions(max_steps=...) and ganesh.ESSOptions(max_steps=...) control run limits.

References

Goodman, J. & Weare, J. (2010). Ensemble samplers with affine invariance. CAMCoS 5(1), 65–80. <https://doi.org/10.2140/camcos.2010.5.65>

Karamanis, M. & Beutler, F. (2021). Ensemble slice sampling. Stat Comput 31(5). <https://doi.org/10.1007/s11222-021-10038-2>

minimize(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

Minimize the LikelihoodTerm with respect to the free parameters in the model

This method “runs the fit”. Given an initial position p0, this method performs a minimization over the likelihood term, optimizing the model over the stored signal data and Monte Carlo.

Parameters:

p0 (array_like or ganesh.NelderMeadInit or ganesh.PSOInit or ganesh.CMAESInit or ganesh.DifferentialEvolutionInit) – Initial state for the selected minimizer. Use a length-n_free vector for lbfgsb, adam, conjugate-gradient, and trust-region; either a vector or ganesh.NelderMeadInit for nelder-mead; a ganesh.CMAESInit for cma-es; a ganesh.DifferentialEvolutionInit for differential-evolution, and either a 2D swarm array or ganesh.PSOInit for pso.
method ({'lbfgsb', 'adam', 'conjugate-gradient', 'trust-region', 'nelder-mead', 'cma-es', 'differential-evolution', 'pso'}) – The minimization algorithm to use
config (ganesh config object, optional) – Method-specific Ganesh configuration, such as ganesh.LBFGSBConfig or ganesh.PSOConfig. Bounds are configured here when supported by the method.
options (ganesh options object, optional) – Method-specific Ganesh options object controlling step limits, built-in observers, and built-in terminators.
observers (MinimizerObserver or list of MinimizerObserver, optional) – User-defined observers which are called at each step
terminators (MinimizerTerminator or list of MinimizerTerminator, optional) – User-defined terminators which are called at each step
threads (int, default=0) – The number of threads to use (setting this to 0 uses the current global or context-managed default; any positive value overrides that default for this call only)

Returns:

A summary of the minimization algorithm at termination

Return type:

MinimizationSummary

Raises:

Exception – If there was an error building the thread pool

Examples

>>> import ganesh
>>> expression.minimize(
...     [1.0],
...     method='lbfgsb',
...     options=ganesh.LBFGSBOptions(max_steps=150),
... )

Notes

config and options use Ganesh’s Python API directly. For example, pass ganesh.LBFGSBConfig(bounds=[...]) for bounded L-BFGS-B, or ganesh.AdamOptions(max_steps=500) to cap the number of Adam iterations.

References

Gao, F. & Han, L. (2010). Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Comput. Optim. Appl. 51(1), 259–277. <https://doi.org/10.1007/s10589-010-9329-3>

Lagarias, J. C., Reeds, J. A., Wright, M. H., & Wright, P. E. (1998). Convergence Properties of the Nelder–Mead Simplex Method in Low Dimensions. SIAM J. Optim. 9(1), 112–147. <https://doi.org/10.1137/S1052623496303470>

Singer, S. & Singer, S. (2004). Efficient Implementation of the Nelder–Mead Search Algorithm. Appl. Numer. Anal. & Comput. 1(2), 524–534. <https://doi.org/10.1002/anac.200410015>

rename_parameter(old, new)

Rename a parameter.

Parameters:

old (str) – Current parameter name.
new (str) – Desired parameter name.

rename_parameters(mapping)

Rename multiple parameters at once.

Parameters:: mapping (dict[str, str]) – Mapping from old names to new names.

n_fixed: Number of fixed parameters in the expression.

n_free: Number of free parameters in the expression.

n_parameters: Total number of parameters (free + fixed).

parameters: Parameters referenced by the expression.

class laddu.extensions.MCMCObserver

abstractmethod observe(step: int, status: EnsembleStatus) → None

class laddu.extensions.MCMCSummary

Python-facing typed wrapper for [MCMCSummary].

Notes

Numeric arrays are exposed as NumPy arrays. Chain post-processing methods use keyword-only burn and thin arguments so the retained-chain view is explicit at each call site.

chain(*, burn=None, thin=None, flat=False)

Get the retained chain after optional burn-in and thinning.

Parameters:

burn (int | None, optional) – Number of retained steps discarded from the front of each walker chain.
thin (int | None, optional) – Retain every thin-th sample after burn-in.
flat (bool, default=False) – If False, return the chain with shape (n_walkers, n_steps, n_dim). If True, flatten the walker and step dimensions and return shape (n_samples, n_dim).

Return type:

numpy.ndarray

chain_storage()

Chain-storage description.

Return type:: dict

diagnostics(*, burn=None, thin=None)

Compute convergence and efficiency diagnostics.

Parameters:

burn (int | None, optional) – Number of retained steps discarded from the front of each walker chain.
thin (int | None, optional) – Retain every thin-th sample after burn-in.

Returns:

Dictionary with r_hat, ess, acceptance_rates, and mean_acceptance_rate entries.

Return type:

dict

to_dict()

Export the MCMCSummary as a plain Python dictionary.

Return type:: dict

bounds

Optional parameter bounds.

Return type:: list[tuple[float | None, float | None]] | None

dimension

Retained-chain dimensions.

Returns:: Dimensions in (n_walkers, n_steps, n_variables) order.
Return type:: tuple[int, int, int]

message_text

Human-readable status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

n_g_evals

Number of gradient evaluations.

Return type:: int

n_h_evals

Number of Hessian evaluations.

Return type:: int

parameter_names

Optional parameter names.

Return type:: list[str] | None

status_type

Summary status type.

Return type:: str

success

Whether the run reported success.

Return type:: bool

class laddu.extensions.MCMCTerminator

abstractmethod check_for_termination(step: int, status: EnsembleStatus) → ControlFlow

class laddu.extensions.MinimizationObserver

abstractmethod observe(step: int, status: GradientStatus | GradientFreeStatus | SwarmStatus) → None

class laddu.extensions.MinimizationSummary

The result of a minimization algorithm.

to_dict()

Export the MinimizationSummary as a plain Python dictionary.

Return type:: dict

bounds

Optional parameter bounds.

Returns:: Bounds represented as (lower, upper) pairs, where None means unbounded on that side.
Return type:: list[tuple[float | None, float | None]] | None

covariance

Estimated covariance matrix.

Return type:: numpy.ndarray

fx

Final objective value.

Return type:: float

message_text

Human-readable status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

n_g_evals

Number of gradient evaluations.

Return type:: int

n_h_evals

Number of Hessian evaluations.

Return type:: int

parameter_names

Optional parameter names.

Return type:: list[str] | None

status_type

Summary status type.

Return type:: str

std

Parameter standard deviations.

Return type:: numpy.ndarray

success

Whether the run reported success.

Return type:: bool

x

Final parameter vector.

Return type:: numpy.ndarray

x0

Initial parameter vector.

Return type:: numpy.ndarray

class laddu.extensions.MinimizationTerminator

abstractmethod check_for_termination(step: int, status: GradientStatus | GradientFreeStatus | SwarmStatus) → ControlFlow

class laddu.extensions.NLL(expression, ds_data, ds_accmc, *, n_mc=None)

A (extended) negative log-likelihood evaluator.

activate(arg, *, strict=True)

activate_all()

deactivate(arg, *, strict=True)

deactivate_all()

evaluate(parameters, *, threads=None)

evaluate_gradient(parameters, *, threads=None)

fix_parameter(name, value)

free_parameter(name)

isolate(arg, *, strict=True)

mcmc(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

minimize(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

project_weights(parameters, *, subset=None, subsets=None, strict=False, mc_evaluator=None, threads=None)

project_weights_and_gradients(parameters, *, subset=None, subsets=None, strict=False, mc_evaluator=None, threads=None)

rename_parameter(old, new)

rename_parameters(mapping)

to_expression()

to_stochastic(batch_size, *, seed=None)

accmc

accmc_evaluator

compiled_expression

data

data_evaluator

expression

n_fixed

n_free

n_parameters

parameters

class laddu.extensions.StochasticNLL

A stochastic (extended) negative log-likelihood evaluator.

mcmc(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

minimize(p0, *, method=Ellipsis, config=None, options=None, observers=None, terminators=None, threads=0)

compiled_expression

expression

nll

class laddu.extensions.SwarmStatus

The status of an particle swarm algorithm.

to_dict()

Export the SwarmStatus as a plain Python dictionary.

Return type:: dict

gbest

The current global best point across all swarm members.

Return type:: Point

initial_gbest

The initial global best point across all swarm members.

Return type:: Point

message

The status message emitted by the algorithm at the current iteration.

Return type:: StatusMessage

message_text

The text content of the status message.

Return type:: str

n_f_evals

Number of cost-function evaluations.

Return type:: int

status_type

The type of status.

Return type:: str

success

An indicator of the convergence/success state of an algorithm.

Return type:: bool

swarm

The current state of the swarm.

Return type:: Swarm

laddu.extensions.LikelihoodOne()

A convenience constructor for a unit-valued [LikelihoodExpression].

Returns:: An expression that evaluates to 1 for any parameter values.
Return type:: LikelihoodExpression

See also

LikelihoodZero, likelihood_product

Examples

>>> from laddu import LikelihoodOne
>>> LikelihoodOne().evaluate([])
1.0

laddu.extensions.LikelihoodScalar(name)

A parameterized scalar term which can be converted into a [LikelihoodExpression].

Parameters:: name (str) – The name of the new scalar parameter.
Returns:: A [LikelihoodExpression] representing a single free scaling parameter.
Return type:: LikelihoodExpression

See also

likelihood_sum, likelihood_product

Examples

>>> from laddu import LikelihoodScalar, likelihood_sum
>>> expr = likelihood_sum([LikelihoodScalar('alpha')])
>>> expr.evaluate([1.25])
1.25

laddu.extensions.LikelihoodZero()

A convenience constructor for a zero-valued [LikelihoodExpression].

Returns:: An expression that evaluates to 0 for any parameter values.
Return type:: LikelihoodExpression

See also

LikelihoodOne, likelihood_sum

Examples

>>> from laddu import LikelihoodZero
>>> expression = LikelihoodZero()
>>> expression.parameters
[]
>>> expression.evaluate([])
0.0

laddu.extensions.integrated_autocorrelation_times(samples, *, c=None)

Calculate the integrated autocorrelation time for each parameter according to Karamanis & Beutler (2021).

Parameters:

samples (array_like) – An array of dimension (n_walkers, n_steps, n_parameters).
c (float, default = 7.0) – The time window for Sokal’s autowindowing function. If omitted, the default window size of 7.0 is used.

Return type:

array of shape (n_parameters,)

Examples

>>> import numpy as np
>>> from laddu import integrated_autocorrelation_times
>>> samples = np.random.randn(4, 16, 2).tolist()
>>> integrated_autocorrelation_times(samples).shape
(2,)

References

Karamanis, M. & Beutler, F. (2021). Ensemble slice sampling. Stat. Comput. 31(5). <https://doi.org/10.1007/s11222-021-10038-2>

Sokal, A. (1997). Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms. NATO ASI Series, 131–192. <https://doi.org/10.1007/978-1-4899-0319-8_6>

laddu.extensions.likelihood_product(terms)

A convenience method to multiply sequences of [LikelihoodExpression]s.

Parameters:: terms (sequence of LikelihoodExpression) – A non-empty sequence whose elements are multiplied. Single-element sequences are returned unchanged while empty sequences evaluate to LikelihoodOne.
Returns:: A new expression representing the product of all inputs.
Return type:: LikelihoodExpression

See also

likelihood_sum, LikelihoodOne

Examples

>>> from laddu import LikelihoodScalar, likelihood_product
>>> expression = likelihood_product([LikelihoodScalar('alpha'), LikelihoodScalar('beta')])
>>> expression.parameters
['alpha', 'beta']
>>> expression.evaluate([2.0, 3.0])
6.0

Notes

When parameters overlap between inputs, the parameter definition from the earliest term is used.

laddu.extensions.likelihood_sum(terms)

A convenience method to sum sequences of [LikelihoodExpression]s or identifiers.

Parameters:: terms (sequence of LikelihoodExpression) – A non-empty sequence whose elements are summed. Single-element sequences are returned unchanged while empty sequences evaluate to LikelihoodZero.
Returns:: A new expression representing the sum of all inputs.
Return type:: LikelihoodExpression

See also

likelihood_product, LikelihoodZero

Examples

>>> from laddu import LikelihoodScalar, likelihood_sum
>>> expression = likelihood_sum([LikelihoodScalar('alpha')])
>>> expression.evaluate([0.5])
0.5
>>> likelihood_sum([]).evaluate([])
0.0

Notes

When multiple inputs share the same parameter name, the value and fixed/free status from the earliest term in the sequence take precedence.