Package: loo 2.9.0

Jonah Gabry

loo: Efficient Leave-One-Out Cross-Validation and WAIC for Bayesian Models

Efficient approximate leave-one-out cross-validation (LOO) for Bayesian models fit using Markov chain Monte Carlo, as described in Vehtari, Gelman, and Gabry (2017) <doi:10.1007/s11222-016-9696-4>. The approximation uses Pareto smoothed importance sampling (PSIS), a new procedure for regularizing importance weights. As a byproduct of the calculations, we also obtain approximate standard errors for estimated predictive errors and for the comparison of predictive errors between models. The package also provides methods for using stacking and other model weighting techniques to average Bayesian predictive distributions.

Authors:Aki Vehtari [aut], Jonah Gabry [cre, aut], Måns Magnusson [aut], Yuling Yao [aut], Paul-Christian Bürkner [aut], Topi Paananen [aut], Andrew Gelman [aut], Ben Goodrich [ctb], Juho Piironen [ctb], Bruno Nicenboim [ctb], Leevi Lindgren [ctb], Visruth Srimath Kandali [ctb]

loo_2.9.0.tar.gz
loo_2.9.0.zip(r-4.7)loo_2.9.0.zip(r-4.6)loo_2.9.0.zip(r-4.5)
loo_2.9.0.tgz(r-4.6-any)loo_2.9.0.tgz(r-4.5-any)
loo_2.9.0.tar.gz(r-4.7-any)loo_2.9.0.tar.gz(r-4.6-any)
loo_2.9.0.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
loo/json (API)

# Install 'loo' in R:
install.packages('loo', repos = c('https://staging.r-multiverse.org', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/stan-dev/loo/issues

Pkgdown/docs site:https://mc-stan.org

Datasets:
  • Kline - Datasets for loo examples and vignettes
  • milk - Datasets for loo examples and vignettes
  • voice - Datasets for loo examples and vignettes
  • voice_loo - Datasets for loo examples and vignettes

On CRAN:

Conda:

bayesbayesianbayesian-data-analysisbayesian-inferencebayesian-methodsbayesian-statisticscross-validationinformation-criterionmodel-comparisonstan

18.67 score 155 stars 314 packages 4.1k scripts 761k downloads 29 mentions 65 exports 19 dependencies

Last updated from:299a4c7af1. Checks:9 OK. Indexed: no.

TargetResultTimeFilesSyslog
linux-devel-x86_64OK826
source / vignettesOK893
linux-release-x86_64OK294
macos-release-arm64OK147
macos-oldrel-arm64OK187
windows-develOK1400
windows-releaseOK339
windows-oldrelOK177
wasm-releaseOK230

Exports:.compute_point_estimate.ndraws.thin_drawscomparecrpsE_looelpdexample_loglik_arrayexample_loglik_matrixextract_log_likfind_model_namesgpdfitis.kfoldis.loois.psisis.psis_loois.sisis.tisis.waickfoldkfold_split_groupedkfold_split_randomkfold_split_stratifiedlooloo_approximate_posteriorloo_approximate_posterior.arrayloo_approximate_posterior.functionloo_approximate_posterior.matrixloo_compareloo_crpsloo_iloo_model_weightsloo_model_weights.defaultloo_moment_matchloo_moment_match.defaultloo_predictive_metricloo_scrpsloo_subsampleloo_subsample.functionloo.arrayloo.functionloo.matrixmcse_loonlistobs_idxpareto_k_idspareto_k_influence_valuespareto_k_tablepareto_k_valuespointwiseprint_dimspseudobma_weightspsispsis_n_eff_valuespsislwrelative_effscrpssisstacking_weightstiswaicwaic.arraywaic.functionwaic.matrixweights.importance_sampling

Dependencies:abindbackportscheckmateclidistributionalgenericsgluelifecyclemagrittrmatrixStatsnumDerivpillarpkgconfigposteriorrlangtensorAtibbleutf8vctrs

Holdout validation and K-fold cross-validation of Stan programs with the loo package
Introduction | Example: Eradication of Roaches using holdout validation approach | Coding the Stan model | Setup | Holdout validation | Splitting the data between train and test | Fitting the model with RStan | Computing holdout elpd: | K-fold cross validation | Splitting the data in folds | Fitting and extracting the log pointwise predictive densities for each fold | Computing K-fold elpd: | References
Rendered fromloo2-elpd.Rmdusingknitr::rmarkdown

Last update: 2025-12-22
Started: 2020-12-04

Avoiding model refits in leave-one-out cross-validation with moment matching
Introduction | Example: Eradication of Roaches | Coding the Stan model | Setup | Fitting the model with RStan | Moment matching correction for importance sampling | Using loo_moment_match() directly | References
Rendered fromloo2-moment-matching.Rmdusingknitr::rmarkdown

Last update: 2025-12-22
Started: 2020-03-05

Using the loo package (version >= 2.0.0)
Introduction | Setup | Example: Poisson vs negative binomial for the roaches dataset | Background and model fitting | Roaches data | Fit Poisson model | Using the loo package for model checking and comparison | Computing PSIS-LOO and checking diagnostics | Plotting Pareto $k$ diagnostics | Marginal posterior predictive checks | Try alternative model with more flexibility | Comparing the models on expected log predictive density | References
Rendered fromloo2-example.Rmdusingknitr::rmarkdown

Last update: 2025-12-22
Started: 2018-04-04

Approximate leave-future-out cross-validation for Bayesian time series models
Introduction | $M$-step-ahead predictions | Approximate $M$-SAP using importance-sampling | Autoregressive models | Case Study: Annual measurements of the level of Lake Huron | 1-step-ahead predictions leaving out all future values | Exact 1-step-ahead predictions | Approximate 1-step-ahead predictions | $M$-step-ahead predictions leaving out all future values | Exact $M$-step-ahead predictions | Approximate $M$-step-ahead predictions | Conclusion | References | Appendix | Appendix: Session information | Appendix: Licenses
Rendered fromloo2-lfo.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2018-10-16

Bayesian Stacking and Pseudo-BMA weights using the loo package
Introduction | Setup | Example: Primate milk | Example: Oceanic tool complexity | Simpler coding using loo_model_weights function | References
Rendered fromloo2-weights.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2018-04-04

Leave-one-out cross-validation for non-factorized models
Introduction | LOO-CV for multivariate normal models | Approximate LOO-CV using integrated importance-sampling | Exact LOO-CV with re-fitting | Lagged SAR models | Case Study: Neighborhood Crime in Columbus, Ohio | Fit lagged SAR model | Approximate LOO-CV | Exact LOO-CV | Working with Stan directly | Conclusion | References
Rendered fromloo2-non-factorized.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2020-06-15

Mixture IS leave-one-out cross-validation for high-dimensional Bayesian models
Introduction | Setup: load packages and set seed | Model | Dataset | PSIS estimators and Pareto-$k$ diagnostics | Mixture estimators | Comparison with benchmark values obtained with long simulations | References
Rendered fromloo2-mixis.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2023-03-23

Using Leave-one-out cross-validation for large data
Introduction | Setup | Example: Well water in Bangladesh | Coding the Stan model | Fitting the model with RStan | Approximate LOO-CV using PSIS-LOO and subsampling | Adding additional subsamples | Specifying estimator and sampling method | Approximate LOO-CV using PSIS-LOO with posterior approximations | Combining the posterior approximation method with subsampling | Comparing models | References
Rendered fromloo2-large-data.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2019-09-17

Writing Stan programs for use with the loo package
Introduction | Example: Well water in Bangladesh | Coding the Stan model | Fitting the model with RStan | Computing approximate leave-one-out cross-validation using PSIS-LOO | Comparing models | References
Rendered fromloo2-with-rstan.Rmdusingknitr::rmarkdown

Last update: 2024-04-15
Started: 2018-04-04

Readme and manuals

Help Manual

Help pageTopics
Efficient LOO-CV and WAIC for Bayesian modelsloo-package
Pareto smoothed importance sampling (PSIS) using approximate posteriorsap_psis ap_psis.array ap_psis.default ap_psis.matrix
Model comparison (deprecated, old version)compare
Continuously ranked probability scorecrps crps.matrix crps.numeric loo_crps loo_crps.matrix loo_scrps loo_scrps.matrix scrps scrps.matrix scrps.numeric
Compute weighted expectationsE_loo E_loo.default E_loo.matrix
Generic (expected) log-predictive densityelpd elpd.array elpd.matrix
Objects to use in examples and testsexample_loglik_array example_loglik_matrix
Extract pointwise log-likelihood from a Stan modelextract_log_lik
Estimate parameters of the Generalized Pareto distributiongpdfit
A parent class for different importance sampling methods.importance_sampling importance_sampling.array importance_sampling.default importance_sampling.matrix
Generic function for K-fold cross-validation for developersis.kfold kfold kfold-generic
Helper functions for K-fold cross-validationkfold-helpers kfold_split_grouped kfold_split_random kfold_split_stratified
Efficient approximate leave-one-out cross-validation (LOO)is.loo is.psis_loo loo loo.array loo.function loo.matrix loo_i
Efficient approximate leave-one-out cross-validation (LOO) for posterior approximationsloo_approximate_posterior loo_approximate_posterior.array loo_approximate_posterior.function loo_approximate_posterior.matrix
Model comparisonloo_compare loo_compare.default print.compare.loo print.compare.loo_ss
Model averaging/weighting via stacking or pseudo-BMA weightingloo_model_weights loo_model_weights.default pseudobma_weights stacking_weights
Moment matching for efficient approximate leave-one-out cross-validation (LOO)loo_moment_match loo_moment_match.default
Split moment matching for efficient approximate leave-one-out cross-validation (LOO)loo_moment_match_split
Estimate leave-one-out predictive performance..loo_predictive_metric loo_predictive_metric.matrix
Efficient approximate leave-one-out cross-validation (LOO) using subsampling, so that less costly and more approximate computation is made for all LOO-fold, and more costly and accurate computations are made only for m<N LOO-folds.loo_subsample loo_subsample.function
Datasets for loo examples and vignettesKline loo-datasets milk voice voice_loo
LOO package glossaryloo-glossary
The number of observations in a 'psis_loo_ss' object.nobs.psis_loo_ss
Get observation indices used in subsamplingobs_idx
Diagnostics for Pareto smoothed importance sampling (PSIS)mcse_loo pareto-k-diagnostic pareto_k_ids pareto_k_influence_values pareto_k_table pareto_k_values plot.loo plot.psis plot.psis_loo psis_n_eff_values
Convenience function for extracting pointwise estimatespointwise pointwise.loo
Print methodsprint.importance_sampling print.importance_sampling_loo print.loo print.psis print.psis_loo print.psis_loo_ap print.waic
Pareto smoothed importance sampling (PSIS)is.psis is.sis is.tis psis psis.array psis.default psis.matrix
Pareto smoothed importance sampling (deprecated, old version)psislw
Convenience function for computing relative efficienciesrelative_eff relative_eff.array relative_eff.default relative_eff.function relative_eff.importance_sampling relative_eff.matrix
Standard importance sampling (SIS)sis sis.array sis.default sis.matrix
Truncated importance sampling (TIS)tis tis.array tis.default tis.matrix
Update 'psis_loo_ss' objectsupdate.psis_loo_ss
Widely applicable information criterion (WAIC)is.waic waic waic.array waic.function waic.matrix
Extract importance sampling weightsweights.importance_sampling