Although likelihood-based methods are theoretically appealing, deep generative models (DGMs) often produce unreliable likelihood estimates in practice, particularly for out-of-distribution (OOD) detection. We reinterpret variational autoencoders (VAEs) through the lens of fast and slow weights. Our approach is guided by the proposed Likelihood Path (LPath) Principle, which extends the classical likelihood principle. A critical decision in our method is the selection of statistics for classical density estimation algorithms. The sweet spot should contain just enough information that’s sufficient for OOD detection but not too much to suffer from the curse of dimensionality. Our LPath principle achieves this by selecting the sufficient statistics that form the “path” toward the likelihood. We demonstrate that this likelihood path leads to SOTA OOD detection performance, even when the likelihood itself is unreliable.
Bibtex
@inproceedings{
huang2024inference,
title={Inference, Fast and Slow: Reinterpreting {VAE}s for {OOD} Detection},
author={Sicong Huang and Jiawei He and Kry Yik-Chau Lui},
booktitle={Neurips Safe Generative AI Workshop 2024},
year={2024},
url={https://openreview.net/forum?id=K1VpgaYPnX}
}
Related Research
-
Unsupervised Event Outlier Detection in Continuous Time
Unsupervised Event Outlier Detection in Continuous Time
S. Nath, K. Y. C. Lui, and S. Liu. Workshop at Conference on Neural Information Processing Systems (NeurIPS)
Publications
-
LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling
LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling
C. Chen, G. Oliveira, H. Sharifi, and T. Sylvain. Workshop at Conference on Neural Information Processing Systems (NeurIPS)
Publications
-
Do LLMs Build World Representations? Probing Through the Lens of State Abstraction
Do LLMs Build World Representations? Probing Through the Lens of State Abstraction
Z. Li, Y. Cao, and J. C.K. Cheung. Conference on Neural Information Processing Systems (NeurIPS)
Publications