Post-hoc Uncertainty Calibration for Domain Drift Scenarios

Abstract

We address the problem of uncertainty calibration. While standard deep neural net-works typically yield uncalibrated predic-tions, calibrated confidence scores that arerepresentative of the true likelihood of a pre-diction can be achieved using post-hoc cali-bration methods. However, to date the focusof these approaches has been on in-domain calibration. Our contribution is two-fold.First, we show that existing post-hoc cali-bration methods yield highly over-confidentpredictions under domain shift. Second, weintroduce a simple strategy where perturba-tions are applied to samples in the valida-tion set before performing the post-hoc cal-ibration step. In extensive experiments, wedemonstrate that this perturbation step re-sults in substantially better calibration underdomain shift on a wide range of architecturesand modelling tasks.

Publication
In 2021 IEEE conference on Computer Vision and Pattern Recognition (CVPR)
Sebastian Gruber
Sebastian Gruber
Ph.D. Candidate

My research mostly revolve around Deep Learning topics with a strong theoretical background.

Florian Buettner
Florian Buettner
Professor for Bioinformatics in Oncology