| Téléchargement | - Voir la version finale : Region-dependent temperature scaling for certainty calibration and application to class-imbalanced token classification (PDF, 2.9 Mio)
|
|---|
| Auteur | Rechercher : Dawkins, Hillary1; Rechercher : Nejadgholi, Isar1 |
|---|
| Affiliation | - Conseil national de recherches du Canada. Technologies numériques
|
|---|
| Format | Texte, Article |
|---|
| Conférence | 60th Annual Meeting of the Association for Computational Linguistics, May 22-27, 2022, Dublin, Ireland |
|---|
| Résumé | Certainty calibration is an important goal on the path to interpretability and trustworthy AI. Particularly in the context of human-in-the-loop systems, high-quality low to mid-range certainty estimates are essential. In the presence of a dominant high-certainty class, for instance the non-entity class in NER problems, existing calibration error measures are completely insensitive to potentially large errors in this certainty region of interest. We introduce a region-balanced calibration error metric that weights all certainty regions equally. When low and mid certainty estimates are taken into account, calibration error is typically larger than previously reported. We introduce a simple extension of temperature scaling, requiring no additional computation, that can reduce both traditional and region-balanced notions of calibration error over existing baselines. |
|---|
| Date de publication | 2022-05 |
|---|
| Maison d’édition | Association for Computational Linguistics |
|---|
| Licence | |
|---|
| Dans | |
|---|
| Langue | anglais |
|---|
| Publications évaluées par des pairs | Oui |
|---|
| Exporter la notice | Exporter en format RIS |
|---|
| Signaler une correction | Signaler une correction (s'ouvre dans un nouvel onglet) |
|---|
| Identificateur de l’enregistrement | f016dcbe-c78a-4297-92fc-77a8b3d4cc20 |
|---|
| Enregistrement créé | 2022-06-03 |
|---|
| Enregistrement modifié | 2022-06-03 |
|---|
