| DOI | Resolve DOI: https://doi.org/10.1007/978-3-030-86520-7_12 |
|---|
| Author | Search for: Guo, Hongyu1 |
|---|
| Affiliation | - National Research Council Canada. Digital Technologies
|
|---|
| Format | Text, Book Chapter |
|---|
| Conference | European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2021), Sept. 13-17, 2021, Bilbao, Spain [Held Online] |
|---|
| Subject | label smoothing; model regularization; mixup |
|---|
| Abstract | Label Smoothing (LS) improves model generalization through penalizing models from generating overconfident output distributions. For each training sample the LS strategy smooths the one-hot encoded training signal by distributing its distribution mass over the non-ground truth classes. We extend this technique by considering example pairs, coined PLS. PLS first creates midpoint samples by averaging random sample pairs and then learns a smoothing distribution during training for each of these midpoint samples, resulting in midpoints with high uncertainty labels for training. We empirically show that PLS significantly outperforms LS, achieving up to 30% of relative classification error reduction. We also visualize that PLS produces very low winning softmax scores for both in and out of distribution samples. |
|---|
| Publication date | 2021-09-10 |
|---|
| Publisher | Springer |
|---|
| In | |
|---|
| Series | |
|---|
| Language | English |
|---|
| Peer reviewed | Yes |
|---|
| Export citation | Export as RIS |
|---|
| Report a correction | Report a correction (opens in a new tab) |
|---|
| Record identifier | 626d0ead-4e83-43b5-aede-66bbaed53047 |
|---|
| Record created | 2021-10-25 |
|---|
| Record modified | 2021-10-25 |
|---|
