Téléchargement | - Voir la version finale : Calibration and context in human evaluation of machine translation (PDF, 4.5 Mio)
- Voir les données supplémentaires : Calibration and context in human evaluation of machine translation (PDF, 10.1 Mio)
|
---|
DOI | Trouver le DOI : https://doi.org/10.1017/nlp.2024.5 |
---|
Auteur | Rechercher : Knowles, Rebecca1Identifiant ORCID : https://orcid.org/0000-0002-1647-584X; Rechercher : Lo, Chi-kiu1Identifiant ORCID : https://orcid.org/0000-0001-8714-7846 |
---|
Affiliation | - Conseil national de recherches du Canada. Technologies numériques
|
---|
Format | Texte, Article |
---|
Sujet | machine translation; evaluation |
---|
Résumé | Human evaluation of machine translation is considered the “gold standard” for evaluation, but it remains a challenging task for which to define best practices. Recent work has focused on incorporating intersentential context into human evaluation, to better distinguish between high-performing machine translation systems and human translations. In this work, we examine several ways that such context influences evaluation and evaluation protocols. We take a close look at annotator variation through the lens of calibration sets and focus on the implications for context-aware evaluation protocols. We then demonstrate one way in which degraded target-side intersentential context can influence annotator scores of individual sentences, a finding that supports the context-aware approach to evaluation and which also has implications for best practices in evaluation protocols. |
---|
Date de publication | 2024-06-03 |
---|
Maison d’édition | Cambridge University Press (CUP) |
---|
Licence | |
---|
Dans | |
---|
Langue | anglais |
---|
Publications évaluées par des pairs | Oui |
---|
Exporter la notice | Exporter en format RIS |
---|
Signaler une correction | Signaler une correction (s'ouvre dans un nouvel onglet) |
---|
Identificateur de l’enregistrement | 38f5f3ec-1a13-4100-bb65-f21273d1bccb |
---|
Enregistrement créé | 2024-06-04 |
---|
Enregistrement modifié | 2024-06-04 |
---|