| Téléchargement | - Voir la version finale : ChatGPT for suicide risk assessment on social media: quantitative evaluation of model performance, potentials and limitations (PDF, 1.1 Mio)
|
|---|
| Auteur | Rechercher : Ghanadian, Hamideh; Rechercher : Nejadgholi, Isar1; Rechercher : Al Osman, Hussein |
|---|
| Affiliation | - Conseil national de recherches du Canada. Technologies numériques
|
|---|
| Format | Texte, Article |
|---|
| Conférence | 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis (WASSA 2023), July 14th, 2023, Toronto, Ontario |
|---|
| Résumé | This paper presents a novel framework for quantitatively evaluating the interactive ChatGPT model in the context of suicidality assessment from social media posts, utilizing the University of Maryland Reddit suicidality dataset. We conduct a technical evaluation of ChatGPT’s performance on this task using Zero-Shot and Few-Shot experiments and compare its results with those of two fine-tuned transformer-based models. Additionally, we investigate the impact of different temperature parameters on ChatGPT’s response generation and discuss the optimal temperature based on the inconclusiveness rate of ChatGPT. Our results indicate that while ChatGPT attains considerable accuracy in this task, transformer-based models fine-tuned on human-annotated datasets exhibit superior performance. Moreover, our analysis sheds light on how adjusting the ChatGPT’s hyperparameters can improve its ability to assist mental health professionals in this critical task. |
|---|
| Date de publication | 2023-07-14 |
|---|
| Maison d’édition | Association for Computational Linguistics |
|---|
| Licence | |
|---|
| Dans | |
|---|
| Langue | anglais |
|---|
| Publications évaluées par des pairs | Oui |
|---|
| Exporter la notice | Exporter en format RIS |
|---|
| Signaler une correction | Signaler une correction (s'ouvre dans un nouvel onglet) |
|---|
| Identificateur de l’enregistrement | 4d659f67-ceea-44ee-92ef-59f6290af830 |
|---|
| Enregistrement créé | 2023-08-09 |
|---|
| Enregistrement modifié | 2023-08-10 |
|---|
