Alfiyah Qothrunnada Zulfa, Syukron Abu Ishaq Alfarozi, Sri Kusrohmaniah, Sunu Wibirama
Mental disorder recognition using Natural Language Processing based on social media texts has increased as they provide a potentially scalable and cost-effective way to assess mental health status. However, most existing approaches have employed cross-entropy fine-tuning methods to improve model accuracy by leveraging pre-trained language models. Although this approach has shown great promise in terms of model accuracy, it also has several significant disadvantages when applied to datasets with noisy labels, class imbalance, or semantic overlap. These factors are common to many datasets in the mental health field. To solve this research gap, this study proposes a method to investigate whether a different type of training objective can be used to generate more robust classifications of six classes of mental disorders using social media data posts. We used MentalBERT as our baseline approach to compare three types of training objectives: (1) cross-entropy loss (CE); (2) N-pair loss (NPair); and (3) supervised contrastive learning (SupCon). The experimental results in the imbalance scenario demonstrated that contrastive learning approaches generally outperformed CE-based fine-tuning methods for all evaluation criteria. SupCon achieved the best performance with a macro F1-score of 85.8%, followed closely by NPair at 84.8%, and CE at 84.3%. Furthermore, SupCon demonstrated a more balanced performance among each of the mental disorder categories, including minority classes. Beyond quantitative evaluations, we also utilized LIME as a post-hoc explanation technique to provide insight into why specific mental disorders were classified into one category. © 2026 IEEE.
Faculty of Engineering, Universitas Gadjah Mada, Department of Electrical and Information Engineering, Yogyakarta, 55281, Indonesia; Vocational Faculty, Universitas Negeri Yogyakarta, Department of Electrical and Electronic Engineering, Yogyakarta, 55652, Indonesia; Faculty of Psychology, Universitas Gadjah Mada, Yogyakarta, 55281, Indonesia
