Andi Abdurrahman Manggaberani, Samsul Hadi, Nur Hidayanto Pancoro Setyo Putro, Abrar Syahrul Fajri, Heri Retnawati
This study investigates the comparative efficacy of Summated Rating Scales (SRS) and traditional ordinal scales (raw Likert-type responses) in measuring critical thinking skills among vocational students, employing Item Response Theory (IRT) to evaluate their psychometric properties. Addressing the limitations of ordinal scales notably inconsistent intervals between response categories the research adopts a descriptive quantitative methodology involving 269 students from state vocational high schools in Yogyakarta, Indonesia. Data were collected using a five-point Likert scale instrument, validated for content (Aiken’s V = 0.94), and analyzed through two IRT frameworks: Polytomous IRT for unscaled ordinal data and Continuous Response Model (CRM) IRT for SRS-transformed interval data. Key findings reveal that SRS enhances measurement precision by normalizing response distributions into proportional intervals (e.g., recalibrated scores: 0.00, 0.73, 1.46, 2.07, 2.84), thereby resolving issues of unequal category spacing inherent to ordinal scales. Polytomous IRT demonstrated robust item fit (e.g., Partial Credit Model fit for 5/6 items) and strong difficulty parameter invariance (r = 0.84), yet exhibited instability in ability estimates (r = 0.37) due to extreme response patterns. Conversely, CRM IRT applied to scaled data produced stable ability estimates (r = 0.46) and eliminated infinite values in Maximum Likelihood Estimation, underscoring its superiority in handling continuous metrics. However, ordinal scales retained higher consistency in difficulty calibration across subgroups. The study concludes that integrating SRS with CRM IRT offers a refined approach for critical thinking assessments, balancing precision and fairness, while ordinal scales remain pragmatic for contexts prioritizing simplicity. These insights advocate for the adoption of advanced scaling techniques in vocational education to improve the validity of competency evaluations, with recommendations for future research to explore hybrid models and longitudinal applications. © 2025 Syarif Hidayatullah State Islamic University (UIN) Jakarta. All rights reserved.
Department of Educational Research and Evaluation, Graduate School, Universitas Negeri Yogyakarta, Yogyakarta, Indonesia
