Validity and Reliability of HOTS (High Order Thinking Skills) Instrument for Voltaic Cell Subject Using the Classical Method
pp. 53-64 | Published Online: August 2024 | DOI: 10.22521/unibulletin.2024.131.4
Mizzan Ayubi , Jaslin Ikhsan , Vegha Dwi Arthamena , Muhammad Akbar Chaniago , Endar Pradesta
Full text PDF | 340 | 96
Abstract
Background/purpose. Higher order thinking skills (HOTS) are conceptually qualified critical-thinking skills, based on Bloom’s revised taxonomy. Frequently used assessments in schools may not always optimally help learners deal with problems conceptually. The habit of teachers preparing test questions without being based on a test grid, but relying only upon questions sourced from textbooks, can result in students not being adequately trained in higher-level thinking skills. Especially in chemistry subjects, it appears that the ability of students to analyze and solve problems is very low. Materials/methods. This research and development (R&D) study employed the Borg and Gall development model on the research subject of HOTS instruments. The study population was from the Muara Beliti sub-district of Musi Rawas, South Sumatra, Indonesia, and was conducted with 102 students from seventh-grade IPA (International Program on Science Education) classes 1, 2, and 3. Data analysis included tests for validity, reliability, proportion of questions, and discrimination of questions were analyzed according to the classical method. Results. The research found 10 questions to be valid and reliable. The developed HOTS instrument may therefore be effectively employed to measure the high-level thinking ability of high school students in learning chemistry, especially on Voltaic cell material. Tests that utilize HOTS instruments have the potential to encourage students to develop higher order thinking skills in the subject area. |
Conclusion. This study concluded that: 1) The developed HOTS assessment instrument consisted of 10 essay questions that were found to be valid and appropriate in terms of construct, language, and material aspects; 2) Reliability of the HOTS assessment instrument was categorized as high, with a coefficient of .7420.
Keywords: HOTS, Borg and Gall, validity, RnD
ReferencesAbidin, A., Istiyono, E., Fadilah, N., & Dwandaru, W. (2019). A computerized adaptive test for measuring the physics critical thinking skills. International Journal of Evaluation and Research in Education, 8(3), 376-383. http://doi.org/10.11591/ijere.v8i3.19642
Abosalem, Y. (2016). Assessment Techniques and Students’ Higher-Order Thinking Skills. International Journal of Secondary Education, 4(1), 1-11. http://doi.org/10.11648/j.ijsedu.20160401.11
Afflerbach, P., Cho, B.-Y., & Kim, J.-Y. (2015). Conceptualizing and Assessing Higher-Order Thinking in Reading Order Thinking in Reading. Theory Into Practice, 54(3), 203-212. http://dx.doi.org/10.1080/00405841.2015.1044367
Aman. (2019). Final Examination Test Instruments for History Subject in Yogyakarta, Indonesia: A Quality Analysis. Universal Journal of Educational Research, 7(12), 2857-2866. http://doi.org/10.13189/ujer.2019.071237
Arifin, Z. (2016). Evaluasi Pembelajaran [Learning Evaluation]. Remaja Rosdakarya.
Atitullah, K., Fitriati, S., & Rukmini, D. (2019). Using Revised Bloom’s Taxonomy to Evaluate Higher Order Thinking Skills (Hots) in Reading Comprehension Questions of English Textbook for Year X of High School. English Education Journal, 9(4), 428-436. https://doi.org/10.15294/eej.v9i4.31794.
Avargil, S., Herscovitz, O., & Dori, Y. (2012). Teaching Thinking Skills in Context-Based Learning: Teachers’ Challenges and Assessment Knowledge. Journal of Science Education and Technology, 21, 207-225. http://dx.doi.org/10.1007/s10956-011-9302-7
Ayubi, M., Arthamena, V., & Aznam, N. (2023). Development of HOTS (High Order Thinking Skill) Question Instrument on Thermochemistry Material. Jurnal Penelitian Pendidikan IPA, 9(11), 9945-9954. https://doi.org/10.29303/jppipa.v9i11.5162
Azwar, S. (2016). Reliabilitas dan Validitas Aitem [Reliability and Validity of Aitems]. Buletin Psikologi, 3(1), 19-26. https://jurnal.ugm.ac.id/buletinpsikologi/article/view/13381
Barbara, L., & Wendy, W. (2010). Developing Higher Level Thinking. Journal of Instructional Pedagogies, 3(1), 1-9. https://www.aabri.com/manuscripts/09423.pdf
Barnett, J., & Francis, A. (2011). Using higher order thinking questions to foster critical thinking: a classroom study. Educational Psychology, 32(2), 201-211. https://doi.org/10.1080/01443410.2011.638619
Boone, W., Staver, J., & Yale, M. (2014). Rasch Analysis in the Human Sciences. SpringerLink.
Boopathiraj, C., & Chellamani, K. (2013). Analysis of Test Items on Difficulty Level and Discrimination Index in the Test for Research in Education. International Journal of Social Science & Interdisciplinary Research, 2(2), 189-193.
Brown, G., Irving, S., & Keegen, P. (2018). An Introduction to Educational Assessment, Measurement, and Evaluation: Improving the Quality of Teacher-Based Assessment. Pearson Education.
Cavallo, M., Potter, W., & Rozman, M. (2010). Gender Differences in Learning Constructs, Shifts in Learning Constructs, and Their Relationship to Course Achievement in a Structured Inquiry, Yearlong College Physics Course for Life Science Majors. School Science and Mathematics, 104(6), 288-300. https://doi.org/10.1111/j.1949-8594.2004.tb18000.x
Chalkiadaki, A. (2018). A Systematic Literature Review of 21st Century Skills and Competencies in Primary Education. International Journal of Instruction, 11(3), 1-16. https://doi.org/10.12973/iji.2018.1131a
Chinedu, C., Kamin, Y., & Olabiyi, O. (2015). Strategies for Improving Higher Order Thinking Skills in Teaching and Learning of Design and Technology Education. Journal of Technical Education and Training, 7(2), 35-43. https://publisher.uthm.edu.my/ojs/index.php/JTET/article/view/1081
Clotfelter, C., Ladd, H., & Vigdor, J. (2005). Who teaches whom? Race and the distribution of novice teachers. Economics of Education Review, 24(4), 377-392. https://doi.org/10.1016/j.econedurev.2004.06.008
Conklin, W. (2011). Higher-Order Thinking Skills to Develop 21st Century Learners. Shell Education.
Cooper, C. (2023). An Introduction to Psychometrics and Psychological Assessment. Routledge.
D’Alessio, F., Avolio, B., & Charles, V. (2019). Studying the impact of critical thinking on the academic performance of executive MBA students. Thinking Skills and Creativity, 31, 275-283. https://doi.org/10.1016/j.tsc.2019.02.002
Demirhan, E., & Koklukaya, A. (2014). The Critical Thinking Dispositions of Prospective Science Teachers Procedia-Social and Behavioral Sciences, 116, 1551-1555. https://doi.org/10.1016/j.sbspro.2014.01.433
Drake, F., & Nelson, L. (2008). Engagement in Teaching History: Theory and Practices for Middle and Secondary Teachers (2nd ed.). Pearson.
Febrina, F., Usman, B., & Muslem, A. (2019). Analysis of Reading Comprehension Questions by Using Revised Bloom’s Taxonomy on Higher Order Thinking Skill (HOTS). English Education Journal, 10(1), 1-15. https://jurnal.usk.ac.id/EEJ/article/view/13253/10123
Febriyani, R., Yunita, W., & Damayanti, I. (2020). An Analysis on Higher Order Thinking Skill (HOTS) in Compulsory English Textbook for the Twelfth Grade of Indonesian Senior High Schools. Journal of English Education and Teaching, 4(2), 170-183. https://doi.org/10.33369/jeet.4.2.170-183
Fischer, C., Bol, L., & Pribesh, S. (2011). An Investigation of Higher-Order Thinking Skills in Smaller Learning Community Social Studies Classrooms. American Secondary Education, 39(2), 5-26.
Gall, M. D., Gall J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed.). Allyn, & Bacon/Pearson.
Gojkov, G., Stojanovic, A., & Rajic, A. (2015). Critical Thinking Of Students – Indicator Of Quality In Higher Education. Procedia-Social and Behavioral Sciences, 191, 591-596. https://doi.org/10.1016/j.sbspro.2015.04.501
Guskey, T. (2007). Closing Achievement Gaps: Revisiting Benjamin S. Bloom’s “Learning for Mastery”. Journal of Advanced Academics, 19(1), 8-31. https://doi.org/10.4219/jaa-2007-704
Haley, S., Coster, W., Dumas, H., Fragala-Pinkham, M., Kramer, J., Ni, P., Tian, F., Kao, Y.-C., Moed, R., & Hudlow, L. (2011). Accuracy and precision of the Pediatric Evaluation of Disability Inventory computer-adaptive tests (PEDI-CAT). Developmental Medicine and Child Neurology, 53(12), 1100-1106. http://doi.org/10.1111/j.1469-8749.2011.04107.x
Halpern, D. (1998). Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449-455. https://psycnet.apa.org/doi/10.1037/0003-066X.53.4.449
Heong, Y., Othman, W., Yunos, J., Kiong, T., Hassan, R., & Mohamad, M. (2011). The Level of Marzano Higher Order Thinking Skills among Technical Education Students. International Journal of Social Science and Humanity, 1(2), 121-125. http://dx.doi.org/10.7763/IJSSH.2011.V1.20
Hmeli, C., & Ferrari, M. (1997). The Problem-Based Learning Tutorial: Cultivating Higher Order Thinking Skills. Journal for the Education of the Gifted, 20(4), 401-422. https://doi.org/10.1177/016235329702000405
Kinay, I., & Bagceci, B. (2016). The Investigation of the Effects of Authentic Assessment Approach on Prospective Teachers’ Problem-Solving Skills. International Education Studies, 9(8), 51-59. http://dx.doi.org/10.5539/ies.v9n8p51
Kusuma, M., Rosidin, U., Abdurrahman, & Suyanta, A. (2017). The Development of Higher Order Thinking Skill (Hots) Instrument Assessment In Physics Study. IOSR Journal of Research & Method in Education, 7(1), 26-32. http://doi.org/10.9790/7388-0701052632
Madhuri, G., Kantamreddi, V., & Goteti, L. (2012). Promoting higher order thinking skills using inquiry-based learning. European Journal of Engineering Education, 37(2), 117-123. https://doi.org/10.1080/03043797.2012.661701
Mahoney, J., & Harris-Reeves, B. (2019). The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education, 20(1), 25-37. https://doi.org/10.1177/1469787417723243
Mardapi, D. (2008). Teknik Penyusunan Instrumen Tes dan Nontes [Techniques for Preparing Test and Non-Test Instruments]. Mitra Cendekia Press.
McCormick, N. (2015). Engaging Students in Critical Thinking and Problem Solving: A Brief Review of the Literature. Journal of Studies in Education, 5(4), 100-113. http://dx.doi.org/10.5296/jse.v5i4.8249
McDonald, G. (2012). Teaching Critical & Analytical Thinking in High School Biology? The American Biology Teacher, 74(3), 178-181. https://doi.org/10.1525/abt.2012.74.3.9
McLaughlin, J., Roth, M., Glatt, D., Gharkholonarehe, N., Davidson, C., Griffin, L., & Mumper, R. (2014). The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236-243. https://doi.org/10.1097/acm.0000000000000086
Miterianifa, M. (2016). Evaluasi Pembelajaran Kimia [Evaluation of Chemistry Learning]. Cahaya Firdaus.
Polit, D., & Beck, C. (2006). The content validity index: Are you sure you know what’s being reported? critique and recommendations. Research in Nursing & Health, 29(5), 489-497. https://doi.org/10.1002/nur.20147
Pommerich, M. (2004). Developing Computerized Versions of Paper-and-Pencil Tests. The Journal of Technology, Learning and Assessment, 2(6), 3-44. https://ejournals.bc.edu/index.php/jtla/article/view/1666
Qing, Z., Jing, G., & Yan, W. (2010). Promoting preservice teachers’ critical thinking skills by inquiry-based chemical experiment. Procedia-Social and Behavioral Sciences, 2(2), 4597-4603. https://doi.org/10.1016/j.sbspro.2010.03.737
Quieng, M., Lim, P., & Lucas, M. (2015). 21st Century-based Soft Skills: Spotlight on Non-cognitive Skills in a Cognitive-laden Dentistry Program. European Journal of Contemporary Education, 11(1), 72-81. http://doi.org/10.13187/ejced.2015.11.72
Retnawati, H. (2016). Validitas, Reliabilitas dan Karakteristik item [Validity, Reliability and Characteristics of items]. Parama Publikasi.
Sepriyanti, N., Nelwati, S., Kustati, M., & Afriadi, J. (2022). The Effect of 21st-Century Learning on Higher-Order Thinking Skills (Hots) and Numerical Literacy of Science Students in Indonesia Based on Gender. Jurnal Pendidikan IPA Indonesia, 11(2), 314-321. http://doi.org/10.15294/jpii.v11i2.36384
Setiawan, J., Aman, A., & Wulandari, T. (2020). Understanding Indonesian history, interest in learning history and national insight with nationalism attitude. International Journal of Evaluation and Research in Education, 9(2), 364-373. http://doi.org/10.11591/ijere.v9i2.20474
Sudijono, A. (2012). Pengantar evaluasi pendidikan [Introduction to educational evaluation]. Rajawali Pers.
Sudijono, A. (2013). Pengantar Evaluasi Pendidikan [Introduction to educational evaluation]. Raja Grafindo Persada.
Szilagyi, J., Clements, D., & Sarama, J. (2013). Young Children’s Understandings of Length Measurement: Evaluating a Learning Trajectory. Journal for Research in Mathematics Education, 44(3), 581-620. https://doi.org/10.5951/jresematheduc.44.3.0581
Udompong, L., & Wongwanich, S. (2014). Diagnosis of the Scientific Literacy Characteristics of Primary Students. Procedia-Social and Behavioral Sciences, 116, 5091-5096. https://doi.org/10.1016/j.sbspro.2014.01.1079
Wardany, K., Sajidan, & Ramli, M. (2017). Pengembangan Penilaian Untuk Mengukur Higher Order Thinking Skills Siswa [Development of Assessments to Measure Students' Higher Order Thinking Skills]. Jurnal Inkuiri, 6(2), 1-16. https://jurnal.uns.ac.id/inkuiri/article/view/16214
Weiss, D. (2011). Better Data From Better Measurements Using Computerized Adaptive Testing. Journal of Methods and Measurement in the Social Sciences, 2(1), 1-27. https://doi.org/10.2458/v2i1.12351
UNIBULLETIN News!
► New issue coming soon! (Volume 13 Issue 2, 2024)
► Call for Papers
UNIBULLETIN is calling for submissions. Authors are invited to submit papers from the all fields of the Education (General) and Social Sciences (General) in the international context. All submissions should be presented only in English.