Confirmatory factor analysis of a rubric for assessing algorithmic thinking on undergraduate students
DOI:
https://doi.org/10.18861/cied.2024.15.2.3797Keywords:
algorithmic thinking, higher education students, educational assessment, confirmatory factor analysis, STEM educationAbstract
Algorithmic thinking is a key element for individuals to be aligned with the computer era. Its study is important not only in the context of computer science but also in mathematics education and all STEAM contexts. However, despite its importance, a lack of research treating it as an independent construct and validating its operational definitions or rubrics to assess its development in university students through confirmatory factor analysis has been discovered. The aim of this paper is to conduct a construct validation through confirmatory factor analysis of a rubric for the algorithmic thinking construct, specifically to measure its level of development in university students. Confirmatory factor analysis is performed on a series of models based on an operational definition and a rubric previously presented in the literature. The psychometric properties of these models are evaluated, with most of them being discarded. Further research is still needed to expand and consolidate a useful operational definition and the corresponding rubric to assess algorithmic thinking in university students. However, the confirmatory factor analysis confirms the construct validity of the rubric, as it exhibits very good psychometric properties and leads to an operational definition of algorithmic thinking composed of four components: Problem analysis, algorithm construction, input case identification, and algorithm representation.
Downloads
References
Bacelo, A., & Gómez-Chacón, I. M. (2023). Characterizing algorithmic thinking: A university study of unplugged activities. Thinking Skills and Creativity, 48, Article 101284. https://doi.org/10.1016/j.tsc.2023.101284
Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Cognitive domain (Vol. 1). McKay.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilford Press.
Bubica, N., & Boljat, I. (2021). Assessment of Computational Thinking – A Croatian Evidence-Centered Design Model. Informatics in Education, 21(3), 425-463. https://doi.org/10.15388/infedu.2022.17
Cebrián de la Serna, M., & Monedero Moya, J. J. (2014). Evolución en el diseño y funcionalidad de las rúbricas: Desde las rúbricas «cuadradas» a las erúbricas federadas. REDU. Revista de Docencia Universitaria, 12(1), 81-98. https://doi.org/10.4995/redu.2014.6408
Chen, P., Yang, D., Metwally, A. H. S., Lavonen, J., & Wang, X. (2023). Fostering computational thinking through unplugged activities: A systematic literature review and meta-analysis. International Journal of STEM Education, 10(1), Article 47. https://doi.org/10.1186/s40594-023-00434-7
Chowdhury, F. (2019). Application of Rubrics in the Classroom: A Vital Tool for Improvement in Assessment, Feedback and Learning International Education Studies, 12(1), 61–68. International Education Studies, 12(1), 61–68. https://doi.org/10.5539/ies.v12n1p61
Futschek, G. (2006). Algorithmic Thinking: The Key for Understanding Computer Science. In R. T. Mittermeir (Ed.), International conference on informatics in secondary schools—Evolution and perspectives (pp. 159–168). Springer-Verlag.
Greiff, S., Wüstenberg, S., Goetz, T., Vainikainen, M.-P., Hautamäki, J., & Bornstein, M. H. (2015). A longitudinal study of higher-order thinking skills: Working memory and fluid reasoning in childhood enhance complex problem-solving in adolescence. Frontiers in Psychology, 6, Article 1060. https://doi.org/10.3389/fpsyg.2015.01060
Grover, S. (2017). Assessing algorithmic and computational thinking in K-12: Lessons from a middle school classroom. In P. J. Rich & C. B. Hodges (Eds.), Emerging research, practice, and policy on computational thinking (pp. 269–288). Springer. https://doi.org/10.1007/978-3-319-52691-1_17
Grover, S., & Pea, R. (2018). Computational thinking: A competency whose time has come. In S. Sentance, E. Barendsen, & C. Schulte (Eds.), Computer science education: Perspectives on teaching and learning in school. Bloomsbury Publishing.
Grozdev, S., & Terzieva, T. (2015). A didactic model for developmental training in computer science. Journal of Modern Education Review, 5(5), 470-480. https://doi.org/10.15341/jmer(2155-7993)/05.05.2015/005
Jordan Muiños, F. M. (2021). Cut-off value of the fit indices in Confirmatory Factor Analysis. {PSOCIAL}, 7(1), 66-71. https://publicaciones.sociales.uba.ar/index.php/psicologiasocial/article/view/6764
Juškevičienė, A. (2020). Developing Algorithmic Thinking Through Computational Making. In D. Gintautas, B. Jolita, & K. Janusz (Eds.), Data Science: New Issues, Challenges and Applications (Vol. 869, pp. 183-197). Springer. https://doi.org/10.1007/978-3-030-39250-5_10
Juškevičienė, A., & Dagienė, V. (2018). Computational Thinking Relationship with Digital Competence. Informatics in Education, 17(2), 265-284. https://doi.org/10.15388/infedu.2018.14
Kadijevich, D. M., Stephens, M., & Rafiepour, A. (2023). Emergence of Computational/Algorithmic Thinking and Its Impact on the Mathematics Curriculum. In Y. Shimizu & R. Vithal (Eds.), Mathematics Curriculum Reforms Around the World (pp. 375-388). Springer International Publishing. https://doi.org/10.1007/978-3-031-13548-4_23
Kanaki, K., & Kalogiannakis, M. (2022). Assessing Algorithmic Thinking Skills in Relation to Age in Early Childhood STEM Education. Education Sciences, 12(6), Article 380. https://doi.org/10.3390/educsci12060380
Kayam, M., Fuwa, M., Kunimune, H., Hashimoto, M., & Asano, David. K. (2016). Assessing your algorithm: A program complexity metrics for basic algorithmic thinking education. In 11th International Conference on Computer Science & Education (ICCSE), 309–313. https://doi.org/10.1109/ICCSE.2016.7581599
Kirçali, A. Ç., & Özdener, N. (2023). A Comparison of Plugged and Unplugged Tools in Teaching Algorithms at the K-12 Level for Computational Thinking Skills. Technology, Knowledge and Learning, 28(4), 1485-1513. https://doi.org/10.1007/s10758-021-09585-4
Knuth, D. E. (1974). Computer Science and its Relation to Mathematics. The American Mathematical Monthly, 81(4), 323–343. https://doi.org/10.1080/00029890.1974.11993556
Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558-569. https://doi.org/10.1016/j.chb.2017.01.005
Lafuente Martínez, M., Lévêque, O., Benítez, I., Hardebolle, C., & Zufferey, J. D. (2022). Assessing Computational Thinking: Development and Validation of the Algorithmic Thinking Test for Adults. Journal of Educational Computing Research, 60(6), 1436-1463. https://doi.org/10.1177/07356331211057819
Lehmann, T. H. (2023). How current perspectives on algorithmic thinking can be applied to students’ engagement in algorithmatizing tasks. Mathematics Education Research Journal. https://doi.org/10.1007/s13394-023-00462-0
Lockwood, E., DeJarnette, A. F., Asay, A., & Thomas, M. (2016). Algorithmic Thinking: An Initial Characterization of Computational Thinking in Mathematics. In M. B. Wood, E. E. Turner, M. Civil, & J. A. Eli (Eds.), Proceedings of the 38th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (PMENA 38) (pp. 1588–1595). ERIC. https://eric.ed.gov/?id=ED583797
Martins-Pacheco, L. H., da Cruz Alves, N., & von Wangenheim, C. G. (2020). Educational Practices in Computational Thinking: Assessment, Pedagogical Aspects, Limits, and Possibilities: A Systematic Mapping Study. In H. C. Lane, S. Zvacek, & J. Uhomoibhi (Eds.), Computer Supported Education (pp. 442-466). Springer International Publishing. https://doi.org/10.1007/978-3-030-58459-7_21
Mertler, C. A. (2001). Designing scoring rubrics for your classroom. Practical assessment, research, and evaluation, 7(1), Article 25. https://doi.org/10.7275/gcy8-0w24
Moreira Gois, M., Eliseo, M. A., Mascarenhas, R., Carlos Alcântara De Oliveira, I., & Silva Lopes, F. (2023). Evaluation rubric based on Bloom's taxonomy for assessment of students learning through educational resources. In 15th International Conference on Education and New Learning Technologies, (pp. 7765-7774). https://doi.org/10.21125/edulearn.2023.2021
Moss, S. (2016, June 27). Fit indices for structural equation modelling. Sicotest.com. https://www.sicotests.com/newpsyarticle/Fit-indices-for-structural-equation-modeling
Navas-López, E. A. (2021). Una Caracterización del Desarrollo del Pensamiento Algorítmico de los Estudiantes de las carreras de Licenciatura en Matemática y Licenciatura en Estadística de la sede central de la Universidad de El Salvador en el período 2018-2020 [Master’s Thesis, Universidad de El Salvador]. https://hdl.handle.net/20.500.14492/12041
Navas-López, E. A. (2024). Relaciones entre la matemática, el pensamiento algorítmico y el pensamiento computacional. IE Revista de Investigación Educativa de la REDIECH, 15, e1929. https://doi.org/10.33010/ie_rie_rediech.v15i0.1929
Noor, N. M. M., Mamat, N. F. A., Mohemad, R., & Mat, N. A. C. (2023). Systematic Review of Rubric Ontology in Higher Education. International Journal of Advanced Computer Science and Applications, 14(10), 147-155. https://doi.org/10.14569/IJACSA.2023.0141016
Nordby, S. K., Bjerke, A. H., & Mifsud, L. (2022). Computational Thinking in the Primary Mathematics Classroom: A Systematic Review. Digital Experiences in Mathematics Education, 8(1), 27-49. https://doi.org/10.1007/s40751-022-00102-5
Oomori, Y., Tsukamoto, H., Nagumo, H., Takemura, Y., Iida, K., Monden, A., & Matsumoto, K. (2019). Algorithmic Expressions for Assessing Algorithmic Thinking Ability of Elementary School Children. In 2019 IEEE Frontiers in Education Conference (FIE), 1-8. https://doi.org/10.1109/FIE43999.2019.9028486
Ortega Ruipérez, B., & Asensio Brouard, M. (2021). Evaluar el Pensamiento Computacional mediante Resolución de Problemas: Validación de un Instrumento de Evaluación. Revista Iberoamericana de Evaluación Educativa, 14(1), 153-171. https://doi.org/10.15366/riee2021.14.1.009
Otero Avila, C., Foss, L., Bordini, A., Simone Debacco, M., & da Costa Cavalheiro, S. A. (2019). Evaluation Rubric for Computational Thinking Concepts. In IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 279-281. https://doi.org/10.1109/ICALT.2019.00089
Park, H., & Jun, W. (2023). A Study of Development of Algorithm Thinking Evaluation Standards. International Journal of Applied Engineering and Technology, 5(2), 53-58.
Poulakis, E., & Politis, P. (2021). Computational Thinking Assessment: Literature Review. In T. Tsiatsos, S. Demetriadis, A. Mikropoulos, & V. Dagdilelis (Eds.), Research on E-Learning and ICT in Education (pp. 111-128). Springer International Publishing. https://doi.org/10.1007/978-3-030-64363-8_7
Román-González, M., Moreno-León, J., & Robles, G. (2019). Combining Assessment Tools for a Comprehensive Evaluation of Computational Thinking Interventions. In S. C. Kong & H. Abelson (Eds.), Computational Thinking Education (pp. 79-98). Springer. https://doi.org/10.1007/978-981-13-6528-7_6
Sadykova, O. V., & Usolzev, A. (2018). On the concept of algorithmic thinking. In SHS Web Conferences – International Conference on Advanced Studies in Social Sciences and Humanities in the Post-Soviet Era (ICPSE 2018), 55. https://doi.org/10.1051/shsconf/20185503016
Selby, C. C., & Woollard, J. (2013). Computational Thinking: The Developing Definition. In 18th annual conference on innovation and technology in computer science education. https://eprints.soton.ac.uk/356481/
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003
Stephens, M., & Kadijevich, D. M. (2020). Computational/Algorithmic Thinking. In S. Lerman (Ed.), Encyclopedia of Mathematics Education (pp. 117-123). Springer International Publishing. https://doi.org/10.1007/978-3-030-15789-0_100044
Sung, J. (2022). Assessing young Korean children’s computational thinking: A validation study of two measurements. Education and Information Technologies, 27(9), 12969–12997. https://doi.org/10.1007/s10639-022-11137-x
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 1-22. https://doi.org/10.1016/j.compedu.2019.103798
Tsai, M. J., Liang, J. C., Lee, S. W. Y., & Hsu, C. Y. (2022). Structural Validation for the Developmental Model of Computational Thinking. Journal of Educational Computing Research, 60(1), 56-73. https://doi.org/10.1177/07356331211017794
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining Computational Thinking for Mathematics and Science Classrooms. Journal of Science Education and Technology, 25(1), 127-147. https://doi.org/10.1007/s10956-015-9581-5
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Wing, J. M. (2017). Computational thinking’s influence on research and education for all. Italian Journal of Educational Technology, 25(2), 7–14. https://doi.org/10.17471/2499-4324/922
Zúñiga Muñoz, R. F., Hurtado Alegría, J. A., & Robles, G. (2023). Assessment of Computational Thinking Skills: A Systematic Review of the Literature. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 18(4), 319-330. https://doi.org/10.1109/RITA.2023.3323762
