Think-Aloud Protocols in process studies revisited: Suggestions for veridicality

Sugirin Sugirin, Jamilah Jamilah, Joko Priyana, Indah Permatasari

Abstract


Thinking-Aloud or Think-Aloud Protocols have been used to study thinking processes in varied fields such as psychology, business, sports, industry, and education. Recently, studies have tried to see its application in teaching in both offline and online modes. Despite its success, as demonstrated by these studies, controversies concerning the believability of thinking aloud and the process of reactivity have been raised. However, no explicit solution has been provided. In response to the issues, this paper critically reviewed examples of think-aloud protocols using Bowles’s guide to TAP (2008). The results show that (1) problems of the reactivity process stem from the nature of the tasks given, and (2) to minimize the problems of reactivity, the TAP data collection procedure should move from the least to the most intervention of the researcher.

Keywords


Think-aloud; think-aloud protocols; thinking processes

Full Text:

PDF

References


Afflerbach, P. A., & Cho, B. Y. (2010). Determining and describing reading strategies: Internet and traditional forms of reading.” In Wolfgang Schneider (Ed.), Metacognition, Strategy Use, and Instruction (pp. 201–255). Guilford Press.

Alhejaili, A., Wharrad, H. & Windle, R. (2022). A pilot study conducting online think aloud qualitative method during social distancing: Benefits and challenges.Healthcare 2022, 10, 1700. https://doi.org/10.3390/ healthcare10091700

Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educational Psychologist, 40(4), 199-209.

Bowles, M. A. (2008). Task type and reactivity of verbal reports in SLA: A first look at an L2 task other than reading. Studies in Second Language Acquisition, 30 (4).

Carioli, S. & Peru, A. (2016). The Think-Aloud approach: A promising tool for online reading comprehension. Journal of Media Literacy Education 8(1), 49-61

Charters, E. (2010). The use of think-aloud methods in qualitative research: An introduction to think-aloud methods. Brock Education Journal 12(2), 68-82.

Coiro, J. (2011). Talking about reading as thinking: Modeling the hidden complexities of online reading comprehension. Theory Into Practice, 50(2), 107–115.

Coiro, J., and Dobler, E. 2007. Exploring the online reading comprehension strategies used by sixth grade skilled readers to search for and locate information on the internet. Reading Research Quarterly, 42(2), 214-257.

Davey, B. (1983). Think aloud-modeling of the cognitive processes of reading comprehension. Journal of Reading, 27(1), 44-47, 1983.

Ebner, R.J., & Ehri, L.C. (2013). Vocabulary learning on the internet: Using a structured think-aloud procedure. Journal of Adolescent & Adult Literacy, 56(6), 480-489.

Ellis, R. (2001). Introduction: Investigating form-focused instruction. Language Learning, 51(Suppl. 1), 1– 46.

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. MIT Press.

Fan, M., Shi, S., Truong, K. N. (2020). Practices and challenges of using think-aloud protocols in industry: an international survey. Journal of Usability Studies Vol. 15, No. 2, 1 Feb 2020, pp. 85-102.

Goldstein, E.B. (2011). Cognitive psychology: Connecting mind, research, and everyday experience. Third edition. Wadsworth.

Gov. UK. (2021). Guidance. Think aloud study: qualitative studies. How to use a think aloud study to evaluate your digital health product. Gov. UK.

Guss, C.D. (2018). What is going through your mind? Thinking aloud as a method in cross-cultural psychology. Sec. Cultural Psychology Volume 9 – 2018. https://doi.org/10.3389/fpsyg.2018.01292

Hasibuan, S., Puspita, H., & Arasuli, A. (2022). An analysis of reading strategies used by the fourth semester students of English Education Study Program in Muhammadiyah University. Journal of English Education and Teaching, 6(1), 73–88. https://doi.org/10.33369/jeet.6.1.73-88

Hesse-Biber, S. (2015). Introduction: Navigating a turbulent research landscape: Working the boundaries, tensions, diversity, and contradictions of multimethod and mixed methods inquiry. In Sharlene Hesse-Biber & R. Burke Johnson (Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp.xxxiii-liii). Oxford University Press.

Jourdenais, R. (2001). Cognition, instruction, and protocol analysis. In P. Robinson (Ed.), Cognition and second language instruction. Cambridge University Press.

Katamadze, N., Tavadze, L., & Diasamidze, I. (2022). The role of checking retelling in teaching English. IJAEDU- International E-Journal of Advances in Education, Vol. VIII, Issue 24, December 2022.

Liu, P., & Li, L. (2015). An overview of metacognitive awareness and L2 reading strategies. The Routledge International Handbook of Research on Teaching Thinking. R. Wegerif, L. Li, & J. C. Kaufman (Eds.), pp. 290–303, Routledge.

Malek, H. B., Berna, F., and D’Argembeau, A. (2017). Reconstructing the times of past and future personal events. Memory 25, 1402–1411. https://doi.org/10.1080/09658211.2017.1310251

Maslawati, M., Hamdan, N. A., Shaharuddin, S., & Khalid, F. (2015). Hypermedia reading strategies of TESL undergraduate students in Malaysia. Turkish Online Journal of Educational Technology, Special Issue 1 for IETC 2015, pp. 301-310.

Morrow, L. (1988). Retelling stories as a diagnostic tool. In reexamining reading diagnosis: new trends and procedures, eds S. M. Glazer, L. W. Searfoss & M. Lance, International Reading Association, Newark, DE, pp. 128-49.

Nicholls, A. R., & Polman, R. J. (2008). Think aloud: Acute stress and coping strategies during golf performances. Anxiety, Stress & Coping, 21, 283–294.

OECD. (2021). 21st-Century readers: Developing literacy skills in a digital world. https://www.oecd.org/publications/21st-century-readers-a83d84cb-en.htm

Ortlieb, E. & Norris, M. (2012). Using the think-aloud strategy to bolster reading comprehension of science concepts. Current Issues in Education, 15(1), 2012.

Parise, A., and Güss, C. D. (2006). TAP: A thinking-aloud analysis program. University of North Florida.

Pratama, M. I. F., Az-Zahra, H. M., & Setiawan, N. Y. Evaluasi usability menggunakan metode think aloud dan heuristic evaluation pada aplikasi mobile Padiciti,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 3, no. 9, hlm. 8390–8399, 2019.

Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction design: Beyond human-computer interaction. Wiley.

Rahmi, L. (2020). Evaluasi usability fitur webshare pada aplikasi Share It menggunakan metode Thinking-Aloud. Ultima InfoSys-Jurnal Sistem Informasi, vol. 10, no. 2, hlm. 111–118, 2020. https://doi.org/10.31937/si.v10i2.1199.

Rehman, A.U., Khan, K., Almas, A., Mohamad, M., & Ismail, N.H. (2020). Teachers’ teaching reading strategies at public elementary schools in Haripur District. Creative Education, 11(4), April 2020.

Reinhart, A. (2022), Think-aloud interviews: A tool for exploring student statistical reasoning. Journal of Statistics and Data Science Education, 30(2), 100-113 https://doi.org/11310.1080/26939169.2022.2063209

Samson, A., Simpson, D., Kamphoff, C., and Langlier, A. (2017). Think aloud: an examination of distance runners’ thought processes. Int. J. Sport Exerc. Psychol. 15, 176–189. https://doi.org/1612197X.2015.1069877

Sebele-Mpofu, F. Y. (2020). Saturation controversy in qualitative research: Complexities and underlying assumptions. A literature review, Cogent Social Sciences, 6:1, 1838706, http://doi.org/10.1080/23311886.2020.1838706

Sönmez, Y. & Sulak, S. E. (2018). The effect of the thinking-aloud strategy on the reading comprehension skills of 4th grade primary school students. Universal Journal of Educational Research 6(1), 168-172.

Sugirin. (2002). Think-aloud protocol analysis, a path to process studies: what, why and how. The 50th TEFLIN International Conference: Asian Odyssey 2002: Explorations in TEFL, 29-31 Oct. 2002.

Sugirin. (2023). A study on reading strategies of undergraduate students of English Education. Research report, Universitas Negeri Yogyakarta.

Sulistiya, M., et al. (2021). Penerapan metode think aloud untuk evaluasi usability pada website Dinas Pendidikan dan Kebudayaan Kota. Jurnal Telematika, Vol 16, No 1 (2021).

Sya’roni, M. I., Kharisma, A. P., & Amalia, F. (2018). Perbandingan hasil metode evaluasi usability antara heuristic evaluation dengan think aloud pada kasus web Filkom Apps untuk mahasiswa. J. Pengemb. Teknol. Inf. dan Ilmu Komputer, vol. 2, no. 2, hlm. 674–678, 2018.

Wang, Y. H. (2016). Reading strategy use and comprehension performance of more successful and less successful readers: a think-aloud study. Educ. Sci. 16, 1789–1813. https://doi.org/10.12738/estp.2016.5.0116

Wilson, R. M., Gambrell, L., & Pfeiffer, W. R. (2015). The effects of retelling upon reading comprehension and recall of text information. The Journal of Educational Research 78(4), 216-220. https://doi.org/10.1080/ 00220671.1985.10885604

Wundt, W. (1888). Selbstbeobachtung und innere wahrnehmung [Self-observation and inner perception]. Philos. Stud. 1, 615–617.




DOI: http://dx.doi.org/10.30659/e.9.2.73-87

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Sugirin Sugirin, Jamilah Jamilah, Joko Priyana, Indah Permatasari

License URL: https://creativecommons.org/licenses/by/4.0/