Main Article Content
In the literature, response style is one of the factors causing an achievement-attitude paradox and threatens the validity of the results obtained from studies. In this regard, the aim of this study is two-fold. Firstly, it attempts to determine which item response tree (IRTree) models based on the generalized linear mixed model (GLMM) approach (random intercept, random intercept with fixed effect of extreme response and random intercept-slope model) best fit the Trends in International Mathematics and Science Study (TIMSS) 2015 data. Secondly, it purports to explore how the extreme response style affects students’ attitudes toward mathematics of students. This study is both basic research and descriptive research in terms of seeking for answers for two different research questions. For the sample of this research, 15 countries were randomly selected among countries participated in TIMSS 2015. The students’ responses to items measuring attitude in the student questionnaire were analyzed with the packages “lme4” and “irtrees” in R software. When the model fit indices were evaluated, the random intercept-slope model was found to be the best fit to the data. According to this model, the extreme response style explains a significant amount of variances in the students’ attitude toward mathematics. Additionally, students with a negative attitude toward mathematics were found to have an extreme response style. It was concluded that an extreme response style had an effect on students’ attitude.
International Journal of Assessment Tools in Education
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64, 359-373.
Bachman, J. G., O’Malley, P. M., & Freedman-Doan, P. (2010). Response styles revisited: Racial/ethnic and gender differences in extreme responding (Monitoring the Future Occasional Paper No. 72). Ann Arbor, MI: Institute for Social Research.
Bates, D., Maechler, M., Bolker, B. & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01
Bofah, E. A. and Hannula, M. S. (2015). TIMSS data in an African comparative perspective: Investigating the factors influencing achievement in mathematics and their psychometric properties. Large-Scale Assessments in Education, 3(1), doi:1.1186/s40536-015-0014-y
Bolt, D. M., & Newton, J. (2011). Multiscale measurement of extreme response style. Educational and Psychological Measurement, 71, 814-833.
Bolt, D., Wollack, J., & Suh, Y. (2012). Application of a multidimensional nested logit model to multiple-choice test items. Psychometrika, 77, 339–357.
Böckenholt, U. (2012). Modeling multiple response processes in judgment and choice. Psychological Methods, 17(4), 665-678.
Böckenholt, U. (2017). Measuring response styles in Likert items. Psychological Methods, 22(1), 69–83. doi:10.1037/met0000106
Böckenholt, U. & Meiser, T. (2017). Response style analysis with threshold and multi-process IRT models: A review and tutorial. British Journal of Mathematical and Statistical Psychology, 70, 159–181. doi:10.1111/bmsp.12086
Buckley, J. (2009, June). Cross-national response styles in international educational assessment: Evidence from PISA 2006. NCES Conference on the Program for International Student Assessment: What we can learn from PISA, Washington, DC.
Büyüköztürk, Ş. (2005). Sosyal Bilimler için veri analizi el kitabı[Data analysis handbook for social sciences]. 5. baskı. Pagem A Yayıncılık.
Bybee, R.,& McCrae, B. (2007). Scientific literacy and student attitudes: Perspectives from PISA 2006 science. International Journal of Science Education, 33, 7-26.
Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological Measurement, 6, 475-494.
Culpepper, S. (2014). If at first you don’t succeed, try, try again: Applications of sequential IRT models to cognitive assessments. Applied Psychological Measurement, 38, 632– 644.
De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28.
De Boeck P & Wilson M (2004). Explanatory item response models: A generalized linear and nonlinear approach. Springer-Verlag, New York.
Harter, S. (1999). The construction of the self: A developmental perspective. New York: Guildford Press.
Heide, M. & Gronhaug, K. (1992) The impact of response styles in surveys: a simulation study. Journal of the Market Research Society, 34, 215-231.
Hofstede, G. H. (2001). Cultures consequences: Comparing values, behaviors, institutions, and organizations across nations (2nd ed.). Thousand Oaks, California: Sage Publications, Inc.
Hooper, M, Mullis. I. V. S., & Martin, M.O. (2013). TIMSS 2015 Context Questionnaire Framework. Mullis, I.V.S. and Martin, M.O. (Eds.) TIMSS 2015 Assessment Frameworks. Retrieved January 15, 2019, from Boston College, TIMSS and PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2015/frameworks.html
Hox J. 2002. Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum
Huang H-Y. (2016) Mixture random-effect IRT models for controlling extreme response style on rating scales. Frontiers Psychology, 7(1706), 1-15. doi: 10.3389/fpsyg.2016.01706
Ilgun Dibek, M., Bulut, O., Sahin Kursad, M., & Yavuz, H. C. (2018, July). Should students with disabilities have multiple opportunities in answering items? Paper presented at the International Testing Commission Conference, Montreal, QC, Canada
Jeon, M., & De Boeck, P. (2016). A generalized item response tree model for psychological assessments. Behavior research methods, 48(3), 1070–1085. doi: 10.3758/s13428-015-0631-y
Johnson, T.R. (2007). Discrete choice models for ordinal response variables: A generalization of the stereotype model. Psychometrika, 72, 489–504.
Johnson, R.B. and Christensen, L.B. (2008) Educational Research: Quantitative, Qualitative, and Mixed Approaches. 3rd Edition, Sage Publications, Inc., Los Angeles.
Kadijevich, D. (2008). TIMSS 2003: Relating dimensions of mathematics attitude to mathematics achievement. Zbornik instituta za Pedagogical Research, 40(2), 327–346. doi: 1.2298/ZIPI0802327K
Kidd, C. V. (1959). Basic research: Description versus definition. Science, 129, 368-371.
LaRoche, S. & Foy, P. (2016). Sample Implementation in TIMSS 2015. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and Procedures in TIMSS 2015 (pp. 5.1-5.175). Retrieved January 8, 2019, from Boston College, TIMSS & PIRLS International Study Center website: http://timss.bc.edu/publications/timss/2015-methods/chapter-5.html
LaRoche, S., Joncas, M., and Foy, P. (2016). Sample Design in TIMSS 2015. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and Procedures in TIMSS 2015 (pp. 3.1-3.37). Retrieved January 10, 2019, from Boston College, TIMSS & PIRLS International Study Center website: http://timss.bc.edu/publications/timss/2015-methods/chapter-3.html
Leventhal, B.C & Stone, C.A (2018). Bayesian analysis of multidimensional item response theory models: A discussion and illustration of three response style models, Measurement: Interdisciplinary Research and Perspective, 16(2), 114-128, doi: 10.1080/15366367.2018.1437306
Liu, M. (2015). Response Style and Rating Scales: The Effects of Data Collection Mode, Scale Format, and Acculturation (Unpublished doctoral dissertation). The University of Michigan.
Marsh, H. W., Trautwein, U., Lüdtke, O., Köller, O & Baumert, J. (2005). Academic self-concept, interest, grades and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397-416.
Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
Mooi, E., Sarstedt, M., & Mooi-Reci, I. (2018). Market research: The process, data, and methods using Stata. Singapore: Springer.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 International Results in Mathematics.Retrieved January 10, 2019, from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2015/international-results/
Nakagawa, S., and H. Schielzeth. 2013. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods in Ecology and Evolution 4(2): 133-142. doi: 10.1111/j.2041-210x.2012.00261.x
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill
Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver & L. S. Wrightman (Eds.), Measures of Personality and Social Psychological Attitudes (Vol. 1). San Diego, CA: Academic Press.
Peterson, R.A, Rhi-Perez, P. & Albaum, G. (2012). A cross-national comparison of extreme response style measures. International Journal of Market Research, 56(1), 89-110.
Thissen-Roe, A., & Thissen, D. (2013). A two-decision model for responses to Likert-type Items. Journal of Educational and Behavioral Statistics, 38(5), 522-547.
Tutz, G. (1990). Sequential item response models with an ordered response. British Journal of Mathematical and Statistical Psychology, 43, 39–55.
Van de Gaer, E. & Adams, R. (2010, May). The Modeling of Response Style Bias: An Answer to the Attitude-Achievement Paradox?, paper presented at the annual conference of the American Educational Research Association, Denver, Colorado, USA.
Van de gaer, E., Grisay, A., Schulz, W. & Gebhardt, E. (2012). The reference group effect an explanation of the paradoxical relationship between academic achievement and self-confidence across countries. Journal of Cross-Cultural Psychology, 43(8), 1205-1228
Yavuz, H. C., Bulut, O., Ilgun Dibek, M., & Sahin Kursad, M. (2018, July). Providing revision opportunities in alternate assessments: An application of sequential IRT. Paper presented at the International Testing Commission Conference, Montreal, QC, Canada.