Nested multiple imputation in large-scale assessments

详细信息    查看全文

• 作者：Sebastian Weirich (1)
  Nicole Haag (1)
  Martin Hecht (1)
  Katrin B枚hme (1)
  Thilo Siegle (1)
  Oliver L眉dtke (2)
  
  1. Institute for Educational Quality Improvement ; Humboldt-Universit盲t zu Berlin ; Unter den Linden 6 ; 10099 ; Berlin ; Germany
  2. Leibniz Institute for Science and Mathematics Education (IPN) ; Centre for International Student Assessment ; Olshausenstra脽e 62 ; 24118 ; Kiel ; Germany
  
• 关键词：Large ; scale assessment ; Missing data ; Imputation ; Simulation ; Item response theory
• 刊名：Large-scale Assessments in Education
• 出版年：2014
• 出版时间：December 2014
• 年：2014
• 卷：2
• 期：1
• 全文大小：329 KB
• 参考文献：1. Adams, RJ, Wilson, M, Wang, W-C (1997) The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement 21: pp. 1-23 CrossRef
  2. Adams, RJ, Wu, ML The Mixed-Coefficients Multinomial Logit Model: A Generalized Form of the Rasch Model. In: Davier, M, Carstensen, CH eds. (2007) Multivariate and Mixture Distribution Rasch Models. Springer, New York, pp. 57-75 CrossRef
  3. Allen, NL, Donoghue, JR, Schoeps, TL (2001) The NAEP 1998 Technical Report. National Center for Educational Statistics, Washington
  4. Allison, PD (2002) Missing Data. Sage Publications, Thousand Oaks, CA
  5. Bock, RD, Aitkin, M (1981) Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika 46: pp. 443-459 CrossRef
  6. Boeck, P (2008) Random item IRT models. Psychometrika 73: pp. 533-559 CrossRef
  7. Embretson, SE, Reise, SP (2000) Item Response Theory for Psychologists. Erlbaum, Mahwah, NJ
  8. Enders, CK (2010) Applied Missing Data Analysis. Guilford Press, New York
  9. Foy, P, Galia, J, Li, I Scaling the Data from the TIMSS 2007 Mathematics and Science Asssessment. In: Olson, JF, Martin, MO, Mullis, IVS eds. (2008) TIMSS 2007 Technical Report. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College, Chestnut Hill, MA, pp. 225-280
  10. Frey, A, Hartig, J, Rupp, AA (2009) An NCME instructional module on booklet designs in large-scale assessments of student achievement: theory and practice. Educational Measurement: Issues and Practice 28: pp. 39-53 CrossRef
  11. Gonzalez, E, Rutkowski, L (2010) Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. IEA-ETS Research Institute Monograph 3: pp. 125-156
  12. Graham, JW (2009) Missing data analysis: making it work in the real world. Annual Review of Psychology 60: pp. 549-576 CrossRef
  13. Harel, O (2007) Inferences on missing information under multiple imputation and two-stage multiple imputation. Statistical Methodology 4: pp. 75-89 CrossRef
  14. Harel, O, Schafer, JL (2003) Multiple Imputation in two Stages. Paper presented at the Proceedings of Federal Committee on Statistical Methodology Research Conference, Washington DC
  15. Kiefer, T, Robitzsch, A, & Wu, M. (2013). TAM: test analysis modules (R package version 0.5-21). Retrieved from http://CRAN.R-project.org/package=TAM.
  16. Kuhl, P, Siegle, T, Lenski, AE Soziale Disparit盲ten [Social disparities]. In: Pant, HA, Stanat, P, Schroeders, U, Roppelt, A, Siegle, T, P枚hlmann, C eds. (2013) IQB-L盲ndervergleich 2012. Mathematische und naturwissenschaftliche Kompetenzen am Ende der Sekundarstufe I. Waxmann, M眉nster
  17. Little, RJA (1992) Regression with missing X鈥檚: A review. Journal of the American Statistical Association 87: pp. 1227-1237
  18. Lord, FM (1974) Estimation of latent ability and item parameters when there are omitted responses. Psychometrika 39: pp. 247-264 CrossRef
  19. Lord, FM (1983) maximum likelihood estimation of item response parameters when some responses are omitted. Psychometrika 48: pp. 477-482 CrossRef
  20. Ludlow, LH, O鈥檒eary, M (1999) Scoring omitted and not-reached items: practical data analysis implications. Educational and Psychological Measurement 59: pp. 615-630 CrossRef
  21. Mazzeo, J, Davier, M (2008) Review of the Programme for International Student Assessment (PISA) Test Design: Recommendations for Fostering Stability in Assessment Results.
  22. Mislevy, RJ, Beaton, AE, Kaplan, B, Sheehan, KM (1992) Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement 29: pp. 133-161 CrossRef
  Assessing Scientific, Reading and Mathematical Literacy. A Framework for PISA 2006. OECD, Paris
  PISA 2006 Technical Report. OECD, Paris CrossRef
  R: A Language and Environment for Statistical Computing. (Version 3.1.0). R Foundation for Statistical Computing, Vienna, Austria
  23. Reiter, JP, Drechsler, J (2007) Releasing multiply-imputed synthetic data generated in two stages to protect confidentiality. IAB Discussion Paper 2007: pp. 1-18
  24. Reiter, JP, Raghunathan, TE (2007) The multiple adaptions of multiple imputation. Journal of the American Statistical Association 102: pp. 1462-1471 CrossRef
  25. Rubin, DB (1987) Multiple Imputation for Nonresponse in Surveys. Wiley, New York CrossRef
  26. Rubin, DB (2003) Nested multiple imputation of NMES via partially incompatible MCMC. Statistica Neerlandica 57: pp. 3-18 CrossRef
  27. Rutkowski, L (2011) The impact of missing background data on subpopulation estimation. Journal of Educational Measurement 48: pp. 293-312 CrossRef
  28. Schafer, JL, Graham, JW (2002) Missing data: our view of the state of the art. Psychological Methods 7: pp. 147-177 CrossRef
  Stanat, P, Pant, HA, B枚hme, K, Richter, D eds. (2012) Kompetenzen von Sch眉lerinnen und Sch眉lern am Ende der vierten Jahrgangsstufe in den F盲chern Deutsch und Mathematik. Waxmann, M眉nster
  29. Tuerlinckx, F, Rijmen, F, Molenberghs, G, Verbeke, G, Briggs, D, Noortgate, W, Boeck, P Estimation and Software. In: Boeck, P, Wilson, M eds. (2004) Explanatory Item Response Models. Springer, New York, pp. 343-373 CrossRef
  30. Buuren, S (2007) Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research 16: pp. 219-242 CrossRef
  31. Buuren, S, Groothuis-Oudshoorn, K (2011) mice: Multivariate imputation by chained equations in R. Journal of Statistical Software 45: pp. 1-67
  32. Davier, M, Sinharay, S, Oranje, A, Beaton, AE The Statistical Procedures Used in National Assessment of Educational Progress: Recent Developments and Future Directions. In: Rao, CR, Sinharay, S eds. (2007) Handbook of Statistics: Vol. 26. Psychometrics. Elsevier, Amsterdam
  33. Davier, M, Gonzalez, E, Mislevy, RJ (2009) What are plausible values and why are they useful?. IERI Monograph Series: Issues and Methodologies in Large Scale Assessments 2: pp. 9-36
  34. Warm, TA (1989) Weighted likelihood estimation of ability in item response theory. Psychometrika 54: pp. 427-450 CrossRef
  35. Weirich, S, Haag, N, Roppelt, A Testdesign und Auswertung des L盲ndervergleichs: Technische Grundlagen [Test design and analysis of IQB national assessment: Technical fundamentals]. In: Stanat, P, Pant, HA, B枚hme, K, Richter, D eds. (2012) Kompetenzen von Sch眉lerinnen und Sch眉lern am Ende der vierten Jahrgangsstufe in den F盲chern Deutsch und Mathematik. Waxmann, M眉nster
  36. White, IR, Royston, P, Wood, AM (2010) Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine 30: pp. 377-399 CrossRef
  37. Wilson, M, Boeck, P Descriptive and Explanatory Item Response Models. In: Boeck, P, Wilson, M eds. (2004) Explanatory Item Response Models. Springer, New York, pp. 43-74 CrossRef
  38. Wu, M (2005) The role of plausible values in large-scale surveys. Studies in Educational Evaluation 31: pp. 114-128 CrossRef
  
• 刊物主题：Assessment, Testing and Evaluation; Statistics for Social Science, Behavorial Science, Education, Public Policy, and Law; Educational Policy and Politics;
• 出版者：Springer US
• ISSN：2196-0739

文摘

Background In order to measure the proficiency of person populations in various domains, large-scale assessments often use marginal maximum likelihood IRT models where person proficiency is modelled as a random variable. Thus, the model does not provide proficiency estimates for any single person. A popular approach to derive these proficiency estimates is the multiple imputation of plausible values (PV) to enable subsequent analyses on complete data sets. The main drawback is that all variables that are to be analyzed later have to be included in the imputation model to allow the distribution of plausible values to be conditional on these variables. These background variables (e.g., sex, age) have to be fully observed which is highly unlikely in practice. In several current large-scale assessment programs missing observations on background variables are dummy coded, and subsequently, dummy codes are used additionally in the PV imputation model. However, this approach is only appropriate for small proportions of missing data. Otherwise the resulting population scores may be biased. Methods Alternatively, single imputation or multiple imputation methods can be used to account for missing values on background variables. With both imputation methods, the result is a two-step procedure in which the PV imputation is nested within the background variable imputation. In the single+multiple-imputation (SMI), each missing value on background variables is replaced by one value. In the multiple+multiple-imputation (MMI), each missing value is replaced by a set of imputed values. MMI is expected to outperform SMI as SMI ignores the uncertainty due to missing values in the background data. Results In a simulation study, both methods yielded unbiased population estimates under most conditions. Still, the recovery proportion was slightly higher for the MMI method. Conclusions The advantages of the MMI method are apparent for fairly high proportions of missing values in combination with fairly high dependency between the latent trait and the probability of missing data on background variables.