ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study

Jiri Dvorak

2012

Abstract

In Educational Measurement field Item Response Theory is a dominant test evaluation method. A few years ago Lawrence Rudner has introduced an alternative method providing better results than IRT in some cases of measurement. However, his method called Measurement Decision Theory did not get much of interest in the community. In this article we would like to give MDT some of the focus we believe it deserves. In particular we are focusing on the practical issues necessary to successfully implement MDT into a daily life of Educational Measurement. We will summarize classification abilities. After that in the main part of this paper we will explain in depth calibration process which is a crucial part of MDT implementation. A basic calibration process will be described as well as its characteristics. Then, as a main result, an improvement of this basic process will be introduced.

References

  1. Caruana, R. and Niculescu-mizil, A. (2006). An empirical comparison of supervised learning algorithms. In In Proc. 23 rd Intl. Conf. Machine learning (ICML06), pages 161-168.
  2. Caruana, R. and Niculescu-mizil, A. (2006). An empirical comparison of supervised learning algorithms. In In Proc. 23 rd Intl. Conf. Machine learning (ICML06), pages 161-168.
  3. Cronbach, L. J. and Gleser, G. C. (1957). Psychological tests and personnel decisions. Urbana: University of Illinois Press.
  4. Cronbach, L. J. and Gleser, G. C. (1957). Psychological tests and personnel decisions. Urbana: University of Illinois Press.
  5. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. In Journal of the Royal Statistical Society, Series B, 39 (1), pages 1-38.
  6. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. In Journal of the Royal Statistical Society, Series B, 39 (1), pages 1-38.
  7. Hambleton, R. and M, M. N. (1973). Toward an integration of theory and method for criterion-referenced tests. In Journal of Educational Measurement, 10, pages 159- 170.
  8. Hambleton, R. and M, M. N. (1973). Toward an integration of theory and method for criterion-referenced tests. In Journal of Educational Measurement, 10, pages 159- 170.
  9. Nigam, K., Mccallum, A. K., Thrun, S., and Mitchell, T. (2000). Text classification from labeled and unlabeled documents using em. In Machine Learning, 39 (2/3), pages 103-134.
  10. Nigam, K., Mccallum, A. K., Thrun, S., and Mitchell, T. (2000). Text classification from labeled and unlabeled documents using em. In Machine Learning, 39 (2/3), pages 103-134.
  11. Rudner, L. M. (2002). An examination of decision-theory adaptive testing procedures. In Paper presented at the annual meeting of the American Educational Research Association, April 2002.
  12. Rudner, L. M. (2002). An examination of decision-theory adaptive testing procedures. In Paper presented at the annual meeting of the American Educational Research Association, April 2002.
  13. Rudner, L. M. (2009). Scoring and classifying examinees using measurement decision theory. In Practical Assessment Research & Evaluation, 14(8).
  14. Rudner, L. M. (2009). Scoring and classifying examinees using measurement decision theory. In Practical Assessment Research & Evaluation, 14(8).
  15. Rudner, L. M. (2010). Measurement decision theory (a measurement decision theory tutorial). In http://echo.edres.org:8080/mdt/.
  16. Rudner, L. M. (2010). Measurement decision theory (a measurement decision theory tutorial). In http://echo.edres.org:8080/mdt/.
  17. van der Linden, W. J. and Mellenbergh, G. J. (1978). Coefficients for tests from a decision-theoretic point of view. In Applied Psychological Measurement, 2, pages 119- 134.
  18. van der Linden, W. J. and Mellenbergh, G. J. (1978). Coefficients for tests from a decision-theoretic point of view. In Applied Psychological Measurement, 2, pages 119- 134.
Download


Paper Citation


in Harvard Style

Dvorak J. (2012). ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study . In Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-8565-07-5, pages 94-99. DOI: 10.5220/0003901300940099


in Harvard Style

Dvorak J. (2012). ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study . In Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-8565-07-5, pages 94-99. DOI: 10.5220/0003901300940099


in Bibtex Style

@conference{csedu12,
author={Jiri Dvorak},
title={ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study},
booktitle={Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2012},
pages={94-99},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003901300940099},
isbn={978-989-8565-07-5},
}


in Bibtex Style

@conference{csedu12,
author={Jiri Dvorak},
title={ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study},
booktitle={Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2012},
pages={94-99},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003901300940099},
isbn={978-989-8565-07-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study
SN - 978-989-8565-07-5
AU - Dvorak J.
PY - 2012
SP - 94
EP - 99
DO - 10.5220/0003901300940099


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - ON PRACTICAL ISSUES OF MEASUREMENT DECISION THEORY - An Experimental Study
SN - 978-989-8565-07-5
AU - Dvorak J.
PY - 2012
SP - 94
EP - 99
DO - 10.5220/0003901300940099