The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients

Nuri DOĞAN, Abdullah Faruk KILIÇ

Abstract


This study aims to compare reliability coefficients according to sample size (250, 500, 1,000, 2,500, 5,000, and 9,773), EFA factor extraction methods (PCA, PA, ULS, WLS, and MLE), CFA estimation methods (UL, ML, and GL), and correlation matrices (Pearson, phi, and tetrachoric). Therefore, it employs a basic research method. The study was conducted with real data, and the data were collected from students’ answers to a Turkish sub-test in the Test for Transition from Basic Education into Secondary Education administered in 2014. Within the scope of the study, McDonald ω, McDonald ωh, maximal reliability, Armor Ɵ, Heise and Bohrnstedt Ω, Revelle β, and standardized alpha coefficients were compared. Consequently, it was found that sample size in the same correlation matrices did not lead to serious changes. It was also found that McDonald ωh and Revelle β coefficients calculated with a tetrachoric correlation were bigger than 1 in some conditions. It was recommended in consequence that those coefficients should be calculated through phi correlations for congeric one-factor structures. Other findings obtained support the literature, and necessary suggestions are made.


Keywords


reliability; McDonald ω; maximal reliability ;Armor Ɵ; Heise and Bohrnstedt Ω; Revelle β; standardized alpha

Full Text:

PDF

References


Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey: Brooks/Cole Publishing Company.

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement In Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Armor, D. F. (1974). Theta reliability and factor scaling. In H. L. Costner (Ed.), Sociological Methodology (5. Ed., pp. 17–50). San Francisco: Jossey-Bass.

Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Beverly Hills, California: SAGE. https://doi.org/10.1037/018269

Chou, C. P., & Bentler, P. M. (1995). Estimates and tests in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA: SAGE.

Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56(5), 754–761. https://doi.org/10.1037/0022-006X.56.5.754

Crocker, L., & Algina, J. (2008). Introduction of classical and modern test theory. Ohio: Cengage Learning.

Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. https://doi.org/10.1037/1082-989X.1.1.16

Ercan, I., Yazici, B., Sigirli, D., Ediz, B., & Kan, I. (2007). Examining Cronbach Alpha, theta, omega reliability coefficients according to sample size. Journal of Modern Applied Statistical Methods, 6(1), 291–303. Retrieved from http://digitalcommons.wayne.edu/jmasm/vol6/iss1/27

Fox, J. (2016). polycor: Polychoric and polyserial correlations. Retrieved from https://cran.r-project.org/package=polycor

Fraenkel, J. R., Wallen, N. E., & Huyn, H. H. (2012). How to design and evaluate research in education (8. Baskı). New York: McGraw-Hill.

Gay, L. R., Mills, G. E., & Airasian, P. (2012). Educational research: Competencies for analysis and applications (10th Ed.). New Jersey: Pearson.

Gorsuch, R. L. (1974). Factor Analysis (1st Ed.). Toronto: W. B. Saunders Company.

Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275.

Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In Structural Equation Modeling: Present and Future: A Festschrift in Honor of Karl Jöreskog (pp. 195–216).

Hancock, G. R., & Mueller, R. O. (2013). Structural equating modelling: A second course. B. G. Tabachnick & L. S. Fidell (Orgs.), Using multivariate statistics (2nd Ed.). Charlotte, NC: Information Age Publishing.

Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. In E. F. Borgatta & G. W. Bohrnstedt (Eds.), Sociological Methodology (pp. 104–129). San Francisco: Jossey-Bass.

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: LEA Publisher.

Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343–355. https://doi.org/10.1037/1082-989X.5.3.343

Price, L. R. (2017). Psychometric methods: Theory and practice. New York, NY: The Guilford Press.

R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.r-project.org/.

Revelle, W. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14, 57–74. https://doi.org/10.1207/s15327906mbr1401_4

Revelle, W. (2016). psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, Illinois. Retrieved from https://cran.r-project.org/package=psych

Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modelling: Multilevel, longitudinal, and structural equation models. New York: Chapman & Hall.

Streiner, D. L. (1994). Figuring out factors: the use and misuse of factor analysis. Canadian Journal of Psychiatry, 39(3), 135–140.

Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s, α Revelle’s β and McDonald’s ω H: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123–133. https://doi.org/10.1007/s11336-003-0974-7




DOI: https://doi.org/10.24106/kefdergi.413303

Refbacks

  • There are currently no refbacks.


indexed by TR-DİZİN, DOAJ, SOBİAD

 


Creative Commons License  
 Kastamonu Eğitim'de yayınlanan tüm içerik ve makaleler "Creative Commons Alıntı 4.0 Uluslararası Lisansı" ile lisanslanmıştır.