Abstract
Coefficientκ is generally defined in terms of procedures of computation rather than in terms of a population. Here a population definition is proposed. On this basis, the interpretation ofκ as a measure of diagnostic reliability in characterizing an individual, and the effect of reliability, as measured byκ, on estimation bias, precision, and test power are examined. Factors influencing the magnitude ofκ are identified. Strategies to improve reliability are proposed, including that of combining multiple unreliable diagnoses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brown, W. Some experimental results in the correlation of mental abilities.British J. of Psychology, 1910,3, 296–322.
Cochran, W. G. Errors of measurement in statistics.Techmometrics, 1968,10, 637–666.
Cohen, J. A coefficient of agreement for nominal scales.Educational and Psychological Measurement, 1960,20, 37–46.
Cronbach, L. J., Gleser, G. C., Nanda, H. & Rajaratnam, N.The dependability of behavioral measurements. New York: John Wiley & Sons, Inc., 1972.
Everitt, B. S. Moments of the statistics kappa and weighted kappa.British Journal of Mathematical and Statistical Psychology, 1968,21, 97–103.
Fleiss, J. L.Statistical methods for rates and proportions. New York: John Wiley & Sons, 1973.
Fleiss, J. L. Measuring agreement between two judges on the presence or absence of a trait.Biometrics, 1975,31, 651–659.
Fleiss, J. L., Cohen, J. & Everitt, B. S. Large sample standard errors of kappa and weighted kappa.Psychological Bulletin, 1969,72, 323–327.
Galen, R. S. & Gambino, S. R.Beyond normality: The predictive value and efficiency of medical diagnosis. New York: John Wiley & Sons, 1975.
Helzer, J. E., Robins, L. N., Tarbleson, M., Woodruff, R. A., Reich, T. & Wish, E. D. Reliability of psychiatric diagnosis: I. A methodological review.Archives of General Psychiatry, 1977,34, 129–133.
Helzer, J. E., Clayton, P. J., Pambakian, R., Reich, T., Woodruff, R. A. & Reveley, M. A. Reliability of diagnostic classification.Archives of General Psychiatry, 1977,34, 136–141.
Hubert, L. Kappa revisited.Psychological Bulletin, 1977,84, 289–297.
Kirk, D. B. On the numerical approximation of the bivariate normal (tetrachoric) correlation coefficient.Psychometrika, 1973,38, 259–267.
Koran, L. M. The reliability of clinical methods, data and judgments.N.E. Journal of Medicine, 1975,293, 695–701.
Kraemer, H. C. On estimation and hypothesis testing problems for correlation coefficients.Psychometrika, 1975,40, 473–485.
Landis, J. R. & Koch, G. G. The measurement of observer agreement for categorical data.Biometrics, 1977,33, 159–174.
Light, R. J. Measures of agreement for qualitative data: Some generalizations and alternatives.Psychological Bulletin, 1971,76, 365–377.
Scheffé, H.The analysis of variance. New York: John Wiley & Sons, Inc., 1959.
Spearman, C. Correlation calculated from faulty data.British Journal of Psychology, 1910,3, 271–295.
Sptizer, R. L. & Fleiss, J. L. A re-analysis of the reliability of psychiatric diagnosis.British Journal of Psychiatry, 1974,125, 341–347.
Walker, M. H. & Lev, J.Statistical inference. New York: Henry Holt & Company, 1953.
Author information
Authors and Affiliations
Additional information
This investigation was supported in part by the National Institute of Mental Health Specialized Research Center Grant # MH-30854.
Rights and permissions
About this article
Cite this article
Kraemer, H.C. Ramifications of a population model forκ as a coefficient of reliability. Psychometrika 44, 461–472 (1979). https://doi.org/10.1007/BF02296208
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02296208