- Home
- About Us
- Chapters
- Study Design and Organizational Structure
- Study Management
- Tenders, Bids, and Contracts
- Sample Design
- Questionnaire Design
- Instrument Technical Design
- Translation
- Adaptation
- Pretesting
- Interviewer Recruitment, Selection, and Training
- Data Collection
- Paradata and Other Auxiliary Data
- Data Harmonization
- Data Processing and Statistical Adjustment
- Data Dissemination
- Statistical Analysis
- Survey Quality
- Ethical Considerations
- Resources
- How to Cite
- Help
Appendix B (Cohen’s Kappa)
Cohen’s kappa can be used to assess the reliability of an individual (item) code.
- In the following formula for kappa ([latex]\kappa[/latex]), [latex]Pr(a)[/latex] is the relative observed agreement among coders for a given item, and [latex]Pr(e)[/latex] is the hypothetical probability of chance agreement in the observed data calculated from the probabilities of each coder randomly reporting each possible code category for that item: [latex]\kappa=\frac{Pr(a)-Pr(e)}{1-Pr(e)}[/latex].
- If the coders are in complete agreement, then [latex]\kappa[/latex] equals 1. If there is no agreement among the coders (other than what would be expected by chance) then [latex]\kappa[/latex] is less than or equal to 0.