Here is the CMT Uptime check phrase

Appendix C (Coder Design Effect)

The coder design effect (deffc) applies much of the same logic as the interviewer design effect (see Interviewer Recruitment, Selection, and Training) [zotpressInText item="{2265844:BLZRKBZS}"].
  • In the following formula for coder design effect, [latex]\rho_{c}[/latex] is the intraclass coefficient for coders, [latex]m[/latex] is the average number of cases coded per coder, and [latex]r[/latex] is the reliability of a particular (item) code: [latex]\text{deff}_{c}=1+\rho_{c}(m-1)(1-r)[/latex].
  • The intraclass coefficient for coders is a measure of the ratio of coder variance to the total variance and is defined as [latex]\rho_{c}=\frac{\text{between-coder variance}}{\text{between-coder variance}+\text{within-coder variance}}[/latex].
Reference [zotpressInTextBib style="apa" sortby="author"]