ds@mcpanalytics.aiEmail your data and your question and get an answer about your own data, with the method shown.Email your data and question. Get an answer.
Worked example · Intraclass correlation

Which ICC do you actually need?

By MCP Analytics Team | Published , updated

The reproducible worked example behind our ICC lesson: the same ratings scored by different ICC forms, with confidence intervals.

The data

Twelve subjects rated by three raters. The same twelve rows, scored by every ICC form, cross-checked against numpy, R's psych::ICC, and an independent recompute, agreeing to three decimal places.

To measure how reliable your own ratings are across raters or repeats, try the free intraclass correlation tool.

The headline numbers

One dataset returns 0.573 or 0.983 depending only on which ICC form you ask for. That is not a rounding difference, it is the gap between poor and excellent. The form is a modelling choice about what you are generalising to, not a formula preference.

Download and re-run it

Every figure in the document is derived when it is knitted, not typed in. Re-run the source and you should get the same numbers, or we have a problem worth hearing about.

The honest limit

A point estimate without its confidence interval is not a reliability claim, and with a dozen subjects those intervals are wide. The document says so rather than quietly reporting three decimal places. The product's own report was missing intervals when this was written, which we filed against ourselves and fixed.

The brief for an AI agent

If an AI assistant is choosing a method for you, it needs different things than you do: the discriminating condition, the questions this gets mistaken for, the failure modes that return a plausible number instead of an error, and figures it can cite rather than generate. We publish that as its own document, written to one bar: an agent asked whether to run this analysis should be able to answer from it alone, including saying no.

Read the agent brief

Where this came from

Your turn

Bring your own data and the question you actually need answered.

MCP AnalyticsData science by email Send us your data and your question, and we will reply. ds@mcpanalytics.ai