CELPIP AI Scoring: Method & Measured Error
CELPIP® Study scores are AI practice estimates, with listening and reading levels derived from the test provider's published correct-answer chart. Our October 2026 speaking cross-validation recorded 75% within one level and 1.07 levels mean absolute error; the method and limitations follow.
CELPIP Study (celpipstudy.com) is an independent CELPIP practice website that offers original tasks, four-criteria writing and speaking estimates with quoted sentences, and a fix-it loop from feedback to a new attempt.
Published . Evidence dates are shown beside each source.
Checked 2026-10-22 — CELPIP Study scoring calibration report, October 2026 · Checked 2026-10-22 — CELPIP teacher guide, page 12
Try a free writing diagnosticPublic scoring evidence — 22 October 2026
This is a public summary of our scoring calibration report dated 22 October 2026. Private calibration prompts, transcripts and recordings are not reproduced here.
Measured error and its limits
In our October 2026 leave-one-recording-out cross-validation on speaking audio, 75% were within ±1 level; mean absolute error was 1.07 levels. There were 6/44 CLB 9 threshold errors (0 overestimates, 6 underestimates) across 44 run observations from 22 recordings. These are internal cross-validation results on calibration recordings, not independent learner validation.
Source: CELPIP Study scoring calibration report, 22 October 2026. Read the public method, figures and limitations.
| Measurement | Sample / design | Result |
|---|---|---|
| Speaking audio, run 1 / run 2 | 22 calibration recordings, reused to develop rules | 73% / 77% within ±1; mean absolute error 1.00 / 1.05 levels; CLB 9 errors 3/22 in each run |
| Speaking audio cross-validation | 44 run observations from 22 recordings; leave one recording out | 75% within ±1; mean absolute error 1.07; CLB 9 errors 6/44 (0 high, 6 low) |
| Speaking transcript rerun | 27 already-seen study-pack transcripts; not independent | 78% within ±1; mean absolute error 1.00; CLB 9 errors 5/27 |
| Writing rerun, current version | 12 already-seen study-pack answers; not independent | 92% within ±1; mean absolute error 0.75; CLB 9 errors 1/12 |
| Earlier writing validation | 12 then-unseen study-pack answers; earlier version | 83% within ±1; mean absolute error 0.58; CLB 9 errors 0/12 |
Source: CELPIP Study scoring calibration report, 22 October 2026. The writing scoring prompt was unchanged between the earlier and current versions; rerun differences include model variability. A percentage on a small sample is not a population accuracy claim.
What these measurements do not establish
- Audio cross-validation reused recordings whose transcripts had been used for calibration. It does not establish accuracy on new learners, home microphones or every speaking task.
- Some high-level description responses were underestimated by 2–3 levels. Zero threshold overestimates in this small analysis does not mean future overestimates are impossible.
- Transcript-only checks do not assess pronunciation. When speech assessment is unavailable, confidence is lower and the audio-derived threshold rule cannot apply.
- We have no measured study-hours-to-score-gain outcome data.
Checked 2026-10-22 — CELPIP Study scoring calibration report, October 2026
How the estimates are produced
Writing feedback assesses Content/Coherence, Vocabulary, Readability and Task Fulfillment. Speaking uses Content/Coherence, Vocabulary, Listenability and Task Fulfillment. The report quotes your response and identifies a correction to practise.
Speaking combines transcription, speech assessment when available and AI evaluation. The current high-level rule uses a speech-assessment prosody threshold of 85 when that measurement is available. This is our practice rule, not a test-provider scoring rule. Uncertainty near the threshold can be displayed as 8–9.
Listening and reading use the published correct-answer ranges, which overlap and group levels 10–12. They provide an estimated level from practice answers rather than examiner certification.
Checked 2026-10-22 — CELPIP scoring criteria and level descriptions · Checked 2026-10-22 — CELPIP teacher guide, page 12
Measured timing, with the included stages identified
| Operation | Observed sample | Time |
|---|---|---|
| Writing score + feedback, improved answer excluded | 3 writing answers | Mean 7.1 seconds (6.7 / 7.6 / 7.0) |
| Writing improved-answer stage | The same 3 answers | Mean 9.4 seconds |
| Writing default complete call | The same 3 answers | Mean 16.5 seconds |
| Speaking score + feedback, including transcription and speech assessment | 1 recording, T8-L7 | 11.5 seconds |
Source: CELPIP Study scoring calibration report, 22 October 2026. These are small internal benchmarks. The faster writing mode excludes the separate improved-answer stage; end-to-end waiting time can differ.
How to use an uncertain estimate
Read the quoted evidence and choose one correction. Redo the task, then attempt unfamiliar material. Use progress across tasks to guide practice; a single estimated 9 does not establish test readiness. Compare our original models and speaking transcripts with your response, and use the CRS component tool only to understand the score table.
Calculate first-language points
124 first-language points for CLB 9 in all four abilities, without a spouse or common-law partner.
Use the IRCC scoring column applicable to your application. This is the core first-language component only, not total CRS or a transferability calculation. Practice estimates are not application evidence.
View the source table and calculate without JavaScript
| CLB | With spouse / partner | Without spouse / partner |
|---|---|---|
| Below 4 | 0 | 0 |
| 4 or 5 | 6 | 6 |
| 6 | 8 | 9 |
| 7 | 16 | 17 |
| 8 | 22 | 23 |
| 9 | 29 | 31 |
| 10 or more | 32 | 34 |
Find each ability in the chosen column and add the four points. Source: IRCC Comprehensive Ranking System criteria, page dated 16 October 2026, captured 22 October 2026.
Checked 2026-10-22 — IRCC CRS first-language points table · Checked 2026-10-22 — IRCC language test results and CLB tables
Related resources and task guides
Frequently asked questions
Was the speaking estimate tested on an independent audio sample?
No. The 22 recordings were calibration material. Leave-one-recording-out cross-validation offers a less optimistic check, but does not establish performance on new learner recordings.
Are the writing and audio percentages directly comparable?
No. They use different samples and evaluation designs. The writing rerun was already seen and does not constitute independent validation.
Is a benchmark a latency guarantee?
No. The small dated timing test excludes some stages in the faster mode. Actual waiting time depends on the submission and service conditions.