Only easy pages
A fictional audit tests articles but also claims to cover search and forms. Add missing functions and journeys before generalising.
In-depth practical guides
Document a sample before publishing a measurement
Define population, selection, repetitions and exclusions so test results can be interpreted.
Updated :
Selected cards and measurement ledger. Illustrative scene.
Which pages or observations does the result cover?
A rate from convenient cases describes those cases, not the whole site. NIST covers measurement plans; WCAG-EM structures conformance sampling. Their purposes and rules remain distinct.
Define population
Specify URLs, languages, dates, queries and devices. Name the unit: page, journey, response or session. Retain the initial list and version.
Describe selection
Separate deliberately critical cases from non-preferential draws. Cover templates, functions and complete processes. Document mechanism, order and replacements.
Separate repetitions
Ten repeats of one question are not ten different intentions. Keep case ID, repeat, time and context; describe dependence before estimating uncertainty.
Publish exclusions
Record technical failures and untested cases rather than silently removing them. Report subsets before aggregation; explain weighting and limits.
Situations and decisions
Typical situations for preparing a check. They do not describe completed assignments or actual observations.
One repeated query
Ten illustrative responses to one query measure its variability, not coverage of ten independent user needs.
Record to retain
- Population and initial list
- Selection rule and units
- Repeats, exclusions and context
- Denominator, weights and limits
Official references
- NIST — Sampling plan
- W3C — WCAG-EM 2.0
- NIST — Generative AI Profile
- Statistics Canada — Read a survey’s limitations
Related protocols
Checks before reaching a conclusion
- Is population defined?
- Are units distinct?
- Are exclusions visible?
- Does the conclusion respect scope?
A selection plan and observation log with explicit denominator.
Frequently asked questions
Should critical and random cases become one rate?
Not without explaining selection and weighting. Critical cases find issues; their share is not automatically their population frequency.
Can a panel be reused after redesign?
Yes for comparison, tracking changes. Extend it for new templates and declare removed cases.
TOOLS / REGISTERS
Write a record
No universal sample size fits every audit. An interval cannot repair biased selection or ill-defined populations. Sampling cannot establish that every page was tested.
- Access date and time zone
- Exact URL
- Sample / population
- Selection unit and case identifier
- Frame, mechanism and selection order
- Repetitions and dependence between observations
- Excluded and untested cases, with reasons
- Metric / measurement
- Limit / uncertainty
- Decision / next action
- Review owner
- Next review