Accessibility Audits Need a Repeatable Evaluation Method

An accessibility audit can produce a long spreadsheet and still leave a business unsure what was tested, how representative the sample was, or whether the next audit can be compared fairly. The problem is not always a lack of tools. It is the absence of a repeatable evaluation method with a defined scope, sample, standard, and evidence trail.

In July 2026, W3C published WCAG Evaluation Methodology 2.0 as a Group Note. WCAG-EM 2.0 extends structured evaluation beyond websites and pages to digital products including applications. It offers organizations a timely reason to improve how audits are planned and reported.

Define the product and scope first

Name the environments, domains, applications, versions, account roles, languages, and third-party flows included. State what is excluded and why. An audit of five public pages should not be presented as an evaluation of an authenticated service, mobile app, or complete ecommerce journey.

Record the conformance target, technologies relied upon, testing dates, and known constraints. If the evaluation supports a legal, procurement, or contractual decision, involve the appropriate specialists before work begins.

Choose a representative sample

Include common templates, critical processes, distinct components, media, documents, error states, and pages with high customer impact. Random selection alone can miss the checkout, application form, account recovery, or custom widget where barriers are concentrated.

Use analytics and product knowledge to find representative content, but do not let traffic hide low-volume essential services. A rarely visited disability accommodation page or payment failure state may still require careful evaluation.

Follow a documented evaluation process

The W3C WAI announcement describes WCAG-EM 2.0 as a step-by-step methodology for evaluating conformance across web and non-web digital products. Adopt a consistent sequence for scope, exploration, sampling, auditing, and reporting.

Document tools, browser and assistive-technology combinations, test accounts, data conditions, and evaluator judgment. Another qualified person should be able to understand what produced the result without guessing.

Combine automation with manual task testing

Automated tests efficiently detect certain markup, name, contrast, and structural issues. Manual review is needed for meaningful alternatives, logical focus, instructions, error handling, keyboard operation, zoom, reading order, and whether a task can actually be completed.

Include testing by people with disabilities when the project can support it. Conformance evaluation and usability research answer different questions; together they produce a more credible view of the experience.

Report impact, evidence, and ownership

For every finding, record the affected criterion, location or component, reproduction steps, observed result, expected result, user impact, evidence, severity rationale, and recommended direction. Link repeated instances to one component-level root cause when appropriate.

• Blockers preventing task completion
• Serious barriers creating substantial difficulty
• Moderate issues with workable alternatives
• Minor defects and consistency improvements
• Systemic causes affecting multiple screens

Assign an owner and target date after the business reviews effort and risk. A severity label without a decision path does not create remediation.

Make the next evaluation comparable

Save the scope, sample, product version, test environment, and unresolved exceptions. When retesting, distinguish fixed findings, regressions, newly sampled areas, and changes caused by the methodology. Avoid claiming trend improvement when two audits measured different products.

Begin by rewriting the brief for your next accessibility audit. Define the critical tasks, representative sample, evidence format, and retest expectations before requesting a quote. A repeatable method turns accessibility reporting from a one-off score into dependable quality management.

Ask evaluators to distinguish direct observation from inference. A finding should show what happened, under which conditions, and which requirement it affects. Recommendations can offer implementation directions, but the product team should confirm the chosen fix within the complete design and technical context.

Protect test evidence appropriately. Screenshots, recordings, account data, and assistive-technology output may contain customer or internal information. Define secure storage, access, retention, and redaction before the audit begins, especially when external evaluators or production accounts are involved.

Provide a plain-language summary for product owners and leadership. It should name the affected customer tasks, highest risks, systemic causes, decisions required, and confidence limits. This makes the technical evidence usable without reducing accessibility to a single percentage.

Connect audit evidence to the release process

Move repeat findings into the design system and automated test suite. If several screens fail because one dialog component traps focus or one field pattern lacks instructions, fix and verify the shared component rather than closing instances individually. Add regression tests at the layer where the defect originated.

Define which severities block release, who can approve a temporary exception, what customer alternative must exist, and when the exception expires. Store this decision with the finding. Accessibility debt should not disappear into a general backlog where it competes without context against cosmetic enhancements.

Schedule focused retests after remediation and a representative evaluation at an appropriate interval or major product change. Use the same methodology where possible, while documenting improvements to the sample. Report task outcomes and remaining risk to leadership, not only counts. Ten fixed low-impact issues do not compensate for one unresolved barrier that prevents payment, application, or account access.

Photo by Darlene Alderson on Pexels.