Skip to main content
AI Accessibility Benchmark

Beta

Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

Claude Haiku 5.5

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
MeasureSkill offSkill on
Unique failed acceptance criteria564
of which WCAG362
of which Best practice202
Total cost (US dollars)$0.14$0.69

Table with irregular headers

Results, totals across all generations

Unique failed acceptance criteria

Skill off
3 fails
WCAG
3 fails
Best practice
0 fails
Skill on
1 fail
WCAG
1 fail
Best practice
0 fails

Efficiency, average per generation

Input tokens include cache reads.

Cost in USD

Input tokens

Skill off
4,708 tokens
Skill on
88,408 tokens

Output tokens

Skill off
2,273 tokens
Skill on
7,154 tokens

Turns

Skill off
2 turns
Skill on
6 turns

Time

Skill off
9 seconds
Skill on
29 seconds

Cost

Skill off
$0.0016
Skill on
$0.0068
Prompt given to the model
Build a data table in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). Give the table the title "Quarterly units sold by product line, 2024". Directly above or with the table, include this introductory sentence: "Figures show units sold, grouped into the first and second half of the year, with one row per product line." The table has a first column headed "Product line", then two groups of columns: a group headed "First half" covering two columns "Q1" and "Q2", and a group headed "Second half" covering two columns "Q3" and "Q4". So the top of the table reads: Product line, then First half spanning Q1 and Q2, then Second half spanning Q3 and Q4. The rows, in this order, are: - Notebooks: 120, 135, 150, 160 - Pens: 300, 280, 310, 290 - Folders: 90, 110, 95, 130 (The four numbers in each row are Q1, Q2, Q3, Q4 in that order.) Output the complete code in one HTML code block.

Generated code

Skill off
Skill on
Show

Showing 10 of 10 criteria

Labels and messaging1 criterion

  • None of the table header cells are blank.Best practiceAutomated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Semantic markup6 criteria

  • The table's semantics (table, row and cell roles) are included in the accessibility tree.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The caption is correctly associated with the table in the accessibility tree, with the table being labelled by the caption element.WCAG 1.3.1Automated
    Skill off

    1/3 failed

    • Generation 1: Fail
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The summary is programmatically associated with the table.WCAG 1.3.1Judgement
    Skill off

    3/3 failed

    • Generation 1: Fail
    • Generation 2: Fail
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The column headers are programmatically associated with the data cells they describe.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • For tables with irregular headers, the header cells are programmatically associated with the data cells they describe.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The irregular header cells are associated with the correct data cells for the table's layout (not merely valid references).WCAG 1.3.1Judgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    2/3 failed

    • Generation 1: Fail
    • Generation 2: Pass
    • Generation 3: Fail

Visual design2 criteria

  • The table uses styling to differentiate the header cells from the data cells.Best practiceJudgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • All text meets the minimum contrast ratio of 4.5:1 against the background colours, or 3:1 for large-scale text.WCAG 1.4.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Adaptive UI1 criterion

  • The table can be viewed at 320px without loss of content or functionality; the table is allowed to scroll horizontally within its own container.WCAG 1.4.10Browser
    Skill off

    3/3 failed

    • Generation 1: Fail
    • Generation 2: Fail
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Back to top