Skip to main content
AI Accessibility Benchmark

Beta

Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

Claude Haiku 5.5

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
MeasureSkill offSkill on
Unique failed acceptance criteria564
of which WCAG362
of which Best practice202
Total cost (US dollars)$0.14$0.69

Modal dialog

Results, totals across all generations

Unique failed acceptance criteria

Skill off
1 fail
WCAG
1 fail
Best practice
0 fails
Skill on
0 fails
WCAG
0 fails
Best practice
0 fails

Efficiency, average per generation

Input tokens include cache reads.

Cost in USD

Input tokens

Skill off
3,714 tokens
Skill on
104,727 tokens

Output tokens

Skill off
3,029 tokens
Skill on
10,542 tokens

Turns

Skill off
1 turn
Skill on
6 turns

Time

Skill off
11 seconds
Skill on
44 seconds

Cost

Skill off
$0.0019
Skill on
$0.0094
Prompt given to the model
Build a page with a delete-confirmation dialog in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). The page shows a short paragraph "Your monthly report is ready to download." and a button labelled "Delete file". Choosing "Delete file" opens a dialog window over the page. The dialog has the heading "Delete this file?", one sentence of body text "This will permanently remove the file from your account.", and two buttons: "Cancel" and "Delete". Both buttons dismiss the dialog (no actual deletion needs to happen). The dialog is closed when the page first loads, and while it is open the rest of the page cannot be used until it is dismissed. Output the complete code in one HTML code block.

Generated code

Skill off
Skill on
Show

Showing 15 of 15 criteria

Labels and messaging1 criterion

  • The modal dialog has a clear, descriptive heading that identifies its purpose.WCAG 2.4.6Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Semantic markup3 criteria

  • The modal dialog container element has role="dialog" in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The modal dialog has an accessible name in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Content outside the active modal dialog is programmatically inert.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Keyboard7 criteria

  • When the modal dialog opens, keyboard focus is moved to the modal.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • While the modal dialog is open, focus is trapped inside of the modal container and does not move to elements in the background.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • All interactive elements within the modal dialog have a focus style when they receive focus using a keyboard.WCAG 2.4.7Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The keyboard focus style on the dialog's controls still renders in forced-colours mode (e.g. Windows High Contrast Mode).Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The modal dialog can be closed with a keyboard, such as using a close or cancel button.WCAG 2.1.1Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Pressing the Escape key closes the modal dialog.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the modal dialog is closed, focus is returned to a logical position, such as the button that opened it.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Visual design3 criteria

  • Text within the modal dialog meets the minimum contrast ratio of 4.5:1 against its background, or 3:1 for large-scale text.WCAG 1.4.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • If interactive elements within the modal dialog use custom focus styles, these styles have a contrast ratio of 3:1 against the background colour.WCAG 1.4.11Browser
    Skill off

    3/3 failed

    • Generation 1: Fail
    • Generation 2: Fail
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Content outside an active modal dialog is dimmed or blurred to indicate its inert state.Best practiceJudgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Adaptive UI1 criterion

  • The modal content can be viewed at smaller screen widths (320px) without loss of content or functionality, and without requiring horizontal scrolling.WCAG 1.4.10Browser
    Skill off

    0/3 passed

    • Generation 1: Unmeasured
    • Generation 2: Unmeasured
    • Generation 3: Unmeasured
    Skill on

    0/3 passed

    • Generation 1: Unmeasured
    • Generation 2: Unmeasured
    • Generation 3: Unmeasured

Back to top