Skip to main content
AI Accessibility Benchmark

Beta

Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

Gemini 4 Argon

Skill version 2.3

Results for 22 components, each generated 3 times.Each acceptance criterion is counted once per component if it fails in any generation.
MeasureSkill offSkill on
Failed acceptance criteria220
of which WCAG150
of which Best practice70
Total cost (US dollars)$4.84$10.85

Menu button

Results, totals across all generations

Failed acceptance criteria

Each acceptance criterion is counted once per component if it fails in any generation.

Skill off
1 fail
WCAG
0 fails
Best practice
1 fail
Skill on
0 fails
WCAG
0 fails
Best practice
0 fails

Efficiency, average per generation

Input tokens include cache reads.

Cost in USD

Input tokens

Skill off
8,484 tokens
Skill on
90,690 tokens

Output tokens

Skill off
6,529 tokens
Skill on
10,802 tokens

Turns

Skill off
3 turns
Skill on
7 turns

Time

Skill off
72 seconds
Skill on
129 seconds

Cost

Skill off
$0.08
Skill on
$0.18

Generated components

Prompt given to the model
Build a menu button in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). It is an "Actions" button that opens a menu of three commands: Duplicate, Rename, and Delete. The menu is closed when the page loads. Choosing a command closes the menu and shows the command's name in a "Last action:" line below the button (initially "Last action: none"). Output the complete code in one HTML code block.
Skill off
Skill on

Acceptance criteria

Show

Showing 27 of 27 criteria

Labels and messaging1 criterion

  • The button has a visible label.WCAG 3.3.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Semantic markup12 criteria

  • The button's role is included in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button's accessible name is included in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The button's accessible name contains the exact text of its visible label.WCAG 2.5.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The button's accessible name matches its visible label exactly, or at least starts with the exact text of the visible label.Best practiceAutomated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button has aria-haspopup set to menu or true.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button's expanded or collapsed state is included in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The element that contains the menu items has the menu role in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button has aria-controls set to the id of the menu popup element.Best practiceAutomated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the menu is expanded, it is included in the accessibility tree.WCAG 1.3.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu items have the menuitem role in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu items come directly after the menu button in the accessibility tree.WCAG 1.3.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the menu is collapsed, it is not included in the accessibility tree.WCAG 1.3.2Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Keyboard10 criteria

  • The menu button is focusable using a keyboard.WCAG 2.1.1Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button is activated by pressing the Space or Enter keys on the keyboard, opening the menu.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the menu button is activated, keyboard focus moves to the first menu item.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu items are operable using a keyboard.WCAG 2.1.1Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu items are navigable using arrow keys.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Pressing the Escape key when the menu is open closes the menu.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Pressing the Tab key to move away from the menu closes the menu.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When closing the menu using the keyboard, focus is returned to the menu button.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The button has a focus style when it receives focus using a keyboard.WCAG 2.4.7Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The keyboard focus style still renders in forced-colours mode (e.g. Windows High Contrast Mode).Best practiceBrowser
    Skill off

    2/3 failed

    • Generation 1: Fail
    • Generation 2: Pass
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Visual design2 criteria

  • All text meets the minimum contrast ratio of 4.5:1 against the background colour, or 3:1 for large-scale text (graded with the menu open, so menu item text is included).WCAG 1.4.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • If the button uses a custom focus style, the focus indicator has a contrast ratio of at least 3:1 against its background colour.WCAG 1.4.11Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Adaptive UI2 criteria

  • The menu button and its menu can be viewed at smaller screen widths (320px) without loss of content or functionality. The user does not have to scroll horizontally to use them.WCAG 1.4.10Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The menu button and menu can be viewed at smaller screen widths (320px) without loss of content or functionality, and without requiring horizontal scrolling.WCAG 1.4.10Browser
    Skill off

    0/3 passed

    • Generation 1: Unmeasured
    • Generation 2: Unmeasured
    • Generation 3: Unmeasured
    Skill on

    0/3 passed

    • Generation 1: Unmeasured
    • Generation 2: Unmeasured
    • Generation 3: Unmeasured

Back to top