Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

GPT-6 Luna

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
Measure	Skill off	Skill on
Failed acceptance criteria	143	12
of which WCAG	100	7
of which Best practice	43	5
Total cost (US dollars)	$0.09	$0.36
Input tokens	199,442	6.2 million
Output tokens	132,160	369,858

Combobox with list autocomplete

Results, totals across all generations

Cost in USD

Failed acceptance criteria

Skill off

15 fails

WCAG: 9 fails
Best practice: 6 fails

Skill on

2 fails

WCAG: 0 fails
Best practice: 2 fails

Total cost

Skill off: $0.0059
Skill on: $0.03

Efficiency, average per generation

Input tokens include cache reads.

Input tokens

Skill off: 5,624 tokens
Skill on: 151,564 tokens

Output tokens

Skill off: 3,341 tokens
Skill on: 10,622 tokens

Turns

Skill off: 2 turns
Skill on: 9 turns

Time

Skill off: 27 seconds
Skill on: 115 seconds

Cost

Skill off: $0.0020
Skill on: $0.0090

Prompt given to the model

Build a search field labelled 'Country" in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). Below the label, show the hint "Start typing to filter the list." The field starts empty. Inside the field on the right, place a downward chevron button that opens a suggestions dropdown. When the field is focused, the full list of countries appears; as the user types, the list filters to the countries whose names match the typed text. The countries are: Argentina, Australia, Austria, Brazil, Canada, France, Germany, India, Japan, and Spain. Just above the list, show a count of how many countries currently match — for example "10 results", or "No results" when none match — and update this count as the user types. No suggestion is highlighted until the user moves into the list. Output the complete code in one HTML code block.

Generated code

Skill off: Generation 1, skill off (opens in a new tab)
Generation 2, skill off (opens in a new tab)
Generation 3, skill off (opens in a new tab)
Skill on: Generation 1, skill on (opens in a new tab)
Generation 2, skill on (opens in a new tab)
Generation 3, skill on (opens in a new tab)

Showing 32 of 32 criteria

Labels and messaging3 criteria

The combobox has a permanent visible label. It does not use placeholder text for its label.WCAG 3.3.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
A live region (role=status / aria-live) is present to announce the number of results.WCAG 4.1.3Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
When typing updates the suggestions, the result count is announced via a live region without moving focus.WCAG 4.1.3Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Semantic markup11 criteria

The combobox's role is included in the accessibility tree.WCAG 4.1.2Automated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox has an accessible name in the accessibility tree.WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox's accessible name contains the text of its visible label.WCAG 2.5.3Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox's accessible name matches its visible label, or at least starts with the text of the visible label.Best practiceAutomated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox's expanded or collapsed state is included in the accessibility tree.WCAG 4.1.2Automated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox has an accessible description (the hint) included in the accessibility tree.WCAG 1.3.1Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The list of options has the listbox role in the accessibility tree.WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Each option has the option role and an accessible name in the accessibility tree.WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox is programmatically associated with its listbox (aria-controls / aria-owns).Best practiceAutomated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The option with visual focus is programmatically determinable via aria-activedescendant on the combobox.WCAG 1.3.1Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox indicates that suggestions are filtered by the typed text (aria-autocomplete="list").WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Keyboard13 criteria

The combobox is focusable using a keyboard.WCAG 2.1.1Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The option with visual focus has a focus style.WCAG 2.4.7Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
People can enter a value into the combobox using a keyboard.WCAG 2.1.1Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
People can open the listbox and choose an option using only a keyboard.WCAG 2.1.1Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Pressing the Down arrow opens the listbox and moves visual focus to the first option.Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The arrow keys move visual focus through the options while DOM focus stays on the input.Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
After navigating options, people can keep editing without losing the typed value (DOM focus stays on the input).Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
No suggestion receives visual focus or becomes selected until the person moves into the list with the arrow keys.Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Pressing Escape when the listbox is open closes it without changing the value.Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Pressing Tab to move away from the combobox closes the listbox.Best practiceBrowser
Skill off
3/3 failed
- Generation 1: Fail
- Generation 2: Fail
- Generation 3: Fail
Skill on
2/3 failed
- Generation 1: Fail
- Generation 2: Pass
- Generation 3: Fail
A separate open (arrow) button is not in the Tab order because it duplicates the combobox's keyboard interface.Best practiceBrowser
Skill off
3/3 failed
- Generation 1: Fail
- Generation 2: Fail
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox has a focus style when it receives focus using a keyboard.WCAG 2.4.7Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The keyboard focus style still renders in forced-colours mode (e.g. Windows High Contrast Mode).Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Visual design4 criteria

All text meets the minimum contrast ratio of 4.5:1 against the background colour, or 3:1 for large-scale text.WCAG 1.4.3Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The option focus style has a contrast ratio of at least 3:1 against the option's resting appearance.WCAG 1.4.11Browser
Skill off
3/3 failed
- Generation 1: Fail
- Generation 2: Fail
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
If the combobox uses a custom focus style, the focus indicator has a contrast ratio of at least 3:1 against its background colour.WCAG 1.4.11Browser
Skill off
2/3 failed
- Generation 1: Fail
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The combobox's visible boundary (its border, or its fill where there is no border) has a contrast ratio of at least 3:1 against the adjacent background colour.WCAG 1.4.11Browser
Skill off
2/3 failed
- Generation 1: Fail
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Adaptive UI1 criterion

The combobox can be viewed at smaller screen widths (320px) without loss of content or functionality, and without requiring horizontal scrolling.WCAG 1.4.10Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Notes on this run

Skill off

Generation 3 does include aria-expanded however it's not included in the accessibility tree because it's missing the combobox role.