Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

Grok 4.7

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
Measure	Skill off	Skill on
Failed acceptance criteria	90	9
of which WCAG	60	8
of which Best practice	30	1
Total cost (US dollars)	$7.93	$51.79
Input tokens	2.6 million	35.3 million
Output tokens	805,511	3.1 million

Carousel

Results, totals across all generations

Cost in USD

Failed acceptance criteria

Skill off

17 fails

WCAG: 5 fails
Best practice: 12 fails

Skill on

0 fails

WCAG: 0 fails
Best practice: 0 fails

Total cost

Skill off: $0.46
Skill on: $4.37

Efficiency, average per generation

Input tokens include cache reads.

Input tokens

Skill off: 42,131 tokens
Skill on: 1.1 million tokens

Output tokens

Skill off: 14,555 tokens
Skill on: 85,041 tokens

Turns

Skill off: 4 turns
Skill on: 17 turns

Time

Skill off: 169 seconds
Skill on: 1,133 seconds

Cost

Skill off: $0.15
Skill on: $1.46

Prompt given to the model

Build a "Featured offers" carousel in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). Show one promotional slide at a time from a set of three. Include a left-pointing arrow icon button to go to the previous slide, a right-pointing arrow icon button to go to the next slide, and a row of three small round icon buttons below the slide — one per slide — to jump directly to any of them. The first slide is shown when the page first loads. Use this exact content for the three slides, in this order: 1. Heading: "Summer sale". Body: "Save 20% on all outdoor furniture until the end of the month." Link: "Learn more" (href="#"). 2. Heading: "New arrivals". Body: "Browse this week's new homeware collection." Link: "Learn more" (href="#"). 3. Heading: "Free shipping". Body: "Free shipping on orders over $50, this week only." Link: "Learn more" (href="#"). Output the complete code in one HTML code block.

Generated code

Skill off: Generation 1, skill off (opens in a new tab)
Generation 2, skill off (opens in a new tab)
Generation 3, skill off (opens in a new tab)
Skill on: Generation 1, skill on (opens in a new tab)
Generation 2, skill on (opens in a new tab)
Generation 3, skill on (opens in a new tab)

Showing 28 of 28 criteria

Labels and messaging1 criterion

Each slide's "Learn more" link has a visible label.WCAG 3.3.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Semantic markup14 criteria

A carousel control's role of button is included in the accessibility tree.WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Every carousel control's accessible name is included in the accessibility tree.WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The "Learn more" link's accessible name contains the exact text of its visible label.WCAG 2.5.3Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The "Learn more" link's accessible name matches its visible label exactly, or at least starts with the exact text of the visible label.Best practiceAutomated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The carousel container has the region role in the accessibility tree.Best practiceAutomated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Fail
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The carousel container has the carousel role description via aria-roledescription="carousel" in the accessibility tree.Best practiceAutomated
Skill off
2/3 failed
- Generation 1: Fail
- Generation 2: Fail
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The carousel container has an accessible name in the accessibility tree.Best practiceAutomated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Each slide has the group role in the accessibility tree.Best practiceAutomated
Skill off
2/3 failed
- Generation 1: Fail
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Each slide has the slide role description via aria-roledescription="slide" in the accessibility tree.Best practiceAutomated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Each slide has an accessible name in the accessibility tree.Best practiceAutomated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The reading order of the slides in the accessibility tree matches the logical visual order.WCAG 1.3.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The slide-picker button for the current slide exposes its current state programmatically (aria-current, or aria-disabled in the APG grouped pattern).WCAG 4.1.2Automated
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
When the slide changes, assistive technology announces the newly displayed slide.Best practiceBrowser
Skill off
2/3 failed
- Generation 1: Pass
- Generation 2: Fail
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
If there is a visual pagination indicator that shows the updated position in the carousel, then assistive technology announces the new position.WCAG 4.1.3Judgement
Skill off
2/3 failed
- Generation 1: Pass
- Generation 2: Fail
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Keyboard6 criteria

The carousel controls are focusable using a keyboard.WCAG 2.1.1Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
A carousel control is activated by pressing the Space or Enter keys on the keyboard.Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
A carousel control has a focus style when it receives focus using a keyboard.WCAG 2.4.7Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The keyboard focus style still renders in forced-colours mode (e.g. Windows High Contrast Mode).Best practiceBrowser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
When a control is activated and the slide changes, focus remains on the activated control so the user can continue navigating.WCAG 2.4.3Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
All carousel controls (Previous, Next, and each slide-picker button) appear in the Tab order before the slide content.Best practiceBrowser
Skill off
3/3 failed
- Generation 1: Fail
- Generation 2: Fail
- Generation 3: Fail
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Visual design6 criteria

All text meets the minimum contrast ratio of 4.5:1 against the background colour, or 3:1 for large-scale text.WCAG 1.4.3Automated
Skill off
1/3 passed
- Generation 1: Pass
- Generation 2: Not applicable
- Generation 3: Not applicable
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
The slide-picker button for the current slide does not rely on colour alone to indicate the current state.WCAG 1.4.1Automated
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Fail
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Each control has a target size of at least 24 by 24 CSS pixels, or has sufficient spacing from adjacent targets.WCAG 2.5.8Automated
Skill off
1/3 failed
- Generation 1: Fail
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
If a carousel control uses a custom focus style, the focus indicator has a contrast ratio of at least 3:1 against its background colour.WCAG 1.4.11Browser
Skill off
1/3 failed
- Generation 1: Pass
- Generation 2: Fail
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Visual information needed to identify the Previous and Next buttons has a contrast ratio of at least 3:1 against adjacent colours.WCAG 1.4.11Automated
Skill off
0/3 passed
- Generation 1: Unmeasured
- Generation 2: Unmeasured
- Generation 3: Unmeasured
Skill on
0/3 passed
- Generation 1: Unmeasured
- Generation 2: Unmeasured
- Generation 3: Unmeasured
Visual information needed to identify each slide-picker button, in its default state and its current state, has a contrast ratio of at least 3:1 against adjacent colours.WCAG 1.4.11Automated
Skill off
0/3 passed
- Generation 1: Unmeasured
- Generation 2: Unmeasured
- Generation 3: Unmeasured
Skill on
0/3 passed
- Generation 1: Unmeasured
- Generation 2: Unmeasured
- Generation 3: Unmeasured

Adaptive UI1 criterion

The carousel can be viewed at smaller screen widths (320px) without loss of content or functionality, and without requiring horizontal scrolling.WCAG 1.4.10Browser
Skill off
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass
Skill on
3/3 passed
- Generation 1: Pass
- Generation 2: Pass
- Generation 3: Pass

Notes on this run

Skill off

In Generation 1 aria-roledescription="carousel" is not on the <section> element.