Skip to main content

Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

GPT-6 Astra

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
MeasureSkill offSkill on
Failed acceptance criteria410
of which WCAG200
of which Best practice210
Total cost (US dollars)$8.02$32.68
Input tokens169,3797.6 million
Output tokens129,025256,032

Modal dialog

Results, totals across all generations

Cost in USD

Failed acceptance criteria

Skill off
3 fails
WCAG
0 fails
Best practice
3 fails
Skill on
0 fails
WCAG
0 fails
Best practice
0 fails

Total cost

Skill off
$0.30
Skill on
$1.44

Efficiency, average per generation

Input tokens include cache reads.

Input tokens

Skill off
2,704 tokens
Skill on
107,773 tokens

Output tokens

Skill off
1,456 tokens
Skill on
3,633 tokens

Turns

Skill off
2 turns
Skill on
8 turns

Time

Skill off
23 seconds
Skill on
72 seconds

Cost

Skill off
$0.10
Skill on
$0.48
Prompt given to the model
Build a page with a delete-confirmation dialog in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). The page shows a short paragraph "Your monthly report is ready to download." and a button labelled "Delete file". Choosing "Delete file" opens a dialog window over the page. The dialog has the heading "Delete this file?", one sentence of body text "This will permanently remove the file from your account.", and two buttons: "Cancel" and "Delete". Both buttons dismiss the dialog (no actual deletion needs to happen). The dialog is closed when the page first loads, and while it is open the rest of the page cannot be used until it is dismissed. Output the complete code in one HTML code block.

Generated code

Skill off
Skill on
Show

Showing 16 of 16 criteria

Labels and messaging1 criterion

  • The modal dialog has a clear, descriptive heading that identifies its purpose.WCAG 2.4.6Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Semantic markup3 criteria

  • The modal dialog container element has role="dialog" in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The modal dialog has an accessible name in the accessibility tree.WCAG 4.1.2Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Content outside the active modal dialog is programmatically inert.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Keyboard8 criteria

  • When the modal dialog opens, keyboard focus is moved to the modal.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the modal dialog opens, initial focus is sent to the modal heading.Best practiceBrowser
    Skill off

    3/3 failed

    • Generation 1: Fail
    • Generation 2: Fail
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • While the modal dialog is open, focus is trapped inside of the modal container and does not move to elements in the background.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • All interactive elements within the modal dialog have a focus style when they receive focus using a keyboard.WCAG 2.4.7Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The keyboard focus style on the dialog's controls still renders in forced-colours mode (e.g. Windows High Contrast Mode).Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The modal dialog can be closed with a keyboard, such as using a close or cancel button.WCAG 2.1.1Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Pressing the Escape key closes the modal dialog.Best practiceBrowser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • When the modal dialog is closed, focus is returned to a logical position, such as the button that opened it.WCAG 2.4.3Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Visual design3 criteria

  • Text within the modal dialog meets the minimum contrast ratio of 4.5:1 against its background, or 3:1 for large-scale text.WCAG 1.4.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • If interactive elements within the modal dialog use custom focus styles, these styles have a contrast ratio of 3:1 against the background colour.WCAG 1.4.11Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • Content outside an active modal dialog is dimmed or blurred to indicate its inert state.Best practiceJudgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Adaptive UI1 criterion

  • The modal content can be viewed at smaller screen widths (320px) without loss of content or functionality, and without requiring horizontal scrolling.WCAG 1.4.10Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Back to top