Skip to main content

Test results

Pick a model to see how accessible its components are, with the accessibility skill off and on.

GPT-6 Astra

Skill version 2.3

Totals for the whole run, across 3 generations of each of the 22 components
MeasureSkill offSkill on
Failed acceptance criteria410
of which WCAG200
of which Best practice210
Total cost (US dollars)$8.02$32.68
Input tokens169,3797.6 million
Output tokens129,025256,032

Table with irregular headers

Results, totals across all generations

Cost in USD

Failed acceptance criteria

Skill off
1 fail
WCAG
1 fail
Best practice
0 fails
Skill on
0 fails
WCAG
0 fails
Best practice
0 fails

Total cost

Skill off
$0.31
Skill on
$1.13

Efficiency, average per generation

Input tokens include cache reads.

Input tokens

Skill off
2,058 tokens
Skill on
74,166 tokens

Output tokens

Skill off
1,626 tokens
Skill on
3,273 tokens

Turns

Skill off
2 turns
Skill on
7 turns

Time

Skill off
27 seconds
Skill on
61 seconds

Cost

Skill off
$0.10
Skill on
$0.38
Prompt given to the model
Build a data table in a single self-contained HTML file (inline CSS and JS, no build step, no external libraries). Give the table the title "Quarterly units sold by product line, 2024". Directly above or with the table, include this introductory sentence: "Figures show units sold, grouped into the first and second half of the year, with one row per product line." The table has a first column headed "Product line", then two groups of columns: a group headed "First half" covering two columns "Q1" and "Q2", and a group headed "Second half" covering two columns "Q3" and "Q4". So the top of the table reads: Product line, then First half spanning Q1 and Q2, then Second half spanning Q3 and Q4. The rows, in this order, are: - Notebooks: 120, 135, 150, 160 - Pens: 300, 280, 310, 290 - Folders: 90, 110, 95, 130 (The four numbers in each row are Q1, Q2, Q3, Q4 in that order.) Output the complete code in one HTML code block.

Generated code

Skill off
Skill on
Show

Showing 10 of 10 criteria

Labels and messaging1 criterion

  • None of the table header cells are blank.Best practiceAutomated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Semantic markup6 criteria

  • The table's semantics (table, row and cell roles) are included in the accessibility tree.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The caption is correctly associated with the table in the accessibility tree, with the table being labelled by the caption element.WCAG 1.3.1Automated
    Skill off

    1/3 failed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Fail
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The summary is programmatically associated with the table.WCAG 1.3.1Judgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The column headers are programmatically associated with the data cells they describe.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • For tables with irregular headers, the header cells are programmatically associated with the data cells they describe.WCAG 1.3.1Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • The irregular header cells are associated with the correct data cells for the table's layout (not merely valid references).WCAG 1.3.1Judgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Visual design2 criteria

  • The table uses styling to differentiate the header cells from the data cells.Best practiceJudgement
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
  • All text meets the minimum contrast ratio of 4.5:1 against the background colours, or 3:1 for large-scale text.WCAG 1.4.3Automated
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Adaptive UI1 criterion

  • The table can be viewed at 320px without loss of content or functionality; the table is allowed to scroll horizontally within its own container.WCAG 1.4.10Browser
    Skill off

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass
    Skill on

    3/3 passed

    • Generation 1: Pass
    • Generation 2: Pass
    • Generation 3: Pass

Back to top