Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/locator-reads-wrong-element.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'e2e': minor
'@e2e-dev/web': patch
---

**Breaking:** `inputValue()` on a node with no value, such as a heading or a button, and `isChecked()` on a node with no checked state throw `ACTION_FAILED`, as in Playwright, instead of returning `""` and `false`. A field with no value still reads `""`, and a checkable role with no state reported still reads `false`. `uncheck()` on an ARIA `role="radio"` now clicks it, as in Playwright; a native radio still refuses.
4 changes: 2 additions & 2 deletions apps/testbed/tests/check.e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ test.describe('page check', { requires: ['browser'], tags: ['playwright'] }, ()
expect(await browser.evaluate("() => window.checkbox.getAttribute('aria-checked')")).toBe('true');
});

test('should uncheck radio by aria role', { fail: '`uncheck` refuses a `role=radio`, where Playwright clicks it and refuses only a native radio' }, async ({ browser }) => {
test('should uncheck radio by aria role', async ({ browser }) => {
await browser.goto(content(`<div role='radio' id='checkbox' aria-checked="true">CHECKBOX</div>
<script>
checkbox.addEventListener('click', () => checkbox.setAttribute('aria-checked', 'false'));
Expand All @@ -79,7 +79,7 @@ test.describe('page check', { requires: ['browser'], tags: ['playwright'] }, ()
}
});

test('should uncheck the box by aria role', { fail: '`uncheck` refuses a `role=radio`, where Playwright clicks it and refuses only a native radio' }, async ({ browser }) => {
test('should uncheck the box by aria role', async ({ browser }) => {
for (const role of ['checkbox', 'menuitemcheckbox', 'option', 'radio', 'switch', 'menuitemradio', 'treeitem']) {
await browser.goto(content(`<div role='${role}' id='checkbox' aria-checked="true">CHECKBOX</div>
<script>
Expand Down
4 changes: 2 additions & 2 deletions apps/testbed/tests/locator-convenience.e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ test.describe('locator convenience', { requires: ['browser'], tags: ['playwright
expect(await locator.getAttribute('foo')).toBe(null);
});

test('inputValue should work', { fail: '`inputValue()` on an element that is no input reads "" instead of throwing' }, async ({ browser }) => {
test('inputValue should work', async ({ browser }) => {
await browser.goto('/dom.html');

await browser.locator('#select').selectOption({ value: 'foo' });
Expand Down Expand Up @@ -74,7 +74,7 @@ test.describe('locator convenience', { requires: ['browser'], tags: ['playwright
expect(await button2.isDisabled()).toBe(false);
});

test('isChecked should work', { fail: '`isChecked()` on an element that is no checkbox reads false instead of throwing' }, async ({ browser }) => {
test('isChecked should work', async ({ browser }) => {
await browser.goto(content(`<input type='checkbox' checked><div>Not a checkbox</div>`));
const element = browser.locator('input');
expect(await element.isChecked()).toBe(true);
Expand Down
15 changes: 7 additions & 8 deletions docs/migrate/playwright.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -169,16 +169,16 @@ uses it declares `requires: ['browser']` and other targets skip it. Calls on
| `pressSequentially(text, { delay })` | the same; keystrokes through the platform keyboard, a `Secret` is refused | same |
| `press(key)` | the same; modifiers join with `+`, see the [key grammar](/reference/engine#keys) | same |
| `clear()`, `focus()`, `hover()`, `setInputFiles()`, `dragTo()` | the same verbs, each with `timeout` as the only option: no `force`, `trial`, `position`, `modifiers`, or `sourcePosition`. Another key is `INVALID_ARGUMENT`. `setInputFiles` is web only, and `setInputFiles([])` clears the input | renamed |
| `check()`, `uncheck()` | the same, except when the state updates late or the click removes the control, see [What differs on purpose](#what-differs-on-purpose). `uncheck()` on a checked radio, `role=radio` included, is `NOT_ACTIONABLE` | by design |
| `check()`, `uncheck()` | the same, except when the state updates late or the click removes the control, see [What differs on purpose](#what-differs-on-purpose). `uncheck()` on a checked native radio is `ACTION_FAILED`; a `role=radio` is clicked, as in Playwright | by design |
| `selectOption('pro')` | `selectOption('Pro')`: a string matches the option's label only, where Playwright matches its value or its label; `selectOption({ value: 'pro' })` for the value | by design |
| `selectOption(['a', 'b'])` | one option per call. An array is `INVALID_ARGUMENT` | missing |
| `setInputFiles({ name, buffer })` | paths only, resolved from the project root | missing |
| `scrollIntoViewIfNeeded()` | `scrollIntoView()` | renamed |
| `locator.innerText()` | `textContent()`, whitespace collapsed to single spaces | renamed |
| `locator.textContent()` | the rendered text, what `innerText()` reads, see [What differs on purpose](#what-differs-on-purpose) | by design |
| `inputValue()` | the same on an input, textarea, or select; `""` on any other element, where Playwright throws | missing |
| `getAttribute()`, `isVisible()`, `isHidden()`, `isEnabled()`, `isDisabled()`, `boundingBox()` | the same | same |
| `isChecked()` | the same on a checkbox, radio, or `aria-checked` node; `false` on any other, see [What differs on purpose](#what-differs-on-purpose) | by design |
| `getAttribute()`, `isVisible()`, `isHidden()`, `isEnabled()`, `isDisabled()`, `isChecked()`, `boundingBox()` | the same | same |
| `inputValue()` | the same, and reads a `contenteditable` editor, where Playwright throws | by design |
| `inputValue()`, `isChecked()` on a `<label>` | throws, where Playwright reads its control. Locate the control with `getByLabel` | missing |
| `isEditable()` | not yet | missing |
| `waitFor({ state: 'visible' })`, `'hidden'`, `'attached'`, `'detached'` | the same | same |
| `locator.evaluate(fn)`, `evaluateAll(fn)` | `browser.evaluate(fn, selector)` with a CSS selector argument | missing |
Expand Down Expand Up @@ -414,10 +414,9 @@ context.
await expect(screen.getByTestId('total')).toHaveText('Total 42'); // Playwright: 'Total debug42'
await expect(screen.getByTestId('address')).toHaveText('Line one Line two'); // Playwright: 'Line oneLine two'
```
- A node with no checked state is unchecked. `isChecked()` is `false` and
`.not.toBeChecked()` passes on a button or a heading, where Playwright
throws. Query a checkable control by role, so the query fails when the
control changes kind.
- `.not.toBeChecked()` passes on a button or a heading, where Playwright
throws, as `isChecked()` does. Query a checkable control by role, so the
query fails when the control changes kind.

```ts
await expect(screen.getByRole('checkbox', 'Subscribe')).not.toBeChecked();
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/errors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ category a blocked verdict reports as the obstacle.
| `APP_NOT_OPEN` | An operation needed an open app: `app.open()` had not run, or the engine reported invalid state | test | 1 |
| `LOCATOR_NOT_FOUND` | No matching node within the deadline, or an agent action refers to a node that is no longer available. For a filtered locator the message can also list how many candidates each filter step left, counted after the failure, when that diagnostic pass can read the chain | test | 1 |
| `LOCATOR_AMBIGUOUS` | More than one match for a single-node operation. Fails at once, with no polling, and for a filtered locator can list how many candidates each filter step left, counted after the failure, when that diagnostic pass can read the chain | test | 1 |
| `ACTION_FAILED` | Actionability failure, operation timeout, or an action that may already have committed | test | 1 |
| `ACTION_FAILED` | Actionability failure, operation timeout, or an action that may already have committed. Also `inputValue()` on a node with no value and `isChecked()` on a node with no checked state | test | 1 |
| `STEP_BUDGET_EXHAUSTED` | The model-call or action-step budget ran out before a conclusion | test | 1 |
| `STEP_TIMEOUT` | The invocation deadline expired, including an operation that timed out within a poll tick of it. Carries the last judgment where one exists, with the [vision hint](/reference/agent#agent-assert) when that judgment was inconclusive | test | 1 |
| `CONTEXT_OVERFLOW` | A request exceeded the model's context window. `act` shortens its history and retries once if that makes the request smaller; judgments fail at once. Lower `agents.<name>.maxObservationBytes`, or split the instruction into shorter steps. Reported as blocked: automation | test | 1 |
Expand Down
16 changes: 14 additions & 2 deletions docs/reference/screen.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -771,6 +771,10 @@ uncheck(options?: ActionOptions): Promise<void>;
await screen.getByRole('checkbox', 'Remember me').uncheck();
```

| Throws | When |
| --- | --- |
| `ACTION_FAILED` | A checked native radio button, which only another radio in its group clears. |

#### `selectOption`

```ts
Expand Down Expand Up @@ -934,6 +938,10 @@ On a checkbox or radio, `inputValue()` and `toHaveValue` read its value
attribute, `on` when it has none, whatever its checked state. `isChecked()`
reads the state.

| Throws | When |
| --- | --- |
| `ACTION_FAILED` | A node with no value, such as a heading or a button. A field with no value reads `""`. |

#### `getAttribute`

```ts
Expand Down Expand Up @@ -1022,8 +1030,12 @@ isChecked(): Promise<boolean>;
const remembered = await screen.getByRole('checkbox', 'Remember me').isChecked();
```

`false` on a node with no checked state, such as a button or a heading. It
does not throw.
A checkable role with no state reported, such as an ARIA switch without
`aria-checked`, reads `false`.

| Throws | When |
| --- | --- |
| `ACTION_FAILED` | A node with no checked state, such as a button or a heading. |

#### `boundingBox`

Expand Down
22 changes: 21 additions & 1 deletion packages/e2e/src/internal/roles.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/** The role sets the runner reads a node against: what takes typed text, what carries a value. */
/** The role sets the runner reads a node against: what takes typed text, what carries a value, what has a checked state. */

import type { SemanticNode } from '../engine/surface.ts';

Expand All @@ -12,6 +12,17 @@ const EDITABLE_ROLES: ReadonlySet<string> = new Set(['textbox', 'searchbox', 'co
*/
const VALUE_ROLES: ReadonlySet<string> = new Set([...EDITABLE_ROLES, 'spinbutton', 'slider', 'listbox', 'option']);

/** Roles with a checked state, the ARIA roles that take `aria-checked`, as Playwright lists them. */
const CHECKED_ROLES: ReadonlySet<string> = new Set([
'checkbox',
'menuitemcheckbox',
'option',
'radio',
'switch',
'menuitemradio',
'treeitem',
]);

/** True for a node of an editable role. */
export function isEditable(node: Pick<SemanticNode, 'role'>): boolean {
return EDITABLE_ROLES.has(node.role ?? '');
Expand All @@ -25,3 +36,12 @@ export function isEditable(node: Pick<SemanticNode, 'role'>): boolean {
export function isValueControl(node: Pick<SemanticNode, 'role'>): boolean {
return VALUE_ROLES.has(node.role ?? '');
}

/**
* True for a node that has a checked state: one the platform reported, or a
* role that carries one, since a node of that role with no state reported is
* unchecked. A button or a heading has none.
*/
export function isCheckable(node: Pick<SemanticNode, 'role' | 'states'>): boolean {
return node.states?.checked !== undefined || CHECKED_ROLES.has(node.role ?? '');
}
6 changes: 5 additions & 1 deletion packages/e2e/src/locator/screen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { locatorBrand } from '../internal/brands.ts';
import { isSecret } from '../secrets.ts';
import { TestError } from '../internal/errors.ts';
import { requireFinitePoint } from '../internal/geometry.ts';
import { isCheckable, isValueControl } from '../internal/roles.ts';
import { isPlainObject, rejectUnknownOptions } from '../internal/options.ts';
import { realmSlot } from '../internal/realm-slot.ts';
import { obj } from '../internal/objects.ts';
Expand Down Expand Up @@ -561,7 +562,9 @@ class LocatorImpl extends ScreenImpl implements Locator {

async inputValue(): Promise<string> {
const node = await this.readGuarded();
return node.value ?? '';
if (node.value !== undefined) return node.value;
if (isValueControl(node)) return '';
throw new TestError('ACTION_FAILED', `Node is not an <input>, <textarea> or <select> element: ${this.label}`);
}

async getAttribute(name: string): Promise<string | null> {
Expand All @@ -588,6 +591,7 @@ class LocatorImpl extends ScreenImpl implements Locator {

async isChecked(): Promise<boolean> {
const node = await this.readGuarded(false);
if (!isCheckable(node)) throw new TestError('ACTION_FAILED', `Not a checkbox or radio button: ${this.label}`);
return node.states?.checked === true;
}

Expand Down
4 changes: 2 additions & 2 deletions packages/e2e/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,7 @@ export interface Locator extends Omit<Screen, 'tree' | 'debug'> {
swipe(options: SwipeOptions & ActionOptions): Promise<void>;
/** Reads normalized text once a match exists, without retrying for a particular value. */
textContent(): Promise<string | null>;
/** Reads the current input value. */
/** Reads the current input value; throws `ACTION_FAILED` on a node with no value, such as a heading. */
inputValue(): Promise<string>;
/** Reads one attribute. */
getAttribute(name: string): Promise<string | null>;
Expand All @@ -596,7 +596,7 @@ export interface Locator extends Omit<Screen, 'tree' | 'debug'> {
isEnabled(): Promise<boolean>;
/** Reads current disabled state, the negation of `isEnabled`. */
isDisabled(): Promise<boolean>;
/** Reads current checked state. */
/** Reads current checked state; throws `ACTION_FAILED` on a node with no checked state, such as a button. */
isChecked(): Promise<boolean>;
/** Reads the current viewport-relative rectangle. */
boundingBox(): Promise<{ x: number; y: number; width: number; height: number } | null>;
Expand Down
6 changes: 4 additions & 2 deletions packages/e2e/tests/integration/scripted-reads.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ test('reads answer from the current tree', async ({ app, screen }) => {
expect(await screen.getByRole('heading', { level: 1 }).textContent()).toBe('Dashboard');
expect(await screen.getByRole('image', { name: 'Map' }).textContent()).toBeNull();
expect(await screen.getByLabel('Search').inputValue()).toBe('hello-value');
expect(await screen.getByRole('heading', { level: 1 }).inputValue()).toBe('');
const noValue = await screen.getByRole('heading', { level: 1 }).inputValue().catch((error: unknown) => error);
expect(noValue).toMatchObject({ code: 'ACTION_FAILED' });
const noState = await screen.getByRole('button', { name: 'Submit' }).isChecked().catch((error: unknown) => error);
expect(noState).toMatchObject({ code: 'ACTION_FAILED' });
expect(await screen.getByTestId('card').getAttribute('class')).toBe('card active');
expect(await screen.getByTestId('card').getAttribute('missing')).toBeNull();
expect(await screen.getByRole('button', { name: 'Submit' }).isVisible()).toBe(true);
Expand Down Expand Up @@ -49,7 +52,6 @@ test('reads answer from the current tree', async ({ app, screen }) => {
test('secure fields still answer state reads', async ({ app, screen }) => {
await app.open('/');
expect(await screen.getByLabel('Password').isEnabled()).toBe(true);
expect(await screen.getByLabel('Password').isChecked()).toBe(false);
expect(await screen.getByLabel('Password').isVisible()).toBe(true);
expect(await screen.getByLabel('Password').boundingBox()).toEqual({ x: 10, y: 140, width: 200, height: 30 });
});
Expand Down
28 changes: 26 additions & 2 deletions packages/e2e/tests/unit/locator-basics.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,35 @@ describe('isHidden and isDisabled', () => {
});
});

describe('inputValue', () => {
it('reads the value, and a value role with none reported as empty', async () => {
const field: SemanticNode = { ref: { id: 'name', revision: '' }, role: 'textbox', value: 'Ada' };
const empty: SemanticNode = { ref: { id: 'empty', revision: '' }, role: 'searchbox' };
expect(await createScreenFixture([field]).getByRole('textbox').inputValue()).toBe('Ada');
expect(await createScreenFixture([empty]).getByRole('searchbox').inputValue()).toBe('');
});

it('fails on a node with no value, as Playwright does', async () => {
await expect(createScreenFixture([SHOWN]).getByRole('listitem').inputValue()).rejects.toMatchObject({
code: 'ACTION_FAILED',
message: expect.stringContaining('Node is not an <input>, <textarea> or <select> element'),
});
});
});

describe('isChecked and isEnabled', () => {
it('isChecked reads the checked state: true, false, and no state at all as false', async () => {
it('isChecked reads the checked state, and a checkable role with no state as unchecked', async () => {
expect(await createScreenFixture([item('on', 'On', { checked: true })]).getByRole('listitem').isChecked()).toBe(true);
expect(await createScreenFixture([item('off', 'Off', { checked: false })]).getByRole('listitem').isChecked()).toBe(false);
expect(await createScreenFixture([SHOWN]).getByRole('listitem').isChecked()).toBe(false);
const toggle: SemanticNode = { ref: { id: 'toggle', revision: '' }, role: 'switch', name: 'Wi-Fi' };
expect(await createScreenFixture([toggle]).getByRole('switch').isChecked()).toBe(false);
});

it('isChecked fails on a node with no checked state, as Playwright does', async () => {
await expect(createScreenFixture([SHOWN]).getByRole('listitem').isChecked()).rejects.toMatchObject({
code: 'ACTION_FAILED',
message: expect.stringContaining('Not a checkbox or radio button'),
});
});

it('isChecked reads a secure field, since a state is not a value', async () => {
Expand Down
21 changes: 14 additions & 7 deletions packages/web/src/actions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,16 @@ async function setChecked(target: ActionTarget, checked: boolean, operation: Con
const verb = checked ? 'check' : 'uncheck';
try {
if ((await element.isChecked()) === checked) return;
if (!checked && (await isRadio(element))) {
const radio = checked ? undefined : await radioKind(element);
if (radio === 'native') {
throw new EngineError('NOT_ACTIONABLE', 'uncheck cannot clear a radio button; select another radio in its group', {
retryable: false,
});
}
await element.click({ timeout: remaining() });
const hint = radio === 'aria' ? '; a radio usually clears only when another radio in its group is selected' : '';
const unchanged = (detail = ''): EngineError =>
new EngineError('NOT_ACTIONABLE', `${verb} clicked the control but its checked state did not change${detail}`, {
new EngineError('NOT_ACTIONABLE', `${verb} clicked the control but its checked state did not change${detail}${hint}`, {
retryable: false,
});
let firstRead = true;
Expand All @@ -89,11 +91,16 @@ async function setChecked(target: ActionTarget, checked: boolean, operation: Con
}
}

/** Whether a checkable element is a radio, which a click can select but never clear. */
function isRadio(element: ElementHandle<Element>): Promise<boolean> {
return element.evaluate(
(node) => (node instanceof HTMLInputElement && node.type === 'radio') || node.getAttribute('role') === 'radio',
);
/**
* Which kind of radio a checkable element is, if any. A native radio can be
* selected but never cleared by a click. An ARIA radio is clicked like any
* control, as Playwright does: its script decides what a click does.
*/
function radioKind(element: ElementHandle<Element>): Promise<'native' | 'aria' | undefined> {
return element.evaluate((node) => {
if (node instanceof HTMLInputElement && node.type === 'radio') return 'native';
return node.getAttribute('role') === 'radio' ? 'aria' : undefined;
});
}

/**
Expand Down
16 changes: 16 additions & 0 deletions packages/web/tests/integration/check-actions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,22 @@ describe.each(KINDS)('through the %s target', (kind) => {
expect(await clicks()).toBe(0);
});

it('unchecks an ARIA radio by clicking it, leaving the radio behavior to its script', async () => {
await page.setContent(`
<div role="radio" id="pro" aria-checked="true" tabindex="0"
onclick="this.setAttribute('aria-checked', 'false')">Pro</div>`);
expect(await perform(await targetOf(kind, '#pro'), { kind: 'uncheck' })).toBeUndefined();
expect(await page.getAttribute('#pro', 'aria-checked')).toBe('false');
});

it('fails an ARIA radio whose click keeps it checked, naming the radio group', async () => {
await page.setContent('<div role="radio" id="pro" aria-checked="true" tabindex="0">Pro</div>');
expect(await perform(await targetOf(kind, '#pro'), { kind: 'uncheck' })).toMatchObject({
code: 'NOT_ACTIONABLE',
message: expect.stringContaining('another radio in its group'),
});
});

it('refuses a control that cannot be checked', async () => {
await page.setContent('<button id="go">Go</button>');
expect(await perform(await targetOf(kind, '#go'), { kind: 'check' })).toMatchObject({ code: 'NOT_ACTIONABLE' });
Expand Down
Loading
Loading