diff --git a/packages/ruleset/README.md b/packages/ruleset/README.md index 29cde70..d2470c0 100644 --- a/packages/ruleset/README.md +++ b/packages/ruleset/README.md @@ -22,6 +22,10 @@ ruleset.test({ url: "https://www.example.com/" }); // true ruleset.test({ url: "https://www.example.net/" }); // false ``` +## Documentation + +See the [ruleset specification](https://github.com/ublacklist/packages/blob/main/packages/ruleset/docs/spec.md) for the full syntax. + ## License [MIT](https://github.com/ublacklist/packages/blob/main/LICENSE) diff --git a/packages/ruleset/docs/spec.md b/packages/ruleset/docs/spec.md new file mode 100644 index 0000000..2e27d5b --- /dev/null +++ b/packages/ruleset/docs/spec.md @@ -0,0 +1,155 @@ +# Ruleset Specification + +Reference for the uBlacklist ruleset syntax. + +## Overview + +A ruleset is a newline-separated list of rules. Each rule is one of: + +- A **match pattern** (optionally with an `@if(...)` guard) +- An **expression** (using variables, string matchers, regex, and logical operators) +- A **regular expression literal** (shorthand for `url =~ /.../`) +- An **unblock rule** (prefixed with `@`) +- A **highlight rule** (prefixed with `@N` where N = 1, 2, 3, ...) +- A **comment** (starting with `#`) + +Empty lines and unparseable lines are effectively ignored, but explicit `#` comments are preferred for forward compatibility. + +## Match patterns + +Match patterns are URLs containing wildcards, following the [WebExtensions match pattern](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Match_patterns) specification. + +| Pattern | Matches | +| ------------------------ | -------------------------------------------------- | +| `*://example.com/*` | URLs hosted at `example.com` | +| `*://*.example.net/*` | `example.net` and any subdomain | +| `*://example.org/hoge/*` | `example.org` URLs whose path starts with `/hoge/` | + +**Invalid:** `*://www.qinterest.*/` — `*` may not appear in the middle of the host. + +### Match patterns with guards + +Append `@if()` to scope a match pattern to a condition: + +``` +*://*.amazon.com/* @if($category="images") +``` + +## Expressions + +Expressions evaluate against the current search result. + +### Variables + +Built-in variables for the result URL and title: + +- `url` — full URL +- `title` — result title +- `scheme`, `host`, `path` — URL parts + +Built-in SERP-context variables (prefixed with `$`): + +- `$site` — search engine identifier (e.g. `"google"`, `"bing"`) +- `$category` — result category (e.g. `"web"`, `"images"`) + +``` +url *= "example" # URL contains "example" +title ^= "Something" # Title starts with "Something" +scheme = "http" # HTTP results +host $= ".example.com" # Host ends with ".example.com" +path *= "blah" i # Path contains "blah", case-insensitive +$site = "google" & host = "youtube.com" +$category = "images" & host = "www.amazon.com" +``` + +### String matchers + +Modeled after CSS attribute selectors: + +| Operator | Meaning | +| -------- | -------------- | +| `=` | exactly equals | +| `^=` | starts with | +| `$=` | ends with | +| `*=` | contains | + +Append the `i` flag for case-insensitive matching: + +``` +title $= "domain" i +``` + +### Regular expressions + +Use JavaScript regex **literals** (must be surrounded by `/`): + +``` +url =~ /example\.(net|org)/ # explicit +url /example\.(net|org)/ # "=~" omitted +/example\.(net|org)/ # "url =~" omitted +title =~ /example domain/i # with flags +``` + +**Invalid:** + +- `^https?:\/\/example\.com\/` — not surrounded by `/` +- `/^https?://example\.com//` — inner `/` must be escaped + +### Logical operators + +| Operator | Meaning | +| -------- | ------- | +| `!` | not | +| `&` | and | +| `\|` | or | + +``` +!scheme = "https" +$category = "images" & host = "www.amazon.com" +title *= "example" i | title *= "domain" i +``` + +## Unblock rules + +A match pattern or regex prefixed with `@` _unblocks_ the matching results. Mainly used to override entries pulled in by a subscription. + +``` +@*://example.com/* +@/example\.(net|org)/ +``` + +## Highlight rules + +A match pattern or regex prefixed with `@N` (N = 1, 2, 3, ...) highlights matching results in color slot N. Color slots are configured in the options page; only `@1` exists by default. + +``` +@1*://example.com/* +@2/example\.(net|org)/ +``` + +## Comments + +Lines starting with `#` are comments. Trailing `#` comments after a rule are also allowed: + +``` +# Block example.com and its subdomains +*://*.example.com/* + +/example\.(net|org)/ # Block example.net or example.org +``` + +Although any unparseable line is effectively ignored, prefer explicit `#` comments: + +1. They are guaranteed to remain comments if new syntax is added later. +2. They can be placed inline after a rule. + +## Subscription frontmatter + +A ruleset published as a subscription may begin with a YAML frontmatter block. The `name` field is recommended: + +``` +--- +name: Your ruleset name +--- +*://*.example.com/* +``` diff --git a/packages/serpinfo/README.md b/packages/serpinfo/README.md index d881a6c..0fe8d4c 100644 --- a/packages/serpinfo/README.md +++ b/packages/serpinfo/README.md @@ -29,6 +29,10 @@ if (result.success) { } ``` +## Documentation + +See the [SERPINFO specification](https://github.com/ublacklist/packages/blob/main/packages/serpinfo/docs/spec.md) for the full format. + ## License [MIT](https://github.com/ublacklist/packages/blob/main/LICENSE) diff --git a/packages/serpinfo/docs/spec.md b/packages/serpinfo/docs/spec.md new file mode 100644 index 0000000..ed1995e --- /dev/null +++ b/packages/serpinfo/docs/spec.md @@ -0,0 +1,165 @@ +# SERPINFO Specification + +Reference for the SERPINFO YAML format used to describe how uBlacklist parses search engine result pages (SERPs). + +## File format + +A SERPINFO file is a YAML document with this shape: + +```yaml +name: SearchEngineName +version: 1.0.0 +homepage: https://example.com/ +lastModified: 2023-09-01T00:00:00Z + +pages: + - name: PageType + matches: + - https://search.example.com/search?* + results: + - root: .result-selector + url: a.link + props: + title: .title + commonProps: + $site: example + $category: web +``` + +## Top-level properties + +| Required | Property | Type | Description | +| :------: | -------------- | ------ | -------------------------------- | +| ✓ | `name` | string | Name of the SERPINFO | +| ✓ | `pages` | array | Array of page definitions | +| | `version` | string | Version of the SERPINFO | +| | `description` | string | Description | +| | `homepage` | string | Homepage URL | +| | `lastModified` | string | Last modification time, ISO 8601 | + +## Page definition + +Each entry in `pages` describes a category of SERP (web, images, news, ...). + +| Required | Property | Type | Description | +| :------: | ---------------- | ----------------- | ---------------------------------------------------------------------------- | +| ✓ | `name` | string | Name of the page definition | +| ✓ | `matches` | array | Match patterns to include pages | +| ✓ | `results` | array | Result definitions | +| | `excludeMatches` | array | Match patterns to exclude pages | +| | `includeRegex` | string | Regex to include pages | +| | `excludeRegex` | string | Regex to exclude pages | +| | `userAgent` | string | `"any"`, `"desktop"`, or `"mobile"` | +| | `commonProps` | object | Properties merged into every result on this page (e.g. `$site`, `$category`) | +| | `delay` | number or boolean | ms delay after page load, or boolean to enable/disable | + +## Result definition + +Each entry in `results` describes how to extract a single result from a page. + +| Required | Property | Type | Description | +| :------: | --------------- | ---------------- | ------------------------------------------------------------------------------ | +| ✓ | `root` | root command | Locates result root elements | +| ✓ | `url` | property command | Extracts the result URL | +| | `name` | string | Name of the result definition | +| | `props` | object | Map of property name → property command (e.g. `title`) | +| | `button` | button command | Adds a block button to each result | +| | `preserveSpace` | boolean | Keep blocked result's layout space to avoid layout shift | +| | `extraSelector` | string | CSS selector list for extra elements to hide or color together with the result | + +In `extraSelector`, `&` refers to the result root element, and every selector in the list must contain `&` (e.g. `"& + tr, & + tr + tr"`). + +## Commands + +Commands are written either as a bare CSS selector string or as a YAML array whose first element is the command name. Commands fall into four categories. + +### Root commands + +Locate the set of result root elements within the page. + +- `` — shorthand for `[selector, ]` +- `[selector, ]` — find all elements matching the selector +- `[upward, , ]` — for each element found by the inner root command, walk `` ancestors up the DOM + +### Element commands + +Resolve a single element relative to the current root element. + +- `` — shorthand for `[selector, ]` +- `[selector, , ?]` — query within the optional inner element (defaults to the current root) +- `[upward, , ?]` — walk `` ancestors up + +When the trailing `` is omitted, the current root element is used. + +### Property commands + +Produce a string value (or filter/transform an existing one). Used for `url` and for each entry in `props`. + +- `` — shorthand: `textContent` of the matched element, or `href` when used as the `url` extractor +- `[attribute, , ?]` — element attribute value +- `[property, , ?]` — JS property value +- `[const, ]` — constant string +- `[domainToURL, ]` — convert a bare domain into a URL +- `[regexInclude, , ]` — keep only if it matches +- `[regexExclude, , ]` — drop if it matches +- `[regexSubstitute, , , ]` — regex replace +- `[or, [*], ?]` — try commands in order, take the first that yields a value + +When the trailing `` is omitted, the current root element is used. + +### Button commands + +Add a block button to each matched result. + +- `[icon, ?, ?]` — icon button + - `style`: inline CSS (e.g. `"top: 16px; right: 32px; --ub-icon-size: 16px;"`) +- `[text, ?, ?]` — text button + - `position`: `"beforebegin"` | `"afterbegin"` | `"beforeend"` (default) | `"afterend"` + - `style`: inline CSS +- `[inset, ?, ?]` — absolutely-positioned button inside the target element + - `top`, `right`, `bottom`, `left`: CSS length or percentage + - `zIndex`: z-index (default `1`) + +When the trailing `` is omitted, the current root element is used. + +## Interaction with the rule language + +`commonProps` (and per-result `props`) populate the variables that ruleset expressions can match against. Variables prefixed with `$` (e.g. `$site`, `$category`) are conventionally set from `commonProps` so users can write rules like: + +``` +$site = "google" & host = "youtube.com" +$category = "images" & host = "www.amazon.com" +``` + +See the [ruleset specification](../../ruleset/docs/spec.md) for the rule language. + +## Example + +Bing image search: + +```yaml +name: Bing +version: 0.1.0 +homepage: https://github.com/ublacklist/builtin#readme +license: MIT +lastModified: 2023-04-05T11:11:20Z + +pages: + - name: Images (desktop) + matches: + - https://www.bing.com/images/search?* + userAgent: desktop + results: + - root: [upward, 1, ".iuscp"] + url: + - regexSubstitute + - '"purl":"([^"]+)' + - "\\1" + - [attribute, "m", ".iusc"] + props: + title: [attribute, "title", "li > a"] + button: [inset, { top: "32px", right: 0 }, ".iuscp"] + commonProps: + $site: bing + $category: images +```