Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/ruleset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ ruleset.test({ url: "https://www.example.com/" }); // true
ruleset.test({ url: "https://www.example.net/" }); // false
```

## Documentation

See the [ruleset specification](https://github.com/ublacklist/packages/blob/main/packages/ruleset/docs/spec.md) for the full syntax.

## License

[MIT](https://github.com/ublacklist/packages/blob/main/LICENSE)
155 changes: 155 additions & 0 deletions packages/ruleset/docs/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Ruleset Specification

Reference for the uBlacklist ruleset syntax.

## Overview

A ruleset is a newline-separated list of rules. Each rule is one of:

- A **match pattern** (optionally with an `@if(...)` guard)
- An **expression** (using variables, string matchers, regex, and logical operators)
- A **regular expression literal** (shorthand for `url =~ /.../`)
- An **unblock rule** (prefixed with `@`)
- A **highlight rule** (prefixed with `@N` where N = 1, 2, 3, ...)
- A **comment** (starting with `#`)

Empty lines and unparseable lines are effectively ignored, but explicit `#` comments are preferred for forward compatibility.

## Match patterns

Match patterns are URLs containing wildcards, following the [WebExtensions match pattern](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Match_patterns) specification.

| Pattern | Matches |
| ------------------------ | -------------------------------------------------- |
| `*://example.com/*` | URLs hosted at `example.com` |
| `*://*.example.net/*` | `example.net` and any subdomain |
| `*://example.org/hoge/*` | `example.org` URLs whose path starts with `/hoge/` |

**Invalid:** `*://www.qinterest.*/` — `*` may not appear in the middle of the host.

### Match patterns with guards

Append `@if(<expression>)` to scope a match pattern to a condition:

```
*://*.amazon.com/* @if($category="images")
```

## Expressions

Expressions evaluate against the current search result.

### Variables

Built-in variables for the result URL and title:

- `url` — full URL
- `title` — result title
- `scheme`, `host`, `path` — URL parts

Built-in SERP-context variables (prefixed with `$`):

- `$site` — search engine identifier (e.g. `"google"`, `"bing"`)
- `$category` — result category (e.g. `"web"`, `"images"`)

```
url *= "example" # URL contains "example"
title ^= "Something" # Title starts with "Something"
scheme = "http" # HTTP results
host $= ".example.com" # Host ends with ".example.com"
path *= "blah" i # Path contains "blah", case-insensitive
$site = "google" & host = "youtube.com"
$category = "images" & host = "www.amazon.com"
```

### String matchers

Modeled after CSS attribute selectors:

| Operator | Meaning |
| -------- | -------------- |
| `=` | exactly equals |
| `^=` | starts with |
| `$=` | ends with |
| `*=` | contains |

Append the `i` flag for case-insensitive matching:

```
title $= "domain" i
```

### Regular expressions

Use JavaScript regex **literals** (must be surrounded by `/`):

```
url =~ /example\.(net|org)/ # explicit
url /example\.(net|org)/ # "=~" omitted
/example\.(net|org)/ # "url =~" omitted
title =~ /example domain/i # with flags
```

**Invalid:**

- `^https?:\/\/example\.com\/` — not surrounded by `/`
- `/^https?://example\.com//` — inner `/` must be escaped

### Logical operators

| Operator | Meaning |
| -------- | ------- |
| `!` | not |
| `&` | and |
| `\|` | or |

```
!scheme = "https"
$category = "images" & host = "www.amazon.com"
title *= "example" i | title *= "domain" i
```

## Unblock rules

A match pattern or regex prefixed with `@` _unblocks_ the matching results. Mainly used to override entries pulled in by a subscription.

```
@*://example.com/*
@/example\.(net|org)/
```

## Highlight rules

A match pattern or regex prefixed with `@N` (N = 1, 2, 3, ...) highlights matching results in color slot N. Color slots are configured in the options page; only `@1` exists by default.

```
@1*://example.com/*
@2/example\.(net|org)/
```

## Comments

Lines starting with `#` are comments. Trailing `#` comments after a rule are also allowed:

```
# Block example.com and its subdomains
*://*.example.com/*

/example\.(net|org)/ # Block example.net or example.org
```

Although any unparseable line is effectively ignored, prefer explicit `#` comments:

1. They are guaranteed to remain comments if new syntax is added later.
2. They can be placed inline after a rule.

## Subscription frontmatter

A ruleset published as a subscription may begin with a YAML frontmatter block. The `name` field is recommended:

```
---
name: Your ruleset name
---
*://*.example.com/*
```
4 changes: 4 additions & 0 deletions packages/serpinfo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ if (result.success) {
}
```

## Documentation

See the [SERPINFO specification](https://github.com/ublacklist/packages/blob/main/packages/serpinfo/docs/spec.md) for the full format.

## License

[MIT](https://github.com/ublacklist/packages/blob/main/LICENSE)
165 changes: 165 additions & 0 deletions packages/serpinfo/docs/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# SERPINFO Specification

Reference for the SERPINFO YAML format used to describe how uBlacklist parses search engine result pages (SERPs).

## File format

A SERPINFO file is a YAML document with this shape:

```yaml
name: SearchEngineName
version: 1.0.0
homepage: https://example.com/
lastModified: 2023-09-01T00:00:00Z

pages:
- name: PageType
matches:
- https://search.example.com/search?*
results:
- root: .result-selector
url: a.link
props:
title: .title
commonProps:
$site: example
$category: web
```

## Top-level properties

| Required | Property | Type | Description |
| :------: | -------------- | ------ | -------------------------------- |
| ✓ | `name` | string | Name of the SERPINFO |
| ✓ | `pages` | array | Array of page definitions |
| | `version` | string | Version of the SERPINFO |
| | `description` | string | Description |
| | `homepage` | string | Homepage URL |
| | `lastModified` | string | Last modification time, ISO 8601 |

## Page definition

Each entry in `pages` describes a category of SERP (web, images, news, ...).

| Required | Property | Type | Description |
| :------: | ---------------- | ----------------- | ---------------------------------------------------------------------------- |
| ✓ | `name` | string | Name of the page definition |
| ✓ | `matches` | array | Match patterns to include pages |
| ✓ | `results` | array | Result definitions |
| | `excludeMatches` | array | Match patterns to exclude pages |
| | `includeRegex` | string | Regex to include pages |
| | `excludeRegex` | string | Regex to exclude pages |
| | `userAgent` | string | `"any"`, `"desktop"`, or `"mobile"` |
| | `commonProps` | object | Properties merged into every result on this page (e.g. `$site`, `$category`) |
| | `delay` | number or boolean | ms delay after page load, or boolean to enable/disable |

## Result definition

Each entry in `results` describes how to extract a single result from a page.

| Required | Property | Type | Description |
| :------: | --------------- | ---------------- | ------------------------------------------------------------------------------ |
| ✓ | `root` | root command | Locates result root elements |
| ✓ | `url` | property command | Extracts the result URL |
| | `name` | string | Name of the result definition |
| | `props` | object | Map of property name → property command (e.g. `title`) |
| | `button` | button command | Adds a block button to each result |
| | `preserveSpace` | boolean | Keep blocked result's layout space to avoid layout shift |
| | `extraSelector` | string | CSS selector list for extra elements to hide or color together with the result |

In `extraSelector`, `&` refers to the result root element, and every selector in the list must contain `&` (e.g. `"& + tr, & + tr + tr"`).

## Commands

Commands are written either as a bare CSS selector string or as a YAML array whose first element is the command name. Commands fall into four categories.

### Root commands

Locate the set of result root elements within the page.

- `<css-selector>` — shorthand for `[selector, <css-selector>]`
- `[selector, <css-selector>]` — find all elements matching the selector
- `[upward, <level>, <root-command>]` — for each element found by the inner root command, walk `<level>` ancestors up the DOM

### Element commands

Resolve a single element relative to the current root element.

- `<css-selector>` — shorthand for `[selector, <css-selector>]`
- `[selector, <css-selector>, <element-command>?]` — query within the optional inner element (defaults to the current root)
- `[upward, <level>, <element-command>?]` — walk `<level>` ancestors up

When the trailing `<element-command>` is omitted, the current root element is used.

### Property commands

Produce a string value (or filter/transform an existing one). Used for `url` and for each entry in `props`.

- `<css-selector>` — shorthand: `textContent` of the matched element, or `href` when used as the `url` extractor
- `[attribute, <name>, <element-command>?]` — element attribute value
- `[property, <name>, <element-command>?]` — JS property value
- `[const, <value>]` — constant string
- `[domainToURL, <property-command>]` — convert a bare domain into a URL
- `[regexInclude, <pattern>, <property-command>]` — keep only if it matches
- `[regexExclude, <pattern>, <property-command>]` — drop if it matches
- `[regexSubstitute, <pattern>, <replacement>, <property-command>]` — regex replace
- `[or, [<property-command>*], <element-command>?]` — try commands in order, take the first that yields a value

When the trailing `<element-command>` is omitted, the current root element is used.

### Button commands

Add a block button to each matched result.

- `[icon, <options>?, <element-command>?]` — icon button
- `style`: inline CSS (e.g. `"top: 16px; right: 32px; --ub-icon-size: 16px;"`)
- `[text, <options>?, <element-command>?]` — text button
- `position`: `"beforebegin"` | `"afterbegin"` | `"beforeend"` (default) | `"afterend"`
- `style`: inline CSS
- `[inset, <options>?, <element-command>?]` — absolutely-positioned button inside the target element
- `top`, `right`, `bottom`, `left`: CSS length or percentage
- `zIndex`: z-index (default `1`)

When the trailing `<element-command>` is omitted, the current root element is used.

## Interaction with the rule language

`commonProps` (and per-result `props`) populate the variables that ruleset expressions can match against. Variables prefixed with `$` (e.g. `$site`, `$category`) are conventionally set from `commonProps` so users can write rules like:

```
$site = "google" & host = "youtube.com"
$category = "images" & host = "www.amazon.com"
```

See the [ruleset specification](../../ruleset/docs/spec.md) for the rule language.

## Example

Bing image search:

```yaml
name: Bing
version: 0.1.0
homepage: https://github.com/ublacklist/builtin#readme
license: MIT
lastModified: 2023-04-05T11:11:20Z

pages:
- name: Images (desktop)
matches:
- https://www.bing.com/images/search?*
userAgent: desktop
results:
- root: [upward, 1, ".iuscp"]
url:
- regexSubstitute
- '"purl":"([^"]+)'
- "\\1"
- [attribute, "m", ".iusc"]
props:
title: [attribute, "title", "li > a"]
button: [inset, { top: "32px", right: 0 }, ".iuscp"]
commonProps:
$site: bing
$category: images
```