Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
version: 2
updates:
- package-ecosystem: npm
directory: /
schedule:
interval: weekly
day: monday
time: "09:00"
timezone: Europe/Moscow
open-pull-requests-limit: 1
versioning-strategy: auto
labels:
- dependencies
ignore:
- dependency-name: "*"
update-types:
- version-update:semver-major
groups:
npm-minor-and-patch:
applies-to: version-updates
patterns:
- "*"
update-types:
- minor
- patch
npm-security:
applies-to: security-updates
patterns:
- "*"
29 changes: 29 additions & 0 deletions .github/workflows/dependency-audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: 🔐 Dependency Audit

on:
schedule:
- cron: "0 5 * * 1"
workflow_dispatch:

permissions:
contents: read

jobs:
audit:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: package.json
cache: npm

- name: Install dependencies
run: npm ci

- name: Run npm audit
run: npm audit --audit-level=moderate
73 changes: 73 additions & 0 deletions .github/workflows/wreq-upstream.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: 📦 Sync wreq Upstream

on:
schedule:
- cron: "15 5 * * 1"
workflow_dispatch:

permissions:
contents: write
pull-requests: write

concurrency:
group: monitor-wreq-upstream
cancel-in-progress: false

jobs:
bump-wreq:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: package.json
cache: npm

- name: Setup Rust
uses: dtolnay/rust-toolchain@stable

- name: Check crates.io for upstream wreq releases
run: node ./scripts/update-wreq-upstream.mjs

- name: Detect dependency changes
id: changes
run: |
if git diff --quiet -- rust/Cargo.toml rust/Cargo.lock; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Install dependencies
if: steps.changes.outputs.changed == 'true'
run: npm ci

- name: Run test suite
if: steps.changes.outputs.changed == 'true'
run: npm test

- name: Create pull request
if: steps.changes.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v7
with:
branch: chore/update-wreq-upstream
delete-branch: true
commit-message: "chore: bump upstream wreq crates"
title: "chore: bump upstream wreq crates"
body: |
Automated upstream bump for the Rust crates.
This workflow checks `crates.io` for the latest published `max_version` of:
- `wreq`
- `wreq-util`
labels: |
dependencies
add-paths: |
rust/Cargo.toml
rust/Cargo.lock
129 changes: 113 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ TLS and HTTP/2 fingerprinting is actively used by major bot protection and WAF p
npm install node-wreq
```

Node.js 20+ is required.

## contents

#### ⚡   **[quick start](#quick-start)**
Expand All @@ -57,7 +59,7 @@ npm install node-wreq
#### 📊   **[observability](#observability)**
#### 🚨   **[error handling](#errors)**
#### 🔌   **[websockets](#websockets)**
#### 🧪   **[networking / transport knobs](#networking)** — TLS, HTTP/1, HTTP/2 options; header ordering.
#### 🧪   **[networking / transport knobs](#networking)** — TLS, HTTP/1, HTTP/2 options; header ordering, mTLS and custom CAs; DNS controls.

## ⚡ <a id="quick-start"></a>quick start

Expand Down Expand Up @@ -137,6 +139,24 @@ const response = await fetch('https://api.example.com/items', {
console.log(await response.json());
```

### upload `FormData`

`FormData` request bodies work like `fetch`: the multipart boundary and `content-type` header are generated automatically.

```ts
const formData = new FormData();

formData.append('alpha', '1');
formData.append('upload', new File(['hello'], 'hello.txt', { type: 'text/plain' }));

const response = await fetch('https://api.example.com/upload', {
method: 'POST',
body: formData,
});

console.log(await response.json());
```

### build a `Request` first

```ts
Expand Down Expand Up @@ -697,10 +717,44 @@ const response = await fetch('https://httpbin.org/anything', {
});
```

If you want to bypass env/system proxy detection for a specific request, use `proxy: false`:

```ts
await fetch('https://example.com', {
proxy: false,
});
```

### disable default browser-like headers

By default, `node-wreq` may apply profile-appropriate default headers.

`disableDefaultHeaders: true` disables those browser/profile preset headers only.

That means it turns off headers injected by the selected browser emulation, such as:

- `user-agent`
- `accept`
- `accept-language`
- `sec-ch-ua`
- `sec-ch-ua-mobile`
- `sec-ch-ua-platform`
- `sec-fetch-dest`
- `sec-fetch-mode`
- `sec-fetch-site`
- `priority`

The exact set varies by profile.

It does **not** disable protocol or transport-level headers that may still appear automatically, such as:

- `host`
- `accept-encoding` when `compress` is enabled
- `content-length` when the request body requires it
- `content-type` generated by the runtime for bodies like `FormData`

It also does not remove headers you set explicitly yourself.

If you want full manual control:

```ts
Expand All @@ -713,34 +767,25 @@ await fetch('https://example.com', {
});
```

### exact header order

Use tuples when header order matters:
For example, with `browser: 'chrome_137'`, the default request would normally include Chrome-like `sec-ch-*`, `sec-fetch-*`, `user-agent`, `accept`, and `accept-language` headers. With `disableDefaultHeaders: true`, those browser preset headers are skipped, while transport headers like `host` and `accept-encoding` may still be present.

```ts
await fetch('https://example.com', {
headers: [
['x-lower', 'one'],
['X-Mixed', 'two'],
],
});
```
### exact header order

### exact original header names on the wire
Use tuples when header order matters.

Use this only if you really need exact casing / spelling preservation:
Tuple headers also preserve the original header names exactly as you wrote them on the wire:

```ts
await fetch('https://example.com', {
disableDefaultHeaders: true,
keepOriginalHeaderNames: true,
headers: [
['x-lower', 'one'],
['X-Mixed', 'two'],
],
});
```

For example, this will preserve both the tuple order and the exact `x-lower` / `X-Mixed` casing you passed.

### lower-level transport tuning

If a browser preset gets you close but not all the way there:
Expand All @@ -767,14 +812,66 @@ Use these only when:
- you are comparing transport behavior
- you want to debug fingerprint mismatches

### mTLS and custom CAs

Use `tlsIdentity` for client certificate authentication and `ca` for a custom trust store:

```ts
import { fetch } from 'node-wreq';
import { readFileSync } from 'node:fs';

await fetch('https://mtls.example.com', {
tlsIdentity: {
cert: readFileSync('./client-cert.pem'),
key: readFileSync('./client-key.pem'),
},
ca: {
cert: readFileSync('./ca.pem'),
includeDefaultRoots: false,
},
});
```

PKCS#12 / PFX identities are also supported:

```ts
await fetch('https://mtls.example.com', {
tlsIdentity: {
pfx: readFileSync('./client-identity.p12'),
passphrase: 'secret',
},
ca: {
cert: readFileSync('./ca.pem'),
includeDefaultRoots: false,
},
});
```

### compression

Compression is enabled by default.

That includes `gzip`, `br`, `deflate`, and `zstd` response decoding when the server supports them.

Disable it if you need stricter control over response handling:

```ts
await fetch('https://example.com/archive', {
compress: false,
});
```

### DNS controls

Use `dns.hosts` to pin hostnames to specific IPs, or `dns.servers` to send lookups through specific nameservers:

```ts
await fetch('https://api.internal.test/health', {
dns: {
servers: ['1.1.1.1', '8.8.8.8'],
hosts: {
'api.internal.test': ['127.0.0.1'],
},
},
});
```
Loading
Loading