Repository navigation
Expand file tree
/
Copy pathengineering.html
More file actions
338 lines (336 loc) · 82.1 KB
/
Copy pathengineering.html
File metadata and controls
338 lines (336 loc) · 82.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Shifter Web Proxy — Engineering Guide</title><style>*{box-sizing:border-box}html{scroll-behavior:smooth;color-scheme:light}body{margin:0;background:#ffffff;color:#1f2328;font:16px/1.65 system-ui,-apple-system,sans-serif}header,main,footer{max-width:1040px;margin:auto;padding:26px 40px}header{color:#59636e;font-size:12px;letter-spacing:1.5px;font-weight:600;border-bottom:1px solid #d1d9e0}main{padding-top:20px}h1,h2,h3{color:#1f2328;line-height:1.3}h1{font-size:34px;letter-spacing:-.9px;margin:28px 0 16px}h2{margin-top:42px;padding-bottom:10px;border-bottom:1px solid #d1d9e0;font-size:24px}h3{margin-top:28px;font-size:19px}p{margin:16px 0}p[align=center]{text-align:center}p[align=center] img{display:block;width:100%;max-width:960px;height:auto;margin:0 auto}h1[align=center]{text-align:center}a{color:#0969da;text-underline-offset:3px;text-decoration:none}a:hover{text-decoration:underline}code{font:13px/1.7 ui-monospace,SFMono-Regular,Consolas,monospace;overflow-wrap:anywhere;background:#eff1f3;border-radius:4px;padding:2px 5px}pre{padding:20px;overflow:auto;background:#f6f8fa;border:1px solid #d1d9e0;border-radius:8px}pre code{white-space:pre;overflow-wrap:normal;padding:0;background:none}.table{overflow-x:auto}table{width:100%;border-collapse:collapse;font-size:14px}th,td{padding:12px 14px;border:1px solid #d1d9e0;text-align:left;vertical-align:top}th{background:#f6f8fa;color:#1f2328}tr:nth-child(even){background:#f6f8fa}li{margin:6px 0}footer{border-top:1px solid #d1d9e0;color:#59636e;font-size:13px}h2,h3{scroll-margin-top:18px}@media(max-width:650px){header,main,footer{padding:18px}h1{font-size:27px}table{min-width:540px}}@media print{html{color-scheme:light}body{background:white;color:#172235}header,main,footer{max-width:none;padding:12px}h1,h2,h3,th{color:#172235}h2,h3{break-after:avoid}pre,th,tr:nth-child(even){background:#f1f4f8}a{color:#1756a5}}</style></head><body><main><h1 id="shifter-web-proxy--engineering-guide">Shifter Web Proxy — Engineering Guide</h1>
<p>A web proxy that combines <strong>Scramjet in the browser</strong>, <strong>HAProxy (local TCP or HTTPS/WSS termination)</strong>, <strong>Rust Wisp gateways</strong>, <strong>Redis session enforcement</strong>, and <strong>Shifter residential proxies</strong>.</p>
<p>Visitors enter a website, choose an exit country, and browse inside the page without configuring their browser’s proxy settings. Each visitor receives a server-generated sticky session ID. The gateway adds the upstream credentials and country targeting on the server.</p>
<p><strong>Status: shared browser SDK and server-verified reCAPTCHA v2 implemented.</strong> The hosted entry point is <code>/sdk/v1/shifter-web-proxy.js</code>, exposing <code>ShifterWebProxy.create()</code>. Production API access uses site-bound anonymous bearer credentials; the legacy cookie API exists only in synthetic test mode. Production activation still requires real CAPTCHA credentials and final acceptance on both websites. This does not establish a unique-person identity or universal destination compatibility.</p>
<p>For the product overview, see <a href="../README.md">README.md</a>. An offline HTML version of this guide is included at <a href="../docs/engineering.html">docs/engineering.html</a>.</p>
<h2 id="contents">Contents</h2>
<ul><li><a href="#what-is-included">What is included</a></li><li><a href="#shared-sdk-integration">Shared SDK integration</a></li><li><a href="#architecture">Architecture</a></li><li><a href="#repository-and-secret-policy">Repository and secret policy</a></li><li><a href="#requirements">Requirements</a></li><li><a href="#quick-start-with-synthetic-traffic">Quick start with synthetic traffic</a></li><li><a href="#using-a-real-shifter-account">Using a real Shifter account</a></li><li><a href="#httpswss-deployment">HTTPS/WSS deployment</a></li><li><a href="#configuration-reference">Configuration reference</a></li><li><a href="#redis-security-rules">Redis security rules</a></li><li><a href="#session-lifecycle-and-limits">Session lifecycle and limits</a></li><li><a href="#browser-integration-and-isolation">Browser integration and isolation</a></li><li><a href="#api-reference">API reference</a></li><li><a href="#gateway-forwarding-and-destination-policy">Gateway forwarding and destination policy</a></li><li><a href="#haproxy-regions-and-recovery">HAProxy, regions, and recovery</a></li><li><a href="#testing-and-validation">Testing and validation</a></li><li><a href="#operations-and-troubleshooting">Operations and troubleshooting</a></li><li><a href="#project-layout">Project layout</a></li><li><a href="#dependencies-and-reproducibility">Dependencies and reproducibility</a></li><li><a href="#production-work-still-required">Production work still required</a></li></ul>
<h2 id="what-is-included">What is included</h2>
<ul><li>Trusted browsing controls: destination input, country selector, Browse, Back, Forward, Reload, Stop, remaining time, remaining bytes, and status messages.</li><li>An embedded Scramjet runtime using bare-mux and Epoxy transport.</li><li>Two Rust gateway replicas, a Rust session API, Redis, and an HAProxy connection balancer.</li><li>Country selection and a cryptographically random 128-bit SID per browsing assignment.</li><li>Shared time/byte allowances, single-use transport tickets, one active connection per visitor, stream admission limits, revocation, and expiry.</li><li>An authenticated SOCKS5 connector that sends every destination stream through a configured Shifter regional endpoint.</li><li>Synthetic destination/login fixtures, integration checks, outage tests, and a 1,000-session load test.</li><li>Explicit, bounded live smoke tests for country targeting, sampled sticky IP behavior, and TLS forwarding.</li><li>Example configuration files and a bootstrap script that creates ignored local copies.</li></ul>
<p>The SDK includes reCAPTCHA v2; live use requires configured credentials. Fingerprint identity is not part of this integration.</p>
<h2 id="web-proxy-page-example">Web proxy page example</h2>
<p><code>web/control/index.html</code> retains the Shifter landing page and browser toolbar. Its <code>app.js</code> is a UI adapter: it consumes SDK state and invokes methods; API calls, CAPTCHA, runtime messages, polling, expiry, and cleanup live in the SDK. <code>web/control/minimal.html</code> demonstrates a different interface using the same library.</p>
<p>Search validates the address and opens the library-owned reCAPTCHA v2 modal. Closing it or pressing Escape cancels. A successful challenge is verified by the API before browsing starts. Navigation, reload, and location changes within an active session do not require another challenge. Every landing-page Search does. Failed or unavailable CAPTCHA never grants access.</p>
<p>The browser retains Back, Forward, Reload, country selection, the URL field, time/data allowance, End session, return-to-home, and resume controls. The website owns these visuals and the loading overlay. The SDK owns the destination iframe and emits state for the website to render.</p>
<h2 id="shared-sdk-integration">Shared SDK integration</h2>
<p>The examples use reserved origins <code>https://example.com</code>, <code>https://second.example.com</code>, and <code>https://staging.example.com</code>. Set your actual origins only in the ignored production environment file; production startup requires an explicit <code>INTEGRATIONS</code> value. No <code>www</code>, HTTP, wildcard, or other subdomain is implicitly included. Loading the public JavaScript does not grant authorization; API and runtime origins are checked separately.</p>
<pre><code><div id="proxy-view" style="height:70vh"></div>
<script src="https://proxy.example.net/sdk/v1/shifter-web-proxy.js"></script>
<script>
const proxy = ShifterWebProxy.create({
container: document.querySelector('#proxy-view')
});
proxy.subscribe(state => {
// Render your controls using state.active, busy, loading, countries,
// country, url, session, remainingSeconds, persistent and error.
});
proxy.init().then(() => {
// Enable your Search control. Call from its submit handler:
// proxy.search({url: address.value, country: country.value});
}).catch(error => console.error(error.message));
</script></code></pre>
<p><code>create()</code> returns immediately; <code>init()</code> waits for the selected release and configuration. Methods return promises. <code>subscribe(callback)</code> invokes the callback immediately and returns an unsubscribe function. <code>getState()</code> returns a defensive copy. One instance may own a container at a time.</p>
<div class="table"><table><tr><th>Method</th><th>Contract</th></tr><tr><td><code>init()</code></td><td>Fetch allowed-site configuration and restore available allowance; does not start browsing</td></tr><tr><td><code>search({url, country})</code></td><td>Validate, display CAPTCHA, authorize, and browse; rejects with <code>CANCELLED</code> on dismissal</td></tr><tr><td><code>navigate(url)</code></td><td>Navigate an active session without another CAPTCHA</td></tr><tr><td><code>back()</code>, <code>forward()</code></td><td>Traverse destination history</td></tr><tr><td><code>reload()</code></td><td>Reconnect with the same assignment/allowance after lease release</td></tr><tr><td><code>changeCountry(country)</code></td><td>Revoke old assignment, clear website state, preserve allowance, and navigate</td></tr><tr><td><code>stop()</code></td><td>Revoke access and await runtime cleanup; retain anonymous identity</td></tr><tr><td><code>destroy()</code></td><td>Abort SDK work, detach iframe/listeners/timers, and release the container; does not reset server allowance</td></tr><tr><td><code>subscribe(callback)</code>, <code>getState()</code></td><td>Read UI state without exposing credentials or tickets</td></tr></table></div>
<p>State status is <code>idle</code>, <code>initializing</code>, <code>ready</code>, <code>verifying</code>, <code>connecting</code>, <code>browsing</code>, <code>interrupted</code>, <code>stopped</code>, <code>expired</code>, <code>exhausted</code>, or <code>destroyed</code>. Errors contain <code>code</code> and <code>message</code>. Handle rejected promises; only <code>CANCELLED</code> normally needs no error message. Concurrent operations reject with <code>BUSY</code>. A storage-disabled browser uses memory and reports <code>persistent: false</code>.</p>
<p>The default API origin is the loader's origin. <code>apiOrigin</code> is an optional local-integration override. <code>nonce</code> can be supplied for CSP-authorized injected scripts/styles; the loader inherits its script nonce by default. Neither option changes server authorization.</p>
<h3 id="automatic-connection-recovery">Automatic connection recovery</h3>
<p>SDK 1.0.3 silently retries a failed GET document navigation at most twice, keeping <code>loading: true</code> and the website's branded overlay visible. Backoffs are 300 and 600 milliseconds; each reconnect waits 6.5 seconds for the previous gateway lease to retire before requesting a fresh one-use ticket. Stop, a new SDK operation, destruction, or expiry cancels queued recovery. After the retry budget is exhausted, the SDK sets <code>status: interrupted</code>, releases the overlay, and leaves browser controls available without displaying a connection-error banner.</p>
<p>A thrown WebSocket connection failure reconnects with the existing SID. A transient SOCKS connection failure can rotate the SID only when the active gateway has recorded that failure for the requested hostname and session revision. Rotation preserves country, region, expiry, used bytes, and byte limit. The atomic Redis operation rejects stale/concurrent recovery, inactive sessions, and more than two recoveries in a 30-second window. The old lease is never cleared early. Redis failures deny recovery.</p>
<p>Website HTTP errors, CAPTCHAs, and login pages are normal responses and pass through unchanged. Failed POST/PUT submissions, engine errors, destination policy/DNS rejection, explicit upstream authentication rejection, and exhausted/expired authorization are not automatically replayed. Recovery applies to failed document loads, not arbitrary background requests or embedded widgets. Changing a residential SID may affect a destination's IP-bound login or session.</p>
<p>The runtime replaces failed document responses with a neutral placeholder before they reach the screen. A short-lived, one-use capability held by the service worker lets the trusted runtime retrieve the actual request method and failure classification; destination HTML and untrusted window messages cannot manufacture that classification. Existing SDK releases receive a friendly error, while 1.0.2 negotiates automatic recovery over the existing v1 protocol. Reloading a website using the stable hosted SDK loader picks up the update after deployment; site adapters render <code>state.loading</code> directly, without a second loading timeout that could reveal a recovery in progress.</p>
<p>SDK 1.0.3 retries API reads and ticket issuance up to twice (500/1000 ms backoff) on network failures or unstructured gateway 502/503/504 responses. Failed background session checks stay silent, preserve the destination and SID, and back off up to 30 seconds; the local expiry clock keeps running. Explicit authorization, quota, and storage refusals are not treated as transient gateway failures. Mutating session requests and destination form submissions are not blindly replayed. Slow-page timeouts release the overlay without a warning banner.</p>
<p>The runtime routes Scramjet v1's injected WASM script through the engine's JavaScript wrapper; raw WASM fetches and reset assets still bypass engine configuration. On browsers without transferable request streams, the service worker buffers request bodies up to 8 MiB before the first BareMux send, preserving method, headers and bytes. Larger buffered uploads fail before sending. Capable browsers keep streaming. This addresses a reproducible request-stream compatibility gap; it does not guarantee that Google will accept a proxy IP or that every CAPTCHA works. A destination CAPTCHA remains on its existing SID and is never used as a reason to rotate an IP automatically.</p>
<h3 id="captcha-configuration">CAPTCHA configuration</h3>
<p>Register both production website domains and the explicitly enabled staging hostname <code>staging.example.com</code> on the existing standard v2 checkbox key. The <strong>site key is public</strong>, including in GitHub. The <strong>secret is private</strong> and must never enter SDK code, HTML, URLs, repository files, or logs. Set <code>RECAPTCHA_SITE_KEY</code> and mount <code>RECAPTCHA_SECRET_FILE</code> only in the API container. The API needs outbound HTTPS to Google's fixed SiteVerify endpoint. Google responses must be successful, free of error codes, and match the exact caller hostname. Google enforces the response token's two-minute lifetime; the returned challenge timestamp is validated as a load timestamp, not mistaken for when the user solved it. A SHA-256 token reservation in Redis rejects concurrent replay; raw CAPTCHA responses are not stored. See <a href="https://developers.google.com/recaptcha/docs/verify">Google verification</a>.</p>
<p>For local real-CAPTCHA development, add <code>compose.captcha.example.yaml</code> as an overlay, set the public key and local secret-file path in ignored configuration, and register localhost on a development key. The overlay mounts the secret only in the API. Synthetic browser tests instead use the disposable test stack; Google test tokens are not automatically accepted by its Redis-backed mock verifier.</p>
<p>If Google reports a widget error, the SDK keeps its frame and modal open so Google's own message remains visible, and offers Try again or Close. It does not reset/remove the frame automatically on that error: doing so hides domain/key errors and produces misleading canceled network requests. Only a completed challenge can call the session API. Script-load failures and expired challenges also offer an explicit retry. For a localhost rejection, verify that the exact site key returned by the local API is the key whose domain settings you saved in Google; changing another key does not affect the running demo.</p>
<p>Production startup rejects missing credentials and Google's public test credentials. Synthetic tests seed random one-use challenge hashes in private Redis; those records are read only in <code>APP_ENV=test</code>. There is no configurable production verifier URL or bypass switch.</p>
<h3 id="release-and-rollback">Release and rollback</h3>
<p>Maintain source only under <code>web/sdk/src/</code>. <code>node scripts/build-sdk.mjs 1.0.3</code> creates/verifies immutable files and a SHA-256 manifest in <code>web/sdk/releases/1.0.3/</code>. With no argument, the script verifies the release selected by the loader. It rejects changed content for an existing version. To release an update, pass a new semantic version and change <code>RELEASE</code> in <code>web/sdk/v1/shifter-web-proxy.js</code> after validating that release.</p>
<p>The loader uses <code>Cache-Control: public, no-cache</code>; release assets use a one-year immutable cache. Each page imports one fixed release. Rollback changes the loader's release target; existing page instances keep their release until reloaded. Retain old release directories. v1 releases must preserve protocol 1 so the runtime supports the current and previous SDK release; introduce <code>/sdk/v2/</code> for a breaking contract. Release 1.0.3 adds silent status-check retries and removes connection/slow-page notices. The accompanying runtime fixes injected WASM routing and unsupported request-stream transfers. Release 1.0.2 adds bounded proxy recovery. Releases 1.0.1 (CAPTCHA error-frame handling) and 1.0.0 remain available for rollback.</p>
<p>Deploy the API and assets before changing website adapters, and disable the old production session creation at that cutover. Old cookie sessions are not imported: visitors complete a fresh CAPTCHA for a new origin-scoped identity. Update Redis ACLs before deploying; missing permissions fail closed.</p>
<h2 id="architecture">Architecture</h2>
<pre><code>Visitor browser
|
+-- http://localhost:8080
| HAProxy TCP listener
| -> Rust API / trusted browsing controls
| -> Redis: identity-linked daily allowance and tickets
|
+-- http://localhost:8081
HAProxy TCP listener / leastconn
-> Rust gateway A or B
|-- serves Scramjet, bare-mux, Epoxy, and service worker
|-- redeems Wisp ticket and claims connection ownership
|-- validates destination and reserves destination bytes
|
+-- authenticated SOCKS5 -> blr.p.shifter.io:443
-> residential exit
-> destination website</code></pre>
<p>Scramjet rewrites proxied website behavior in the visitor’s browser. Wisp carries multiplexed destination TCP streams over a WebSocket. The Rust gateway provides the session-specific upstream connector; Scramjet does not receive the residential proxy credentials.</p>
<p>The same Rust binary has two roles:</p>
<div class="table"><table><tr><th>Role</th><th>Responsibility</th><th>Secret access</th></tr><tr><td><code>api</code></td><td>Trusted controls, country list, session lifecycle, ticket issuance</td><td>Does not load the upstream credential file</td></tr><tr><td><code>gateway</code></td><td>Runtime assets, authenticated Wisp, DNS/destination checks, SOCKS forwarding</td><td>Reads the mounted upstream credential file</td></tr></table></div>
<p>Redis is shared across the API and gateway replicas. No process-global proxy URL is changed when a visitor selects a country. Upstream authenticated connections are not pooled across visitors.</p>
<h2 id="repository-and-secret-policy">Repository and secret policy</h2>
<p><strong>Only portable examples of deployment and credential configuration belong in this repository.</strong> Real accounts, passwords, machine paths, resolved configuration, operational logs, session records, live IP evidence, and local validation reports remain outside Git.</p>
<div class="table"><table><tr><th>Tracked example</th><th>Generated local file</th><th>Purpose</th></tr><tr><td><code>compose.example.yaml</code></td><td><code>compose.yaml</code></td><td>Local development topology</td></tr><tr><td><code>compose.test.example.yaml</code></td><td><code>compose.test.yaml</code></td><td>Synthetic upstream and isolated test database</td></tr><tr><td><code>compose.production.example.yaml</code></td><td><code>compose.production.yaml</code></td><td>HTTPS ingress and applications connected to existing private Redis</td></tr><tr><td><code>deploy/haproxy.production.example.cfg</code></td><td><code>deploy/haproxy.production.cfg</code></td><td>TLS certificates, HTTPS routing, redirects and WSS</td></tr><tr><td><code>deploy/production.example.env</code></td><td><code>.env.production</code></td><td>Hostnames and private certificate/secret paths</td></tr><tr><td><code>deploy/haproxy.example.cfg</code></td><td><code>deploy/haproxy.cfg</code></td><td>TCP listeners, balancing, DNS resolution, health checks</td></tr><tr><td><code>.env.example</code></td><td><code>.env</code></td><td>Non-secret mock defaults; local credential-file path for live mode</td></tr><tr><td><code>tests/fixtures/credentials.example.toml</code></td><td><code>tests/fixtures/credentials.toml</code></td><td>Public synthetic values accepted only by the mock fixture</td></tr><tr><td><code>examples/shifter-credentials.example.toml</code></td><td>A file under <code>.secrets/</code> or outside the checkout</td><td>Placeholder schema for a real upstream account</td></tr></table></div>
<p>The synthetic fixture values are deliberately non-secret test data; they are not a working Shifter account.</p>
<p>Run <code>sh scripts/configure.sh</code> to generate missing local files. It never overwrites an existing local configuration. Generated files are created with restrictive permissions. Before adopting updated templates, compare them with your local copies and apply changes deliberately.</p>
<p><code>.gitignore</code> excludes local YAML/CFG/TOML files, environment files, credentials directories, certificates/keys, build products, logs, artifacts, and local-only research/report files. Explicit <code>*.example.*</code> templates, dependency manifests, source, tests, and this documentation are tracked. <code>.dockerignore</code> uses a build-context allowlist so local configuration and evidence do not enter Docker builds.</p>
<p>The HTTP server never serves the repository root. Both the API's demo and the gateway serve only explicitly allowed browser assets from their respective <code>web/control</code>, <code>web/runtime</code>, and published <code>web/sdk</code> directories. Unknown files, dotfiles, configuration, backups, SDK source directories, source maps, encoded aliases, traversal paths, and symlinked assets return 404 with <code>Cache-Control: no-store</code>. Adding a new public asset requires updating <code>src/public_assets.rs</code>; SDK releases accept numeric <code>major.minor.patch</code> directories and only the release's client, CAPTCHA module, and manifest. Runtime dependencies remain at their pinned versions.</p>
<p>Keep <code>.env.captcha</code> and <code>.secrets/recaptcha-secret</code> outside <code>web/</code>, ignored by Git, with host permissions 600 (the <code>.secrets</code> directory should be 700). The public site key is intentionally returned by <code>/api/v1/config</code>; the private secret is mounted read-only at <code>/run/secrets/recaptcha_secret</code> only in the API container, never in gateway containers, container environment variables, or images. Docker build exclusions also cover nested secret directories, environment/configuration files, keys, backups, and database files accidentally placed beneath an allowed source directory. Production serves a read-only container filesystem; do not mount the checkout, secret directories, or writable content into public asset roots. Git ignore rules alone do not provide HTTP protection.</p>
<p><code>node tests/public-files.mjs</code> probes the running local demo and runtime using raw GET/HEAD paths (including traversal and percent encoding) and checks that required public assets still load. The same denial probes run against the disposable production HTTPS ingress in <code>tests/https-deployment.mjs</code>. Rust tests plant synthetic sensitive files and symlinks inside a temporary public tree to verify that the guard denies them even when present. <code>node tests/build-context.mjs</code> builds a disposable image with synthetic secret files to verify Docker's actual exclusion rules. Local validation does not verify an existing public deployment; deploy these changes before relying on them at <code>proxy.example.net</code>.</p>
<p>Never put a real password in a template, a command-line argument, a Docker build argument, a frontend bundle, an issue, or a test-output file. Git ignore rules do not remove previously committed data: inspect the staged diff before every push.</p>
<h2 id="requirements">Requirements</h2>
<ul><li>Docker with the Compose plugin, a running local Docker engine, support for BuildKit cache mounts and bridge gateway mode <code>isolated</code> (validated locally with Engine 29.4.0).</li><li>A browser with service workers, SharedWorkers, and WebAssembly support. Browser compatibility beyond the observed local fixture flows is not established.</li><li>Node.js 24 when running the host-side test scripts or vendoring assets outside Docker.</li><li>Rust/Cargo compatible with Rust 1.94.1 when running host-side Rust checks. The Docker build includes the pinned toolchain.</li><li>Python 3.11 or newer for <code>tests/secret-scan.py</code>; Python 3 for the offline README renderer.</li><li>Available loopback ports 8080 and 8081 for the normal stack, or 8180 and 8181 for the isolated SDK test runner.</li></ul>
<p>The build fetches pinned dependencies and base images. After the images and assets exist locally, the mock destination does not require a real upstream account or destination Internet traffic.</p>
<h2 id="quick-start-with-synthetic-traffic">Quick start with synthetic traffic</h2>
<pre><code>git clone https://github.com/leftclick-io/web-proxy.git
cd web-proxy
# Optional explicit bootstrap; stack.sh also runs it automatically.
sh scripts/configure.sh
# Build and start the API, two gateways, Redis, HAProxy, and mock upstream.
sh scripts/stack.sh mock</code></pre>
<p>Open <strong>http://localhost:8080</strong> to inspect the UI. Synthetic backend tests use <code>http://fixture.test/</code> and never consume paid upstream traffic. Browser Search now requires verification: the mock backend accepts only one-use challenge records seeded directly by tests, not arbitrary checkbox state or Google's public test responses. Use the disposable SDK runner below for automated checks; use the local CAPTCHA overlay for real verification.</p>
<p>The fixture displays the configured exit country and a synthetic assignment marker. It provides a JavaScript fetch, navigation links, redirects, a synthetic username form, and a second virtual origin for cookie-isolation checks. Use only synthetic identities with the fixture.</p>
<p>Only the HAProxy listeners are published to the host:</p>
<div class="table"><table><tr><th>URL</th><th>Purpose</th></tr><tr><td><code>http://localhost:8080</code></td><td>Trusted controls and session API</td></tr><tr><td><code>http://localhost:8081</code></td><td>Isolated runtime origin and Wisp endpoint</td></tr></table></div>
<p>Use <code>localhost</code> consistently. Origin checks expect these exact origins; replacing the host with <code>127.0.0.1</code> changes the origin even though both reach loopback.</p>
<p>Localhost HTTP supports the service-worker secure-context exception. It does not validate production HTTPS certificates, WSS termination, or a production origin layout.</p>
<p>To stop the project while retaining its Redis volume:</p>
<pre><code>sh scripts/stack.sh down</code></pre>
<h2 id="using-a-real-shifter-account">Using a real Shifter account</h2>
<p>The repository does not include a real credential file or a path to anyone’s existing secrets.</p>
<p>Create a private local file from the placeholder schema:</p>
<pre><code>mkdir -p .secrets
chmod 700 .secrets
cp examples/shifter-credentials.example.toml .secrets/shifter.toml
chmod 600 .secrets/shifter.toml</code></pre>
<p>Edit that ignored file locally and replace the placeholders with your account and password. The expected TOML structure is:</p>
<pre><code>[shifter]
account = "YOUR_ACCOUNT"
password = "REPLACE_LOCALLY"</code></pre>
<p>The account may include the <code>customer-</code> prefix; the loader removes it before constructing the full username. Supply the base account, not a username already decorated with country/SID flags.</p>
<p>Edit the ignored <code>.env</code> file:</p>
<pre><code>SHIFTER_CREDENTIALS_FILE=./.secrets/shifter.toml
SHIFTER_REGION=blr
SESSION_SECONDS=1800
BYTE_LIMIT=104857600</code></pre>
<p>Then start live development mode:</p>
<pre><code>sh scripts/stack.sh live</code></pre>
<p>The credential file is mounted read-only at <code>/run/secrets/shifter_credentials</code> in each gateway. The non-root container user must be able to read the mount under your Docker engine’s host-file permission mapping. If access fails, provision a suitably owned/group-readable secret for UID 10001; do not solve it by committing the file or making the secret world-readable.</p>
<p>The default regional hostname is <code>blr.p.shifter.io:443</code>. The selected exit country remains independent of this regional ingress setting.</p>
<p>Switching between mock and live recreates the application containers and interrupts current requests. The profiles use separate Redis databases: mock uses DB 1; live development uses DB 0. The live command removes the orphaned mock fixture container.</p>
<h2 id="httpswss-deployment">HTTPS/WSS deployment</h2>
<p>Use <code>compose.production.example.yaml</code> as a <strong>standalone</strong> stack. Do not merge it with <code>compose.example.yaml</code>, which supplies local HTTP and unauthenticated Redis. The production example uses your existing Redis and does not create, publish, or modify it.</p>
<pre><code>sh scripts/configure.sh
mkdir -p .secrets/https .secrets/redis-tls</code></pre>
<p>Edit the ignored <code>.env.production</code> generated from <code>deploy/production.example.env</code>:</p>
<ul><li><code>RUNTIME_HOST=proxy.example.net</code> is the public gateway hostname. Point its DNS record at the ingress host. Only HAProxy publishes ports 80 and 443. The API and gateway listeners stay on the internal ingress network; gateways have a separate outbound network for Shifter.</li><li><code>INTEGRATIONS</code> maps exact HTTPS origins to stable website IDs. The production environment example keeps <code>https://example.com</code> (<code>shifter</code>) and <code>https://second.example.com</code> (<code>ip-info</code>) and adds <code>https://staging.example.com</code> (<code>shifter</code>) for staging. Origins must be unique; explicitly configured aliases may share a site ID and its bearer-identity/quota namespace. Browser storage remains separate per origin, and CAPTCHA verification still checks the exact requesting hostname. Set the same mapping in API and gateways. Website IDs namespace quotas; do not rename them during an ordinary release. There is no built-in production allowlist; production startup requires this explicit setting. No implicit <code>www</code>, other subdomains, or localhost are added.</li><li>Set <code>TLS_CERT_DIR</code> to a directory containing one or more <code>.pem</code> files, each with the full certificate chain followed by its private key. The certificate must cover <code>proxy.example.net</code> (or the configured <code>RUNTIME_HOST</code>). Provision certificates through your existing certificate manager or ACME DNS challenge workflow. The example does not issue or renew certificates automatically. Mount only the needed server PEM files, not an entire CA/ACME account directory.</li><li>Set <code>REDIS_NETWORK</code> to the existing Redis Docker network. It must be an internal bridge with IPv4/IPv6 gateway mode <code>isolated</code>, no Redis published ports, and only Redis/API/gateways as members. HAProxy never joins it. For Redis on separate private infrastructure, adapt the application state-network attachment and firewall policy to that deployment; the supplied Compose topology assumes a shared Docker network.</li><li>Set <code>REDIS_URL_SECRET_FILE</code> to an ignored file containing <code>rediss://USER:URL_ENCODED_PASSWORD@PRIVATE_REDIS_HOST:6379/0</code> for production Redis with verified TLS. The hostname must resolve only to private addresses. Use a dedicated named ACL user, with unauthenticated default access disabled. Production requires TLS with certificate and hostname verification for client and replication traffic, as specified by the Redis security rule.</li><li>Use <code>rediss://</code>. For a private CA place its PEM bundle at <code>.secrets/redis-tls/ca.crt</code> and set <code>REDIS_CA_FILE=/run/redis-tls/ca.crt</code>. For a system-trusted CA, leave this variable empty. TLS certificate and hostname verification cannot be disabled. Mutual TLS client certificates are not configured by this example.</li><li>Set <code>SHIFTER_CREDENTIALS_FILE</code> to the existing account TOML. Ensure application secret files and the CA are readable by container UID 10001, while restricting host access. HAProxy must be able to read its private keys. Never bake these files into images.</li></ul>
<p>The least-privilege Redis ACL tested for application traffic permits <code>~daily:* ~ticket:* ~identity:* ~captcha:*</code> keys and commands <code>+get +set +ping +hello +select +client|setinfo +evalsha +script|load +exists +hset +pexpireat +hget +hincrby +hgetall +hdel +setex +getdel</code>. Apply this to a dedicated application user through your existing Redis administration process; do not replace operational or replication ACLs with this list.</p>
<p>Validate configuration, build, and launch <strong>on the intended deployment host</strong>:</p>
<pre><code>docker compose --env-file .env.production -f compose.production.yaml config --quiet
docker compose --env-file .env.production -f compose.production.yaml run --rm --no-deps haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg
docker compose --env-file .env.production -f compose.production.yaml up -d --build --wait</code></pre>
<p>HAProxy rejects unknown hostnames, redirects HTTP to the gateway HTTPS hostname, and terminates TLS 1.2 or later. On <code>proxy.example.net</code>, <code>/api/*</code> routes to the session API; <code>/wisp/*</code> and runtime assets route to the gateways. The control-page example is not published by this production ingress. WebSockets use HTTP/1.1 with an eleven-minute idle tunnel timeout. The browser automatically chooses WSS. The SDK uses cookie-free bearer authentication and exact-origin CORS. HSTS is set on backend responses. Public <code>/health</code> and <code>/metrics</code> requests are denied; backend health checks remain internal. Access logs are disabled to avoid storing Wisp tickets.</p>
<p>Certificate renewal: have your certificate manager atomically replace the fullchain-plus-key <code>.pem</code> files in the mounted directory, validate with the same HAProxy command, then recreate HAProxy to load them:</p>
<pre><code>docker compose --env-file .env.production -f compose.production.yaml up -d --no-deps --force-recreate haproxy</code></pre>
<p>Recreating HAProxy interrupts active Wisp connections. Schedule renewal deployment accordingly. Production requires server-verified CAPTCHA and site-bound credentials; only synthetic test mode enables the legacy cookie API.</p>
<p>Use the shared SDK integration above on either allowed website. The production ingress serves the library and runtime on <code>proxy.example.net</code>; <code>/api/v1/*</code> routes to the API service. The API has a separate outbound network for Google verification and remains attached to private Redis.</p>
<p>Validate HTTPS locally with a disposable production-mode stack:</p>
<pre><code>docker build -t shifter-web:https-check .
HTTPS_TEST_REDIS_TLS=1 node tests/https-deployment.mjs</code></pre>
<p>The test uses disposable certificates and ACL secrets, the real production HAProxy configuration and application mode, and a private isolated Redis network with ACL/password authentication. The <code>HTTPS_TEST_REDIS_TLS=1</code> setting checks the encrypted Redis path required by production policy. It checks redirects, certificate trust, cookie-free bearer access, exact CORS, production legacy-API rejection, and rejection of foreign Origins, authenticated WSS, ticket reuse rejection, invalid Redis passwords/hostnames/CAs, unauthenticated Redis rejection, application connectivity, HAProxy isolation, and failure when quota storage stops. Only loopback ingress ports are published; Redis has none. Test containers and fixture credentials are removed afterward. Passing these checks does not establish browser compatibility or validate an existing production Redis server.</p>
<p>Configuration references: <a href="https://www.haproxy.com/documentation/haproxy-configuration-tutorials/protocol-support/websocket/">HAProxy WebSocket support</a> and <a href="https://docs.rs/crate/redis/0.32.7">Redis Rust 0.32.7 TLS features</a>.</p>
<h2 id="configuration-reference">Configuration reference</h2>
<p>The example Compose environment supplies the following defaults. Customize the ignored <code>.env</code> and local Compose files as needed.</p>
<div class="table"><table><tr><th>Variable</th><th>Default</th><th>Meaning</th></tr><tr><td><code>SHIFTER_CREDENTIALS_FILE</code></td><td>Mock fixture path in the example</td><td>Host-side path mounted as the live gateway secret</td></tr><tr><td><code>SHIFTER_REGION</code></td><td><code>blr</code></td><td>Explicit Shifter ingress region</td></tr><tr><td><code>SESSION_SECONDS</code></td><td><code>1800</code></td><td>Daily browsing window, maximum 1800 seconds (30 minutes)</td></tr><tr><td><code>BYTE_LIMIT</code></td><td><code>104857600</code></td><td>100 MiB combined destination payload per UTC day</td></tr><tr><td><code>MAX_STREAMS</code></td><td><code>32</code></td><td>Concurrent destination streams per Wisp connection; configuration maximum 128</td></tr><tr><td><code>STREAM_RATE</code></td><td><code>8</code></td><td>New destination streams per second</td></tr><tr><td><code>STREAM_BURST</code></td><td><code>16</code></td><td>Redis token-bucket burst size</td></tr><tr><td><code>CONNECT_TIMEOUT_SECONDS</code></td><td><code>15</code></td><td>DNS plus authenticated upstream establishment timeout</td></tr><tr><td><code>IDLE_TIMEOUT_SECONDS</code></td><td><code>60</code></td><td>Destination-stream idle timeout</td></tr></table></div>
<p>A separate protocol-flood guard terminates a Wisp connection after more than 32 CONNECT packets within its one-second admission window. Raising the token-bucket settings does not remove that guard.</p>
<p>These advanced settings are set in the local Compose environment or supplied when running the binary directly:</p>
<div class="table"><table><tr><th>Variable</th><th>Example/default</th><th>Meaning</th></tr><tr><td><code>APP_ENV</code></td><td><code>development</code>, <code>test</code>, or <code>production</code></td><td>Production policy requires distinct HTTPS origins and private Redis with a named ACL user/password and verified TLS. The API requires real reCAPTCHA credentials; SDK identity is a site-bound bearer credential.</td></tr><tr><td><code>ROLE</code></td><td><code>api</code> or <code>gateway</code></td><td>Binary role</td></tr><tr><td><code>REPLICA</code></td><td><code>api</code>, <code>gateway-a</code>, <code>gateway-b</code></td><td>Operational identity for health/metrics</td></tr><tr><td><code>LISTEN</code></td><td><code>0.0.0.0:3000</code></td><td>Listener inside the container</td></tr><tr><td><code>REDIS_URL</code></td><td><code>redis://redis:6379/</code></td><td>Local shared session store; production policy requires private <code>rediss://</code> with named ACL credentials and verified TLS</td></tr><tr><td><code>REDIS_URL_FILE</code></td><td><code>/run/secrets/redis_url</code> in production</td><td>Read the connection URL from a secret file; takes precedence over <code>REDIS_URL</code></td></tr><tr><td><code>REDIS_CA_FILE</code></td><td>Empty (system trust)</td><td>Optional PEM CA bundle for private Redis TLS; hostname verification always remains enabled</td></tr><tr><td><code>INTEGRATIONS</code></td><td>Two production websites; localhost in development</td><td>JSON array of exact <code>{origin, site}</code> mappings</td></tr><tr><td><code>CONTROL_ORIGIN</code></td><td><code>http://localhost:8080</code></td><td>Legacy synthetic-test origin and local default only</td></tr><tr><td><code>RECAPTCHA_SITE_KEY</code></td><td>Empty outside tests</td><td>Public standard v2 checkbox key; required in production</td></tr><tr><td><code>RECAPTCHA_SECRET_FILE</code></td><td>None</td><td>API-only mounted secret file; required in production</td></tr><tr><td><code>RUNTIME_ORIGIN</code></td><td><code>http://localhost:8081</code></td><td>Exact runtime/WebSocket origin</td></tr><tr><td><code>CREDENTIALS_FILE</code></td><td><code>/run/secrets/shifter_credentials</code></td><td>In-container credential path</td></tr><tr><td><code>WEB_DIR</code></td><td><code>/app/web</code> in Docker</td><td>Static asset root</td></tr></table></div>
<p>Country availability is generated from the <a href="https://github.com/leftclick-io/session-manager/blob/master/config/weights.json">Shifter session-manager weights</a>. Sum only <code>shifter</code> leaf counts within each country in <code>locations</code>; include countries with a total <strong>strictly greater than 250</strong>. The current snapshot yields <strong>54 countries</strong>. This is a weights-based snapshot, not a live availability check. Both the country API and gateway validation use the same generated allowlist. Missing or unlisted countries are rejected; country targeting is never silently omitted.</p>
<p>The searchable country picker groups eligible locations into Tier 1, Tier 2, and Tier 3, with the US, UK, Canada, Germany, Italy, and France first. These are configurable product display priorities in <code>web/control/country-order.js</code>, independent of the IP-count eligibility filter. Filtering search results preserves that order, and future weights refreshes cannot replace it with alphabetical order. Countries not explicitly prioritized fall into Tier 3 alphabetically. The picker defaults to the United States and uses the Astro site’s existing circular SVG flags and country names. Flags appear in the selected value and every menu option, in both the landing page and browser toolbar. Keyboard users can search, move with arrow keys, select with Enter, and dismiss with Escape.</p>
<p>To refresh, obtain the latest weights JSON through authenticated GitHub access into an ignored location or outside the repository, then run:</p>
<pre><code>node scripts/sync-countries.mjs /path/to/weights.json /path/to/Shifter-Astro
node --test tests/countries.test.mjs</code></pre>
<p>The importer updates <code>src/config.rs</code> and copies eligible flags from the Astro checkout. It does not copy provider weights, counts, or operational metadata into this repository. Rebuild the API and gateways together after refreshing the list. The boundary test excludes a country at exactly 250 and ensures other providers cannot make a country eligible.</p>
<p>The regional allowlist is <code>fra</code>, <code>ams</code>, <code>lon</code>, <code>nyc</code>, <code>tor</code>, <code>sgp</code>, <code>blr</code>, and <code>syd</code>. Only the BLR upstream was exercised in the initial live validation.</p>
<h2 id="redis-security-rules">Redis security rules</h2>
<p><strong>Redis must run exclusively on private network addresses in every region. Public Redis endpoints and published Redis ports are prohibited.</strong> This applies to primary/replica instances, Sentinel, and cluster management ports.</p>
<p>The permanent project rule is <a href="../.cursor/rules/redis-security.mdc">Redis security rule</a>, referenced by <a href="../AGENTS.md">AGENTS.md</a>. It applies to application, infrastructure, example configuration, and operational changes.</p>
<p>The local Compose example attaches Redis only to <code>session_state</code>, a bridge network with <code>internal: true</code> and <code>isolated</code> gateway mode for IPv4 and IPv6. The API and both gateways join that network and the separate default network. HAProxy and the synthetic destination stay off the state network. Redis publishes no host port, including loopback port mappings. Its <code>6379/tcp</code> image metadata in <code>docker compose ps</code> is not a published host listener; any <code>HOST:PORT->6379</code> mapping would violate the rule. Docker host administrators retain privileged access to container networks; this isolation does not defend against a compromised host or Docker administrator.</p>
<p>The explicit bridge mode is required: on the development device, <code>internal: true</code> alone still allowed a direct connection from HAProxy to Redis. With <code>isolated</code> mode, the deployed-container check confirms that the API and both gateways can use Redis while HAProxy cannot connect to its private port. Run this check on each deployment engine; do not remove the isolation options to work around an unsupported engine.</p>
<p>For an existing checkout, <code>scripts/configure.sh</code> preserves local files. Merge the network changes from <code>compose.example.yaml</code> into your ignored <code>compose.yaml</code>, then recreate this project’s containers and networks with <code>docker compose down</code> followed by your usual stack start command. Do not pass <code>--volumes</code>: the Redis data volume must survive. This interrupts active local browsing connections.</p>
<p>Production requirements:</p>
<ul><li>Private IPs and private DNS only; no public IP, public load balancer, public DNS endpoint, NAT/port forwarding, or host-network shortcut for Redis.</li><li>Bind Redis only to private interfaces and necessary loopback addresses. Firewall/security groups deny access by default, allowing the regional API/gateways and explicitly approved private administration, monitoring, and replication peers.</li><li>Require named ACL users with least-privilege key and command permissions; disable unauthenticated default access. Keep protected mode enabled.</li><li>Require TLS with certificate and hostname verification for client and replication connections. Private networking and named ACL authorization remain mandatory.</li><li>Keep credentials, ACL material, certificates, private keys, and actual regional configuration in secret storage or ignored local files. Do not log authenticated Redis URLs.</li><li>Continue failing closed when authorization or quota storage is unavailable.</li></ul>
<p>The local configuration uses unauthenticated Redis <strong>only inside its isolated Docker state network in development/test mode</strong>. Production policy requires a named ACL user and password on private Redis with verified TLS. The application also supports plaintext private connections, but that support does not satisfy the production TLS requirement. Redis TLS supports a private CA bundle. Startup rejects insecure TLS verification, default/missing ACL users, and Redis DNS resolving outside RFC1918/ULA space. Network isolation alone must not be described as complete production hardening.</p>
<p>Run the policy rejection checks and the deployed-container checks after networking changes:</p>
<pre><code>node --test tests/redis-network-policy.test.mjs
node tests/configuration.mjs</code></pre>
<p>The configuration check rejects Redis host port mappings, host networking, non-internal networks, missing isolated bridge modes, non-private assigned IPs, and unexpected state-network members. It also confirms application Redis access and requires a direct connection attempt from HAProxy to fail. These are local Docker checks; actual cloud firewall rules, private endpoint settings, ACLs, certificates, and regional replication access require deployment-specific verification.</p>
<p>These requirements follow the <a href="https://redis.io/docs/latest/operate/oss_and_stack/management/security/">Redis security guidance</a> and <a href="https://docs.docker.com/reference/compose-file/networks/#internal">Docker Compose internal network configuration</a>, and <a href="https://docs.docker.com/engine/network/port-publishing/#gateway-modes">Docker isolated gateway mode</a>.</p>
<h2 id="session-lifecycle-and-limits">Session lifecycle and limits</h2>
<h3 id="identity-and-daily-allowance">Identity and daily allowance</h3>
<p>The SDK persists a server-generated 256-bit opaque credential in the embedding website's first-party local storage, scoped to API origin and website ID. Redis stores only its SHA-256 hash and an anonymous visitor/site mapping, with a 30-day expiry refreshed only after successful CAPTCHA-backed session creation. Requests omit cookies. The runtime receives only one-use tickets, never the persistent credential. Daily session keys include both website ID and visitor ID.</p>
<p>Deleting storage or changing browser profiles can obtain another anonymous identity after CAPTCHA. This is not an unresettable per-person quota. Scripts trusted by the embedding website can access its storage, so that website's XSS protections remain important. A stopped session retains its credential and allowance. Legacy <code>shifter_dev</code> cookies are accepted only in <code>APP_ENV=test</code>.</p>
<p>On first session creation, Redis records the visitor’s country, SID, region, expiry, byte usage, limit, revision, and stopped state. The browsing window starts immediately, including time spent idle or stopped. Its expiry is the earlier of the configured duration and the next UTC midnight.</p>
<p>The daily allowance is shared across streams and replicas. Reconnects, country changes, Stop, and resume do not reset time or byte usage. Session records remain long enough to enforce the daily allowance rather than disappearing when browsing stops.</p>
<h3 id="connection-authorization">Connection authorization</h3>
<ol><li>The trusted UI requests a short-lived ticket from the session API.</li><li>The API creates a random ticket containing an immutable session snapshot and a 30-second TTL.</li><li>The trusted page sends the ticket to the runtime using exact-origin <code>postMessage</code> with source-window validation.</li><li>The runtime opens the authenticated Wisp WebSocket.</li><li>The gateway atomically consumes the ticket using Redis GETDEL, then atomically validates the session revision and claims its connection lease.</li><li>An active owner blocks another simultaneous connection for the same daily visitor record.</li><li>Backend checks renew the six-second lease approximately every second and verify expiry, revision, ownership, and available bytes.</li></ol>
<p>A ticket cannot be reused. Country changes, revocation, and explicit reconnect revision changes invalidate previously issued tickets. The browser cannot supply an upstream SID, account, password, or host through the API.</p>
<h3 id="forwarding-quota">Forwarding quota</h3>
<p>Before forwarding each chunk, a Redis script validates authorization and reserves up to 16 KiB from the shared allowance. Both directions count, including destination TLS traffic. Wisp/control framing is not destination payload. A failed write may conservatively consume its reservation; quotas do not depend on a delayed after-the-fact counter.</p>
<p>Storage errors fail authorization/forwarding closed. Existing connections are also monitored while idle. An interface countdown alone does not grant or revoke access.</p>
<h3 id="country-changes-stop-and-reconnect">Country changes, Stop, and reconnect</h3>
<p>Changing country revokes the old connection, generates a fresh SID, preserves the original allowance, and starts a clean proxied website context. Sites may require login again.</p>
<p>Stop revokes active access and requests website-state cleanup. Resuming uses the same daily time/byte budget and creates a new assignment after a stopped session.</p>
<p>Reload performs an explicit reconnect, retains country/SID/region/allowance, waits for the old connection lease to clear, and reloads the displayed address. Interrupted form submissions are not automatically replayed. Browser cookie retention across this reconnect still needs final acceptance testing.</p>
<p>Different SIDs represent independent assignment requests. They do not guarantee globally unique residential IPs. Even a sticky SID can receive a new IP when its residential peer disconnects or its upstream TTL expires.</p>
<h2 id="browser-integration-and-isolation">Browser integration and isolation</h2>
<p>The embedding website supplies its UI. The runtime on a separate origin owns Scramjet, its service worker, bare-mux SharedWorker, Epoxy, and a sandboxed destination iframe. No service worker is installed on either website by the SDK. Runtime and reset documents validate the configured parent origin, actual parent window, protocol version, and per-instance channel identifier. Unknown or stale messages are ignored. <code>frame-ancestors</code> limits browser embedding to self and configured websites.</p>
<p>The integration does not require site-wide COOP/COEP headers. The pinned Scramjet version runs with synchronous XHR disabled. Destination features requiring cross-origin isolation or SharedArrayBuffer are not supported in this mode. Browser service-worker, SharedWorker, WASM, and third-party storage restrictions can still affect compatibility; the SDK reports initialization errors/timeouts. It cannot override a website's restrictive CSP. Allow the SDK script/module origin, API connect origin, runtime frame origin, and Google's reCAPTCHA script/frame/connect resources as appropriate to that website's policy. Use CSP nonces for injected scripts/styles when required.</p>
<p>The runtime clears its local/session storage, service-worker registrations, caches, and IndexedDB contents on reset. Stop removes the engine frame before running reset in an independent offscreen iframe, so a website hiding its viewport cannot interrupt cleanup. Runtime/reset assets bypass the service worker’s engine configuration read, allowing reset to finish after storage is cleared. Stop waits for acknowledgement; a cleanup failure is surfaced. Cleanup is not forensic erasure. Server revocation and quotas remain effective when browser cleanup fails. Runtime cookies and emulated destination origins still need adversarial review: rewriting does not provide browser-enforced origin isolation between every proxied destination. No persistent SDK credential is stored at the runtime origin.</p>
<h2 id="api-reference">API reference</h2>
<p>All SDK API calls use <code>/api/v1</code>. Send the embedding site's exact <code>Origin</code>; authenticated routes additionally require <code>Authorization: Bearer <credential></code>. Browsers omit Origin on same-origin GETs: these are accepted only with same-origin Fetch Metadata and a matching configured Host. Credentialed CORS is not used. JSON requests have a 16 KiB body limit; CAPTCHA response tokens have an 8 KiB field limit.</p>
<div class="table"><table><tr><th>Route</th><th>Body</th><th>Result</th></tr><tr><td><code>GET /api/v1/config</code></td><td>None</td><td>Countries, site ID, runtime origin, public CAPTCHA site key, protocol version</td></tr><tr><td><code>POST /api/v1/sessions</code></td><td><code>{country, captchaToken}</code></td><td><code>{session, credential}</code>; credential returned only for a new identity</td></tr><tr><td><code>GET /api/v1/session</code></td><td>None</td><td>Current allowance and status</td></tr><tr><td><code>POST /api/v1/session/tickets</code></td><td>None</td><td>One-use ticket, session revision, 30-second TTL, runtime origin</td></tr><tr><td><code>POST /api/v1/session/country</code></td><td><code>{country}</code></td><td>Updated assignment; allowance retained</td></tr><tr><td><code>POST /api/v1/session/reconnect</code></td><td>None</td><td>Old connection revision revoked</td></tr><tr><td><code>POST /api/v1/session/recover</code></td><td><code>{revision, url, reason}</code></td><td>Bounded recovery; reason is <code>websocket</code> or <code>upstream</code>; returns <code>{status, retryAfterMs}</code></td></tr><tr><td><code>DELETE /api/v1/session</code></td><td>None</td><td>Stopped access</td></tr></table></div>
<p>Public session fields are <code>country</code>, <code>status</code>, <code>serverTime</code>, <code>expiresAt</code>, <code>remainingBytes</code>, <code>byteLimit</code>, and <code>connected</code>. Times are Unix milliseconds; upstream credentials/SID are private. Session status is <code>active</code>, <code>stopped</code>, <code>expired</code>, or <code>exhausted</code>.</p>
<p>Error responses use <code>{code, error}</code>. Codes include <code>ORIGIN_DENIED</code>, <code>AUTH_REQUIRED</code>, <code>NO_SESSION</code>, <code>INVALID_INPUT</code>, <code>INVALID_COUNTRY</code>, <code>CAPTCHA_REQUIRED</code>, <code>CAPTCHA_REJECTED</code>, <code>CAPTCHA_UNAVAILABLE</code>, <code>SESSION_ENDED</code>, <code>RECOVERY_UNAVAILABLE</code>, and <code>STORAGE_UNAVAILABLE</code>. Verification failures never silently mint another credential. An invalid saved credential returns 401; the SDK discards it and the visitor retries Search with a new challenge.</p>
<p>The runtime keeps <code>/wisp/?ticket=...</code> and <code>/wisp/<ticket>/</code> WebSocket routes. Only its exact Origin can redeem tickets. Forged/reused/expired tickets, unavailable sessions, ownership conflicts, regional mismatches, and Redis failures deny access. <code>/settings</code> exposes only allowed parent origins and protocol versions. <code>/health</code> and <code>/metrics</code> stay internal at the production ingress.</p>
<p>The old <code>/api/*</code> cookie endpoints are available only for isolated synthetic transport regression tests. They are absent in production and development.</p>
<h2 id="gateway-forwarding-and-destination-policy">Gateway forwarding and destination policy</h2>
<p>The gateway uses Tokio and pinned <code>wisp-mux</code> components for framing and multiplexing. It does not implement a second browser or replay website requests on the server.</p>
<p>For each destination stream, the gateway:</p>
<ol><li>Checks stream type, concurrent-stream admission, and the shared rate bucket.</li><li>Allows only TCP destination ports 80 and 443; UDP is rejected.</li><li>Resolves the requested hostname locally, validates the returned addresses, and rejects blocked destinations.</li><li>Constructs the Shifter username using the immutable authorized session context.</li><li>Authenticates with SOCKS5 to the explicit regional Shifter endpoint.</li><li>Requests the validated destination IP through that proxy.</li><li>Forwards bytes while enforcing quota, idle timeout, expiry, and revocation.</li></ol>
<p>The username format is:</p>
<pre><code>customer-<account>-country-<country>-strict-true-sid-<128-bit-random-hex>-ttl-1800-pool-shifter</code></pre>
<p>The final <code>pool-shifter</code> flag restricts upstream assignments to the Shifter pool. The gateway always adds it server-side; visitors cannot choose or override the pool.</p>
<p>Account syntax and SOCKS5 credential lengths are validated. Authentication failures do not trigger a direct-connect fallback. Authenticated upstream sockets are not shared between sessions.</p>
<p>The destination policy rejects private/reserved IPv4/IPv6 ranges, loopback, link-local, multicast, mapped/translation ranges, local/internal names, Shifter infrastructure hostnames, and the configured upstream’s resolved addresses. DNS answers containing a blocked address are rejected rather than selecting a different answer. Connecting to the validated IP prevents a second, unchecked destination DNS lookup by the upstream proxy.</p>
<p>The destination’s original TLS hostname remains available to the browser transport. The live Node TLS smoke test separately checked opaque TLS forwarding with hostname/certificate verification enabled; it does not validate production frontend certificates or browser-wide compatibility.</p>
<p>The reserved <code>fixture.test</code>, <code>second.test</code>, and <code>failed-upstream.test</code> mappings exist only in test mode. The synthetic SOCKS server serves fixture responses or a deliberate SOCKS failure without opening destination Internet connections.</p>
<h2 id="haproxy-regions-and-recovery">HAProxy, regions, and recovery</h2>
<p>Local HAProxy runs in TCP mode; the production example terminates HTTPS in HTTP mode. Both use <code>leastconn</code>, backend health checks, Docker DNS resolution, and an eleven-minute connection/tunnel idle timeout. A gateway sends periodic WebSocket pings. Backend allowance enforcement still ends browsing at its own shorter session deadline.</p>
<p><strong>HAProxy balances connections, not individual requests inside a Wisp connection.</strong> An established WebSocket remains attached to one replica. Independent visitors and later reconnects may select different replicas.</p>
<p>If a replica stops, in-flight requests can fail. SDK 1.0.2 automatically reconnects eligible failed GET document loads within its retry budget; unsafe submissions and exhausted retries require user action. The surviving replica reads shared Redis state and preserves the session’s country, SID, region, and remaining allowance. A crashed replica’s lease may take up to six seconds to expire, in addition to health-check/routing recovery time.</p>
<p>The production design should place a regional HAProxy in front of healthy replicas in that same region. Each replica should use the corresponding explicit Shifter regional hostname. Geographic/latency-aware DNS can select a nearby healthy ingress region; it cannot guarantee the nearest server for every visitor.</p>
<p>The gateway binds sessions to their assigned region and rejects a region mismatch. Cross-region recovery that preserves allowance while creating a fresh residential assignment is not implemented. It must not silently promise IP continuity across regions.</p>
<h2 id="testing-and-validation">Testing and validation</h2>
<h3 id="install-host-side-test-dependencies">Install host-side test dependencies</h3>
<pre><code>npm ci --ignore-scripts
sh scripts/stack.sh mock</code></pre>
<p>The Docker asset build runs the vendoring script itself. To populate local generated assets for inspection:</p>
<pre><code>npm run vendor</code></pre>
<p>Generated vendor files are ignored by Git and excluded from the input Docker build context.</p>
<h3 id="mock-and-code-checks">Mock and code checks</h3>
<pre><code>node tests/integration.mjs
node tests/limits.mjs
node tests/load.mjs
node tests/failover.mjs
node --test tests/redis-network-policy.test.mjs
node tests/configuration.mjs
cargo test --locked
cargo clippy --all-targets -- -D warnings
npm audit --audit-level=high</code></pre>
<div class="table"><table><tr><th>Check</th><th>Coverage</th></tr><tr><td><code>integration.mjs</code></td><td>Country validation, Origin checks, forged/reused tickets, per-visitor country/SID isolation, one active connection, malformed Wisp, forbidden destinations/ports/UDP, reconnect context, country revocation, synthetic form response, shared byte cap, expiry, Stop/resume allowance</td></tr><tr><td><code>limits.mjs</code></td><td>Ticket TTL/expiry, outstanding ticket revocation, unauthorized country changes, sixteen-stream burst, thirty-two-stream concurrency cap</td></tr><tr><td><code>load.mjs</code></td><td>10, 100, 500, and 1,000 simultaneous mock sessions; two 128 KiB requests per session; errors, latency percentiles, throughput, replica counts, sampled Docker CPU/memory</td></tr><tr><td><code>failover.mjs</code></td><td>Different countries on one replica, replica failure/recovery with preserved SID/allowance, Redis outage and fail-closed behavior</td></tr><tr><td><code>configuration.mjs</code></td><td>Production-mode rejection, read-only gateway secret mounts, loopback-only application listeners, private/internal Redis with no published ports, isolated bridge modes, restricted network membership, working application access, and blocked HAProxy access</td></tr><tr><td><code>redis-network-policy.test.mjs</code></td><td>Rejection of published Redis ports, host networking, non-internal networks, missing isolated bridge modes, public addresses, and unauthorized network members</td></tr><tr><td>Rust unit test</td><td>Representative allowed/blocked destination addresses</td></tr></table></div>
<p>Load tests refuse live mode. Failover tests deliberately stop/start gateways and pause/unpause Redis. Use a dedicated local test stack and do not run those tests while using the browsing UI. The scripts restore stopped services in cleanup paths, but inspect the stack after any externally interrupted test.</p>
<p>Results are written under ignored <code>artifacts/</code>. They can contain local resource information, synthetic identifiers, or live IP addresses; do not commit the raw results.</p>
<p>To change mock load stages:</p>
<pre><code>LOAD_STAGES=10,100,500,1000 node tests/load.mjs</code></pre>
<p>These are short synthetic bursts, not production capacity measurements or a sustained saturation benchmark. Docker CPU samples may include setup or idle periods, and browser JavaScript execution is outside this server load test.</p>
<h3 id="reset-synthetic-browsing-state">Reset synthetic browsing state</h3>
<pre><code>node tests/reset-mock.mjs</code></pre>
<p>This command refuses live mode and clears only this project’s mock Redis DB 1. It allows a new fixture run after the daily window ends. Normal user flows do not reset the daily allowance.</p>
<h3 id="explicit-live-smoke-checks">Explicit live smoke checks</h3>
<p>After configuring a real private credential file:</p>
<pre><code>sh scripts/stack.sh live
node tests/live-smoke.mjs
node tests/live-tls.mjs</code></pre>
<p>Set <code>IP_CHECK_HOST</code> to your real IP-information endpoint before running live smoke checks (and <code>IP_CHECK_PATH</code> if its path differs from <code>/json</code>). The country test checks US and DE using that endpoint, with three samples per country including a Wisp reconnect. The TLS test fetches <code>https://example.com</code> through the Wisp/Shifter path with Node certificate verification enabled.</p>
<p>The scripts inspect existing usage in the dedicated live-development Redis DB 0 and reserve bounded per-visitor quotas within a cumulative <strong>20 MiB</strong> test ceiling. Run them serially on a dedicated stack. This is test accounting, not a global production billing limiter; unrelated concurrent visitors or simultaneous test runners are outside that cumulative-test calculation. Do not clear DB 0 to bypass the smoke-test cap.</p>
<p>Live smoke scripts print observed IP addresses and store them in ignored artifacts. No upstream credentials should appear in their output.</p>
<h3 id="credential-scan">Credential scan</h3>
<pre><code># Requires .env to reference the real private credential file and Python 3.11+.
python3 tests/secret-scan.py</code></pre>
<p>The local scan compares exact upstream account/password and configured local CAPTCHA secret values against source, generated assets, test evidence, application/HAProxy logs, image metadata/history, and the final image filesystem without printing those values. It supplements review; it does not prove the absence of every possible secret or replace a staged-file audit.</p>
<h3 id="shared-sdk-validation">Shared SDK validation</h3>
<pre><code>npm test
cargo test
node scripts/build-sdk.mjs
docker build -t shifter-web:sdk-check .
node tests/run-sdk-tests.mjs
HTTPS_TEST_IMAGE=shifter-web:sdk-check HTTPS_TEST_REDIS_TLS=1 node tests/https-deployment.mjs</code></pre>
<p>The SDK runner creates a separate disposable local project on loopback ports 8180/8181, runs the origin/CAPTCHA/identity/cache checks, existing transport/quota integration tests, and Redis network-policy checks, then removes that project. It never rewrites the developer's active configuration. <code>--keep</code> retains it for browser checks and writes its cleanup context under ignored <code>artifacts/</code>.</p>
<p>Recovery tests cover hidden error documents, worker capability validation, website HTTP-error passthrough, POST safety, stale frames, retry limits, Stop/destruction cancellation, gateway-attested SID rotation, concurrent recovery, and unchanged quota/expiry. The TLS fixture also exercises recovery under the production Redis ACL and verifies that storage loss denies recovery. Automated runtime tests use a simulated DOM/service-worker bridge; a real-browser recovery acceptance run remains necessary.</p>
<p>Unit tests cover modal cancellation, concurrent clicks, errors, bearer requests, stale messages, storage denial, cleanup, and Google challenge focus handling. The TLS fixture seeds a verified synthetic identity directly in its private Redis for WSS testing; it does not claim to solve a real CAPTCHA or expose a production bypass. Real credentials are never used by synthetic tests. Local browser checks exercised the branded and minimal UIs on different local site origins, mobile and desktop browsing, cancellation, and Stop/cleanup. Google’s live widget script was unavailable in that browser session; synthetic verification was used for successful session flows. Real Google challenge acceptance on the two production websites remains outstanding.</p>
<h3 id="validation-status-at-the-initial-implementation">Validation status at the initial implementation</h3>
<ul><li>Twenty-two backend integration, limit, and outage checks passed.</li><li>The 1,000-session mock stage completed 2,000 requests with zero errors and distributed connections across both gateways.</li><li>Live US/DE targeting and sampled sticky behavior passed through the BLR endpoint.</li><li>Node TLS forwarding with certificate verification passed.</li><li>Total recorded live validation traffic was 19,314 destination bytes, including an initial incomplete test-client attempt.</li><li>The destination-policy Rust unit test, Clippy, dependency audit, configuration checks, and exact-value credential scan passed at the time of validation.</li><li>Embedded fixture HTML/JavaScript, a synthetic login, and subsequent link navigation were observed in the browser.</li><li>Final browser idle stability, Back/Forward, full redirect/cookie separation, reconnect cookie retention, and Stop/country/expiry cleanup remain pending.</li></ul>
<p>Raw local reports, live IPs, and workstation-specific evidence are intentionally not published in this repository. Rerun the included checks to produce evidence for your checkout and device. The summary above is a historical observation, not a promise that future dependency or environment changes will pass.</p>
<h3 id="remaining-manual-browser-acceptance">Remaining manual browser acceptance</h3>
<ol><li>Start the mock stack, open the trusted page, and browse <code>http://fixture.test/</code>.</li><li>Confirm the JavaScript country message and submit the synthetic username form.</li><li>Follow Next page, then use Back and Forward; confirm navigation and login state.</li><li>Leave the page idle for at least one minute, then navigate again and check connection stability.</li><li>Verify Redirect, visit Second origin, and confirm that the first origin’s login does not appear there.</li><li>Return to the first origin and confirm its login remains; use Reload and check reconnect cookie retention.</li><li>Change country and confirm a fresh website context with the original allowance retained.</li><li>Stop and resume; confirm website state is cleared while time and byte usage remain shared.</li><li>Expire a session while traffic is active; verify both backend closure and the VPN informational state.</li><li>Interrupt/reload the tab and check next-load cleanup. Perform any additional public-site browser tests under a separately bounded live test budget.</li></ol>
<h2 id="operations-and-troubleshooting">Operations and troubleshooting</h2>
<h3 id="inspect-the-local-services">Inspect the local services</h3>
<pre><code>docker compose ps
docker compose logs --tail 50 api gateway-a gateway-b haproxy
curl -fsS http://localhost:8080/health
curl -fsS http://localhost:8081/health</code></pre>
<p>The runtime listener balances independent HTTP connections as well as WebSocket connections, so repeated health requests can report different replicas. Metrics contain counters, not destination URLs or visitor labels. There is intentionally no request URL/access logging; adding it could expose tickets and browsing history.</p>
<h3 id="common-issues">Common issues</h3>
<div class="table"><table><tr><th>Symptom</th><th>Checks and action</th></tr><tr><td>Compose cannot find local configuration</td><td>Run <code>sh scripts/configure.sh</code>. It copies examples only when local files are missing.</td></tr><tr><td>Live secret cannot be read</td><td>Check the ignored <code>.env</code> path, TOML schema, and UID 10001 mount readability. Do not put the secret in an image or template.</td></tr><tr><td>API denies a request Origin</td><td>Use <code>http://localhost:8080</code> for controls and preserve the configured runtime origin. Update both settings together if changing origins.</td></tr><tr><td>A second tab cannot connect</td><td>One connection is allowed per visitor. Stop the other session or use explicit reconnect and wait for lease cleanup.</td></tr><tr><td>Reconnect is briefly denied after a crash</td><td>Allow the six-second lease and HAProxy health checks to converge. The same country/SID/allowance should be retained.</td></tr><tr><td>Daily allowance ended</td><td>This is expected after expiry/exhaustion. Reconnect and country changes must not reset it. Reset DB 1 only for synthetic tests.</td></tr><tr><td>Fixture hostname fails in live mode</td><td>Fixture mappings are test-only. Start the mock profile for fixture checks.</td></tr><tr><td>Redis is paused after interrupted outage testing</td><td>Inspect the project, then use <code>docker compose unpause redis</code> and <code>docker compose start gateway-a gateway-b</code>.</td></tr><tr><td>Runtime cannot initialize</td><td>Check service worker, SharedWorker, WebAssembly support, exact origins, and generated asset availability.</td></tr><tr><td>Browser loses the connection or a website fails</td><td>This remains an open compatibility/regression area. Use explicit Reload; do not automatically replay form submissions.</td></tr><tr><td>Template changes do not appear locally</td><td>Bootstrap preserves local copies. Compare the example with the ignored local file and apply the change yourself.</td></tr></table></div>
<p>Redis uses an AOF-backed named volume and a no-eviction memory policy in the example. <code>stack.sh down</code> preserves that volume. Deleting it removes the local allowance ledger. Redis high availability, backup/restore policy, crash-durability guarantees, and production observability remain deployment work.</p>
<p>Production uses server-verified CAPTCHA and anonymous bearer identity. Clearing first-party storage can still reset anonymous identity; verified per-person identity and stronger abuse controls are separate work.</p>
<h2 id="project-layout">Project layout</h2>
<pre><code>src/
main.rs Role selection, routing, headers, process setup
config.rs Validated settings and server-side credential loading
api.rs Session API, trusted controls, health and metrics
store.rs Redis session state, tickets, leases and quotas
relay.rs Wisp adapter, admission, destination policy and SOCKS5
web/
control/ Shifter UI adapter and minimal integration example
sdk/ SDK source, stable loader, immutable release assets
runtime/ Scramjet setup, service worker and state cleanup
scripts/
configure.sh Create missing ignored local files from examples
stack.sh Local mock/live/down lifecycle
vendor.mjs Copy locked browser dependencies into generated assets
render-readme.py Produce self-contained HTML documentation
examples/
shifter-credentials.example.toml Placeholder live-account schema
deploy/
haproxy.example.cfg Portable TCP balancer template
tests/
fixtures/ Synthetic SOCKS5/destination and mock credential example
client.mjs Wisp protocol client and TLS tunnel helper
integration.mjs Core authorization, isolation, quota and expiry checks
limits.mjs Ticket and stream-admission checks
load.mjs Increasing-concurrency mock benchmark
failover.mjs Gateway/Redis outage checks
configuration.mjs Deployment-default assertions
live-smoke.mjs Bounded real country/sticky-IP checks
live-tls.mjs Bounded real TLS-forwarding check
reset-mock.mjs Reset only isolated mock data
secret-scan.py Local exact-value credential scan
compose.example.yaml Portable development stack example
compose.test.example.yaml Synthetic test-profile example
Dockerfile Multi-stage source/asset build, non-root runtime
Cargo.toml / Cargo.lock Rust dependencies
package.json / package-lock.json Browser and test dependencies
README.md Product overview and quickstart
ENGINEERING.md Detailed engineering documentation
docs/readme.html Offline README
docs/engineering.html Offline engineering guide</code></pre>
<p>Local-only configuration, artifacts, generated dependencies, and pre-existing research documents are omitted from this layout because they are not repository deliverables.</p>
<h2 id="dependencies-and-reproducibility">Dependencies and reproducibility</h2>
<div class="table"><table><tr><th>Component</th><th>Pin</th></tr><tr><td>Scramjet</td><td>Release tarball <code>1.1.0</code>, locked in <code>package-lock.json</code></td></tr><tr><td>bare-mux</td><td><code>2.1.9</code></td></tr><tr><td>Epoxy transport</td><td><code>2.1.19</code></td></tr><tr><td>wisp-mux</td><td>epoxy-tls Git revision <code>0c11678d72a636c3a4bc723db87e03e7b888eaf9</code></td></tr><tr><td>Node build stage</td><td>Node 24 image pinned by digest</td></tr><tr><td>Rust build stage</td><td>Rust 1.94.1 image pinned by digest</td></tr><tr><td>HAProxy / Redis / fixture Python</td><td>Example image tags plus immutable digests</td></tr></table></div>
<p>Use <code>npm ci --ignore-scripts</code> and Cargo’s <code>--locked</code> mode. The Docker build follows those locked inputs and generates browser assets during the build. Container digests and lockfiles improve repeatability; this is not a claim of fully bit-for-bit reproducible images, since operating-system packages are installed during the build.</p>
<p>Review transport/runtime compatibility when updating dependencies. Their contracts, browser isolation behavior, service-worker caching, and Wisp protocol behavior matter as much as version numbers. The pinned versions and known browser gaps need review before any public rollout.</p>
<p>Regenerate the standalone README after editing Markdown:</p>
<pre><code>python3 scripts/render-readme.py</code></pre>
<p>The HTML file embeds its styling, has no CDN dependencies, and opens without a development server.</p>
<p>This repository does not grant a blanket license for third-party components. Upstream licenses, including AGPL terms for relevant proxy components, remain applicable. Review their obligations before distributing or hosting a modified public service.</p>
<h2 id="production-work-still-required">Production work still required</h2>
<ul><li>Complete the pending browser acceptance procedure and investigate any remaining idle/navigation failures.</li><li>Validate representative public websites and supported authentication flows; define a compatibility policy.</li><li>Provision the real v2 CAPTCHA key/secret and register both domains; verify real Google challenge acceptance on both websites. Consider stronger identity and abuse monitoring if anonymous quotas are insufficient.</li><li>Add broader service-level admission controls, operational resource limits, and sustained load testing.</li><li>Review engine isolation and escape risks for the requested original-site integration: controls at the configured Shifter origin and the runtime at <code>proxy.example.net</code> are distinct origins; evaluate the actual deployment domains.</li><li>Provision real-domain certificates and renewal on the deployment host using the HTTPS/WSS example below; complete browser acceptance against those domains.</li><li>Verify the existing production Redis deployment against the rules above: private endpoints, named ACL users, and network access restrictions; verify TLS certificates and hostnames for client and replication connections. Client ACL/TLS support is implemented and covered by local integration checks.</li><li>Establish regional Redis/state availability and an explicit cross-region recovery protocol that preserves allowance but rotates residential assignment.</li><li>Deploy regional HAProxy/gateway groups and geographic/latency-aware DNS, with health-aware routing.</li><li>Add production telemetry that protects browsing privacy and excludes credentials, connection tickets, and destination URLs.</li><li>Configure a real VPN download destination and final expired-session UX.</li></ul>
<p>The HTTPS implementation and tests run locally. Production DNS changes, certificate issuance, and execution on a deployment host are separate operational steps; no remote deployment was performed.</p>
<p>References: <a href="https://github.com/MercuryWorkshop/scramjet">Scramjet</a>, <a href="https://github.com/MercuryWorkshop/epoxy-tls">epoxy-tls and Wisp</a>, <a href="https://www.haproxy.com/documentation/haproxy-configuration-tutorials/protocol-support/tcp/">HAProxy TCP configuration</a>, <a href="https://shifter.io/docs/products/residential-proxies/gateway-and-auth/">Shifter gateway and regions</a>, <a href="https://shifter.io/docs/products/residential-proxies/geo-targeting/">Shifter geo-targeting</a>, and <a href="https://shifter.io/docs/products/residential-proxies/sessions/">Shifter sessions</a>.</p>
<h2 id="license">License</h2>
<p>The project code is available under the <a href="../LICENSE">MIT License</a>. Third-party dependencies, fonts, and assets retain their own licenses.</p></main></body></html>