diff --git a/.github/workflows/sync-check.yml b/.github/workflows/sync-check.yml new file mode 100644 index 0000000..369379b --- /dev/null +++ b/.github/workflows/sync-check.yml @@ -0,0 +1,33 @@ +name: Skill sync check + +# Claude-BugHunter is the canonical home for the two recon skills this repo +# re-exports (offensive-osint, osint-methodology). This guard fails if our copies +# drift from BugHunter's — either because someone edited them here (edit them in +# BugHunter instead) or because BugHunter advanced and we haven't re-synced. + +on: + push: + branches: [main, develop] + pull_request: + branches: [main, develop] + schedule: + - cron: '17 6 * * 1' # weekly Monday — catch upstream drift even without a PR + workflow_dispatch: + +jobs: + drift: + name: Mirrored skills match Claude-BugHunter + runs-on: ubuntu-latest + steps: + - name: Checkout claude-osint + uses: actions/checkout@v4 + + - name: Checkout Claude-BugHunter (canonical source) + uses: actions/checkout@v4 + with: + repository: elementalsouls/Claude-BugHunter + ref: main # pin to a release tag / SHA for reproducible releases + path: _bughunter + + - name: Verify the 2 mirrored skills are in sync + run: bash scripts/sync-from-bughunter.sh --from "$GITHUB_WORKSPACE/_bughunter" --check diff --git a/.gitignore b/.gitignore index b809c33..9355d2f 100644 --- a/.gitignore +++ b/.gitignore @@ -56,3 +56,9 @@ node_modules/ # Local sync artifacts *.local.md *.synced + +# Workspace layout: the sibling Claude-BugHunter repo lives here in this checkout. +# Never let `git add -A` sweep it (or stale self-nested skill dirs) into a commit. +/Bug Hunter/ +skills/offensive-osint/offensive-osint/ +skills/osint-methodology/osint-methodology/ diff --git a/README.md b/README.md index bb18e09..5c26027 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # claude-osint -> 2 paired Claude skills · **90+ recon modules** · 48 secret-regex patterns · 80+ dorks · 9 read-only credential validators · 27 attack-path templates · 4,600+ lines of structured tradecraft. Drop-in `SKILL.md` files that turn Claude into a god-mode external recon operator for authorized red-team and bug-bounty engagements. +> 2 paired Claude skills · **90+ recon modules** · 48 secret-regex patterns · 80+ dorks · 9 read-only credential validators · 27 attack-path templates · 6,000+ lines of structured tradecraft. Drop-in `SKILL.md` files that turn Claude into a god-mode external recon operator for authorized red-team and bug-bounty engagements. Built by **[ElementalSoul](https://github.com/elementalsouls)** — GenAI Security Research. @@ -28,7 +28,7 @@ Built by **[ElementalSoul](https://github.com/elementalsouls)** — GenAI Securi Drop both into your Claude environment and it behaves like a senior recon analyst: it knows the techniques, the tooling, the edge cases, and the escalation paths — and it stays in scope. -~4,600 lines of structured tradecraft · 96.9% PASS on a 32-prompt self-evaluation · ~85–90% practitioner coverage for the recon phase of authorized engagements. +~6,000 lines of structured tradecraft · 96.9% PASS on a 32-prompt self-evaluation · ~85–90% practitioner coverage for the recon phase of authorized engagements. --- @@ -37,11 +37,13 @@ Drop both into your Claude environment and it behaves like a senior recon analys ``` claude-osint/ ├── skills/ -│ ├── osint-methodology/SKILL.md # how to think (455 lines) +│ ├── osint-methodology/SKILL.md # how to think (~1,700 lines) │ └── offensive-osint/ -│ ├── SKILL.md # what to reach for (4,168 lines) -│ ├── scripts/secret_scan.py # stdlib-only secret scanner -│ └── scripts/h1_reference.py # HackerOne disclosed-reports reference agent +│ ├── SKILL.md # what to reach for (lean index, ~400 lines) +│ ├── references/ # 15 modular reference files (~3,900 lines) +│ ├── scripts/secret_scan.py # stdlib-only 48-pattern secret scanner +│ ├── scripts/h1_reference.py # HackerOne disclosed-reports reference agent +│ └── scripts/dashboard.py # local recon console (stdlib web UI) ├── docs/ # architecture · coverage · install · usage ├── examples/ # 4 end-to-end engagement walk-throughs ├── tests/smoke-test-prompts.md # 32-prompt self-evaluation @@ -50,6 +52,8 @@ claude-osint/ Each skill directory is self-contained. Drop into `~/.claude/skills/` and Claude auto-triggers on relevant phrases. +> **Mirrored from [Claude-BugHunter](https://github.com/elementalsouls/Claude-BugHunter).** That repo is the canonical monorepo home for all skills; this repo re-exports the two recon skills. **Edit them in BugHunter, not here** — maintainers re-sync with `scripts/sync-from-bughunter.sh`, and a CI guard (`.github/workflows/sync-check.yml`) fails on drift. + --- ## Skill Index @@ -282,17 +286,20 @@ flowchart TD ### With Claude Code ```bash -# Install both skills (one-time, after clone) git clone https://github.com/elementalsouls/Claude-OSINT.git cd Claude-OSINT -chmod +x ./scripts/sync-skill-content.sh -./scripts/sync-skill-content.sh -mkdir -p ~/.claude/skills -cp -r skills/osint-methodology ~/.claude/skills/ -cp -r skills/offensive-osint ~/.claude/skills/ -ls ~/.claude/skills/ +bash scripts/install.sh # copies both skills + records an install manifest ``` +The installer is idempotent (skips a skill already present and identical) and writes +a manifest so you can later run `bash scripts/install.sh --uninstall` to remove exactly +this bundle's footprint. Prefer a plain copy? `cp -r skills/* ~/.claude/skills/` still works. + +> **Safe alongside [Claude-BugHunter](https://github.com/elementalsouls/Claude-BugHunter).** +> These two recon skills are shared with that bundle (canonically maintained there). Both +> installers record a manifest, so installing both is idempotent and `--uninstall` keeps any +> skill the other bundle still owns — uninstalling one never breaks the other. + Then, in any Claude Code session, ask an OSINT question — both skills auto-load and trigger on relevant phrases (50+ trigger phrases each). ### With the Claude Skills System diff --git a/SECURITY.md b/SECURITY.md index 579203d..d047ac4 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -60,7 +60,7 @@ If you used these skills during an authorized engagement and found a vulnerabili ## Security best practices for users - Pin the skill version (`v2.1`) in any production deployment. -- Run `scripts/sync-skill-content.sh` (or manual cp) only against this repo's bundled `docs/full-skills/` files; don't fetch from arbitrary sources. +- Install skills only by copying this repo's bundled `skills/*/` directories; don't fetch skill content from arbitrary sources. - Verify SHA-256 of any binary helper scripts before execution. - Don't commit your engagement-specific notes into a fork of this repo. - Use sock-puppet GitHub accounts when contributing if your engagement persona shouldn't be linked to your contributor identity. diff --git a/docs/installation.md b/docs/installation.md index 146e4ea..569ac25 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -12,10 +12,8 @@ Claude Code looks for skills in `~/.claude/skills/` by default. git clone https://github.com/elementalsouls/Claude-OSINT.git cd Claude-OSINT -# Optional: populate full SKILL.md content from bundled full-skills (one-time after clone) -./scripts/sync-skill-content.sh - # Copy both skills into your local Claude Code skills directory +# (skills/*/SKILL.md ship with full content — no populate step needed) mkdir -p ~/.claude/skills cp -r skills/osint-methodology ~/.claude/skills/ cp -r skills/offensive-osint ~/.claude/skills/ @@ -29,12 +27,9 @@ mkdir -p ~/.claude/skills ln -sf ~/.local/share/Claude-OSINT/skills/osint-methodology ~/.claude/skills/osint-methodology ln -sf ~/.local/share/Claude-OSINT/skills/offensive-osint ~/.claude/skills/offensive-osint - -cd ~/.local/share/Claude-OSINT -./scripts/sync-skill-content.sh # one-time ``` -Then `git -C ~/.local/share/Claude-OSINT pull && ./scripts/sync-skill-content.sh` periodically. +Then `git -C ~/.local/share/Claude-OSINT pull` periodically to stay current. ### Verify install @@ -116,23 +111,11 @@ The skill's `triggers:` list controls auto-activation. If your prompt's wording - Try rephrasing with a phrase from the SKILL.md `triggers:` list. - If your phrasing is a common practitioner term, [open an issue](https://github.com/elementalsouls/Claude-OSINT/issues) to add it. -### "I get the structured-outline SKILL.md, not the full content" - -By default we ship structured-outline SKILL.md files (small, fast to load). To get full inline content: - -```bash -cd -./scripts/sync-skill-content.sh -``` - -This populates `skills/*/SKILL.md` with the full content from `docs/full-skills/*.SKILL.full.md`. - ### "Skill is too large for my model's context" Both skills together are ~5,500 lines / ~150 KB. This fits comfortably in modern Claude context windows (200K+). If you're using an older model with smaller context: -- Use the structured-outline SKILL.md files (don't run sync-skill-content.sh). -- Or attach only one skill at a time, depending on the task. +- Attach only one skill at a time, depending on the task. - Or run a model with larger context (Claude Sonnet 4.6+, Opus 4.6+). ### "I want to filter the skill content" diff --git a/docs/usage.md b/docs/usage.md index abeaa40..7627c50 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -166,3 +166,29 @@ See [`../examples/`](../examples/) for end-to-end walkthroughs: - `02-bug-bounty-workflow.md` — full HackerOne engagement - `03-identity-fabric-mapping.md` — M365 deep enum - `04-secret-hunting.md` — leaked-credential workflow + +## Local dashboard (optional) + +A zero-dependency (stdlib-only) localhost web UI ships alongside the skill at +[`../skills/offensive-osint/scripts/dashboard.py`](../skills/offensive-osint/scripts/dashboard.py). +It wraps the bundled helpers in a browser console — no Flask, no pip install. + +```bash +python3 skills/offensive-osint/scripts/dashboard.py # http://127.0.0.1:8765 +python3 skills/offensive-osint/scripts/dashboard.py --open # also open a browser tab +python3 skills/offensive-osint/scripts/dashboard.py --port 9000 +``` + +Three tabs: + +- **Secret Scan** — point it at a repo/file path; runs `secret_scan.py` recursively + (binaries, `.git`, `node_modules` skipped) and renders findings with severity/category + aggregation, live filtering, and JSON/CSV export. +- **Paste & Scan** — paste JS, configs, env dumps, or response bodies; scanned fully + **offline** against the 48-pattern catalog. +- **HackerOne Ref** — queries the public disclosed-report corpus via `h1_reference.py` + (this tab is the only one that makes outbound HTTPS requests). + +**Safety:** binds to `127.0.0.1` only by default. The scan API reads local files the +operator points it at, so binding to a non-loopback address prints a warning — only do +that on a trusted, isolated host. diff --git a/scripts/install.sh b/scripts/install.sh new file mode 100644 index 0000000..0f8651b --- /dev/null +++ b/scripts/install.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# ===================================================================== +# install.sh — install the claude-osint recon skills into ~/.claude/skills/ +# +# Copies the two skills (osint-methodology, offensive-osint) and records an +# install manifest so they can be cleanly removed later. These two skills are +# shared with — and canonically maintained in — Claude-BugHunter. Both bundles +# record a manifest, so uninstalling EITHER keeps a skill the other still owns. +# +# bash scripts/install.sh install +# bash scripts/install.sh --uninstall remove (keeps skills BugHunter still owns) +# -h | --help show this help +# ===================================================================== +set -e + +REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )" +SKILLS=(osint-methodology offensive-osint) +DEST="$HOME/.claude/skills" +BACKUP_DEST="$HOME/.claude/install-backups/$(date +%Y%m%d-%H%M%S)" +BUNDLE_NAME="claude-osint" +MANIFEST_DIR="$HOME/.claude/.skill-manifests" +MANIFEST="$MANIFEST_DIR/$BUNDLE_NAME.txt" + +usage() { sed -n '2,/^# ===/p' "$0" | sed 's/^#\{0,1\} \{0,1\}//'; } + +DO_UNINSTALL=0 +while [ $# -gt 0 ]; do + case "$1" in + --uninstall) DO_UNINSTALL=1 ;; + -h|--help) usage; exit 0 ;; + *) echo "Unknown option: $1 (try --help)" >&2; exit 2 ;; + esac + shift +done + +uninstall_bundle() { + if [ ! -f "$MANIFEST" ]; then + echo "No manifest at $MANIFEST — nothing tracked to uninstall." + return 0 + fi + echo "Uninstalling $BUNDLE_NAME using $MANIFEST" + local rel target other owned removed=0 kept=0 + while IFS= read -r rel; do + [ -z "$rel" ] && continue + target="$HOME/.claude/$rel" + owned=0 + for other in "$MANIFEST_DIR"/*.txt; do + [ -e "$other" ] || continue + [ "$other" = "$MANIFEST" ] && continue + if grep -qxF "$rel" "$other" 2>/dev/null; then owned=1; break; fi + done + if [ "$owned" = "1" ]; then kept=$((kept + 1)); else rm -rf "$target"; removed=$((removed + 1)); fi + done < "$MANIFEST" + rm -f "$MANIFEST" + echo " ✓ removed $removed item(s); kept $kept still owned by another bundle (e.g. claude-bughunter)" +} + +if [ "$DO_UNINSTALL" = "1" ]; then uninstall_bundle; exit 0; fi + +mkdir -p "$DEST" "$MANIFEST_DIR" +echo "Installing $BUNDLE_NAME skills → $DEST" +for name in "${SKILLS[@]}"; do + src="$REPO_DIR/skills/$name" + if [ ! -d "$src" ]; then echo " ⚠ missing $src — skipping"; continue; fi + if [ -d "$DEST/$name" ] && [ ! -L "$DEST/$name" ]; then + if diff -rq --exclude=__pycache__ "$src" "$DEST/$name" >/dev/null 2>&1; then + echo " = $name already present and identical — skipped" + else + mkdir -p "$BACKUP_DEST"; mv "$DEST/$name" "$BACKUP_DEST/$name" + cp -r "$src" "$DEST/$name"; echo " ✓ $name installed (previous backed up to $BACKUP_DEST)" + fi + else + cp -r "$src" "$DEST/$name"; echo " ✓ $name installed" + fi +done + +{ for name in "${SKILLS[@]}"; do echo "skills/$name"; done; } > "$MANIFEST" +echo " ✓ Install manifest → $MANIFEST" +echo "" +echo "Done. Both skills auto-trigger in Claude Code. Uninstall later:" +echo " bash scripts/install.sh --uninstall" diff --git a/scripts/sync-from-bughunter.sh b/scripts/sync-from-bughunter.sh new file mode 100644 index 0000000..10241c6 --- /dev/null +++ b/scripts/sync-from-bughunter.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# sync-from-bughunter.sh — mirror the shared recon skills FROM Claude-BugHunter. +# +# Claude-BugHunter is the canonical monorepo home for ALL skills. Claude-OSINT +# re-exports two of them (offensive-osint, osint-methodology). This script copies +# those two skill directories from a BugHunter checkout into this repo so the two +# copies never drift. EDIT THE SKILLS IN BUGHUNTER, NOT HERE. +# +# Usage: +# ./scripts/sync-from-bughunter.sh --from /path/to/Claude-BugHunter # sync (overwrites local copies) +# ./scripts/sync-from-bughunter.sh --from /path/to/Claude-BugHunter --check # drift check only (CI; non-zero on drift) +# BUGHUNTER_DIR=/path/to/Claude-BugHunter ./scripts/sync-from-bughunter.sh # env-var form +# +# With no --from / $BUGHUNTER_DIR, a few common sibling locations are probed. +# --check is intended for CI, where .gitattributes guarantees LF on both sides. + +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +SKILLS=(offensive-osint osint-methodology) + +BUGHUNTER_DIR="${BUGHUNTER_DIR:-}" +CHECK_ONLY=false + +while [ $# -gt 0 ]; do + case "$1" in + --from) BUGHUNTER_DIR="${2:-}"; shift 2 ;; + --check|-c) CHECK_ONLY=true; shift ;; + --help|-h) grep '^#' "$0" | sed 's/^# \?//'; exit 0 ;; + *) echo "Unknown argument: $1 (try --help)" >&2; exit 2 ;; + esac +done + +# Probe common locations if not given explicitly. +if [ -z "$BUGHUNTER_DIR" ]; then + for c in \ + "$REPO_ROOT/Bug Hunter/Claude-BugHunter" \ + "$REPO_ROOT/../Claude-BugHunter" \ + "$HOME/security-research/Claude-BugHunter"; do + if [ -d "$c/skills" ]; then BUGHUNTER_DIR="$c"; break; fi + done +fi + +if [ -z "$BUGHUNTER_DIR" ] || [ ! -d "$BUGHUNTER_DIR/skills" ]; then + echo "✗ Claude-BugHunter repo not found. Pass --from or set BUGHUNTER_DIR." >&2 + exit 2 +fi + +echo "==> Source of truth: $BUGHUNTER_DIR" +drift=0 + +for s in "${SKILLS[@]}"; do + SRC="$BUGHUNTER_DIR/skills/$s" + DST="$REPO_ROOT/skills/$s" + if [ ! -d "$SRC" ]; then + echo " ✗ source missing: $SRC" >&2 + exit 2 + fi + if [ "$CHECK_ONLY" = true ]; then + # Exclude python caches: sync strips them from the destination, so a stray + # __pycache__ in the source would otherwise report spurious drift (locally; + # CI checkouts never have one). + if diff -rq --exclude=__pycache__ --exclude='*.pyc' "$SRC" "$DST" >/dev/null 2>&1; then + echo " ✓ $s: in sync" + else + echo " ✗ $s: DRIFT (run without --check to re-sync, or edit in BugHunter)" + drift=1 + fi + else + rm -rf "$DST" + mkdir -p "$DST" + cp -r "$SRC/." "$DST/" + find "$DST" -name __pycache__ -type d -prune -exec rm -rf {} + 2>/dev/null || true + echo " ✓ $s: synced from BugHunter" + fi +done + +if [ "$CHECK_ONLY" = true ]; then + [ "$drift" -eq 0 ] && echo "==> In sync with BugHunter." || echo "==> DRIFT detected." >&2 + exit "$drift" +fi + +echo "==> Done. The 2 recon skills are mirrored from BugHunter." +echo " Reminder: edit these skills in Claude-BugHunter, then re-run this script." diff --git a/scripts/sync-skill-content.sh b/scripts/sync-skill-content.sh deleted file mode 100644 index 76e9d83..0000000 --- a/scripts/sync-skill-content.sh +++ /dev/null @@ -1,91 +0,0 @@ -#!/usr/bin/env bash -# sync-skill-content.sh — populate skills/*/SKILL.md with full v2.1 inline content -# from the bundled docs/full-skills/ canonical sources. -# -# Run this once after `git clone` to convert the structured-outline SKILL.md files -# into the full inline-content versions that Claude can load directly. -# -# Usage: -# ./scripts/sync-skill-content.sh # interactive, prompts before overwrite -# ./scripts/sync-skill-content.sh --force # non-interactive overwrite -# ./scripts/sync-skill-content.sh --check # verify checksums; no changes - -set -euo pipefail - -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" - -# Mapping: full-skills source → SKILL.md destination -SKILLS=( - "osint-methodology" - "offensive-osint" -) - -FORCE=false -CHECK_ONLY=false - -for arg in "$@"; do - case "$arg" in - --force|-f) FORCE=true ;; - --check|-c) CHECK_ONLY=true ;; - --help|-h) - grep '^#' "$0" | sed 's/^# \?//' - exit 0 - ;; - *) - echo "Unknown argument: $arg" >&2 - echo "Run with --help for usage." >&2 - exit 2 - ;; - esac -done - -echo "==> Syncing SKILL.md content from docs/full-skills/" - -for skill in "${SKILLS[@]}"; do - SRC="$REPO_ROOT/docs/full-skills/$skill.SKILL.full.md" - DST="$REPO_ROOT/skills/$skill/SKILL.md" - - if [ ! -f "$SRC" ]; then - echo " ⚠ Source missing: $SRC" - echo " (Skipping. The structured-outline SKILL.md will remain in place.)" - continue - fi - - if [ "$CHECK_ONLY" = true ]; then - SRC_HASH=$(sha256sum "$SRC" | awk '{print $1}') - if [ -f "$DST" ]; then - DST_HASH=$(sha256sum "$DST" | awk '{print $1}') - if [ "$SRC_HASH" = "$DST_HASH" ]; then - echo " ✓ $skill: in sync" - else - echo " ✗ $skill: DRIFT (run without --check to update)" - fi - else - echo " ✗ $skill: destination missing (run without --check to populate)" - fi - continue - fi - - if [ -f "$DST" ] && [ "$FORCE" = false ]; then - echo - echo " $skill: existing SKILL.md will be overwritten with full v2.1 content." - read -p " Continue? [y/N] " -n 1 -r REPLY - echo - if [[ ! "$REPLY" =~ ^[Yy]$ ]]; then - echo " Skipping $skill." - continue - fi - fi - - cp "$SRC" "$DST" - LINES=$(wc -l < "$DST") - BYTES=$(wc -c < "$DST") - echo " ✓ $skill: $LINES lines, $BYTES bytes installed at $DST" -done - -if [ "$CHECK_ONLY" = false ]; then - echo - echo "==> Done. Skills are ready to load." - echo " Verify: head -5 skills/*/SKILL.md" - echo " Install: cp -r skills/* ~/.claude/skills/" -fi diff --git a/skills/offensive-osint/SKILL.md b/skills/offensive-osint/SKILL.md index f6d7dbf..7ce6754 100644 --- a/skills/offensive-osint/SKILL.md +++ b/skills/offensive-osint/SKILL.md @@ -1,7 +1,7 @@ --- name: offensive-osint -description: "Operational arsenal for external red-team and bug-bounty reconnaissance. Concrete wordlists (28 Swagger paths, 13 GraphQL paths, 35 high-risk ports, 6 missing-header findings, 15 always-on HTTP checks, 5 SAML paths, cloud bucket permutations, JS guess-paths, vendor product fingerprints for Citrix/F5/Pulse/Fortinet/Cisco/PaloAlto/VMware/Exchange, cloud-native service fingerprints, container/K8s exposure paths, CI/CD platform paths, documentation/wiki leak paths, WHOIS/RDAP, DNS record catalog, Wayback CDX recipes), 43+-pattern secret-regex catalog (incl. modern AI API keys: Anthropic/OpenAI/HuggingFace/Cloudflare/DigitalOcean/npm/PyPI/Docker Hub/Atlassian/DataDog/Sentry/ngrok), 80+ dork corpus across 9 categories, GitHub code-search dorks, copy-paste curl/httpie probes for every check, post-discovery enumeration workflows (AWS/GitHub/Slack/JWT/PMAK/Anthropic/OpenAI), endpoint interest scoring rubric (0–100), mobile app ownership confidence, identity-fabric endpoints (Entra/Okta/ADFS/Google/SAML/M365 Teams+SharePoint+OneDrive+OAuth + user-enum), GraphQL field-suggestion enumeration when introspection disabled, 9 read-only secret validators (Postman/AWS/GitHub/Slack/Anthropic/OpenAI/npm/Atlassian/DataDog), Postman workspace search (verified endpoint), Stack Exchange sweep, public SaaS dorks, email security analysis (SPF/DMARC/DKIM/BIMI/MTA-STS/DNSSEC), origin-discovery / CDN bypass techniques, TLS deep audit (sslyze/testssl.sh/JA3/JA4), reverse-DNS sweep + IPv6 enum, vulnerability prioritization data sources (NVD/EPSS/CISA KEV/ExploitDB/Metasploit), 27 attack-path hint templates, 80+ severity-matrix examples, LinkedIn employee enumeration, job posting tech-stack analysis, Slack/Discord workspace discovery, package registry leak hunting (npm/PyPI/Docker Hub/Quay/GHCR), sat imagery for physical recon, tooling quick-install one-liners, sector-specific recon notes (healthcare/finance/ICS-SCADA/IoT/government), runnable stdlib-only secret_scan.py helper, plus the existing tool references for username/email/phone/people/social/breach/infrastructure/crypto/media/geospatial/AI/archiving/automation. Use when you need concrete probe paths, regexes, payloads, scoring rules, curl one-liners, and tool URLs for an authorized external recon engagement." -version: 2.1.1 +description: "Operational arsenal for authorized external red-team and bug-bounty recon. Concrete probes, wordlists, regexes, dorks, curl one-liners for: subdomain enum, GraphQL/Swagger/REST discovery, identity fabric (Entra/Okta/ADFS/Google/SAML/M365 deep — Teams/SharePoint/OneDrive), cloud bucket enum (S3/GCS/Azure), CDN/WAF bypass, origin discovery, vendor fingerprinting (Citrix/F5/Pulse/Fortinet/PaloAlto/Cisco/VMware), CI/CD exposure, 48-pattern secret-scan catalog (AWS/GCP/GitHub/Stripe/Slack/Anthropic/OpenAI/Atlassian/DataDog/npm/PyPI), Postman workspaces, breach correlation (HudsonRock/HIBP/DeHashed/IntelX), TLS/JA3 audit, certificate transparency, JS endpoint extraction, package registry leaks, mobile/APK recon, sat imagery, sector-specific recon (healthcare DICOM, finance SWIFT, ICS/SCADA Modbus/BACnet). Detail content in 15 modular reference files, loaded on demand. Use for any authorized recon: scoping, asset discovery, attack-path mapping, secret triage, severity scoring." +version: 3.0.0 triggers: - external recon - external red team @@ -134,7 +134,8 @@ triggers: # Offensive OSINT — External Red-Team Arsenal -> Companion skill: `osint-methodology` (the "how to think" skill). This skill is the "what to reach for." Use them together. +> **v3.0** — Refactored 2026-05-02 from a 4,168-line monolith into a lean SKILL.md (~400 lines) plus 15 modular reference files in `references/`. Detail content loads on demand — Claude reads only the reference files relevant to the current task. + ## 0. When to use / When NOT @@ -152,3166 +153,140 @@ triggers: ## 1. Authorization & Legal Posture -For assets the operator owns or has written authorization to assess. Soft scope check before acting against an unverified third-party target — see methodology skill §1 for the full posture. - ---- - -## 2. Confidence Levels - -- **TENTATIVE** — plausible based on indirect evidence (snippet-only dork match, single-source asset, inferred email pattern). -- **FIRM** — directly observed (subdomain resolves, HEAD-confirmed bucket exists, banner returned). -- **CONFIRMED** — verified via independent corroboration OR direct verification (live PMAK validation, multiple sources agree, listable bucket with object retrieval). - ---- - -## 3. Output Format Conventions - -Findings should carry: `id`, `module`, `asset_key`, `category`, `severity` (info/low/medium/high/critical), `confidence`, `title`, `description`, `evidence` (url + UTC timestamp + sha256 + raw ≤ 2 KiB), `references`, `remediation`. UTC timestamps everywhere. - ---- - -## 4. Source Hygiene & Citations - -URL + UTC timestamp + SHA-256 + tool version + run_id, every artifact. PNG screenshots, JSONL run logs, raw HTTP captures capped at 2 KiB body. - ---- - -## 5. Do NOT - -- Don't paste creds/PII/session tokens into cloud LLMs. -- Don't run destructive probes outside DEEP/`--aggressive`. -- Don't use validated credentials for anything except read-only liveness check. -- Don't single-source attribute. -- Don't assume vendor labels are ground truth. - ---- - -## 6. General OSINT (curated tool refs) - -- [OSINT Bookmarks](https://tools.myosint.training/) — comprehensive bookmarks. -- [OSINT Framework](https://osintframework.com/) — tool/resource directory. -- [IntelTechniques Tools](https://inteltechniques.com/tools/) — investigative suite. -- [Bellingcat Toolkit](https://www.bellingcat.com/resources/2024/09/24/bellingcat-online-investigations-toolkit/) — investigative journalism. -- [CyberSudo OSINT Toolkit](https://docs.google.com/spreadsheets/d/1EC0sKA_W9znzsxUt0wye9UYtyATXw5m8) — OSINT websites list. -- [Google Dorks](https://dorksearch.com/) — efficient Google searching. -- [Distributed Denial of Secrets](https://ddosecrets.com/) — leaked datasets. -- [Country-Specific Resources](https://digitaldigging.org/osint/) — country-targeted OSINT. - -## 7. Search Engines - -| Tool | Notes | -|------|-------| -| [Carrot2](https://search.carrot2.org/#/search/web) | Clusters results by topic | -| [etools](https://www.etools.ch/) | Metasearch | -| [Kagi](https://kagi.com/) | Privacy-first, non-personalized | -| [Brave Search](https://search.brave.com/) | Independent index; Goggles for custom ranking | -| [PDF Search](https://www.pdfsearch.io/) | PDF + table of contents | -| [Google Fact Check Explorer](https://toolbox.google.com/factcheck/explorer) | Cross-site fact-check | - ---- - -## 8. Username & Email Investigation - -| Tool | Purpose | -|------|---------| -| [Sherlock](https://github.com/sherlock-project/sherlock) | Username search across social networks | -| [Maigret](https://github.com/soxoj/maigret) | Profile collector by username | -| [What's My Name](https://whatsmyname.app/) | Username search | -| [Holehe](https://github.com/megadose/holehe) | Email registration check | -| [Epieos](https://epieos.com/) | Email pivots and metadata | -| [OSINT Industries](https://osint.industries/) | Email/username/phone lookups | -| [Hunter.io](https://hunter.io/) | Domain → emails | -| [EmailRep](https://emailrep.io/) | Email reputation | -| [Emailable](https://emailable.com/) | Email verification | -| [Mugetsu](https://mugetsu.io/) | X/Twitter username history | -| [RocketReach](https://rocketreach.co/) / [Apollo](https://www.apollo.io/) | Email enrichment + pattern guessing | -| [PhoneInfoga](https://github.com/sundowndev/phoneinfoga) | Phone number intelligence | - -Browser extensions: [GetProspect](https://chromewebstore.google.com/detail/email-finder-getprospect/bhbcbkonalnjkflmdkdodieehnmmeknp), [SignalHire](https://chrome.google.com/webstore/detail/signalhire-find-email-or/aeidadjdhppdffggfgjpanbafaedankd). - ---- - -## 9. People Search - -- [TruePeopleSearch](https://www.truepeoplesearch.com/) — free U.S. people search. -- [WhitePages](https://www.whitepages.com/), [Spokeo](https://www.spokeo.com/), [Webmii](https://webmii.com/), [Pipl](https://pipl.com/) (paid). -- [Clearbit](https://clearbit.com/) — company/individual data enrichment. -- [FaceCheck](https://facecheck.id/) / [FaceSeek](https://faceseek.online/) — reverse face search. - ---- - -## 10. Phone Number OSINT - -- [TrueCaller](https://www.truecaller.com/) — caller ID + spam blocking. -- [ThatsThem](https://thatsthem.com/) — reverse phone search. -- [Infobel](https://infobel.com/) — non-USA phone search. -- [FreeCarrierLookup](https://freecarrierlookup.com/) — carrier/type (US). -- [NumlookupAPI](https://numlookupapi.com/) [Freemium] — programmatic carrier checks. -- [CallerIDTest](https://calleridtest.com/), [Advanced Background Checks](https://www.advancedbackgroundchecks.com/). - ---- - -## 11. Email-Pattern Inference (TENTATIVE candidates) - -Given a `(first_name, last_name, domain)`, generate these 8 candidate addresses for breach pre-hits, phishing list curation, and downstream enrichment. Mark as **TENTATIVE** confidence until corroborated. - -``` -{first}.{last}@{domain} # john.doe@example.com -{first}{last}@{domain} # johndoe@example.com -{first}@{domain} # john@example.com -{first[0]}{last}@{domain} # jdoe@example.com -{first}.{last[0]}@{domain} # john.d@example.com -{last}@{domain} # doe@example.com -{first}_{last}@{domain} # john_doe@example.com -{first}-{last}@{domain} # john-doe@example.com -``` - -Lowercase before lookup. Strip diacritics for ASCII fallback. If the org uses a known pattern (e.g., Hunter.io shows `{first}.{last}` is dominant), prioritize that one and mark FIRM. - ---- - -## 12. Email-Harvest Source Stack - -Six parallel sources, dedup at the end: - -1. **IntelX phonebook API** — 2-step search + poll. Largest single source for breach-era addresses. -2. **Hunter.io** — domain-search endpoint. ~25 free/month. Returns verified emails + roles. -3. **crt.sh** — extract X.509 SAN extensions. Many certs include admin/contact emails. -4. **DuckDuckGo SERP scrape** — HTML scrape of `"@{target-domain}"` results. -5. **Bing SERP scrape** — same query, complementary index. -6. **Wayback CDX** — historic snapshots of the target's homepage / contact / about pages often contain emails removed from the live site. - -**Email regex:** -```regex -\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b -``` - -**Noise filter (reject numeric-only locals):** -```regex -^[0-9]+$ -``` -(Discards garbage like `12345@example.com` from random tokens.) - ---- - -## 13. Social Media - -| Platform | Tool | -|----------|------| -| Instagram | [Picuki](https://www.picuki.com/) — profile view without account | -| X/Twitter | [snscrape](https://github.com/snscrape/snscrape) — preferred CLI scraper; Twint as fallback | -| Facebook | [Graph Search](https://inteltechniques.com/tools/Facebook.html), [sowsearch.info](https://sowsearch.info/), [lookup-id.com](https://lookup-id.com/), [whopostedwhat.com](https://whopostedwhat.com/) | -| Facebook (research) | [Meta Content Library](https://transparency.meta.com/researcher) — CrowdTangle successor (researcher-gated) | -| YouTube/Twitch | [Social Blade](https://socialblade.com/) — analytics | -| TikTok | [Tokboard](https://tokboard.com/) — trends + profile analytics | -| Reddit | [Reveddit](https://www.reveddit.com/) — removed content; [RedTrack.social](https://redtrack.social/) — user history | -| Bluesky | [Firesky](https://firesky.tv/) — real-time firehose; [SkyView](https://bsky.jazco.dev/) — follower graphs | -| Mastodon | [FediSearch](https://fedisearch.skorpil.cz/) — cross-instance search; [Fedifinder](https://fedifinder.glitch.me/) — find Twitter users on Mastodon | -| Faces | [Search4Faces](https://search4faces.com/) | - ---- - -## 14. Public Records & Company Information - -- [OpenCorporates](https://opencorporates.com/) — world's largest open company DB. -- [SEC EDGAR](https://www.sec.gov/edgar.shtml) — U.S. company filings. -- [OpenOwnership Register](https://register.openownership.org/) — beneficial ownership. -- [MuckRock](https://www.muckrock.com/) — FOIA repository + request tracking. -- [EU Tenders (TED)](https://ted.europa.eu/) — EU procurement notices. -- [World Bank Projects](https://projects.worldbank.org/) — project + procurement records. -- [UK Companies House](https://find-and-update.company-information.service.gov.uk/) — UK companies + officers + filings. - -### 14.1 RU registries - -[Rusprofile](https://www.rusprofile.ru/), [Kontur.Focus](https://focus.kontur.ru/) (freemium), [zakupki.gov.ru](https://zakupki.gov.ru/) (procurement), EGRUL/EGRIP (official, captcha-gated). - -### 14.2 CN registries + USCC + ICP - -- **GSXT** — [gsxt.gov.cn](https://www.gsxt.gov.cn/) National Enterprise Credit Info; cross-check with Tianyancha / Qichacha. -- **USCC (Unified Social Credit Code)** — 18-character entity ID assigned to all CN legal entities. Format: ``. Useful for joining GSXT records to ICP filings. -- **ICP Beian** — [beian.miit.gov.cn](https://beian.miit.gov.cn/) — every domain serving traffic in mainland CN must register an ICP filing; the filing links the domain to a USCC, which links to the legal entity in GSXT. -- Workflow: `target.cn` domain → ICP lookup → USCC → GSXT → entity name + officers + adjacent registered entities. - -### 14.3 Sanctions & Compliance - -- [OFAC SDN List](https://sanctionssearch.ofac.treas.gov/), [EU Sanctions Map](https://www.sanctionsmap.eu/). -- [OpenSanctions](https://www.opensanctions.org/) — aggregated. -- [OCCRP Aleph](https://aleph.occrp.org/) — investigative documents, leaks, company records. - ---- - -## 15. Breach & Leak Data - -- [Have I Been Pwned](https://haveibeenpwned.com/) — breach lookup; Pwned Passwords API (k-anonymity). -- [Dehashed](https://dehashed.com/) — credential search (paid). -- [IntelX](https://intelx.io/) — data intelligence. -- [LeakCheck](https://leakcheck.io/), [Snusbase](https://snusbase.com/), [BreachDirectory](https://breachdirectory.org/), [Scattered Secrets](https://scatteredsecrets.com/), [Phonebook](https://phonebook.cz/), [LeakPeek](https://leakpeek.com/). -- [Cavalier (Hudson Rock)](https://cavalier.hudsonrock.com/) — **infostealer log lookups; FREE; highest single-source ROI for finding compromised employee credentials in corporate SSO**. - -### 15.0.1 HudsonRock Cavalier — direct API recipe - -The web UI wraps a **public, unauthenticated JSON API**. Hit it directly: - -```bash -# By domain (canonical first call) -curl -sk -m 30 "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-domain?domain=target.com" | jq . - -# By email (single-account check) -curl -sk -m 30 "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-email?email=alice@target.com" | jq . - -# By URL (when target's app is the breach victim) -curl -sk -m 30 "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-url?url=https://app.target.com" | jq . -``` - -PowerShell: -```powershell -$hr = Invoke-RestMethod -Uri "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-domain?domain=$D" -TimeoutSec 30 -"Employees: $($hr.employees) | Users: $($hr.users) | Third-party: $($hr.third_parties) | Total: $($hr.total)" -$hr.data.employees_urls | Sort-Object -Property occurrence -Descending | Select-Object -First 20 -$hr.data.clients_urls | Sort-Object -Property occurrence -Descending | Select-Object -First 15 -``` - -**Top-level JSON fields:** -- `total` — total stealer entries touching this domain. -- `totalStealers` — global stealer-log corpus size (context only). -- `employees` — count of `<*>@` accounts found. -- `users` — count of accounts where the domain appeared as a *visited* URL (customers/vendors). -- `third_parties` — accounts touching adjacent domains in the org. -- `data.employees_urls[]` — `{occurrence, type, url}` — internal apps where employees were logging in when stolen. **Subdomain hits here = recon gold.** -- `data.clients_urls[]` — same shape; user-facing apps (often reveals undocumented public portals). -- `data.stealer_families[]` — `{_key, _value}` → which stealer (RedLine / Lumma / StealC / Vidar / Raccoon). -- `data.dates_compromised[]` — `{_key, _value}` → temporal distribution. - -**Free-tier caveats (CRITICAL to know):** -- Subdomain hostnames in `data.*_urls[]` past the first few are **redacted with asterisks** (`*****.target.com`). Pivot to paid Cavalier tier or other sources for unredacted. -- Free endpoint returns counts + sample URLs only. Cleartext passwords + emails are **never** in the free response. -- Rate limit ~1 req/sec/IP; 429 on burst. Sleep 1s between calls. -- For unredacted creds + bulk enumeration → paid Cavalier portal. - -**Severity mapping (per §15.1 + §15.2):** `employees ≥ 10` → CRITICAL, **regardless of whether the breached service is still online** (legacy Lotus Domino / on-prem mail decommissioned + cloud SSO migration → employees almost always reuse passwords → SSO_EXPOSURE escalates CRITICAL). - -### 15.1 Domain-Level Breach Severity Mapping - -When you query a breach corpus by domain, map the result to severity like so: - -| Stat | Severity | -|---|---| -| ≥ 10 employees compromised | **CRITICAL** | -| 1–9 employees compromised | **HIGH** | -| ≥ 1 end-user (non-employee) compromised | **MEDIUM** | -| Domain seen in breach with 0 named accounts | **INFO** | - -**Employees vs end-users distinction:** an employee account is `@` (the breach victim is the target's own staff). An end-user account is the target's customer who reused a password — useful for credential-stuffing risk awareness but not directly compromising the target's identity fabric. - -### 15.2 SSO_EXPOSURE finding - -When a discovered SSO tenant (Entra GUID / Okta slug / Google Workspace domain) intersects with the breach corpus on its domain → `SSO_EXPOSURE` finding, severity **CRITICAL**. Evidence: tenant ID + product + employee count + per-account source attribution. - -**Legacy-mail-decommissioned pattern (high-value variant):** - -If `mail.` / `webmail.` returns **NXDOMAIN today** but HudsonRock/HIBP corpus still has historical employee credentials against it AND `autodiscover.` resolves to Microsoft IPs (M365) or `aspmx.l.google.com` MX (Workspace), the org migrated from on-prem to cloud — and the stolen passwords almost certainly survived the migration via password reuse. **Escalate to CRITICAL `SSO_EXPOSURE`** even when the legacy host is dead. - -Concrete triggers (all three together): -1. `Resolve-DnsName mail. -Type A` → NXDOMAIN (legacy gone) -2. HudsonRock corpus has employee URLs against the *old* host (e.g. `mail./names.nsf` for Lotus Domino, `mail./owa/` for Exchange, `mail./iwaredir.nsf` for iNotes, `mail./zimbra/` for Zimbra) -3. Current MX → M365 / Google Workspace / Zoho cloud (DNS confirms migration) - -Evidence pack: tenant GUID + breach count + 3+ legacy URLs from corpus + autodiscover Microsoft IPs + current MX. Recommend forced password rotation + MFA audit + Conditional Access review. - ---- - -## 16. Pre-built Wordlists & Probe Paths - -Copy-pasteable arsenals, severity-annotated where relevant. - -### 16.1 Swagger / OpenAPI discovery — 28 paths - -Probe each path on every alive webapp. GET (or HEAD if rate-limited). - -``` -swagger.json -swagger.yaml -swagger/v1/swagger.json -swagger/v2/swagger.json -swagger-ui.html -swagger-ui/ -swagger-resources -api-docs -api-docs.json -api/swagger -api/swagger.json -api/swagger-ui.html -api/v1/swagger.json -api/v2/swagger.json -api/v3/api-docs -v2/api-docs -v3/api-docs -openapi.json -openapi.yaml -openapi/v1 -openapi/v3 -docs -redoc -rapidoc -api/docs -api/documentation -.well-known/openapi -``` - -**Severity:** -- Reachable Swagger/OpenAPI spec without auth → **HIGH** `LEAKY_API_SPEC` (full endpoint enumeration leaks; often reveals undocumented internal APIs). -- Behind auth but accessible to any authenticated user → MEDIUM (still discloses internal API surface). - -### 16.2 GraphQL discovery — 13 paths - -``` -graphql -graphiql -api/graphql -v1/graphql -v2/graphql -query -api/query -gql -altair -playground -subscriptions -graphql/console -api/v1/graphql -``` - -**Standard introspection POST body:** -```json -{ - "operationName": "IntrospectionQuery", - "query": "query IntrospectionQuery { __schema { types { name kind fields { name type { name kind } } } queryType { name } mutationType { name } subscriptionType { name } } }" -} -``` - -**Severity:** -- Introspection returns schema without auth → **HIGH** `OPEN_GRAPHQL_API`. -- Field-suggestion enumeration possible (server returns "did you mean" for typo'd field names) → **MEDIUM** (re-derive partial schema even when introspection is disabled). -- `/graphql` accepts batched queries (`[...]` request body) → MEDIUM (rate-limit bypass surface; auth bypass via mixed batches). - -UI markers (lower severity but still discoverable): -- HTML response contains `graphiql`, `playground`, `apollo studio`, `altair` → GraphiQL UI exposed (often shipped accidentally on prod). - -### 16.3 High-risk ports — 35 services - -For each open port, emit a finding with the severity and "why an attacker cares" below. Source for the open-port observation: Shodan InternetDB (free, 1 req/sec) is the recommended starting point. - -| Port | Service | Severity | Why it matters | -|---|---|---|---| -| 21 | FTP | HIGH | Anonymous read often enabled; cleartext creds. | -| 22 | SSH | LOW | Banner discloses version; brute-force surface. | -| 23 | Telnet | HIGH | Cleartext protocol; should never be exposed. | -| 25 | SMTP | LOW | Open relay risk; version banner. | -| 53 | DNS | LOW | Recursion = DDoS amplifier; AXFR opportunism. | -| 80 | HTTP | INFO | Standard. | -| 110 | POP3 | LOW | Cleartext if no STARTTLS. | -| 111 | rpcbind | MEDIUM | NFS exports enumeration. | -| 135 | MS RPC | HIGH | Enum via Impacket. | -| 139 | NetBIOS-SSN | HIGH | File/printer enum. | -| 143 | IMAP | LOW | Cleartext if no STARTTLS. | -| 161 | SNMP | HIGH | Community strings often `public`/`private`; full device enum. | -| 389 | LDAP | HIGH | Anonymous bind = full directory dump. | -| 443 | HTTPS | INFO | Standard. | -| 445 | SMB | **CRITICAL** | EternalBlue, SMB relay, anonymous shares. | -| 465 | SMTPS | LOW | Banner. | -| 514 | rsyslog | MEDIUM | Log injection / DoS. | -| 587 | SMTP-MSA | LOW | Banner. | -| 631 | IPP/CUPS | MEDIUM | Print server enum / RCE in old CUPS. | -| 873 | rsync | HIGH | Modules often listable; backup data exposure. | -| 1433 | MSSQL | HIGH | Brute-force; xp_cmdshell. | -| 1521 | Oracle TNS | HIGH | Brute-force; SID enum. | -| 2049 | NFS | HIGH | World-readable exports. | -| 2375 | Docker API (unencrypted) | **CRITICAL** | Unauthenticated container/host takeover. | -| 2376 | Docker API (TLS) | HIGH | Cert validation bypass risk. | -| 3000 | Common dev / Grafana | MEDIUM | Often Grafana / Express dev with default creds. | -| 3306 | MySQL | HIGH | Brute-force; default `root:""`. | -| 3389 | RDP | **CRITICAL** | BlueKeep / DejaBlue / NLA bypass. | -| 5432 | PostgreSQL | HIGH | Brute-force; default `postgres:postgres`. | -| 5601 | Kibana | HIGH | Often unauthenticated; Elasticsearch pivot. | -| 5900 | VNC | HIGH | Often unauthenticated or weak password. | -| 5984 | CouchDB | HIGH | Default no auth; admin party. | -| 6379 | Redis | **CRITICAL** | No auth default; write `authorized_keys` for SSH. | -| 7001 | WebLogic | HIGH | Frequent CVEs (CVE-2020-14882, etc.). | -| 8000 | Common dev | MEDIUM | Django, common dev servers. | -| 8080 | HTTP-alt | MEDIUM | Tomcat, Jenkins, common proxy. | -| 8443 | HTTPS-alt | MEDIUM | Same as 8080. | -| 8888 | Common dev / Jupyter | HIGH | Jupyter often exposes interactive shell. | -| 9090 | Cockpit / Prometheus | HIGH | Server admin UI / metrics scraping. | -| 9200 | Elasticsearch | **CRITICAL** | Typically no auth. | -| 9300 | Elasticsearch transport | HIGH | Cluster join + RCE. | -| 11211 | memcached | MEDIUM | UDP DDoS amp; data dump. | -| 27017 | MongoDB | **CRITICAL** | No auth by default. | -| 50070 | Hadoop NameNode | HIGH | HDFS browse. | - -When Shodan InternetDB returns `vulns[]` for a port, escalate the finding severity by one tier and include the CVE list in evidence. - -### 16.4 Missing security headers — 6 findings - -For every alive webapp, audit response headers. Each missing header below = one finding. - -| Header | Severity (default) | Severity (sensitive path) | Notes | -|---|---|---|---| -| `Strict-Transport-Security` | MEDIUM | **HIGH** | Sensitive paths: `/login`, `/signin`, `/sso`, `/admin`, `/auth`. | -| `Content-Security-Policy` | MEDIUM | MEDIUM | XSS impact mitigation gone. | -| `X-Frame-Options` | LOW | LOW | Clickjacking. (CSP `frame-ancestors` is the modern replacement.) | -| `X-Content-Type-Options` | LOW | LOW | MIME-sniff XSS. | -| `Referrer-Policy` | INFO | INFO | Outbound link leakage. | -| `Permissions-Policy` | INFO | INFO | Feature-policy hardening. | - -### 16.5 Always-on HTTP checks — 15 paths - -Run these against every alive webapp regardless of Nuclei availability. Cheap; high signal. - -| Path | Finding | Severity | Match logic | -|---|---|---|---| -| `/.git/config` | Exposed `.git` repo | **CRITICAL** | Body contains `[core]`, `[remote`, `repositoryformatversion` | -| `/.git/HEAD` | Exposed `.git/HEAD` | HIGH | Body matches `^ref:\s` | -| `/.env` | Exposed `.env` | **CRITICAL** | Multiline regex `^\s*[A-Z_][A-Z0-9_]*\s*=` | -| `/server-status` | Apache server-status | MEDIUM | Body contains `Apache Server Status` or matching title | -| `/server-info` | Apache mod_info | MEDIUM | Body contains `Apache Server Information` | -| `/.DS_Store` | Exposed `.DS_Store` | LOW | Byte signature `\x00\x00\x00\x01Bud1` | -| `/phpinfo.php` | phpinfo() leak | HIGH | Body contains `phpinfo()`, `PHP Version`, or matching title | -| `/info.php` | phpinfo() (alt path) | HIGH | Same as above | -| `/actuator/env` | Spring Boot `/actuator/env` | **CRITICAL** | Body contains `"propertySources"`, `systemProperties`, `systemEnvironment` | -| `/actuator/heapdump` | Spring Boot heapdump | **CRITICAL** | HPROF magic bytes / large binary download | -| `/_cat/indices` | Elasticsearch open | HIGH | Returns index list | -| `/console` | Jenkins script console | HIGH | Body contains `Jenkins`/`Script Console` | -| `/manager/html` | Tomcat Manager | HIGH | Body contains `Tomcat Web Application Manager` | -| `/wp-admin/install.php` | Orphaned WP install | LOW | Body contains `WordPress Installation` | -| `/.well-known/security.txt` | Disclosure policy info | INFO | Parse contact + policy fields | - -Plus parse `/robots.txt` for `Disallow:` paths — those become the next-tier wordlist for that target. - -### 16.6 SAML metadata — 5 paths - -``` -/saml/metadata -/FederationMetadata/2007-06/FederationMetadata.xml -/federationmetadata/2007-06/federationmetadata.xml -/simplesaml/saml2/idp/metadata.php -/auth/saml2/metadata -``` - -Reachable SAML metadata XML reveals: `EntityID`, signing certs (often pinned → cert-reuse pivot), `SingleSignOnService` URL, `NameIDFormat`. Mark as `MISCONFIG` (LOW severity unless metadata leaks internal hostnames or non-public certs, then MEDIUM). - -### 16.7 SSO subdomain prefixes — 8 prefixes - -Probe each against root domain + every sibling brand domain: -``` -auth.{domain} -login.{domain} -sso.{domain} -idp.{domain} -iam.{domain} -identity.{domain} -accounts.{domain} -oauth.{domain} -``` - -Plus probe `/.well-known/openid-configuration` on every alive subdomain (regardless of prefix). - -### 16.8 Cloud bucket permutation arsenal - -**6 prefixes:** -``` -"" # bare candidate -backup- -assets- -static- -dev- -prod- -``` - -**15 suffixes:** -``` -"" # bare candidate --backup --assets --static --media --data --uploads --dev --prod --staging --logs --private --public --dump --archive -``` - -**47 generic stems** (filter unless combined with target-identifying token): -``` -www, mail, email, app, apps, web, webmail, ftp, cdn, static, assets, media, img, images, -videos, download, downloads, upload, uploads, data, files, docs, support, help, kb, -blog, news, dev, test, staging, stg, qa, uat, sandbox, preprod, preview, vpn, -mx, smtp, imap, pop, dns, ns, ns1, ns2, mx1, mx2 -``` - -**Provider URL templates:** - -S3: -``` -https://{candidate}.s3.amazonaws.com/ -https://{candidate}.s3-{region}.amazonaws.com/ # try us-east-1, us-west-2, eu-west-1, ap-southeast-1 first -https://s3.{region}.amazonaws.com/{candidate}/ -``` - -GCS: -``` -https://{candidate}.storage.googleapis.com/ -https://storage.googleapis.com/{candidate}/ -``` - -Azure Blob: -``` -https://{candidate}.blob.core.windows.net/ -``` - -**Probe technique:** HEAD first → 200/301 = exists, 403 = exists private, 404 = skip. On exists, GET root → if XML/JSON object listing returns, **CRITICAL** `PUBLIC_CLOUD_BUCKET`. Direct-URL object reads but not listable → **HIGH** `PUBLIC_CLOUD_BUCKET_OBJECT_READ`. - -### 16.9 JS guess-paths for endpoint discovery - -Probe these paths on every alive webapp (in addition to scraped ` + + +""" + + +# -------------------------------------------------------------------------- +# Entry point +# -------------------------------------------------------------------------- + +def main() -> int: + ap = argparse.ArgumentParser(description="Local recon console for claude-osint.") + ap.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1).") + ap.add_argument("--port", type=int, default=8765, help="Bind port (default: 8765).") + ap.add_argument("--open", action="store_true", help="Open a browser tab on startup.") + args = ap.parse_args() + + if args.host not in ("127.0.0.1", "localhost", "::1"): + print( + f"[!] WARNING: binding to {args.host} exposes a file-reading scan API to the " + "network. Only do this on a trusted, isolated host.", + file=sys.stderr, + ) + + try: + httpd = ThreadingHTTPServer((args.host, args.port), Handler) + except OSError as exc: + print(f"[!] Could not bind {args.host}:{args.port} — {exc}", file=sys.stderr) + return 2 + + url = f"http://{args.host}:{args.port}" + print(f"==> claude-osint recon console: {url}") + print(" Ctrl-C to stop.") + if args.open: + threading.Timer(0.6, lambda: webbrowser.open(url)).start() + + try: + httpd.serve_forever() + except KeyboardInterrupt: + print("\n==> shutting down.") + finally: + httpd.server_close() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/osint-methodology/SKILL.md b/skills/osint-methodology/SKILL.md index 8880249..17ecff5 100644 --- a/skills/osint-methodology/SKILL.md +++ b/skills/osint-methodology/SKILL.md @@ -1,7 +1,7 @@ --- name: osint-methodology -description: "Comprehensive OSINT methodology for external red-team operations and authorized attack-surface assessments. Covers the 5-stage recon pipeline, asset-graph discipline, severity rubric, confidence upgrade workflows, time budgeting, identity-fabric mapping, breach×identity correlation, detectability tagging, detection-aware probing, WAF/CDN bypass, vulnerability prioritization, phishing infrastructure planning, bug bounty submission, and client deliverable templates. Use when planning or executing reconnaissance against authorized targets, mapping an organization's external attack surface, investigating a person/entity, or producing client deliverables." -version: 2.2 +description: "Comprehensive OSINT methodology for external red-team operations and authorized attack-surface assessments. Covers the 5-stage recon pipeline (seed discovery, asset expansion, enrichment, exposure analysis, reporting), asset-graph discipline with 29 asset types, severity rubric (CRITICAL/HIGH/MEDIUM/LOW/INFO), confidence upgrade workflows, time budgeting, asset-level triage rules, scale-based tactics, identity-fabric mapping (Entra/Okta/ADFS/Google/SAML/M365 Teams+SharePoint+OAuth), API and auth-map methodology, JavaScript deep analysis, mobile attack surface, cloud attack surface, breach×identity correlation, detectability tagging, detection-aware probing (back-off, persona rotation), read-only validator discipline, WAF/CDN bypass + origin discovery, vulnerability prioritization (CVE/EPSS/KEV), phishing infrastructure planning + pretext development, bug bounty submission templates, client deliverable templates with risk translation, threat-actor investigation (incl. RU/CN pivots), cryptocurrency tracing, image/video forensics, chronolocation. Use when planning or executing reconnaissance against authorized targets, mapping an organization's external attack surface, investigating a person/entity, tracing crypto flows, geolocating media, or performing attribution work." +version: 2.3 triggers: - external recon - external red team @@ -26,6 +26,7 @@ triggers: - mobile recon - APK analysis - cloud bucket enumeration + - bucket enum - breach correlation - secret leak hunt - origin discovery @@ -48,10 +49,14 @@ triggers: - asset triage - detection-aware probing - back-off strategy + - persona rotation - OSINT methodology - open source intelligence - target profiling + - data correlation - OSINT workflow + - intelligence collection + - OSINT campaign - recon methodology - threat actor investigation - attribution @@ -59,107 +64,129 @@ triggers: # OSINT Methodology — External Red-Team Edition -## 0. When to Use / When NOT +## 0. When to use this skill / When NOT -**Use this skill when:** planning or executing authorized external recon (red team, bug bounty, ASM); mapping an org's attack surface; investigating a person/entity/threat-actor; producing client deliverables. +**Use this skill when:** +- Planning or executing external reconnaissance against an authorized target (red team, bug bounty in-scope, ASM engagement). +- Mapping an organization's external attack surface end-to-end (subdomains → assets → exposure → attack paths). +- Investigating a person, entity, or threat actor where evidence discipline matters. +- Tracing cryptocurrency flows, geolocating media, performing image/video forensics, or chronolocating events. +- Building a structured OSINT campaign that needs reproducibility, severity grading, and clean handoffs. +- Producing client-facing deliverables (exec summaries, technical reports, reproduction packages) from offensive engagements. -**Do NOT use this skill when:** the user needs active exploitation, post-exploitation, or malware dev; blue-team/detection content; or the target's authorization is unclear — surface the scope question first. +**Do NOT use this skill when:** +- The user is asking for active exploitation, post-exploitation, lateral movement, AD privilege escalation, malware development, or anything beyond reconnaissance — those are out of scope. +- The user is asking for blue-team / defensive content (SIEM rules, detection engineering) — different domain. +- The target's authorization is unclear and the user is asking you to act against a third-party asset they don't own — see §1 below; gently surface the scope question before proceeding. --- ## 1. Authorization & Legal Posture -Intended for assets the operator owns or has **written authorization** to assess. +This skill is intended for assets the operator owns or has written authorization to assess (red-team rules of engagement, bug-bounty in-scope assets, ASM contracts). -**Soft scope check** — when authorization isn't established, ask once: -> *"Quick scope check: is this a target you own or have written authorization to assess? I want to make sure we stay on the right side of the engagement boundary."* +**Soft scope check:** when a user asks you to act against a target whose authorization isn't established earlier in the conversation, ask once before proceeding: -Once asserted, don't re-ask. If the engagement type is stated ("pentest of acme.com under contract"), proceed. +> *"Quick scope check: is this a target you own or have written authorization to assess (e.g., a red-team engagement, in-scope bug-bounty asset, or your own infrastructure)? I want to make sure we stay on the right side of the engagement boundary."* -**Always-on guardrails:** -- Never weaken auth, rate limits, or safety controls on the target side. -- No destructive probes (SYN scans at line-rate, masscan, fuzzing) outside explicit `--aggressive` mode. -- Never paste real PII, credentials, session tokens, or API keys into cloud-hosted LLMs. -- Never act against assets outside documented scope, even "obviously related" ones. +Once authorization is asserted, proceed without re-asking. If the user explicitly states the engagement type (e.g., "this is for our pentest of acme.com under contract"), you don't need to ask again. + +**Always-on guardrails (regardless of authorization):** +- Never weaken auth, rate limits, banners, or any safety control that enforces scope on the target side. +- Never run destructive probes (true SYN scans on production, masscan at line rate, fuzzing/brute-force) outside an explicit DEEP / `--aggressive` mode. +- Never paste real PII, valid credentials, session tokens, API keys, or other secrets into cloud-hosted LLMs or third-party services. +- Never take action against assets outside the documented scope, even if "obviously related" (subsidiaries, vendors, employees' personal accounts, etc.). --- ## 2. Confidence Levels -Every assertion carries a confidence level. +Every assertion you make during an engagement should carry a confidence level. Three levels: -| Level | Meaning | -|---|---| -| **TENTATIVE** | Plausible from indirect evidence; unverified. Snippet-only dork match, email pattern inferred from name, single passive-source subdomain. | -| **FIRM** | Directly observed, uncorroborated. Subdomain resolves; Shodan banner returned; CT-log entry. | -| **CONFIRMED** | Multiple independent corroborations OR directly verified. Live-validated token; bucket listable; three-source subdomain convergence. | +| Level | Meaning | Examples | +|---|---|---| +| **TENTATIVE** | Plausible based on indirect evidence; unverified. | Snippet-only Google dork match; email pattern inferred from name; subdomain returned by one passive source only; favicon-hash overlap (two hosts share a favicon — could be shared infra, could be a coincidence). | +| **FIRM** | Directly observed but uncorroborated. | Subdomain that resolves to an IP; HEAD-confirmed bucket exists (private); CT-log entry shows certificate; Shodan banner returned. | +| **CONFIRMED** | Multiple independent corroborations OR directly verified. | Live-validated PMAK token (read-only `/me` returned 200); breach corpus + crt.sh + DNS all agree; bucket listable AND files retrievable; user enumerated AND password reset flow returns valid hint. | -**Rule of three for attribution:** 3 independent weak signals, OR 1 strong + 1 weak. Never single-source attribute. +**Rule of three for attribution:** require three independent weak signals, OR one strong + one weak, before asserting linkage. Don't single-source attribute. ### 2.1 Confidence Upgrade Workflows +Confidence isn't static — every TENTATIVE asset should have a documented path to FIRM and to CONFIRMED. Use these per-asset-type rules. + | Asset type | TENTATIVE → FIRM | FIRM → CONFIRMED | |---|---|---| -| Subdomain | ≥2 passive sources OR DNS resolves | Serves on a standard port AND banner/cert returned | -| IP | ≥2 sources (passive DNS, ASN, Shodan) | TCP SYN-ACK or ICMP reply | -| WebApp | URL extracted but not yet hit | HTTP returns 2xx/3xx/4xx AND content-length > 0 | -| Email | Name-pattern inferred OR snippet-only | Listed in Hunter/IntelX/breach, OR SMTP 250 (abort at DATA) | -| Bucket | Permutation candidate + HEAD returns 200/301/403 (exists) | GET listing = CONFIRMED | -| Credential / secret | Regex match in captured text | Read-only validator returns success (scope + account-ID documented) | -| Person | Name from single source | Confirmed by second independent source | -| SSO tenant | OIDC discovery endpoint returns metadata | Tenant GUID extracted AND domain ties back via MX/autodiscover/SP record | - -Default reporting posture: never claim CONFIRMED without explicit corroboration. When in doubt, downgrade. +| **Subdomain** | Returned by ≥2 independent passive sources, OR DNS A/AAAA/CNAME resolves successfully. | Serves on a standard port (80/443/22/etc.) AND HTTP banner / TLS cert / SSH banner returned. | +| **IP** | Discovered via ≥2 sources (passive DNS, ASN lookup, Shodan). | Active probe responds (TCP SYN-ACK on at least one port, or ICMP echo reply). | +| **WebApp** | URL extracted from JS / API / archive but not yet hit. | HTTP request returns 2xx/3xx/4xx (any non-network-error response) AND content-length > 0. | +| **Email** | Generated from a name pattern OR returned by snippet-only dork. | Listed in Hunter.io / EmailRep / IntelX / breach corpus, OR `MAIL FROM`/`RCPT TO` SMTP probe returns 250 (without delivery — abort at DATA). | +| **Bucket (S3/GCS/Azure)** | Permutation candidate; no probe yet. | HEAD returns 200, 301, or 403 (existence confirmed). Then CONFIRMED when GET returns object listing or known object retrieval. | +| **Endpoint (API / wayback)** | Extracted from JS regex / Wayback / Postman. | HTTP request returns non-404 (route exists). Then CONFIRMED when the endpoint's behavior is fingerprinted (auth posture, response shape, rate limits). | +| **Credential / secret** | Matches catalog regex in captured text. | Read-only validator (`/me`, `auth.test`, `sts:GetCallerIdentity`, `/user`) returns success. Then CONFIRMED with documented scope + account ID. | +| **Person** | Name extracted from a single source (LinkedIn / breach / GitHub commit). | Confirmed by a second source (Hunter.io role + LinkedIn profile, or two breach sources with same email). | +| **Repo** | Name match on org keyword in GitHub search. | Repo metadata shows confirmed org/email/website match. Then CONFIRMED when commit-history shows employee involvement. | +| **Mobile app** | Name match in app store. | Ownership-confidence score ≥70 (see companion skill §21). Then CONFIRMED when binary metadata (signing cert, package name, dev account) ties back to target. | +| **Certificate** | Returned by crt.sh once. | CT-log entry confirmed in ≥2 logs. Then CONFIRMED when serving on a discovered host. | +| **SSO tenant** | Discovery-endpoint returns OIDC metadata. | Tenant GUID extracted AND domain resolves through the tenant's expected MX / autodiscover / SP record. | + +**Default reporting posture:** never claim CONFIRMED without explicit corroboration. When in doubt, downgrade. Operators trust under-claims more than over-claims. --- -## 3. Output Format +## 3. Output Format Conventions -Each finding uses this schema (drops cleanly into asset-management tools): +When you produce findings during an active session, structure each finding to match the schema below — it drops cleanly into asset-management tools. ``` Finding: - id: - module: - asset_key: - category: - severity: - confidence: - title: - description: <2-5 sentences> + id: + module: + asset_key: + category: + severity: + confidence: + title: + description: <2-5 sentences> evidence: - url: - timestamp: - sha256: - raw: - references: [] - remediation: + url: + timestamp: + sha256: + raw: + references: + - + remediation: ``` -Always use UTC timestamps. +**Always use UTC timestamps**. Local time creates correlation bugs across notes/screenshots/logs. --- ## 4. Source Hygiene & Citations -For every artifact: **URL + UTC timestamp + SHA-256 + tool version + run_id**. +For every artifact you capture, record: **URL + UTC timestamp + SHA-256 hash + tool version + run_id**. -- Hash all downloads with SHA-256. Screenshot in PNG. -- Raw HTTP captures capped at 2 KiB body. JSONL logs, one line per event. +- Hash all downloaded files with SHA-256. +- Screenshot in PNG (lossless, smaller than full-page WARC for evidence packs). +- Capture raw HTTP requests/responses, capped at 2 KiB body to keep evidence packs small. +- Use JSONL (NDJSON) logs, one line per event, with a `run_id` so the entire engagement is replayable. - Separate evidence read-only from working copies; never edit captured artifacts. -- Prefer durable references (CVE, ATT&CK technique ID, RFC). If ephemeral, archive first (archive.today, Wayback SavePageNow). + +When citing a source in your output, prefer durable references (CVE, vendor advisory, ATT&CK technique ID, RFC) over ephemeral ones (a Twitter post, a forum thread). If the only source is ephemeral, archive it (archive.today, Wayback SavePageNow) before citing. --- -## 5. Do NOT +## 5. Do NOT (hard rules) -- Do NOT paste creds, session tokens, real PII, or unique pivots into cloud LLMs. Use local models for sensitive analysis. -- Do NOT assume vendor labels are ground truth (TRM, Chainalysis, Arkham can disagree). -- Do NOT assert ownership from a single signal (favicon hash, shared NS, shared CT issuer — each is a hypothesis). -- Do NOT run fuzzing, SYN scans, masscan, or `nuclei fuzzing/*` outside explicit `--aggressive` mode. -- Do NOT use a credential validator for anything except read-only verification. -- Do NOT mirror-image the threat actor. Separate capability from intent and sponsorship. -- Do NOT escalate when you hit active defenses — back off and document (§6.4). +- DO NOT paste creds, session tokens, API keys, real PII, infostealer logs, or unique pivots into cloud LLMs (ChatGPT, Claude.ai, Gemini, Perplexity). Use local models (Ollama, LM Studio, GPT4All) for sensitive analysis. +- DO NOT assume vendor labels are ground truth. Cross-label sanity: TRM, Chainalysis, Arkham can disagree. Treat every label as a hypothesis. +- DO NOT assume 1:1 bridge flows. Bridges/mixers/wrappers introduce mint/burn semantics; validate with on-chain proofs. +- DO NOT assert ownership from a single signal. Favicon-hash overlap, shared CT issuer, shared NS — each is a hypothesis. Need rule-of-three. +- DO NOT run fuzzing, SYN scans, masscan, or `nuclei fuzzing/*` templates outside an explicit DEEP / `--aggressive` mode. +- DO NOT use a credential validator to do anything except read-only verification (no create/delete/send). +- DO NOT mirror-image (assume the target thinks like you do). Separate capability from intent and sponsorship. +- DO NOT confuse correlation with control. +- DO NOT escalate when you encounter active defenses; back off and document (see §6.4). --- @@ -167,289 +194,1510 @@ For every artifact: **URL + UTC timestamp + SHA-256 + tool version + run_id**. ### 6.1 Sock Puppets -Build posting history, age the account, use a separate browser profile. Persona generation: Fake Name Generator, This Person Does Not Exist. Browser isolation: Firefox Multi-Account Containers. Disposable numbers for SMS verification. Audit every extension before install. Maintain chain-of-custody: timestamp every action, hash every artifact. +A sock puppet is a fake account that cannot be linked to you. Build a posting history, age the account, use it from a separate browser profile. -### 6.2 Detectability Tagging +Resources & techniques: +- Persona generation: [Fake Name Generator](https://www.fakenamegenerator.com/), [This Person Does Not Exist](https://thispersondoesnotexist.com/). +- Browser isolation: [Firefox Multi-Account Containers](https://addons.mozilla.org/firefox/addon/multi-account-containers/), or dedicated profiles per persona. +- Disposable phone numbers: Burner, Silent Link (some platforms reject VoIP — keep a backlog of numbers). +- Hardware passkeys for any high-value persona; store recovery codes offline. +- Audit every browser extension before installation. Supply-chain attacks on popular extensions have repeatedly targeted investigators — assume the popular ones are at higher risk, not lower. +- Maintain chain-of-custody: timestamp every action, hash every key artifact, record tool versions per case. +- Personas should look like real low-engagement accounts: profile photo (synthetic), bio, a few low-effort posts spread across weeks before the persona is "used." -Tag every operation so you can reason about the trail you leave. +References: +- [Effective Sock Puppets](https://medium.com/@unseeable06/creating-an-effective-sock-puppet-for-your-osint-investigation-95fdbb8b075a) +- [Ultimate Guide to Sock Puppets](https://osintteam.blog/the-ultimate-guide-to-sockpuppets-in-osint-how-to-create-and-utilize-them-effectively-d088c2ed6e36) + +### 6.2 Detectability & OpSec Tagging + +Every probe leaves a footprint. Tag every operation in your notes with a detectability level so you can reason about the SIEM trail you're leaving on the target's side. | Tag | Examples | |---|---| -| **Low** | Passive Shodan InternetDB; crt.sh; Wayback CDX; SecurityTrails PDNS; Hunter.io; HTTP HEAD on public buckets; `getuserrealm.srf`; OIDC metadata fetch. | -| **Medium** | `GetCredentialType` user-enum; Okta `/api/v1/authn` user-enum; credential validation; AWS `sts:GetCallerIdentity`; Swagger/GraphQL probes; targeted favicon-hash + JARM fingerprinting. | -| **High** | Active port scans (naabu/masscan/nmap); Nuclei full runs against production; subdomain brute-force at scale; SMTP `RCPT TO` enum; web fuzzing. | +| **Low** | Passive Shodan InternetDB; CT-log queries (crt.sh); Wayback CDX; passive DNS (SecurityTrails); Hunter.io email enrichment; HTTP HEAD on public buckets; `getuserrealm.srf`; Microsoft OIDC metadata fetch. | +| **Medium** | Microsoft `GetCredentialType` user-enum; Okta `/api/v1/authn` user-enum; Postman API key validation; AWS `sts:GetCallerIdentity` (logs to CloudTrail); Slack `auth.test`; full-page screenshots; Swagger/GraphQL probes against a 28/13-path wordlist; targeted favicon-hash + JARM fingerprinting. | +| **High** | Active port scans (naabu / masscan / nmap); Nuclei full template runs against production; subdomain brute-force at scale; APK download from third-party mirrors; deep-mode user enumeration past N attempts per tenant; SMTP `RCPT TO` enumeration; web fuzzing (ffuf/gobuster). | + +When working with a client, document the operations actually run and their detectability tag in the engagement report — clients appreciate knowing what their detection stack should have caught. -Defaults: passive by default. Active probes only when (a) explicitly authorized, (b) within agreed windows, (c) operator aware of log volume. +**Defaults:** passive by default. Active probes only when (a) explicitly authorized, (b) within agreed maintenance windows, and (c) with the operator's awareness of the resulting log volume. ### 6.3 Validator Discipline -When you find a credential in the wild, confirm liveness with **read-only validators only** (`/me`, `auth.test`, `sts:GetCallerIdentity`). Never create, modify, delete, or send. Record `checked_at` UTC + truncated response + scope/account-ID. Concrete validator endpoints for 9 providers live in `offensive-osint` §23. +When you discover a credential in the wild (a leaked API key, a sourcemap-exposed token, a hard-coded PMAK in a public Postman workspace), you may want to confirm it's live. Do this with **read-only validators only**. -### 6.4 Detection-Aware Probing +Discipline: +- Read-only endpoint only (e.g., `/me`, `/whoami`, `auth.test`, `sts:GetCallerIdentity`). +- Never use the validated credential to create, modify, delete, or send anything. +- Tag the validation attempt with detectability — every validator generates an audit-log entry on the provider side. +- Record `checked_at` (UTC), the response (truncated), and the scope/account-ID returned. +- If the operator's rules of engagement forbid validation, mark the credential `validation_skipped_by_policy` and stop. -**Signs you've been detected (escalating severity):** 429 / `Retry-After`; captcha interstitials; WAF block page; status-code drift (200→403 from your IP only); banner change; NXDOMAIN rollback; honeypot bait (credentials that don't validate); direct contact. +Concrete validator endpoints (Postman, AWS, GitHub, Slack, Anthropic, OpenAI, npm, Atlassian, DataDog) live in the companion `offensive-osint` skill. -**Back-off ladder:** -1. Halve concurrency; add 2–10s jitter. -2. Stop hitting the triggering path; pivot to a different module. -3. New User-Agent / TLS fingerprint. -4. Rotate egress IP (residential proxy, different cloud region). -5. Pause 1–24 hours. -6. If WAF block / status drift / direct contact: **stop and consult the engagement lead.** +### 6.4 Detection-Aware Probing (signs of detection + back-off) ---- +Your probes will eventually hit detection. Recognize the signs and back off **before** you trip an active response. -## 7. External Red-Team Recon Pipeline +**Signs you've been detected (in roughly increasing severity):** -Five sequential stages; modules within a stage can run concurrently. +1. **Rate-limit responses** — `429 Too Many Requests`, `Retry-After` header set, `X-RateLimit-Remaining: 0`. +2. **Captcha interstitials** — Cloudflare interstitial page, hCaptcha challenge, AWS WAF page. +3. **WAF page** — explicit "Access denied" with provider branding (Cloudflare, Akamai, Imperva, F5 ASM, AWS WAF, Sucuri). +4. **Status code drift** — endpoints that previously returned 200/401 now return 403 only from your IP. +5. **Banner change** — server header shape or response timing changes consistently. +6. **DNS poisoning back to NXDOMAIN** — target's authoritative servers stop resolving subdomains (probably their CDN took over). +7. **Honeypot bait** — endpoints that look too good (`/admin/db_dump.sql`, exposed `.env` with credentials that don't validate). Real exposures rarely look this clean. +8. **Direct contact** — your sock-puppet email gets a "we noticed unusual activity" message; or, in extreme cases, your IP gets a courtesy abuse-contact email. -| Stage | What you do | -|---|---| -| **1 — Seed Discovery** | WHOIS, ASN enum (HE BGP Toolkit, RIPEstat), DNS records (A/AAAA/MX/TXT/NS/SOA/CAA), CT history (crt.sh, Censys). | -| **2 — Asset Expansion** | Subdomain enum (passive first → permutations → brute); cloud bucket permutation; typosquat generation; Wayback CDX; mobile app discovery; DNS walking; LinkedIn employee enum. | -| **3 — Enrichment** | Port/service (Shodan InternetDB → naabu); TLS handshakes (cert chain, JARM, favicon mmh3); WAF/CDN inference; origin discovery; security headers; email harvest; email security audit; GitHub dorking; JS deep analysis; SSO/IdP fingerprinting; API discovery; secrets sweep (Postman, Stack Exchange); vendor product fingerprinting; container/CI-CD/cloud-native exposure; job posting harvest. | -| **4 — Exposure Analysis** | Nuclei always-on checks; TLS deep audit; breach × identity correlation → SSO_EXPOSURE findings; targeted misconfig probes (`.git/config`, `.env`, `/actuator/env`, `/_cat/indices`, `/console`); vulnerability prioritization (CVE × EPSS × KEV × POC). | -| **5 — Reporting** | Risk scoring per finding; asset graph export; client-facing report (exec summary + technical detail + remediation); reproduction package; bug bounty submission if applicable. | +**Back-off ladder:** -### 7.1 Pipeline Priority Order (highest signal density first) +1. **Slow down.** Halve your concurrency. Add 2–10s jitter between requests. +2. **Switch endpoints.** Stop hitting the path that triggered. Move to a different module of the recon pipeline. +3. **Switch persona.** New User-Agent (rotate among realistic browsers), new TLS fingerprint (different httpx/curl version). +4. **Switch IP.** Rotate to a new egress (residential proxy, Tor for sensitive lookups, a different cloud region). +5. **Pause.** Wait 1–24 hours. Many WAFs have rolling-window IP-based reputation; passive time often resets it. +6. **Document and consult.** If you've hit (3) WAF, (4) status drift, or (8) direct contact, **stop active probing and consult the engagement lead**. Continued probing past these signals risks scope violation. + +**Persona / IP rotation rules:** +- Never rotate persona to one that's been used in a prior engagement against the same target. +- Use residential proxies (Bright Data, Smartproxy, IPRoyal) for high-detectability work — but be aware they're sometimes IP-blocklisted by Cloudflare. +- Tor exit nodes are useful for **passive lookups** (CT logs, archive sites) but are blocked by most active-probe targets. +- Cloud egress IPs (AWS / GCP / Azure) are often blocklisted aggressively for recon. Use sparingly. +- Document every rotation with timestamp + reason; reviewers will ask. + +**Don't:** +- Don't try to "outsmart" a confirmed WAF block by sending more aggressive payloads. That's how clients get extra logs and how you get caught. +- Don't switch source IPs to evade an explicit block-list — that crosses into evasion territory and may breach the rules of engagement. +- Don't ignore signals because the dashboard says "still up." The probe is being silently logged; the response will come later. -1. **Breaches** — HudsonRock Cavalier + HIBP + DeHashed. Highest ROI; often yields plaintext corp SSO creds. -2. **GitHub recon** — code-search dorks. Fastest path to AWS keys, Slack tokens, JWT secrets. +--- + +## 7. External Red-Team Recon Pipeline + +A 5-stage pipeline for any authorized external assessment. Stages are sequential; modules within a stage can run concurrently. + +### Stage 1 — Seed Discovery +Establish the ground truth of who/what the target is. + +- WHOIS on the seed domain (registrant, dates, name servers). +- ASN enumeration: which AS does the org own/use? (Hurricane Electric BGP Toolkit, RIPEstat, BGPView.) +- DNS records (A/AAAA/MX/TXT/NS/SOA/CAA) — records-only, no walking yet. +- Certificate Transparency history for the root domain (crt.sh, Censys). + +### Stage 2 — Asset Expansion +Discover everything that might belong to the target. + +- Subdomain enumeration (passive sources first: crt.sh, VirusTotal, AlienVault OTX, Shodan, then permutations and bruteforce). +- Cloud bucket enumeration (S3/GCS/Azure permutations from company name + subdomain stems — see §15). +- Typosquat domain generation (dnstwist variants → resolve → WHOIS) — for both phishing risk and adjacent corp assets. +- Wayback CDX archive endpoints for forgotten paths. +- Mobile app discovery (Android via google-play-scraper, iOS via iTunes Search API — see §14). +- DNS deep walking (NSEC walk on misconfigured zones, AXFR opportunism). +- LinkedIn employee enumeration → email-pattern derivation. + +### Stage 3 — Enrichment +Add depth to the discovered assets. + +- Port + service detection (Shodan InternetDB free → naabu/masscan if authorized). +- Live TLS handshakes (cert chain, JARM, favicon mmh3 hash). +- Web tech detection (Wappalyzer-style ~600 signatures via httpx). +- WAF/CDN inference (header markers). +- Origin discovery if behind CDN (see §27). +- Security header audit. +- Bulk screenshots (triage 1000s of hosts visually). +- Email harvesting (6 parallel sources). +- Email security audit (SPF/DMARC/DKIM/BIMI/MTA-STS). +- GitHub code-search dorking (13 dork templates × 29+ secret regexes). +- JavaScript deep analysis (sourcemaps, secrets, endpoints, internal-host leakage). +- SSO/IdP tenant fingerprinting (Entra, Okta, ADFS, Google, SAML, M365 Teams/SharePoint/OAuth — see §11). +- API & auth-map discovery (Swagger/OpenAPI, GraphQL, Postman). +- Secrets-beyond-GitHub sweep (Postman public workspaces, Stack Exchange, Trello/Notion/Atlassian dorks). +- Vendor product fingerprinting (Citrix/F5/PaloAlto/Pulse/Fortinet/Cisco/VMware/Exchange). +- Container / CI-CD / cloud-native exposure check. +- Job posting harvest for tech-stack inference. + +### Stage 4 — Exposure Analysis +Convert assets into findings. + +- Nuclei (15 always-on built-in checks + optional binary). +- TLS deep audit (sslyze / testssl.sh). +- Breach × identity correlation (HudsonRock Cavalier, HIBP, DeHashed, IntelX, local corpus → SSO_EXPOSURE findings). +- Targeted misconfiguration probes (`.git/config`, `.env`, `phpinfo.php`, `/actuator/env`, `/actuator/heapdump`, `_cat/indices`, `/console`, `/manager/html`). +- Vulnerability prioritization (CVE × EPSS × CISA KEV × public-POC availability — see §28). + +### Stage 5 — Reporting +Make the work usable. + +- Risk scoring per finding (CVSS + program-specific weights). +- Asset graph export (D3-friendly nodes/links, GraphML, JSON). +- Client-facing report (executive summary + technical detail + remediation — see §31). +- Reproduction package (run_id, tool versions, raw evidence, JSONL log). +- Bug bounty submission (if applicable — see §30). + +### 7.5 Pipeline Priority Order (highest signal density first) + +When budget is constrained, work in this order: + +1. **Breaches** — infostealer logs (HudsonRock Cavalier free tier) + HIBP + DeHashed. Highest ROI for red teams; often gives valid plaintext creds for corp SSO. Requires emails as input. +2. **GitHub recon** — code-search dorks. Finds AWS keys, Slack tokens, JWT secrets, `.env` files. Fastest path to cloud pivot. 3. **Nuclei misconfig sweep** — exposed admin panels, CVEs with public POCs. -4. **Cloud buckets** — listable = CRITICAL. -5. **Ports** — Shodan InternetDB first. VPN concentrators, RDP, Jenkins, Elasticsearch are high-value pivots. +4. **Cloud buckets** — permutate company name + subdomain stems. Listable bucket = CRITICAL. +5. **Ports** — Shodan InternetDB first (free, keyless). VPN concentrators, RDP, Jenkins, GitLab-CE, Elasticsearch are the high-value pivot points. 6. **Email OSINT** — feeds breaches; feeds phishing list. -7. **Web tech / WAF / screenshots** — triage thousands of hosts. -8. **Wayback** — archived JS for hard-coded keys; removed admin/dev paths. -9. **DNS deep + email security** — SPF/DMARC gaps enable spoofing; TXT tokens reveal SaaS tenancies. -10. **Certificates → TLS** — CT timeline catches forgotten subdomains; weak ciphers = cheap findings. +7. **Web tech / WAF / screenshots** — triage thousands of hosts; know the stack before probing. +8. **Wayback** — archived JS often has hard-coded keys; archived endpoints reveal removed admin/dev paths. +9. **DNS deep + email security** — SPF/DMARC gaps enable email spoofing; TXT verification tokens reveal SaaS tenancies. +10. **Certificates** — CT-log timeline catches forgotten subdomains; weak ciphers = cheap findings. 11. **ASN + reverse DNS** — corporate IP space hosts unadvertised infra. -12. **Typosquats** — registered = finding; unregistered = phishing shortlist. +12. **WHOIS** — registrant PII reveals adjacent corp assets. +13. **Typosquat** — actively-registered squats are findings; unregistered ones go on the phishing-domain shortlist. +14. **Security headers** — low standalone value but required for client reports. + +### 7.6 Time Budgeting & Engagement Profiles -### 7.2 Time Budgeting & Engagement Profiles +Stage and asset count drive how long a recon takes. Rough estimates (single operator on a typical SaaS-style target): -| Stage | Small org (<100) | Medium (100–1K) | Large (1K+) | +| Stage | Small org (<100 employees) | Medium (100–1K) | Large (1K+) | |---|---|---|---| -| 1. Seed | 30 min | 30 min | 30 min | +| 1. Seed discovery | 30 min | 30 min | 30 min | | 2. Asset expansion | 1–2 h | 2–4 h | 4–8 h | | 3. Enrichment (per 100 alive webapps) | ~1 h | ~1 h | ~1 h | | 4. Exposure analysis | 1–3 h | 3–6 h | 6–12 h | | 5. Reporting | 2–4 h | 4–8 h | 1–2 days | -**Profiles:** 1-hour rapid (Stages 1–2 passive + breach + exec summary) · 4-hour focused (adds email harvest, SSO fingerprinting, typosquats) · 1-day standard (full Stages 1–4 in priority order) · 1-week deep (all of standard + JS deep, mobile, cloud-native, vendor product, package registry) · ongoing weekly diff (re-run Stages 1–3, diff against baseline). +**Engagement profiles:** + +- **1-hour rapid recon ("how exposed is X?")** — Stage 1 (15 min) → passive subdomain (crt.sh + Subfinder, 10 min) → Shodan InternetDB on resolved IPs (5 min) → email harvest via Hunter+IntelX (10 min) → breach lookup on emails (10 min) → executive-summary-only output (10 min). +- **4-hour focused recon ("phish-readiness check")** — adds: full email harvest, LinkedIn employee enum, SPF/DMARC analysis, typosquat candidate generation, SSO/IdP fingerprinting. Output: phishing-feasibility report + target email list. +- **1-day standard recon** — full Stages 1–4 with the priority order above. Output: per-asset finding list + asset graph + exec summary. +- **1-week deep recon** — all of standard, plus: deep-mode user enumeration, JS deep analysis at full budget, mobile attack surface, cloud-native fingerprinting, vendor product fingerprinting, package registry leak hunting, vulnerability prioritization. Output: full client deliverable package + reproduction bundle. +- **Ongoing monitoring (weekly diff)** — re-run Stages 1–3 weekly; diff against baseline; alert on new asset / new finding / asset disappeared. -**Abort conditions:** scope mismatch after Stage 1; near-zero attack surface after Stage 2; WAF/detection signs hit during any stage (§6.4). +**When to abort early:** +- After Stage 1 if scope is wrong (target turns out to be subsidiary of unrelated corp; rules of engagement need clarification). +- After Stage 2 if attack surface is below threshold (no public webapps + no exposed services + no leaked emails → little to find externally). +- During any stage if you hit the WAF / detection signs in §6.4. --- ## 8. Asset Graph Discipline -Every discovery is a **typed asset** in a graph, not a free-floating string. +Treat every discovery as a typed asset in a graph, not a free-floating string. -### 8.1 Asset Taxonomy +### 8.1 Asset Taxonomy (29 types) -| Category | Types | +| Category | Asset Types | |---|---| -| DNS / Network | `domain`, `subdomain`, `ip`, `netblock`, `asn` | -| Service | `port`, `service`, `certificate` | -| Identity | `email`, `person`, `credential` | -| Code / Config | `repo`, `secret` | -| Cloud / Storage | `bucket`, `firebase_project` | -| Web | `webapp`, `wayback_endpoint`, `api_endpoint`, `api_spec`, `graphql_schema` | -| Mobile | `mobile_app`, `deep_link`, `exported_component` | -| Phishing | `typosquat_domain` | -| SaaS | `postman_collection`, `postman_workspace`, `postman_api_key`, `stack_post`, `saas_public_surface` | +| **DNS / Network** | `domain`, `subdomain`, `ip`, `netblock`, `asn` | +| **Service** | `port`, `service`, `certificate` | +| **Identity** | `email`, `person`, `credential` | +| **Code / Config** | `repo`, `secret` | +| **Cloud / Storage** | `bucket`, `firebase_project` | +| **Web** | `webapp`, `wayback_endpoint`, `api_endpoint`, `api_spec`, `graphql_schema` | +| **Mobile** | `mobile_app`, `deep_link`, `exported_component` | +| **Phishing / Adversarial** | `typosquat_domain` | +| **Collaboration / SaaS** | `postman_collection`, `postman_workspace`, `postman_api_key`, `stack_post`, `saas_public_surface` | -Every asset carries: `type`, `key` (typed dedup id), `value`, `sources[]`, `confidence`, `first_seen`, `last_seen`, `attrs{}`. +### 8.2 Asset Schema -**Discipline:** create the asset first, then attach the finding. Dedup by key. `sources[]` must list every source. Confidence is per-source, then aggregated. +Every asset carries: +- `type` — one of the 29 above. +- `key` — unique dedup id (typed prefix, e.g. `sub:api.example.com`, `email:alice@example.com`). +- `value` — the actual string/object. +- `sources[]` — every source that confirmed this asset (deduplicated). +- `confidence` — TENTATIVE / FIRM / CONFIRMED. +- `first_seen`, `last_seen` — UTC timestamps. +- `attrs{}` — type-specific metadata (e.g., for a `webapp`: status_code, title, tech-stack list, JARM, favicon mmh3, screenshot path). -### 8.2 Asset-Level Triage Rules +### 8.3 Edge Taxonomy -**WebApp priority (highest first):** auth (`auth.`, `login.`, `sso.`) → admin paths → dev/staging hosts → API (`api.`, `gateway.`) → customer-facing (`portal.`, `app.`) → marketing. +Relationships are typed edges, not text: +`RESOLVES_TO`, `HOSTED_ON`, `IN_NETBLOCK`, `BELONGS_TO_ASN`, `LISTED_IN_CERT`, `OWNED_BY`, `ALIAS_OF`, `BREACHED_FROM`, `EMPLOYED_BY`, `HOSTS_REPO`, `TYPOSQUAT_OF`, `EXPOSES`, `DOCUMENTED_BY`, `BELONGS_TO_HOST`, `REQUIRES_AUTH`, `LEAKS_SCHEMA`, `SHIPPED_BY_ORG`, `CONTAINS_SECRET`, `TALKS_TO_HOST`, `EXPOSES_DEEPLINK`, `HAS_EXPORTED_COMPONENT`, `USES_FIREBASE_PROJECT`, `LACKS_PINNING_FOR`. -**Email priority:** exec (CEO/CFO/CISO) → IT/helpdesk/security → dev/engineer/DBA → sales/HR/finance → generic role accounts. +### 8.4 Discipline rules -**Repo priority:** recently pushed (last 30 days) > stale; public with target name in description > code-only; mentions `prod`/`internal`/`secret` in name → HIGH priority despite being public. +- **Every discovery is an asset.** Don't write findings against free-floating strings; create the asset first, then attach the finding. +- **Dedup by key, not by value.** Same value, different type ≠ same asset (`sub:api.example.com` and `webapp:https://api.example.com/` are different assets with a `BELONGS_TO_HOST` edge). +- **Provenance is non-negotiable.** `sources[]` must list every source. If two sources confirmed it, both go in. +- **Confidence is per-source, then aggregated.** A subdomain returned by 3 passive sources is FIRM; one returned by snippet-only Bing is TENTATIVE. +- **Late binding via sidecars.** When module A produces output that module B needs, write a JSON sidecar (`mobile_endpoints.json`, `secrets_sidecar.json`) — don't block module B on module A. See §24. ---- +### 8.5 Asset-Level Triage Rules -## 9. Findings Rubric & Severity Mapping +When you have a mixed bag of assets and limited probe budget, prioritize by what each asset *enables*: -### Severity Anchors +**WebApp priority by hostname signal (highest first):** -| Severity | Anchor | -|---|---| -| **CRITICAL** | Pre-auth code execution; confirmed valid credentials; listable production data; fundamental trust violations. Examples: `.env` exposed, listable S3 bucket with PII, live-validated AWS admin key, open Kubernetes API with anon-auth, ≥10 employees in breach corpus + tenant identified. | -| **HIGH** | Significant exposure with clear escalation path; high-value info disclosure. Examples: public secret in GitHub repo, subdomain takeover possible, reflected CORS with credentials, exposed Jenkins/phpMyAdmin admin UI, open GraphQL introspection on prod, DMARC `p=none`. | -| **MEDIUM** | Info disclosure, hardening gaps, brute-force exposure. Examples: missing HSTS/CSP, Apache `/server-status`, internal IP/hostname in JS, schema leakage in error pages, `android:allowBackup=true`, wildcard CORS on user-data API, Slack webhook leaked. | -| **LOW** | Cosmetic or marginal gaps. Examples: missing `X-Frame-Options`, `.DS_Store` exposed, Stripe **test** key, cert pinning missing, outdated WordPress (no known active exploit). | -| **INFO** | Worth recording; no immediate action. Examples: `robots.txt` reveals paths, private bucket locked down, DNSSEC not enabled. | +1. Auth-related hostnames (`auth.`, `login.`, `sso.`, `idp.`, `accounts.`, `oauth.`). +2. Admin paths (`/admin`, `/dashboard`, `/console`, `/manager`, `/wp-admin`, `/phpmyadmin`). +3. Dev/staging hosts (`dev.`, `staging.`, `stg.`, `qa.`, `uat.`, `test.`, `sandbox.`, `preprod.`, `preview.`) — lower defenses, often dump prod data. +4. API hostnames (`api.`, `services.`, `gateway.`, `graph.`). +5. Customer-facing hostnames (`portal.`, `app.`, `my.`, `account.`). +6. Marketing / content (`www.`, `blog.`, `news.`, `careers.`, `support.`). + +**Subdomain priority by inferred function:** + +- API > Admin > Dev > Auth > Prod-app > Marketing. + +**IP priority by netblock:** + +- Corporate ASN-owned (most likely to host unadvertised internal infra). +- Cloud netblocks (AWS / GCP / Azure / DO / OVH) — high turnover but interesting for cloud-native services. +- CDN ranges (Cloudflare / Akamai / Fastly) — usually edge, not origin; defer unless doing origin discovery. + +**Email priority by role hint:** + +| Role indicator | Priority | Why | +|---|---|---| +| `ceo@`, `cfo@`, `cto@`, `ciso@` | HIGHEST | Exec accounts have highest breach value (BEC, finance authority, board access). | +| `it@`, `helpdesk@`, `support@`, `security@` | HIGH | IT/security accounts have privileged tool access; helpdesk accounts handle reset workflows. | +| `dev`, `engineer`, `architect`, `dba` | MEDIUM | Developer accounts often have GitHub / cloud / CI access. | +| `sales`, `marketing`, `hr`, `finance` | MEDIUM | SaaS access (Salesforce, HubSpot, Workday); finance enables BEC. | +| Generic role accounts (`info@`, `noreply@`, `contact@`) | LOW | Often unmonitored or alias forwarded; less personal context. | + +**Repo priority by recency + naming:** + +- Recently-pushed (last 30 days) > stale. +- Public repo with target name in description > target name only in code. +- Forked from internal-looking parent > standalone. +- Mentions `prod`, `internal`, `private`, `secret` in name → priority HIGH despite being public (may be misnamed or accidentally exposed). + +**Application order:** when you have N assets and budget for M probes (M < N), apply asset-type priority first, then within-type priority. E.g.: 50 subdomains → probe API + admin + dev first (~15), then auth + prod-app (~20), defer marketing/content to a later pass. -### Severity Escalation Rules +--- + +## 9. Findings Rubric & Severity Mapping + +Severity is operational, not subjective. Use these anchors: + +### 9.1 CRITICAL + +Pre-auth code execution, confirmed valid credentials, listable production data, fundamental trust violations. + +Examples: +- `.git/config` exposed on production webapp (full source-code disclosure). +- `/.env` exposed (credentials in plaintext, often DB / cloud / API). +- Spring Boot `/actuator/env` or `/actuator/heapdump` reachable unauthenticated. +- Listable S3 / GCS / Azure bucket containing user data. +- Unauthenticated POST/PUT/DELETE to a write endpoint that mutates state. +- Open Firebase Realtime Database (`https://{project}.firebaseio.com/.json` returns data). +- `android:debuggable=true` in a production Android app. +- Live-validated credential (PMAK, AWS key, Anthropic/OpenAI key) with broad scope. +- ≥10 employees compromised in a breach corpus + their tenant identified (SSO_EXPOSURE). +- Open Elasticsearch cluster (`/_cat/indices` returns data). +- Open Docker API (`/v1.40/containers/json` returns containers). +- Open Redis (no AUTH; can write `authorized_keys`). +- Open Kubernetes API server with anonymous-auth enabled. +- Open kubelet on 10250 (pod exec without auth). +- Open etcd on 2379 (cluster state and secrets). +- BlueKeep-vulnerable RDP, EternalBlue-vulnerable SMB. +- Citrix Netscaler / F5 BIG-IP with version-specific RCE CVE. + +### 9.2 HIGH + +Significant exposure but not yet RCE; clear path to escalation; high-value information disclosure. + +Examples: +- Public secret in a GitHub repo (PAT, AWS key, Slack token, etc.). +- Sourcemap (`.js.map`) accessible — full original-source disclosure of frontend. +- Open GraphQL introspection on production (full schema leaked → mutations to enum). +- Subdomain takeover possible (CNAME points to unclaimed Heroku/Shopify/etc.). +- Reflected CORS with credentials (`Access-Control-Allow-Origin: ` + `Access-Control-Allow-Credentials: true`). +- Verb tampering: hidden DELETE/PATCH on an endpoint that publicly only allows GET. +- Missing HSTS on a sensitive path (`/login`, `/sso`, `/admin`, `/auth`) — escalated from MED. +- Exposed Jenkins/Tomcat-Manager/phpMyAdmin admin UI (no auth or default creds). +- Telnet (port 23) reachable. +- WebView with JS bridge in a mobile app (XSS → RCE potential). +- Sensitive deep-link handler in a mobile app. +- DMARC policy `p=none` on production sending domain (spoof-feasible). +- Vendor product banner with known unpatched CVE (KEV-listed). + +### 9.3 MEDIUM + +Information disclosure, hardening gaps, brute-force exposure. + +Examples: +- Missing security headers on standard pages: HSTS, CSP. +- Apache `/server-status` or `/server-info` reachable. +- `phpinfo()` or `/info.php` reachable on dev/staging only. +- Internal IP / hostname / K8s service DNS leaked in JS. +- Schema leakage in error pages (stack traces, ORM signatures). +- `android:allowBackup=true` in Android app. +- `android:usesCleartextTraffic=true` in Android app. +- Exported activity/service without `android:permission` protection. +- Missing rate-limit on an API endpoint. +- Wildcard CORS (`Access-Control-Allow-Origin: *`) on an API that returns user-tied data (no creds). +- Slack webhook URL leaked. +- Twilio Account SID leaked (without auth token). +- SPF record permissive (`+all` or many includes). + +### 9.4 LOW + +Cosmetic or marginal hardening gaps. + +Examples: +- Missing `X-Frame-Options`. +- Missing `X-Content-Type-Options`. +- `.DS_Store` exposed. +- Stripe **test** key leaked. +- Firebase URL exposed (URL only, no open RTDB). +- Certificate pinning missing in mobile app. +- Outdated WordPress install detected (no known exploit yet). +- BIMI not configured (brand impersonation risk only). + +### 9.5 INFO + +Worth recording, no action required immediately. + +Examples: +- Missing `Referrer-Policy` / `Permissions-Policy`. +- Discovered `/.well-known/security.txt`. +- `robots.txt` reveals interesting paths. +- Private bucket exists but is locked down. +- Domain detected in a breach corpus with 0 employee accounts. +- DNSSEC not enabled. + +### 9.6 Severity escalation rules - HSTS missing on auth/login/SSO/admin path → **MED → HIGH**. -- Wildcard CORS + credentials header → **MED → HIGH**. -- Endpoint interest score ≥70 (companion skill §20) → at least **HIGH**. -- Domain breach ≥10 employees → **CRITICAL** regardless of stale-data caveats. -- Vendor product version matches CISA KEV → **CRITICAL**. +- Wildcard CORS + credentials → **MED → HIGH**. +- Wildcard CORS + sensitive endpoint → **LOW → MED**. +- API endpoint with score ≥70 on the interest rubric (companion skill §20) → at least **HIGH**. +- Domain breach severity ≥10 employees → **CRITICAL** regardless of stale-data caveats. +- Vendor product version matches CISA KEV entry → **CRITICAL**. +- DMARC `p=reject` + SPF strict + DKIM rotated → no escalation; well-postured. --- -## 10. Pivot Modes & Scale Tactics +## 10. Bug-Bounty / Red-Team Pivot Modes + +Existing investigative work (threat-actor research, doxxing investigations, attribution) operates under different posture than offensive recon. Switch posture explicitly. | Aspect | Investigative Mode | Offensive Recon Mode | |---|---|---| -| Probing rate | Slow, single-threaded, blends with traffic | Bursts, parallel, rate-limited per provider | -| OpSec posture | Sock-puppet only; never reveal investigator | Engagement persona; team may notify SOC | -| Evidence handling | Court-grade chain of custody | Engagement-grade; same hash/timestamp discipline | -| Reporting format | Narrative + sourced timeline | Per-asset findings + remediation + reproduction | +| **Probing rate** | Slow, single-threaded, blend with normal traffic. | Bursts, parallel, but rate-limited per provider. | +| **OpSec posture** | Sock-puppet only, never reveal investigator. | Persona may be the engagement persona; team may notify SOC. | +| **Evidence handling** | Court-grade chain of custody; hashes, timestamps, screenshots. | Engagement-grade; same hashing/timestamp discipline but evidence is for the client report. | +| **Severity in scope** | All severity levels relevant for context. | CRIT/HIGH/MED matter; LOW/INFO often dropped from exec summary. | +| **Authorization posture** | Public-record / OSINT-only; no probing private resources without authorization. | Written rules of engagement; explicit scope; explicit out-of-scope list. | +| **Reporting format** | Narrative + sourced timeline. | Per-asset findings + remediation + reproduction steps. | +| **Stop conditions** | When the question is answered. | When the engagement window closes OR when the report is delivered. | + +When you're working with the user, ask which mode they're in if it's unclear from context. + +### 10.1 Scale-Based Tactics + +Org size shapes which techniques pay off. + +**Small org (< 100 employees):** +- Executive accounts disproportionately matter; one CEO/CFO compromise often hands you the keys. +- Email harvest is small enough to enumerate exhaustively (10–50 emails total). +- Likely Microsoft 365 or Google Workspace; identity fabric is one tenant. +- Code repos often public on GitHub under personal accounts (founders moved from solo dev). +- Cloud presence often single-account AWS or GCP project. +- Tactics: deep on every email + every identity-fabric finding; full LinkedIn enum; check founders' personal GitHub orgs. + +**Medium org (100–1K):** +- Balanced enumeration. Email list is enumerable but not exhaustive. +- Identity fabric likely one IdP but with multiple SaaS tenants (Slack workspace, Notion org, GitHub org). +- Mobile apps possible; check both stores. +- Cloud presence multi-account or multi-region. +- Tactics: full pipeline at standard depth; sample-and-deepen on each asset class; LinkedIn priority by role. + +**Large org (1K–10K):** +- Email enum becomes lossy (sample top roles); breach hits scale up. +- Multi-tenant identity fabric (often Entra + Okta + multiple Auth0 customers). +- Mobile apps, multiple Android packages from different teams. +- Cloud presence sprawling; subsidiaries / acquisitions complicate scope. +- Tactics: breadth-first; rely on automation for asset discovery; manual triage on findings. + +**Very large org (10K+) or conglomerate:** +- Brand-pivot maps before anything else: enumerate every brand domain, every subsidiary. +- Breach corpus dominates: 10K+ employees mean significant past-breach exposure. +- Identity fabric may differ per business unit (legal entity boundaries). +- Tactics: scope pruning is the most important step; sampling + automation throughout; deep dive only on high-priority findings. + +**Cross-scale principle:** the smaller the org, the more individual-account focus pays off. The larger the org, the more systemic posture findings (DMARC gaps, SSO_EXPOSURE breadth, vendor-product version sweeps) pay off. + +--- + +## 11. Identity Fabric Mapping + +An organization's IdP/SSO posture is a high-value target: compromise the identity fabric and you don't need to break into individual apps. Map it methodically. + +### 11.1 Subdomain prefix enumeration + +Probe these prefixes against the target's root domain (and any sibling brand domains discovered): + +``` +auth.{domain} +login.{domain} +sso.{domain} +idp.{domain} +iam.{domain} +identity.{domain} +accounts.{domain} +oauth.{domain} +``` + +Plus generic OIDC discovery on every alive subdomain: +``` +{any-host}/.well-known/openid-configuration +``` + +### 11.2 Microsoft Entra (Azure AD) + +- **OIDC metadata + tenant GUID extraction** — fetch `https://login.microsoftonline.com/{tenant-or-domain}/.well-known/openid-configuration`. The `issuer` field returns a URL containing the tenant GUID (8-4-4-4-12 hex format). Tenant GUID + domain = stable tenant fingerprint. +- **getuserrealm.srf** — `https://login.microsoftonline.com/getuserrealm.srf?login=@` returns NameSpaceType: `Managed` (cloud-native), `Federated` (on-prem ADFS / external IdP), or `Unknown`. Detectability: low. +- **Autodiscover v2** — `https://autodiscover-s.outlook.com/autodiscover/metadata/json/1` POST with email; detects tenant membership. +- **GetCredentialType** (deep-mode user-enum) — `https://login.microsoftonline.com/common/GetCredentialType` POST `{"username": ""}`. Response indicates whether email exists in tenant. Detectability: medium. Cap attempts at 20 per tenant. + +### 11.3 Okta + +- **Org slug derivation** — derive candidate slugs from subdomains + root domain stem; Okta tenants live at `.okta.com` (or `.oktapreview.com`). +- **OIDC fingerprint** — `https://.okta.com/.well-known/openid-configuration`. +- **/api/v1/authn user-enum** (deep-mode) — POST `{"username": "", "password": "invalid"}`. 400 vs 401 response code indicates user existence. Detectability: medium. Cap at 20 per tenant. + +### 11.4 ADFS + +- **Passive fingerprint** — GET `https://{domain}/adfs/idpinitiatedsignon.aspx` → 200 indicates ADFS present. +- **Active mex endpoint** (deep-mode) — `https://{domain}/adfs/Services/Trust/mex` returns SOAP metadata. + +### 11.5 Google Workspace + +- `https://{domain}/.well-known/openid-configuration` — Google-hosted-domain customers expose discovery endpoints with characteristic issuer/JWKS URIs. +- MX records pointing to `*.googlemail.com` / `aspmx.l.google.com` is a strong Google Workspace signal. + +### 11.6 Generic OIDC (Keycloak / Auth0 / Ping / OneLogin / Duo) + +- Probe every alive subdomain for `/.well-known/openid-configuration`. +- The `issuer` and `authorization_endpoint` fields fingerprint the IdP product. +- `*.auth0.com`, `*.onelogin.com`, `*.pingone.com`, `*.duosecurity.com` patterns are characteristic. + +### 11.7 SAML metadata + +Probe these paths on every alive webapp: + +``` +/saml/metadata +/FederationMetadata/2007-06/FederationMetadata.xml +/federationmetadata/2007-06/federationmetadata.xml +/simplesaml/saml2/idp/metadata.php +/auth/saml2/metadata +``` + +SAML metadata XML contains: `EntityID`, signing certs, `SingleSignOnService` URL, `NameIDFormat`. + +### 11.8 AWS account-ID extraction + +- **S3 bucket region header** — HEAD on a known target bucket returns `x-amz-bucket-region`; correlate with bucket-name entropy to infer account. +- **ARN regex in JSON / HTML responses** — search for `arn:aws:[a-z0-9-]+:[a-z0-9-]*:([0-9]{12}):` (the 12-digit AWS account ID is the capture group). +- **`AccountId` property in JS / API responses** — common in IAM-related error messages and CloudFormation outputs. +- **OAuth client_id leaks** — Google OAuth: `-.apps.googleusercontent.com`; MSAL: GUID in `clientId` property. + +### 11.9 Output + +Each discovered IdP becomes a `Service` asset with `attrs.product`, `attrs.tenant_id`, `attrs.discovery_endpoint`. Then in Stage 4, correlate with breach data: every compromised user under a discovered tenant becomes an SSO_EXPOSURE finding (CRITICAL — see §22.3). + +### 11.10 Microsoft 365 Deep Surface + +Beyond plain Entra fingerprinting, M365 exposes a wider attack surface that's worth enumerating in depth. + +**Teams Federation:** + +- `https://login.microsoftonline.com//.well-known/openid-configuration` confirms tenant. +- Teams federation status: `https://teams.microsoft.com/api/mt//beta/users//externalsearchv3` (requires authenticated request from a federated tenant; useful for confirming whether external Teams chat is allowed). +- **External chat enabled** = soft-attack surface (vishing, smishing, "from-IT" pretexts via Teams chat). +- **Open Federation** (any tenant can chat) is the default; check whether the target restricted it. + +**SharePoint subdomains:** + +- `.sharepoint.com` — main tenant SharePoint. +- `-my.sharepoint.com` — OneDrive-for-Business URLs (per-user personal sites). +- `-admin.sharepoint.com` — SharePoint admin center (auth-required, but presence confirms tenancy). +- Where `` is derived from the company name (often the part before `.com`). + +**OneDrive personal site enumeration:** + +- Per-user OneDrive URL: `https://-my.sharepoint.com/personal//Documents/`. +- Replace `@` with `_` and `.` with `_` in the email (e.g., `alice@acme.com` → `alice_acme_com`). +- Authenticated probe; useful for confirming whether the OneDrive personal site has been provisioned (which itself is a presence indicator). + +**M365 OAuth client_id discovery:** + +- Many internal apps register OAuth client_ids in Entra. Search JS bundles, mobile-app strings, and API responses for `client_id=` patterns. +- Microsoft's well-known first-party client_ids (for Office, Graph, etc.) are documented; finding non-Microsoft GUIDs reveals custom internal apps. +- The endpoint `https://login.microsoftonline.com//v2.0/.well-known/openid-configuration` lists supported endpoints; some tenants leave `device_authorization_endpoint` enabled (device-code phishing target). + +**Power Platform / Power Apps:** + +- `https://make.powerapps.com/environments` (auth-required); environment IDs sometimes leak in URLs. +- `*.crm.dynamics.com` (Dynamics 365 / Power Apps default URLs). +- `*.azurewebsites.net` for App Service deployments. + +**M365 OAuth misconfig findings to look for:** + +- `device_authorization_endpoint` enabled on `common` tenant (device-code phishing target) → **MEDIUM** (operational risk; not directly exploitable but enables attack). +- Custom OAuth app with `Public client` flow enabled and broad scopes (offline_access, Mail.Read, Files.Read.All) → **HIGH** if app is approved for the tenant. +- Multi-tenant OAuth app published by the target (others can consent) → check whether scopes include sensitive Graph permissions. + +**Detectability:** all M365 endpoint probes log to Entra sign-in logs / audit logs (medium-low for fetch-only; medium for any auth attempt). + +--- + +## 12. API & Auth-Map Methodology -**Scale tactics:** -- **Small (<100):** Individual-account focus. One exec/CFO compromise often hands you the keys. Deep on every email + every identity-fabric finding. Check founders' personal GitHub orgs. -- **Medium (100–1K):** Balanced enumeration. Full pipeline at standard depth. LinkedIn priority by role. Check both app stores. -- **Large (1K–10K):** Breadth-first; automation for asset discovery; manual triage on findings only. -- **Very large / conglomerate:** Scope pruning is the most important step. Brand-pivot map first. Breach corpus and systemic posture findings (DMARC gaps, SSO_EXPOSURE breadth) dominate over individual accounts. +Modern targets expose REST, GraphQL, and undocumented internal APIs. The OSINT goal is to enumerate them, classify them, and rank by attack interest. + +### 12.1 Discovery paths + +- **Swagger / OpenAPI** — probe a 28-path wordlist (companion skill §16.1) on every alive webapp. Parse YAML/JSON; extract every endpoint (method + path). +- **GraphQL** — probe a 13-path wordlist (companion skill §16.2). POST a standard introspection query. If schema returns, you have full type/query/mutation/subscription enumeration. +- **GraphQL when introspection is disabled** — fall back to field-suggestion enumeration (companion skill §22.9). +- **Postman** — query Postman's public universal-search endpoint with the target name; walk each matching workspace; extract requests, headers, pre-request scripts, test scripts, env vars. +- **JS-extracted endpoints** — every endpoint extracted from JavaScript bundles feeds into the same classifier. +- **Mobile-extracted endpoints** — every endpoint from APK static analysis feeds in via sidecar (`mobile_endpoints.json`). + +### 12.2 Classification + +For each endpoint, capture: + +``` +url, method, source[], auth_required, auth_type, auth_location, +rate_limited, cors_policy, sensitive_path_keywords[], schema_leaks, +verb_tampering_possible, interest_score (0..100), interest_reasons[] +``` + +How to determine each field: +- Send `OPTIONS` → `Allow` header reveals supported methods (verb tampering check). +- Send `GET` without auth → 200 = `auth_required=false`; 401/403 = `auth_required=true`. +- Capture response headers for `WWW-Authenticate` (auth_type), `RateLimit-*` / `X-RateLimit-*`. +- Send a request with `Origin: https://attacker.example` → response `Access-Control-Allow-Origin` reflected = `cors_policy=reflected`. +- Trigger an error → check response for stack traces, ORM hints. + +### 12.3 Interest score (0–100) + +See companion skill §20 for the full rubric. **Score ≥ 70 → HIGH/CRITICAL finding** with `attack_path_hint` in evidence. + +### 12.4 Attack-path hints + +When emitting a HIGH/CRITICAL finding, include a one-sentence attack-path hint in the evidence so the operator knows where to start exploiting. Templates in companion skill §39. + +--- + +## 13. JavaScript Deep Analysis + +For every alive webapp, scrape its JS — it's where modern frontends leak. + +### 13.1 Script discovery + +- Parse HTML for `