diff --git a/HACKING.md b/HACKING.md new file mode 100644 index 0000000..9ba5c88 --- /dev/null +++ b/HACKING.md @@ -0,0 +1,101 @@ +# HACKING + +## Overview + +`wax` is a Rust CLI with provider-based dispatch for Bandcamp and SoundCloud. + +Current provider split: + +- Bandcamp: + - `resolve` supported + - `collectors` supported + - `library` supported + - `dig` supported +- SoundCloud: + - `resolve` supported + - `dig` supported for track URLs as a prototype + - `collectors` unsupported + - `library` unsupported + - playlist `dig` unsupported + +Core implementation lives in: + +- `src/app.rs` +- `src/provider.rs` +- `src/parser.rs` +- `src/soundcloud.rs` +- `src/score.rs` +- `src/output.rs` + +## Development + +Run the test suite: + +```bash +cargo test +``` + +Run the CLI locally without installing: + +```bash +cargo run -- dig https://artist.bandcamp.com/album/example-record +cargo run -- resolve https://soundcloud.com/chvrches/the-mother-we-share +``` + +## Configuration + +Pass a TOML file with `--config`: + +```bash +wax dig https://artist.bandcamp.com/album/example-record --config ./wax.toml +``` + +Example config: + +```toml +cache_dir = "/tmp/wax-cache" +request_delay_ms = 1000 +concurrency = 4 +timeout_ms = 10000 +user_agent = "wax/0.1" +max_collectors = 75 +max_depth = 1 +``` + +Note: the current CLI reads the config file for shared request settings. Command-specific defaults such as `max_collectors` and `max_depth` are present in the config schema but are not yet wired into command execution. + +## SoundCloud Notes + +SoundCloud resolve currently uses the public `api-v2 /resolve` endpoint after deriving a client id from the page or falling back to a known public client id. + +SoundCloud `dig` currently works like this: + +1. Resolve the seed track +2. Fetch public likers for the seed +3. Fetch each liker's public likes feed +4. Find the seed-like event in that feed +5. Use the two previous and two next likes as candidate evidence +6. Rank repeated co-likes + +The live SoundCloud `dig` path is still expensive and may need more work on: + +- crawl limits +- fallback behavior +- sparse public data +- pagination strategy + +## Testing Notes + +Fixture coverage currently includes: + +- Bandcamp overlap flow in `tests/dig_flow.rs` +- SoundCloud likes-based flow in `tests/soundcloud_likes_dig_flow.rs` + +Relevant fixtures: + +- `tests/fixtures/fan_a.html` +- `tests/fixtures/fan_b.html` +- `tests/fixtures/seed.html` +- `tests/fixtures/soundcloud_likers.json` +- `tests/fixtures/soundcloud_user_likes_a.json` +- `tests/fixtures/soundcloud_user_likes_b.json` diff --git a/README.md b/README.md index e3bac89..f9ff7e8 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,17 @@ # wax -`wax` is a Rust CLI for digging through public Bandcamp collector overlap. +`wax` is a Rust CLI for digging through public music discovery signals from Bandcamp and SoundCloud. -Given a Bandcamp album URL, `wax` resolves the seed album, finds public collectors who own it, inspects their public libraries, and ranks related albums worth exploring. +Give it a URL and it will detect the platform automatically: -## What It Does +- Bandcamp album URLs use collector overlap +- SoundCloud track URLs use public likers and nearby likes in each liker feed -- Resolves a Bandcamp album URL into normalized seed metadata -- Finds public collectors for the seed album -- Crawls public libraries for a bounded set of collectors -- Ranks related albums by collector overlap -- Prints human-readable tables or machine-readable JSON and CSV -- Caches fetched pages locally to make repeated runs faster +`wax` keeps the same entrypoint across providers: + +```bash +wax dig +``` ## Install @@ -39,12 +39,19 @@ The binary will be at `target/release/wax`. ## Quick Start -Run the main recommendation flow with a Bandcamp album URL: +Bandcamp: ```bash wax dig https://artist.bandcamp.com/album/example-record ``` +SoundCloud: + +```bash +wax resolve https://soundcloud.com/chvrches/the-mother-we-share +wax dig https://soundcloud.com/chvrches/the-mother-we-share +``` + Inspect the available commands: ```bash @@ -54,46 +61,60 @@ wax dig --help ## Commands -### `wax dig ` +### `wax dig ` + +Runs the recommendation flow for a supported URL. -Runs the full discovery flow and prints ranked recommendations. +Examples: ```bash wax dig https://artist.bandcamp.com/album/example-record wax dig https://artist.bandcamp.com/album/example-record --max-collectors 50 --limit 20 -wax dig https://artist.bandcamp.com/album/example-record --exclude-artist --sort overlap +wax dig https://soundcloud.com/chvrches/the-mother-we-share --max-collectors 25 --limit 10 +wax dig https://soundcloud.com/chvrches/the-mother-we-share --exclude-artist --sort overlap wax dig https://artist.bandcamp.com/album/example-record --json ``` Useful flags: -- `--max-collectors `: maximum collectors to sample from the seed album +- `--max-collectors `: maximum source nodes to sample during discovery - `--max-depth `: reserved crawl-depth flag, currently defaults to `1` - `--limit `: maximum recommendations to print -- `--sample top|random`: keep the first collectors found or shuffle before truncating +- `--sample top|random`: keep sources in discovery order or shuffle before truncating - `--min-overlap `: minimum overlap count required for a recommendation - `--exclude-artist`: drop records by the same artist as the seed - `--exclude-label`: drop records on the same label as the seed - `--tag `: require matching tags on candidate records - `--sort score|overlap`: sort recommendations by score or raw overlap -### `wax resolve ` +Provider notes: -Resolves an album URL and prints seed metadata. +- Bandcamp `dig` uses public collectors and their public libraries +- SoundCloud `dig` currently supports track URLs +- SoundCloud playlist URLs are not supported for `dig` yet + +### `wax resolve ` + +Resolves a supported URL and prints canonical seed metadata. + +Examples: ```bash wax resolve https://artist.bandcamp.com/album/example-record +wax resolve https://soundcloud.com/chvrches/the-mother-we-share ``` ### `wax collectors ` -Lists public collectors discovered for a seed album. +Lists public collectors discovered for a Bandcamp seed album. ```bash wax collectors https://artist.bandcamp.com/album/example-record wax collectors https://artist.bandcamp.com/album/example-record --max-collectors 100 ``` +This command is Bandcamp-only. + ### `wax library ` Prints albums from a public Bandcamp fan page. @@ -103,6 +124,8 @@ wax library https://bandcamp.com/fanname wax library https://bandcamp.com/fanname --limit 50 ``` +This command is Bandcamp-only. + ### `wax cache stats` Prints the cache directory, entry count, and cache size in bytes. @@ -142,6 +165,7 @@ Use `--json` for structured output: ```bash wax dig https://artist.bandcamp.com/album/example-record --json +wax resolve https://soundcloud.com/chvrches/the-mother-we-share --json ``` Use `--csv` for CSV output: @@ -152,47 +176,13 @@ wax collectors https://artist.bandcamp.com/album/example-record --csv wax library https://bandcamp.com/fanname --csv ``` -## Configuration - -Pass a TOML file with `--config`: - -```bash -wax dig https://artist.bandcamp.com/album/example-record --config ./wax.toml -``` - -Example config: - -```toml -cache_dir = "/tmp/wax-cache" -request_delay_ms = 1000 -concurrency = 4 -timeout_ms = 10000 -user_agent = "wax/0.1" -max_collectors = 75 -max_depth = 1 -``` - -Note: the current CLI reads the config file for shared request settings. Command-specific defaults such as `max_collectors` and `max_depth` are present in the config schema but are not yet wired into command execution. - ## Notes -- `wax` only uses publicly visible Bandcamp pages -- private or unavailable fan libraries are skipped -- if no usable public data is available, the command exits with a non-zero status +- `wax` only uses publicly visible Bandcamp and SoundCloud data +- SoundCloud support currently focuses on track resolution and a prototype `dig` flow +- private or unavailable public data is skipped when possible - repeated runs are faster when the cache is warm -## Development - -Run the test suite: - -```bash -cargo test -``` - -Run the CLI without installing: - -```bash -cargo run -- dig https://artist.bandcamp.com/album/example-record -``` +## Hacking -The current implementation is in [Cargo.toml](Cargo.toml), [src/cli.rs](src/cli.rs), and [src/app.rs](src/app.rs). The broader product notes live in [SPEC.md](SPEC.md). +Developer-facing notes live in [HACKING.md](HACKING.md).