Skip to content

Parse extensionless Python and Node scripts by their shebang - #52

Open
FredAmartey wants to merge 1 commit into
dzhng:mainfrom
FredAmartey:shebang-languages
Open

FredAmartey wants to merge 1 commit into
dzhng:mainfrom
FredAmartey:shebang-languages

Conversation

@FredAmartey

Copy link
Copy Markdown

jevgrep picks a file's language from its extension. An executable script with no extension, such as bin/tool starting with #!/usr/bin/env python3, gets the plain-text fallback: no declaration units, no sampled Python preview, no call context and no test-body selection. Repos that ship their CLIs as extensionless scripts lose the structure-aware path for exactly those files.

resolveLanguage(path, source) in source.ts is now the one place that decides a file's language, and every extension check in core calls it. The extension still decides first. Only a file with no extension reads its first line: an interpreter named python, python3 or python3.N means Python, whether it is a direct path or comes through env (which skips env's options, their operands and NAME=VALUE assignments), and node or nodejs means JavaScript. Anything else keeps the text fallback. The preview worker now receives the resolved language instead of the path.

Files with an extension send the same requests as before. For an extensionless script that resolves to a language, request contents change the way they already do for the same file with the extension. Request schemas, the extension field and cache keys are unchanged.

What a caller sees, one question per repo:

  • A private repo with twelve extensionless Python CLIs. On two questions whose answers live in those scripts, the code spans a separate agent needed went from 2 of 7 to 5 of 7, and from 0 to 2 of 4 (two runs) to 3 of 4. On main the largest script was listed as "locations only" with no excerpt.
  • git-filter-repo, whose implementation is one 5,000-line extensionless script. With this branch it returns the same declaration blocks that 0.8.0 returns for the file renamed to git-filter-repo.py (three runs each). That is no better than main here: main's text chunks also land on the option parser and the filtering method, and output grows from 13 to 37 KB to 48 to 57 KB because whole declarations come back. That size is the existing Python path, not something this change adds.

Validation on macOS arm64, Bun 1.3.14, Node 26.7:

  • The three new tests fail on main and pass here. Two in test/parser/source.test.ts check that Python and Node shebang scripts inspect exactly like the same file named .py or .js. One in test/retrieval.test.ts checks that provider requests and returned evidence match the .py or .js file apart from the path and extension field. The fallback table gains cases for no shebang, bash, a shebang below line 1, and a mismatched extension winning.
  • node --experimental-strip-types --test test/parser/*.test.ts: 41 passed.
  • bun test --timeout 30000 ./apps/cli/test ./packages/core/test ./test/discovery.test.ts ./test/evaluator.test.ts ./test/retrieval.test.ts ./test/retrieval-freshness.test.ts: 59 passed, 81 Docker-only skipped.
  • Provider protocol 18, release 4 and evaluation harness 32 passed.
  • bun run check-types passed. bun run lint: 0 errors, the one existing warning in @repo/core. oxfmt --check passed on the 9 changed files.
  • Not run: the Docker suites and test:installed. test/skill.test.mjs fails the same way on main on this machine (installer exit 1 instead of 7).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant