Skip to content

Replace inline source embedding with VLQ-encoded source maps#20

Draft
Copilot wants to merge 3 commits intomasterfrom
copilot/use-vlq-encoding-source-maps
Draft

Replace inline source embedding with VLQ-encoded source maps#20
Copilot wants to merge 3 commits intomasterfrom
copilot/use-vlq-encoding-source-maps

Conversation

Copy link

Copilot AI commented Nov 18, 2025

The compiler was embedding Python source code directly into compiled JavaScript ($currSource="def hello():") at every annotated position, resulting in bloated output and non-standard source tracking.

Changes

New: src/sourcemap.js

  • VLQ encoding/decoding utilities (Base64, variable-length, sign-bit support)
  • SourceMapGenerator class implementing Source Map v3 format
  • Stores original source once in sourcesContent field

Modified: src/compile.js

  • Compiler initializes SourceMapGenerator when annotation enabled
  • annotateSource() emits position references ($currSource='1:0') instead of inline source
  • Compiled output includes:
    • $sourceMap constant with v3 source map
    • $getSource(pos) helper to decode positions to source lines
  • handleTraceback() decodes positions before storing in error tracebacks

Modified: src/main.js

  • Added sourcemap module to build

Impact

Before:

$currSource="    result = x + y"  // Repeated per statement

After:

$currSource='3:4'  // Position reference
// Source stored once: {"sourcesContent":["def foo():\n    result = x + y\n..."]}
// Decoded via $getSource('3:4') when needed

~57% reduction in source-related data size. Error tracebacks automatically display decoded source.

Original prompt

This section details on the original issue you should resolve

<issue_title>Make the compile function use VLQ encoding source maps</issue_title>
<issue_description>Currently the source code is annotated using a clunky approach that embeds the source of the python directly into the JavaScript. Instead, the compiler should use VLQ encoding source maps. We will need to store the encoding as a string still, but we should be able to just keep track of the position instead of the entire source inline. Look at the annotateSource function in compile.js, along with the parser information.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits November 18, 2025 04:29
Co-authored-by: acbart <897227+acbart@users.noreply.github.com>
Co-authored-by: acbart <897227+acbart@users.noreply.github.com>
Copilot AI changed the title [WIP] Make the compile function use VLQ encoding source maps Replace inline source embedding with VLQ-encoded source maps Nov 18, 2025
Copilot AI requested a review from acbart November 18, 2025 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the compile function use VLQ encoding source maps

2 participants