Skip to content

add binary stream format (#168)#173

Merged
TinoDidriksen merged 42 commits into
mainfrom
168-binary-format
Sep 8, 2025
Merged

add binary stream format (#168)#173
TinoDidriksen merged 42 commits into
mainfrom
168-binary-format

Conversation

@mr-martian
Copy link
Copy Markdown
Collaborator

@mr-martian mr-martian commented Jun 24, 2025 •
edited
Loading

TODO:

  • variables
  • relations
  • initial header
  • format detection
  • testing
  • docs

@mr-martian mr-martian requested a review from TinoDidriksen June 27, 2025 15:48
@mr-martian
Copy link
Copy Markdown
Collaborator Author

mr-martian commented Jun 27, 2025

If this looks generally sane, I can make some tests.

@mr-martian
Copy link
Copy Markdown
Collaborator Author

mr-martian commented Aug 22, 2025 •
edited
Loading

remaining issues:

  • runGrammarOnText somehow needs to know that the output is binary so it can allocate a new window after a flush so window-initial text doesn't get lost
  • mapping tags get reordered somehow
  • MOVE, COPYCOHORT, SPLITCOHORT, and SWITCHPARENT have slightly wrong dependencies for some reason
  • T_SpaceInForms loses extra baseform
  • T_Variables loses trailing SETVAR

mr-martian and others added 27 commits September 8, 2025 19:25
@mr-martian @TinoDidriksen
start on binary format
a2e0192
@mr-martian @TinoDidriksen
assorted fixes
db2ac15
@mr-martian @TinoDidriksen
stream header and detection
ff5b438
@mr-martian @TinoDidriksen
variables; account for ID updates
56d68e6
@mr-martian @TinoDidriksen
start on relations
60a5e3d
@mr-martian @TinoDidriksen
write global_number rather than dep_self
a5fd974
@mr-martian @TinoDidriksen
relations
9f114b4
@mr-martian @TinoDidriksen
minor optimizations - don't rehash if we don't need to
9fae2fc
@mr-martian @TinoDidriksen
python binary parser
ef63dce
@mr-martian @TinoDidriksen
fix indentation
f890e75
@mr-martian @TinoDidriksen
stop using CI as compiler
ad0279f
@mr-martian @TinoDidriksen
add docs
9ea2ea7
@mr-martian @TinoDidriksen
typo in python; --dep-delimit for conv
03fe90d
@mr-martian @TinoDidriksen
add baseform properly
1670d65
@mr-martian @TinoDidriksen
also for static tags
c149504
@TinoDidriksen
Add format conversion in main; Add tests for binary format, currently…
f17ac62 
… 39 / 69 tests passing
@mr-martian @TinoDidriksen
handle empty cohorts (41/69)
373bbcd
@mr-martian @TinoDidriksen
add endtag to last cohort (42/69)
151c291
@mr-martian @TinoDidriksen
args to cg-conv (45/69)
5bd496c
@mr-martian @TinoDidriksen
unique_tags (46/69)
329b515
@mr-martian @TinoDidriksen
delimiters in tests (49/69)
628168b
@mr-martian @TinoDidriksen
print end tag
daf2d86
@mr-martian @TinoDidriksen
dep_has_spanned (Omniscan)
e192cbf
@TinoDidriksen
Include Static (57/69)
c6e610f
@mr-martian @TinoDidriksen
fix flag offsets
c6155cc
@mr-martian @TinoDidriksen
split mappings
ad8e641
@mr-martian @TinoDidriksen
ensure tags are mapping tags
32d3b32
mr-martian and others added 14 commits September 8, 2025 19:25
@TinoDidriksen TinoDidriksen merged commit 797d3cd into main Sep 8, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TinoDidriksen TinoDidriksen Awaiting requested review from TinoDidriksen

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mr-martian @TinoDidriksen