Hi, this is a genius idea!
I thought that the token reduction could be grater if repetitive outputs are stored as a template, e.g.
worker-7 request 4832 failed after 30 sec
worker-7 request 4833 failed after 60 sec
worker-8 request 4834 failed after 60 sec
becomes
Template: worker-{worker} request {request} failed after {seconds} sec
Rows:
7,4832,30
7,4833,60
8,4834,60
This step would run after pxpipe decides that a tool result is eligible for imaging, but before layout and image rendering.
The original, unedited text would also be stored so the model could easily access it if needed
I implemented and integrated a naive version for myself and overall token reduction was ~35-40% in a typical dev setting. Every message was successfully decoded and understood by fable, and it queried exact, uncompressed results as needed, all of the other repo tests passed too
it turned out better in terms of token saved and accuracy (jugded by just asking questions about exact deatils of the codebase) than just using rtk or other token compressors, i dont really know why cause i don't know how they work exactly
I also think you can use different heuristics to decide if pxpipe should write a word/phrase out in text or embed it into the image, we can roughly estimate entropy/surprise of phrases and judge accordingly, e.g. random constant like openssl rand hex 32 is high entropy, known words are low entropy, so maybe you can literally use a dictionary or a small local embedding model. Or at least you can increase the font size of high entropy parts.
idk if you're accepting PRs or suggestions so lmk if you want me to create it. Anyway, thanks for the tool!
Hi, this is a genius idea!
I thought that the token reduction could be grater if repetitive outputs are stored as a template, e.g.
becomes
This step would run after pxpipe decides that a tool result is eligible for imaging, but before layout and image rendering.
The original, unedited text would also be stored so the model could easily access it if needed
I implemented and integrated a naive version for myself and overall token reduction was ~35-40% in a typical dev setting. Every message was successfully decoded and understood by fable, and it queried exact, uncompressed results as needed, all of the other repo tests passed too
it turned out better in terms of token saved and accuracy (jugded by just asking questions about exact deatils of the codebase) than just using rtk or other token compressors, i dont really know why cause i don't know how they work exactly
I also think you can use different heuristics to decide if pxpipe should write a word/phrase out in text or embed it into the image, we can roughly estimate entropy/surprise of phrases and judge accordingly, e.g. random constant like openssl rand hex 32 is high entropy, known words are low entropy, so maybe you can literally use a dictionary or a small local embedding model. Or at least you can increase the font size of high entropy parts.
idk if you're accepting PRs or suggestions so lmk if you want me to create it. Anyway, thanks for the tool!