llmzip is a high-performance C++ utility designed to compress technical text into a structured, AI-readable protocol. It achieves up to 90% compression by extracting semantic entities and mapping them to a standardized DSL (Domain Specific Language).
- Deterministic Semantic Compression (Level 1): Uses a rule-based engine to transform raw text into a
Key:Value;protocol. 100% reproducible, zero latency. - LLM-Powered Polish (Level 2): Optional integration with ONNX models (BART/T5) for advanced summarization and noise removal.
- Binary Container (.lz): Custom format with Magic headers, CRC32 integrity checks, and metadata tracking.
- Hybrid Mode: Combines semantic extraction with standard
zlibcompression for maximum efficiency. - Technical Focus: Pre-configured for DevOps, AI/ML, and Software Architecture domains.
llmzip transforms this:
"We are using a microservices architecture with a PostgreSQL database and Python FastAPI. The project is currently in progress and we have finished the authentication module."
Into this:
Proj:llmzip; Status:🔄; Stack:FastAPI/PG/Python; Done:Auth✅; →tech
# Requirements: CMake 3.16+, C++17, zlib
mkdir build && cd build
cmake ..
make -j$(nproc)# Compress a file (defaults to input.txt.lz)
llmzip c input.txt
# Compress with specific output and mode
llmzip c input.txt archive.lz -m lossy
# Decompress (defaults to stdout if output is omitted)
llmzip d archive.lz output.txt
# View compression statistics
llmzip s archive.lz
# Show version
llmzip v| Mode | Description | Ratio | Use Case |
|---|---|---|---|
lossy |
Deterministic Protocol + LLM | 70-95% | AI-to-AI communication, Knowledge bases |
lossless |
Standard zlib deflate | 30-40% | General purpose storage |
hybrid |
Protocol + zlib | 50-60% | Balanced efficiency |
- Core: Binary format and orchestration.
- ProtocolParser: Regex-based entity extraction and template assembly.
- InferenceEngine: ONNX Runtime wrapper for optional LLM layers.
- CLI: Built with CLI11 for robust command-line interaction.
MIT