DeukPack: AI-Native Universal Schema Multi-hub

⚠️ [NOTICE] Large-Scale Internal Refactoring in Progress

During recent integration testing across various languages, several defects were identified, prompting a large-scale internal refactoring. I am intentionally pivoting my focus 'from extreme performance optimizations to ensuring architectural stability and flexibility'. My primary goal is to carefully review the specific characteristics of each language and protocol to improve and fix omissions from the initial implementation. Please keep in mind that there may be unexpected issues during use until this work is complete.

Universal Schema Multi-hub for Protobuf, Thrift, OpenAPI, and .deuk. (High-Performance)

Languages / 언어: English · 한국어

Turn any IDL (Protobuf, OpenAPI, JSON Schema, .deuk) into type-safe, deterministic code across C#, C++, TypeScript, JavaScript, Java, and Elixir — with a single, unified serialization API.

---

📢 [v1.9.0 Update] Official Python (Pure/Rust) Support and Industry Standard BMT Benchmark Revamp

We are pleased to announce the addition of the official Python (3.6+) binary engine (Pure/Rust Accelerated) to the DeukPack ecosystem. Furthermore, to ensure objective reliability in enterprise environments, we have discarded legacy measurement methods and elevated the benchmark environments across all languages to industry-standard frameworks (such as BenchmarkDotNet, mitata, and pytest-benchmark).

You can review the newly validated, highly transparent and strict performance metrics—measured under real-world complex object conditions—in our updated Performance Matrix White Paper.

• Unified API: You no longer need to call verbose factory methods. All languages now intuitively share the same 2-Method struct syntax: Hero.Pack() and Hero.Unpack(). (Legacy APIs are temporarily maintained for backward compatibility).

• ⚠️ CAUTION (C# / Unity Users): If you manually copy generated .cs runtime files, you MUST completely empty that legacy folder prior to importing the new files to prevent duplicate declaration compile errors. (npm / UPM users are automatically updated).

---

Why DeukPack: The AI-Ready Advantage

1. Universal IDL Gateway (OpenAPI, JSON Schema, Protobuf, Thrift, CSV)

Modern systems often struggle with a fragmented mix of Specs—legacy (Thrift), modern (Protobuf/gRPC), and web-native (OpenAPI/JSON Schema). DeukPack acts as a Single Source of Truth that aggregates diverse IDL sources into one unified, semantic model — wire-compatible with your legacy protocols.

2. IDL-to-AI Semantic Mapping

DeukPack extracts metadata from IDL comments and field structures, transforming them into a Semantic Context that AI can instantly grasp. Designers evolve into architects defining data lineage via a machine-readable semantic layer.

3. AI-Native Execution Bridge (MCP Plugin Support)

The Model Context Protocol (MCP) server auto-generation feature (DeukPackMcp) lets AI agents (Cursor, Claude, etc.) browse live documentation and execute backend methods directly.

4. Core Memory Optimization (In-place Reuse)

Engineered for high efficiency. 60–100% reduction in memory allocation and 250% increase in JS parsing speed vs. classic industry flows. See the Performance section for raw numbers.

---

⚡ Two Words. Every Language.

DeukPack v1.7.6 introduces a universal 2-method serialization API: Pack and Unpack.
Regardless of language or wire format — binary, JSON, In-place — you only need to remember two verbs.

Pack    → Serialize (data out)
Unpack  → Deserialize (data in)

Pass a format parameter to switch protocols. Call Unpack on an existing instance for in-place overwrite.
That's the entire API surface.

Unity / C# Notice (GC Spike Prevention): NEVER use var h = Hero.Unpack(bin); (Factory method) for high-frequency network packets (Hotpath). It implicitly triggers new allocations, causing GC spikes and severe frame drops. You MUST use Hero.Unpack(cachedHero, bin); to overwrite pre-allocated (pooled) objects if you want to achieve true in-place architecture without GC stutters.

// C# / Unity: 1.Create  2.Pack  3.Unpack (In-place)
var hero = new Dto.Hero { id = 1, name = "Deuk" };
byte[] bin = Dto.Hero.Pack(hero);          // Serialize (Static)
Dto.Hero.Unpack(hero, bin);                // In-place Update

// TypeScript / JavaScript: 1.Create  2.Pack  3.Unpack (No Class Wrappers)
const hero = Dto.Hero.create({ id: 1, name: "Deuk" });
const bin = Dto.Hero.pack(hero);           // Serialize
Dto.Hero.unpack(hero, bin);                // In-place Update

// C++ (Native): 1.Create  2.Pack  3.Unpack (Memory Safe)
Dto::Hero hero; hero.id = 1; hero.name = "Deuk";
auto bin = Dto::Hero::Pack(hero);          // Serialize
Dto::Hero::Unpack(hero, bin);              // In-place Deserialize

// Java: 1.Create  2.Pack  3.Unpack (High-Performance)
Dto.Hero hero = new Dto.Hero(1, "Deuk");
byte[] bin = Dto.Hero.pack(hero);          // Serialize (Static)
Dto.Hero.unpack(hero, bin);                // In-place Overwrite (Static)

# Elixir: 1.Create  2.Pack  3.Unpack (BEAM Native)
hero = %Dto.Hero{id: 1, name: "Deuk"}      # Immutable Struct
bin = Dto.Hero.pack(hero)                  # Serialize
hero_parsed = Dto.Hero.unpack(bin)         # BEAM Pattern Match

---

🚀 Quick Start

npx deukpack init

1. Define your schema (or import OpenAPI / Protobuf)

namespace Dto

struct Hero {
    1> int32 id
    2> string name
    3> float hp
}

---

🔄 Backward Compatibility — Existing Code is Safe

All legacy method names are preserved as deprecated aliases. No breaking changes.

Old API (still works)	New equivalent
Hero.toBinary(obj)	Hero.pack(obj)
Hero.toJson(obj)	Hero.pack(obj, 'json')
Hero.fromBinary(buf)	Hero.unpack(buf)
Hero.fromJson(str)	Hero.unpack(str, 'json')
Hero.unpackInto(obj, buf)	Hero.unpack(obj, buf)
DeukPackCodec.UnpackInto(obj, data)	obj.Unpack(data) (C#)

Your existing project will compile without errors. IDE will show soft deprecation warnings to guide migration at your own pace.

---

🎮 Real-World Pattern: Unity Game Client (In-place Reuse)

Dto.Hero cachedHero = new Dto.Hero(); // Allocated exactly ONCE at startup

void OnNetworkMessage(byte[] inputData) {
    // Zero-Garbage Deserialization — NO new class allocations, NO GC spikes!
    cachedHero.Unpack(inputData);
    Debug.Log($"Hero: {cachedHero.name}, HP: {cachedHero.hp}");

    // Mutate and re-serialize (Note: byte[] reuse requires Stream API)
    cachedHero.hp -= 10f;
    byte[] outputData = Dto.Hero.Pack(cachedHero);
    network.Send(outputData);
}

---

🚀 Release Roadmap

Version	Key Milestones	Status
v1.4.0	MCP Protobuf expansion, C#/C++/JS core runtime stabilization	DONE
v1.5.0	Java & Core Parity: Inheritance, Compact/TJSON protocols, MCP Core Decoupling	DONE
v1.5.1	C++ In-place Reuse Optimization: Arena allocator, C++ DDL Generator	DONE
v1.6.0	V8 JIT Codegen & In-place Reuse Architecture: JS/C# memory optimizations	DONE
v1.7.0	Elixir Engine Support: Native BEAM pattern matching & Universal Protocol Security Shield	DONE
v1.8.0	Unified 2-Method API: Pack/Unpack standard across all 6 languages	DONE
v1.8.1	Dialyzer & CI Security: Strict Elixir typing and GitHub Actions sample.deuk pure migration	DONE
v1.9.0	Python Official Parity & BMT: Python 3.6+ Engine and industrial benchmark suite	Current

---

⚡ Performance: The Zero-Bottleneck Foundation

Notice: Benchmark Data Under Revision We have identified an inconsistency between the highly-optimized protocol implementations and the aggregated result table. The performance metrics matrix is currently being revised and will be republished once the evaluation is complete.

Figures based on decoding a 10,000-row payload. Results vary by environment.
Detailed cross-language matrix · Benchmarking Guide

---

Feature Matrix

Category	Feature	TS / JS	C# / Unity	C++	Java	Elixir	Python
IDL Core	Basic Types / Aliases	✅	✅	✅	✅	✅	✅
Inheritance	extends support	✅	✅	✅	✅ (v1.5)	✅	✅
Unified API	Pack / Unpack (2-method)	✅ v1.8.0	✅ v1.8.0	✅ v1.8.0	✅ v1.8.0	✅ v1.8.0	✅ v1.9.0
Protocols	Native Pack (.dpk)	✅	✅	✅	✅	✅	✅ (Pure/Rust)
	Protobuf Compatible	✅	✅	✅	✅	-	🚧
	Thrift Compatible (T-Series)	✅	✅	✅ (v1.5)	✅ (v1.5)	-	-
	JSON (Tagged / POJO)	✅	✅	✅ (v1.5)	✅	-	✅
	YAML / CSV	✅	✅ (v1.2.7)	🚧	🚧	-	-
Optimizations	In-place / JIT	✅ (v1.6)	✅	✅ (v1.4.2)	🚧	✅ (BEAM)	✅ (v1.9)
	Write Logic Overrides	✅	✅	✅ (v1.5)	✅ (v1.5)	-	-
Data/Meta	tablelink / MetaTable	✅	✅	✅ (v1.5)	✅	-	-
AI Integration	MCP Tool Auto-Generation	✅ (v1.5)	🚧	-	-	-	-
	Intelligent Context	✅	✅	✅	✅	✅	✅
	IDE IntelliSense	✅	✅	✅	✅	✅	✅

---

Installation

npm install deukpack
npx deukpack init
npx deukpack run         # default: ./deukpack.pipeline.json

Tutorials and OS-specific notes: deukpack.app/tutorial.

---

🛡️ Security & Reliability (OOM Defense / Anti-DDoS)

DeukPack implements strict defense-in-depth against network-layer parsing vulnerabilities (OOM, Buffer Flooding, Infinite Recursion):

• Universal OOM Defense (v1.7.0+): Enforced MAX_SAFE_LENGTH (10MB) and MAX_ELEMENT_COUNT (1,000,000) limits across all engines. Malicious packets are discarded (Fail-Fast) before any memory allocation.
• Progressive Chunk Validation: Replaces legacy ReadToEnd() with length pre-evaluation, neutralizing JSON stack bombing in Node.js and Java backends.
• Continuous DDoS Fuzzer Suite: CI-integrated test-fuzz-oom.js injects parsers with 2GB+ abnormal buffers.

---

Documentation & Links

Type	Link
This README	Clone-time summary
Feature overview	DEUKPACK_FEATURES.md · KO
deukpack.app	Install, tutorials, protocol, API reference
Korean README	README.ko.md
Releases	RELEASING.md
Full doc index	docs/README.ko.md

Contact: contact@deukpack.app

---

Development

npm ci
npm run build
npm test
npm run benchmark                     # Node serialize smoke
npm run bench:format-parity           # parser comparison
npm run bench:cross-lang              # Node vs .NET pack

---

☕ Support & Contact

DeukPack is completely open-source (Apache 2.0). Built to solve fundamental GC bottleneck and model synchronization issues from 30 years of server architecture experience.

• 📩 Contact / Technical Inquiries: joygram@gmail.com
• ☕ Support the Project: Sponsor via Ko-fi

Starring the repo or sharing it with teams juggling Protobuf/Thrift helps immensely.

---

Works well with (Deuk Family)

Want agents to do more with your specs? Use DeukPack — IDL in, deterministic types and serializers out.
Want agents to behave predictably in your repo? Use DeukAgentRules — versioned AGENTS.md and rule templates.

npm install -D deuk-agent-rule
npx deuk-agent-rule init --non-interactive

---

⚡ Performance & Benchmarks

Notice: Benchmark Data Under Revision We have identified an inconsistency between the highly-optimized protocol implementations and the aggregated result table. The performance metrics matrix is currently being revised and will be republished once the evaluation is complete.

To ensure objective performance testing, we have adopted industry-standard benchmarks such as BenchmarkDotNet, mitata, and pytest-benchmark. Achieving 60-100% reduction in memory allocation and 2.5x to 5x faster parsing speeds compared to legacy industry formats. If you find any errors or have suggestions regarding the benchmark scenarios, please let us know.

• Performance test technical document

---

Ecosystem (Deuk Family)

Contributing

1. Fork → feature branch → PR.
2. See RELEASING.md for release layout.

---

License

Apache License 2.0 — LICENSE · NOTICE.

---

Acknowledgments

The broader IDL / OpenAPI / schema communities; DeukPack is a standalone pipeline (not an Apache Thrift subproject).