Paperclip Token Optimizer

A Paperclip adapter that reduces token consumption across agent operations by 50–70% through intelligent model routing, prompt caching, batch optimization, and conversation compression.

Open-source contribution from Odience Network, Paris, France — helping reduce the cost of running AI agents.

Features

• Model Routing — Automatically select optimal model (Haiku vs Opus) based on task complexity
• Prompt Caching — Cache expensive system prompts and tool definitions to reduce redundant tokens
• Batch API Integration — Queue requests for batch processing when latency allows
• Conversation Compression — Intelligently compress conversation history while preserving context
• Thinking Optimization — Extended thinking cost controls with smart fallback routing
• Stream Processing — Real-time token counting and optimization during streaming responses
• Tool Translation — Efficient tool use and result formatting with stable JSON stringification

Quick Start

Installation

npm install @odience-network/paperclip-token-optimizer
# or
yarn add @odience-network/paperclip-token-optimizer
# or
pnpm add @odience-network/paperclip-token-optimizer

Basic Usage

import { AnthropicAILayer } from "@odience-network/paperclip-token-optimizer";

const layer = new AnthropicAILayer({
  apiKey: process.env.ANTHROPIC_API_KEY,
  defaultModel: "claude-opus-4-1-20250805",
});

// Use for completions with automatic token optimization
const response = await layer.stream({
  messages: [{ role: "user", content: "Hello, Claude!" }],
});

for await (const event of response) {
  // Process streaming events
}

Architecture

Core Modules

• stream.ts — Streaming response handler with real-time token counting and cache hit reporting
• model-selector.ts — Intelligent model selection based on prompt complexity and cost thresholds
• prompt.ts — System prompt building with caching directives
• thinking-optimizer.ts — Extended thinking controls and cost optimization
• conversation-compressor.ts — Context-preserving conversation history compression
• batch-api-optimizer.ts — Batch request queuing and optimization
• translate.ts — Tool use translation and result formatting
• types.ts — TypeScript interfaces and type definitions

Optimization Strategies

1. Model Routing

Routes tasks to Haiku for simple operations and Opus for complex ones based on:

• Prompt token count
• Tool complexity
• Context window requirements
• Cost thresholds

2. Prompt Caching

Leverages Anthropic's prompt caching to avoid re-processing:

• System prompts (often 2KB+)
• Tool definitions
• Common context chunks

Expected savings: 90% on cached tokens after first request

3. Batch API

Queues non-urgent requests for batch processing:

• Reduces per-request overhead
• 50% cost discount on batch requests
• Automatic fallback to standard API if latency exceeds threshold

4. Conversation Compression

Intelligently compresses conversation history:

• Preserves recent turns
• Summarizes older interactions
• Maintains semantic context

Expected savings: 30–40% on history tokens for long conversations

5. Thinking Optimization

Controls extended thinking cost:

• Routes complex reasoning to cheaper Haiku when possible
• Falls back to Opus with thinking disabled for cost-sensitive tasks
• Caches thinking output across requests

Configuration

const layer = new AnthropicAILayer({
  apiKey: string;                           // Anthropic API key
  defaultModel?: string;                    // Default Claude model
  cacheControlEnabled?: boolean;            // Enable prompt caching (default: true)
  batchAPIEnabled?: boolean;                // Enable batch API (default: true)
  compressionEnabled?: boolean;             // Enable conversation compression (default: true)
  modelRouterEnabled?: boolean;             // Enable intelligent model selection (default: true)
  thinkingOptimizerEnabled?: boolean;       // Enable thinking optimization (default: true)
  haikusThreshold?: number;                 // Token count for Haiku/Opus routing (default: 2000)
  batchLatencyThreshold?: number;           // Max latency for batch (default: 30000ms)
  logger?: Logger;                          // Custom logger instance
});

Token Savings Example

Before optimization (standard Claude Opus):

Single turn: 4,256 tokens
  - System: 2,048 (cache_creation)
  - User: 512
  - Response: 87
  - Output: 1,609

After optimization (with caching + batch):

Turn 1: 4,256 tokens (cache written)
Turn 2: 788 tokens (2,048 cached, not billed)
  - User: 512
  - Response: 87
  - Output: 189

Overall savings: 81% on turn 2
Sustained: 50–70% across multi-turn conversations

Testing

npm run test        # Run all tests
npm run typecheck   # Check TypeScript types
npm run build       # Build type declarations

Tests cover:

• Stream processing and event handling
• Token counting accuracy
• Cache hit tracking
• Error handling and fallbacks
• Tool translation
• Prompt compression

API Reference

AnthropicAILayer

Main class for handling Claude completions with optimization.

async stream(input: TurnInput): Promise<AsyncIterable<AIStreamEvent>>

Streams a completion response with automatic token optimization.

Parameters:

• input — Turn input with messages, tools, and optional system prompt

Returns:

• Async iterable of AIStreamEvent objects (text, tool_use, message_start, etc.)

Events:

type AIStreamEvent = 
  | { type: "message_start"; message: ContentBlock[] }
  | { type: "text"; text: string }
  | { type: "tool_use"; name: string; input: unknown }
  | { type: "message_done"; telemetry: CallTelemetry }
  | { type: "error"; error: Error };

Exports

• AnthropicAILayer — Main adapter class
• buildSystemPrompt() — Build cached system prompts
• appendSystemReminder() — Append dynamic reminders
• toolUseToCall() — Convert Claude tool_use to function call
• toolResultToBlock() — Convert function result to tool result block
• getToolDefinitions() — Format tools for Claude
• cacheHitFraction() — Calculate cache hit rate from telemetry
• appendCompletedTurn() — Add completed turn to history

Types

interface AILayer {
  stream(input: TurnInput): Promise<AsyncIterable<AIStreamEvent>>;
}

interface TurnInput {
  messages: ContentBlock[];
  tools?: Tool[];
  systemPrompt?: string;
  maxTokens?: number;
  temperature?: number;
  topP?: number;
  topK?: number;
  stopSequences?: string[];
}

interface CallTelemetry {
  inputTokens: number;
  outputTokens: number;
  cacheCreationInputTokens: number | null;
  cacheReadInputTokens: number | null;
  stopReason: StopReason;
  model: string;
  cost?: number; // Estimated cost in dollars
}

Integration with Paperclip

This adapter is designed to drop into Paperclip's AI layer:

1. Install the package
2. Initialize AnthropicAILayer with your Anthropic API key
3. Use .stream() method in place of standard Claude API calls
4. Monitor CallTelemetry for token and cost tracking

Example Paperclip adapter setup:

import { AnthropicAILayer } from "@odience-network/paperclip-token-optimizer";

export class PaperclipTokenOptimizerAdapter {
  private layer: AnthropicAILayer;

  constructor(config: AdapterConfig) {
    this.layer = new AnthropicAILayer({
      apiKey: config.anthropicKey,
      defaultModel: "claude-opus-4-1-20250805",
      cacheControlEnabled: true,
      batchAPIEnabled: config.enableBatch,
      compressionEnabled: true,
    });
  }

  async complete(request: CompletionRequest) {
    return this.layer.stream({
      messages: request.messages,
      tools: request.tools,
      systemPrompt: request.systemPrompt,
      maxTokens: request.maxTokens,
    });
  }
}

Performance & Cost

Token Reduction by Strategy

Strategy	Reduction	Use Case
Model Routing (Haiku)	20–30%	Simple tasks, summaries
Prompt Caching	50–90%	Repeated contexts, tools
Batch API	50% cost	Non-urgent bulk operations
Compression	30–40%	Long conversations
Extended thinking fallback	40–60%	Complex reasoning
Combined	50–70%	Full optimization stack

Latency Impact

• Stream processing: <5ms overhead per event
• Model selection: <10ms decision overhead
• Caching: Transparent (0ms when using cache)
• Batch API: +30s–5min delay for non-urgent requests (configurable)

Troubleshooting

Cache not being used

• Verify cacheControlEnabled: true in config
• Check that system prompt is >1024 tokens
• Review cacheReadInputTokens in telemetry

High latency with batch API

• Increase batchLatencyThreshold or disable batch API
• Check Anthropic batch queue status

Model router choosing unexpected model

• Adjust haikusThreshold value
• Review prompt complexity calculation logic
• Check available token context window

Contributing

We welcome contributions! Please:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/your-feature)
3. Write tests for new functionality
4. Run npm run test and npm run typecheck
5. Submit a pull request

License

Apache License 2.0 — see LICENSE for details.

This is an open-source contribution from Odience Network, Paris, France, to help reduce the cost of running AI agents in the Paperclip community.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: See docs/ directory

Acknowledgments

Built with Anthropic Claude API and Paperclip.