Peer-to-peer AI Inference & Distributed Execution with PyTorch

Table of Contents

• What is Tensorlink?
• Quick Start
• Learn More
• Contributing

---

What is Tensorlink?

Tensorlink is a Python library and decentralized compute platform for running PyTorch and Hugging Face models across peer-to-peer networks. It enables you to easily distribute and remotely access models across devices through PyTorch-facing wrappers or HTTP endpoints via your own hardware or by tapping into public peer-to-peer resources. Hardware owners can leverage their own GPUs for private remote AI services or contribute resources to the public network and earn rewards.

Key Features

• Run Large Models - Automatic offloading and model sharding across peers
• Native PyTorch & REST API - Use models directly in Python or via HTTP endpoints
• Streaming Generation - Token-by-token streaming for real-time responses
• Privacy Controls - Route queries exclusively to your own hardware
• Earn Rewards - Contribute GPU resources to the network and get compensated

Early Access: Tensorlink is under active development. APIs and internals may evolve. Join our Discord for updates, support, and roadmap discussions.

---

Quick Start

There are three ways to interact with Tensorlink. Choose the path that fits your use case:

• Distributed Models in Python - run PyTorch/Hugging Face models directly from Python
• HTTP API - OpenAI-style REST endpoints for distributed inference
• Run a Node - contribute GPU compute or host your own private cluster

Distributed Models in Python

Installation

pip install tensorlink

Requirements: Python 3.10+, UNIX/macOS (Windows: use WSL). No GPU required to use the public network.

Inference

from tensorlink.ml import DistributedModel
from transformers import AutoTokenizer

MODEL_NAME = "Qwen/Qwen3-14B"

model = DistributedModel(model=MODEL_NAME)

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
inputs = tokenizer("Explain the theory of relativity.", return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Distributed Training

from tensorlink.ml import DistributedModel

model = DistributedModel(model="Qwen/Qwen3-14B", training=True)
optimizer = model.create_optimizer(optimizer_type="adamw", lr=1e-4, weight_decay=0.01)

# Training loop works like standard PyTorch
outputs = model(**inputs, labels=inputs["input_ids"])
outputs.loss.backward()
optimizer.step()
optimizer.zero_grad()

For private clusters, custom architectures, and full parameter reference, see docs/distributed-models.md.

---

HTTP API

Access models via HTTP - either through the public network or your own private node. The API is OpenAI-compatible and requires no GPU or Python on the client side.

Simple generation

import requests

response = requests.post(
    "http://smartnodes.ddns.net/tensorlink-api/v1/generate",
    json={
        "hf_name": "Qwen/Qwen2.5-7B-Instruct",
        "message": "Explain quantum computing in one sentence.",
        "max_new_tokens": 50,
        "stream": False,
    }
)
print(response.json()["generated_text"])

OpenAI-compatible chat

import requests

response = requests.post(
    "http://localhost:64747/v1/chat/completions",
    json={
        "model": "Qwen/Qwen2.5-7B-Instruct",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What are the benefits of distributed computing?"}
        ],
        "max_tokens": 150,
        "stream": False,
    }
)
print(response.json()["choices"][0]["message"]["content"])

For all endpoints, streaming, the responses API, and model preloading, see docs/api.md.

---

Run a Node

Run worker or validator nodes to contribute compute to the public network, host a private cluster, or expose models as API endpoints.

1. Download the latest tensorlink-node from Releases
2. Edit config.json to configure your node
3. Run ./run-node.sh

The default config runs a public worker node, where your GPU will process network jobs and earn rewards on the public network via Smartnodes.

For configuration reference, private cluster setup, and network architecture patterns, see docs/nodes.md.
To contribute your GPU in the fastest way possible, see docs/worker-guide.md.

---

Learn More

Resource	Description
Getting Started	Installation, requirements, and first steps
Distributed Models	DistributedModel, DistributedOptimizer, private clusters
API Reference	HTTP endpoints, parameters, and examples
Node Setup	Workers, validators, config reference, network topologies
Worker Quick Start	Contribute GPU compute in minutes
Discord Community	Get help and connect with developers
Live Demo	Try a chatbot powered by Tensorlink
Litepaper	Technical overview and architecture

---

Contributing

Read our contribution guide.

Tensorlink is released under the MIT License.