Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,11 @@ These instructions help AI agents work productively in this repo. Focus on concr
- Initialize infra: `twinkle.initialize(mode='local', seed=42)`
- Inspect device placement: call `twinkle.infra.get_device_placement()`.
- **Ray Serve demo (HTTP services):**
- Config and launcher: [cookbook/client/server.py](cookbook/client/server.py), [cookbook/client/server_config.yaml](cookbook/client/server_config.yaml)
- Config and launcher: [cookbook/client/tinker/megatron/server.py](https://github.com/modelscope/twinkle/blob/main/cookbook/client/tinker/megatron/server.py), [cookbook/client/tinker/megatron/server_config.yaml](https://github.com/modelscope/twinkle/blob/main/cookbook/client/tinker/megatron/server_config.yaml)
- Start:
- `python cookbook/client/server.py`
- Endpoints print on startup (default `localhost:8000`).
- `cd cookbook/client/tinker/megatron`
- `python server.py`
- Endpoints print on startup (default `localhost:8000` or `https://www.modelscope.cn/twinkle`).
- Model app binds `MultiLoraTransformersModel` and exposes routes like `/add_adapter_to_model`, `/forward`, `/calculate_loss`, etc. See [src/twinkle/server/twinkle/model.py](src/twinkle/server/twinkle/model.py).
- **vLLM inference:** Use `VLLMEngine` with engine args; LoRA weight sync via `patch.vllm_lora_weights`. See [src/twinkle/sampler/vllm_engine.py](src/twinkle/sampler/vllm_engine.py).

Expand Down
2 changes: 1 addition & 1 deletion README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ pip install -e .
| | [deepseek-ai/DeepSeek-R1](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1) | transformers>=4.39.3 | ✅ | [deepseek-ai/DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1) |
| deepSeek-r1-distill | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://modelscope.cn/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) ~32B | transformers>=4.37 | ✅ | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) |

更详细的模型支持列表 👉 [快速开始.md](https://github.com/modelscope/twinkle/blob/dev/docs/source/%E4%BD%BF%E7%94%A8%E6%8C%87%E5%BC%95/%E5%BF%AB%E9%80%9F%E5%BC%80%E5%A7%8B.md)
更详细的模型支持列表 👉 [快速开始.md](docs/source_zh/使用指引/快速开始.md)

## 示例代码

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
We adopt the google style docstring format as the standard, please refer to the following documents.
1. Google Python style guide docstring [link](http://google.github.io/styleguide/pyguide.html#381-docstrings)
2. Google docstring example [link](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html)
3. sample:torch.nn.modules.conv [link](https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#Conv1d)
3. sample:torch.nn.modules.conv [link](https://docs.pytorch.org/docs/stable/generated/torch.nn.Conv1d.html)
4. load function as an example:

```python
Expand Down
14 changes: 7 additions & 7 deletions docs/source_en/Usage Guide/NPU-Support.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ This document describes how to install and use the Twinkle framework in Huawei A

Before getting started, please ensure your system meets the following requirements:

| Component | Version Requirement | Description |
|------|---------|------|
| Python | >= 3.11, < 3.13 | Twinkle framework requirement |
| Ascend Firmware Driver (HDK) | Latest version recommended | Hardware driver and firmware |
| CANN Toolkit | 8.3.RC1 or higher | Heterogeneous Computing Architecture |
| PyTorch | 2.7.1 | Deep learning framework |
| torch_npu | 2.7.1 | Ascend PyTorch adapter plugin |
| Component | Version Requirement | Description |
|------------------------------|----------------------------|--------------------------------------|
| Python | >= 3.11, < 3.13 | Twinkle framework requirement |
| Ascend Firmware Driver (HDK) | Latest version recommended | Hardware driver and firmware |
| CANN Toolkit | 8.3.RC1 or higher | Heterogeneous Computing Architecture |
| PyTorch | 2.7.1 | Deep learning framework |
| torch_npu | 2.7.1 | Ascend PyTorch adapter plugin |

**Important Notes**:
- torch and torch_npu versions **must be exactly the same** (e.g., both 2.7.1)
Expand Down
1 change: 1 addition & 0 deletions tests/docs/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Copyright (c) ModelScope Contributors. All rights reserved.
Loading
Loading