Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/get-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ torchrun --nproc_per_node=2 examples/run_simple_mcore_train_loop.py

```bash
# 8 GPUs, FP8 precision, mock data
./examples/llama/train_llama3_8b_fp8.sh
./examples/open_models/llama/train_llama3_8b_fp8.sh
```

## Data Preparation
Expand Down
10 changes: 4 additions & 6 deletions docs/models/llms.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,10 @@ See the [Megatron Bridge supported models list](https://github.com/NVIDIA-NeMo/M
## Example Scripts

Training examples for these models can be found in the `examples/` directory:
- `examples/gpt3/` - GPT-3 training scripts
- `examples/llama/` - LLaMA training scripts
- `examples/mixtral/` - Mixtral MoE training
- `examples/mamba/` - Mamba training scripts
- `examples/bert/` - BERT training scripts
- `examples/t5/` - T5 training scripts
- `examples/open_models/gpt3/` - GPT-3 training scripts
- `examples/open_models/llama/` - LLaMA training scripts
- `examples/open_models/mamba/` - Mamba training scripts
- `examples/open_models/t5/` - T5 training scripts

## Model Implementation

Expand Down
4 changes: 2 additions & 2 deletions docs/models/multimodal.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Megatron Core supports multimodal models that combine language with vision, audi
- Unified embedding space across modalities
- Support for both vision-language and audio-vision-language models

See [examples/mimo](https://github.com/NVIDIA/Megatron-LM/tree/main/examples/mimo) for training scripts and examples.
See [examples/open_models/mimo](https://github.com/NVIDIA/Megatron-LM/tree/main/examples/open_models/mimo) for training scripts and examples.

## Vision-Language Models

Expand Down Expand Up @@ -52,7 +52,7 @@ For multimodal diffusion models (image generation, text-to-image, etc.), see [Ne
Multimodal training examples can be found in the following directories:

**MIMO Framework:**
- `examples/mimo/` - Multimodal In/Out training with support for vision-language and audio-vision-language models
- `examples/open_models/mimo/` - Multimodal In/Out training with support for vision-language and audio-vision-language models

**Specific Multimodal Models:**
- `examples/multimodal/` - LLaVA-style training with Mistral + CLIP
Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/training-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ This example:
Train LLaMA-3 8B model with FP8 mixed precision on 8 GPUs:

```bash
./examples/llama/train_llama3_8b_fp8.sh
./examples/open_models/llama/train_llama3_8b_fp8.sh
```

**Configuration:**
Expand Down
53 changes: 0 additions & 53 deletions examples/bert/README.md

This file was deleted.

79 changes: 0 additions & 79 deletions examples/bert/train_bert_340m_distributed.sh

This file was deleted.

132 changes: 0 additions & 132 deletions examples/mixtral/README.md

This file was deleted.

Loading
Loading