diff --git a/README.md b/README.md new file mode 100644 index 00000000..d3104226 --- /dev/null +++ b/README.md @@ -0,0 +1,141 @@ +# ODC - Open Dev Cloud + +**High-Performance GPU Development Platform** + +ODC (formerly OSDC) provides on-demand GPU reservations for machine learning development and training, running on Kubernetes with persistent storage, multinode support, and seamless IDE integration. + +--- + +## Quick Start + +```bash +# Install the CLI +pip install gpu-dev + +# Configure your GitHub username +gpu-dev config set github_user your-github-username + +# Reserve GPUs +gpu-dev reserve + +# Connect +gpu-dev connect +``` + +--- + +## Features + +- **On-Demand GPU Access** - Reserve B200, H200, H100, A100, and more with a single command +- **Persistent Storage** - Your work survives across reservations with automatic snapshots +- **IDE Integration** - Seamless VS Code and Cursor remote development +- **Multinode Support** - Distributed training across multiple GPU nodes with EFA networking +- **Custom Environments** - Bring your own Docker images or Dockerfiles +- **Jupyter Lab** - Built-in notebook support +- **Claude Code Pre-installed** - AI pair programming out of the box + +--- + +## Documentation + +**Full documentation:** [odc-docs.nest.x2p.facebook.net](https://odc-docs.nest.x2p.facebook.net) *(internal)* + +### Quick Links + +- [User Guide](docs/USER_GUIDE.md) - Comprehensive guide to all features +- [CLI Reference](cli-tools/gpu-dev-cli/README.md) - Complete command reference +- [GitHub Repository](https://github.com/wdvr/osdc) + +--- + +## Installation + +```bash +# Install from GitHub +pip install git+https://github.com/wdvr/osdc.git + +# Or install from local clone +git clone https://github.com/wdvr/osdc.git +cd osdc +pip install -e . +``` + +**Requirements:** +- Python 3.10+ +- AWS credentials configured +- GitHub account with SSH keys + +--- + +## Example Usage + +```bash +# Reserve 4 H100 GPUs for 8 hours with Jupyter +gpu-dev reserve -t h100 -g 4 -h 8 --jupyter + +# Connect via SSH +gpu-dev connect + +# Check availability +gpu-dev avail + +# List your reservations +gpu-dev list + +# Extend a reservation +gpu-dev edit --extend +``` + +--- + +## GPU Types + +| Type | Memory/GPU | Best For | +|------|-----------|----------| +| B200 | 192 GB | Latest NVIDIA Blackwell architecture | +| H200 | 141 GB | Large models, high memory workloads | +| H100 | 80 GB | Production training | +| A100 | 40 GB | General ML training | +| T4 | 16 GB | Development and testing | + +See the [full GPU reference](docs/USER_GUIDE.md#gpu-types-reference) for all available types. + +--- + +## Architecture + +ODC runs on Amazon EKS with: +- Auto-scaling GPU node groups +- EBS persistent storage with snapshots +- EFS shared storage +- EFA networking for multinode +- NVIDIA GPU Operator +- Custom reservation processor + +See [Architecture Overview](docs/USER_GUIDE.md#architecture-overview) for details. + +--- + +## Contributing + +Contributions welcome! Please: +1. Fork the repository +2. Create a feature branch +3. Submit a pull request + +--- + +## License + +[Add license information] + +--- + +## Support + +- Report issues: [GitHub Issues](https://github.com/wdvr/osdc/issues) +- Internal documentation: [odc-docs.nest.x2p.facebook.net](https://odc-docs.nest.x2p.facebook.net) + +--- + +**Project Status:** Active development | Production ready for internal use diff --git a/cli-tools/gpu-dev-cli/README.md b/cli-tools/gpu-dev-cli/README.md index 7280bf4b..becf6500 100644 --- a/cli-tools/gpu-dev-cli/README.md +++ b/cli-tools/gpu-dev-cli/README.md @@ -1,6 +1,8 @@ # GPU Developer CLI -A command-line tool for reserving and managing GPU development servers on AWS EKS. +**ODC (Open Dev Cloud)** - Command-line tool for reserving and managing GPU development servers on AWS EKS. + +*Formerly known as OSDC (Open Source Developer Cloud)* ## Table of Contents @@ -286,6 +288,18 @@ gpu-dev disk rename ``` Renames an existing disk. +#### `gpu-dev disk clone` +```bash +gpu-dev disk clone +``` +Creates a copy of an existing disk from its latest snapshot. + +#### `gpu-dev disk unlock` +```bash +gpu-dev disk unlock +``` +Unlocks a disk that may be locked due to failed operations or cleanup issues. + ### `gpu-dev help` Show help information. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index b406b62e..1b64240a 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -1,6 +1,8 @@ -# ODC User Guide -- v1 +# ODC User Guide -**Open Developer Cloud** - High-Performance GPU Development Platform +**Open Dev Cloud** - High-Performance GPU Development Platform + +*Formerly known as OSDC (Open Source Developer Cloud)* --- @@ -534,6 +536,12 @@ gpu-dev disk list-content my-project # Rename a disk gpu-dev disk rename old-name new-name +# Clone a disk (creates a copy) +gpu-dev disk clone source-disk new-disk-name + +# Unlock a locked disk (if needed for cleanup) +gpu-dev disk unlock my-project + # Delete a disk (snapshots kept for 30 days) gpu-dev disk delete my-project ``` @@ -986,6 +994,24 @@ gpu-dev edit --extend - Maximum extension: 24 additional hours - Total maximum: 48 hours +### Edit Reservation Features + +You can modify several aspects of an active reservation: + +```bash +# Enable Jupyter Lab on an existing reservation +gpu-dev edit --enable-jupyter + +# Disable Jupyter Lab +gpu-dev edit --disable-jupyter + +# Add a collaborator (by GitHub username) +gpu-dev edit --add-user colleague-username + +# Extend the duration +gpu-dev edit --extend +``` + ### Cancel a Reservation ```bash @@ -1403,6 +1429,8 @@ A: Reserve more GPUs, or use a GPU type with better CPU ratio (like A10G or T4). | `gpu-dev disk delete NAME` | Delete a disk | | `gpu-dev disk list-content NAME` | View disk contents | | `gpu-dev disk rename OLD NEW` | Rename a disk | +| `gpu-dev disk clone SOURCE NEW` | Clone a disk | +| `gpu-dev disk unlock NAME` | Unlock a locked disk | | `gpu-dev config show` | Show configuration | | `gpu-dev config set KEY VALUE` | Set a config value | | `gpu-dev config ssh-include enable` | Enable SSH config integration |