From f06675638a9057d05bb0e4f8199d4023a4785507 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 12 Feb 2026 13:19:31 +0000 Subject: [PATCH 1/2] Initial plan From b3cea6aa4bc922e83f1c7bd1d8af1834602185a5 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 12 Feb 2026 13:20:52 +0000 Subject: [PATCH 2/2] Update README with comprehensive project description, technologies, and model info Co-authored-by: Matthew-Grayson <106278637+Matthew-Grayson@users.noreply.github.com> --- readme.md | 49 +++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 43 insertions(+), 6 deletions(-) diff --git a/readme.md b/readme.md index 2603569..caf7b39 100644 --- a/readme.md +++ b/readme.md @@ -1,7 +1,44 @@ -run: -cp .env.example .env +# LLM Infrastructure on Oracle Cloud -edit .env and fill values -set -a; source .env; set +a -terraform init -terraform apply -auto-approve +**Terraform infrastructure-as-code to deploy a free-tier LLM server on Oracle Cloud Infrastructure (OCI)** + +This project automatically provisions and configures an ARM64 virtual machine on OCI's Always Free tier, running the [llama.cpp](https://github.com/ggml-org/llama.cpp) server with the **TinyLlama-1.1B-Chat-v1.0** model (GGUF format, Q4_K_M quantization). + +## 🚀 What It Does + +- Provisions a complete OCI infrastructure (VCN, subnet, security groups, compute instance) +- Uses OCI's free-tier **VM.Standard.A1.Flex** ARM64 shape (4 OCPUs, 24GB RAM) +- Automatically installs and configures llama.cpp +- Downloads and deploys the TinyLlama-1.1B model +- Starts an LLM inference server on port 8000 +- Ready for REST API requests and chat interactions + +## 🛠️ Technologies Used + +- **Terraform** - Infrastructure as Code +- **Oracle Cloud Infrastructure (OCI)** - Cloud platform (free tier) +- **llama.cpp** - High-performance LLM inference engine +- **TinyLlama-1.1B-Chat-v1.0** - Lightweight chat model (GGUF Q4_K_M) +- **cloud-init** - Automated instance configuration + +## 📋 Quick Start + +1. **Copy and configure environment variables:** + ```bash + cp .env.example .env + ``` + +2. **Edit `.env` and fill in your OCI credentials and settings** + +3. **Load environment variables:** + ```bash + set -a; source .env; set +a + ``` + +4. **Deploy infrastructure:** + ```bash + terraform init + terraform apply -auto-approve + ``` + +After deployment, the LLM server will be accessible at `http://:8000`