DepthWizard: Single-View Height Estimation and 3D Flythrough

Smart India Hackathon (SIH 2026)
Problem Statement ID: 26175
Organization: Indian Space Research Organisation (ISRO) / Department of Space
Theme: Disaster Management
Category: Software

---

🎥 Official SIH Submission Demo Video

---

📌 Executive Summary

DepthWizard is an end-to-end web platform designed for the Indian Space Research Organisation (ISRO) to transform single-view 2D optical satellite imagery and Digital Elevation Models (DEMs) into interactive, metric-accurate 3D terrain assets.

By combining deep learning monocular depth estimation backbones (ZoeDepth and DepthAnythingV2) with high-performance C-level GIS ingestion (Rasterio / GDAL) and GPU-accelerated WebGL rendering (Three.js / GLSL Shaders), DepthWizard bridges the gap between static remote-sensing rasters and real-time interactive 3D flight reconnaissance in the browser.

---

📸 Product Renders & Visual Showcase

Release v1.4.0 (Latest: Disaster Slope Hazard, GCP Calibration & Multi-Format Export)

Features real-time GPU slope hazard classification, least-squares GCP affine calibration, 1024x1024 high-fidelity mesh engine, and 1-click export to .OBJ, .PNG, and 32-bit GeoTIFF.

Kedarnath Valley — Hypsometric Elevation	Kedarnath Valley — Topographic Contours
Structural Wireframe Overlay	Dynamic Solar Hillshading

Figure 1: High-resolution 1-channel GeoTIFF elevation raster of the Kedarnath Valley rendered with GPU-accelerated procedurals.

---

Release v1.2.0 (Aerospace UI & Real-Time Telemetry HUD)

Introduced dark-mode glassmorphism interface and 60 FPS raycasting telemetry.

Tactical Dashboard Layout	Martian Mountains Topography
Martian Surface Angled View	Lunar Surface Telemetry HUD

---

Release v1.0.0 (Core Engine Prototype Renders)

Initial proof-of-concept pipeline translating optical 2D satellite rasters into 3D meshes.

Global Topography	UK Coastal Raster	Burj Khalifa Overhead

---

🚀 Release History & Version Evolution

Version 1.4.1 (FPV Drone & UI Polishing)

• Autonomous FPV Drone Flythrough: Resolved frame-delta locks allowing seamless First-Person View (FPV) flight through the 3D terrain using keyboard and mouse controls.
• GCP UI/UX Enhancements: Streamlined calibration pinning workflow and real-time metrics telemetry.

Version 1.4.0 (Full Disaster Management & Geospatial Export Suite)

• Real-Time 3D Slope Hazard Heatmap (Landslide Risk Assessment):
  • Instant GPU-accelerated terrain gradient calculation classifying slopes into:
    • 🟩 Safe Zone ($0^\circ - 30^\circ$): #22c55e (Emerald Green)
    • 🟨 Warning Zone ($30^\circ - 45^\circ$): #eab308 (Amber Yellow)
    • 🟥 High Landslide Risk ($> 45^\circ$): #ef4444 (Crimson Red)
  • Compensates for artificial $Z$-scale visual exaggeration so physical slope angles remain geometrically accurate.
  • Includes a dedicated tactical glassmorphic HUD legend card.
• Interactive GCP (Ground Control Point) Metric Scale Calibration:
  • Interactive pinning tool allowing operators to place 3+ known elevation GCPs directly on the 3D terrain.
  • Fits an absolute least-squares affine transformation model ($\text{Height} = \alpha \cdot d_{\text{rel}} + \beta$).
  • Real-time $R^2$ goodness-of-fit and RMSE (Root Mean Square Error) telemetry readout.
• 1-Click Multi-Format Geospatial Export Suite:
  • 📦 3D Mesh (.OBJ): Exports reconstructed Wavefront 3D terrain geometry with calibrated affine scaling for Blender, Unity, and Unreal Engine.
  • 🖼️ DEM Raster (.PNG): High-contrast 8-bit elevation raster matrix.
  • 🗺️ Scientific GeoTIFF (.tif): 32-bit floating-point metric GeoTIFF with embedded Ground Sample Distance (GSD) for QGIS and ArcGIS.
• High-Fidelity Mesh Engine: Dynamic toggle between standard 512x512 and ultra-dense 1024x1024 vertex resolution grids for micro-terrain fidelity.
• Zero-Latency Client WebGPU Fallback: Integrated ONNX Runtime Web for local in-browser depth inference.

Version 1.3.1 (Heavy-Duty GIS & Metric Engine Update)

• 500MB+ GeoTIFF Scale Unlock: Completely dismantled file-size constraints to support high-density satellite rasters, Sentinel-2 imagery, and SRTM DEM datasets up to 500MB+.
• Native DEM Bypass Pipeline: Automatically detects single-band floating-point GeoTIFFs, bypassing neural depth estimation to preserve absolute metric height scales without squashing terrain features.
• Quad Texture Shading Modes:
  1. Colored Mode: Procedural 7-color Hypsometric Tinting (#1b382b emerald valley to #f8f9fa glacial peak).
  2. Grayscale Mode: Mathematical Rec. 601 luminance desaturation (dot(topoColor, vec3(0.299, 0.587, 0.114))) preserving identical contour features in high-contrast black-and-white.
  3. Original Mode: Overlays true-color optical RGB satellite textures on the physical mesh.
  4. Slope Hazard Mode: Real-time slope steepness classification for disaster reconnaissance.
• Expanded Vertical Exaggeration: Dynamic Z-Scale slider expanded up to 25.0x for deep ravine and cliff visualization.

Version 1.3.0 (The Kedarnath Scientific GeoTIFF Upgrade)

• Rasterio GIS Engine Integration: Replaced legacy image parsers with GDAL-backed rasterio to safely spool single-band 32-bit floating-point TIFFs and extract spatial Ground Sample Distance (GSD).
• GPU Shaders (onBeforeCompile): Offloaded procedural colormapping, topographic contour rendering (fwidth derivatives), and tactical grid overlays to custom WebGL GLSL fragment shaders.
• Physical CPU Vertex Displacement & Lighting: Physical CPU/GPU vertex displacement (512x512 / 1024x1024 grid) and procedural normal computation for realistic directional solar hillshading.

Version 1.2.0 (Aerospace Dashboard & Telemetry HUD)

• Real-Time Telemetry HUD: 60 FPS mouse-hover raycasting displaying 3D world coordinates $(X, Y, Z)$, relative elevation percentage, estimated height ($m$), and UV coordinates.
• Offscreen Canvas Caching: Zero-latency pixel caching bypassing WebGL read-pixel GPU bottlenecks.
• Aerospace Dark-Mode UI: Revamped glassmorphism tactical control panel built for high legibility during operations.

Version 1.0.0-Alpha (Core Proof of Concept)

• End-to-End Pipeline: Established FastAPI + PyTorch backend communicating with React/Vite Three.js frontend.
• Monocular Inference: Relative depth estimation powered by DepthAnythingV2.

---

✅ Solved Milestones & Capability Status

Unlike early prototype drafts, the following capabilities have been fully resolved and deployed in production:

• Real-Time Slope & Landslide Hazard Shader: Solved in v1.4.0. Physical surface gradient classification ($0^\circ-30^\circ$ safe, $30^\circ-45^\circ$ warning, $>45^\circ$ danger) with interactive HUD card.
• Interactive GCP Metric Scale Calibration: Solved in v1.4.0. Least-squares affine solver ($\text{Height} = \alpha \cdot d_{\text{rel}} + \beta$) with live $R^2$ and RMSE telemetry.
• Multi-Format Export Suite: Solved in v1.4.0. 1-click export to .OBJ 3D mesh, 8-bit DEM .PNG, and 32-bit floating-point .tif GeoTIFF for QGIS/ArcGIS.
• ZoeDepth & Real-World Metric Depth Calibration: Solved in v1.1.0/v1.3.0 via backend/services/depth_engine.py. Operates natively alongside DepthAnythingV2 for absolute metric depth estimation.
• Aerospace UI Rebuild: Solved in v1.2.0. Complete dark-mode glassmorphism interface overhaul with responsive sidebars and dynamic HUD telemetry.
• 1-Band Scientific GeoTIFF Support: Solved in v1.3.0/v1.3.1. High-throughput rasterio GDAL C-engine integration for single-channel elevation rasters.
• Automated GIS Math Testing & CI: Pytest unit test suite validating GeoTIFF ingestion, GSD resolution scaling, and GCP affine calibration.
• Autonomous FPV Drone Flythrough: Solved in v1.4.1. Fully functional WASD + Mouse First-Person camera controls synced with real-time delta framerates.

---

🏗️ System Architecture & Pipeline

                              [ User Upload ]
                       (JPG, PNG, or 1-Band GeoTIFF)
                                     │
                                     ▼
                ┌────────────────────────────────────────┐
                │          FastAPI Backend Engine        │
                ├────────────────────────────────────────┤
                │ 1. Rasterio / GDAL GIS Parser          │
                │ 2. Single-Band vs RGB Auto-Detection   │
                │ 3. ZoeDepth / DepthAnythingV2 PyTorch  │
                │ 4. Native Metric DEM Preservation      │
                │ 5. Least-Squares GCP Affine Solver     │
                │ 6. 32-bit GeoTIFF Export Generator     │
                └────────────────────────────────────────┘
                                     │
                                     ▼
                      [ Normalized Height Array ]
                       + Texture Base64 Payload
                                     │
                                     ▼
                ┌────────────────────────────────────────┐
                │          Three.js WebGL Frontend       │
                ├────────────────────────────────────────┤
                │ 1. 512x512 / 1024x1024 Grid Geometry   │
                │ 2. Procedural View-Space Normals       │
                │ 3. Quad GLSL Shader Pipeline:          │
                │    - 7-Color Hypsometric Topography    │
                │    - Rec. 601 Luminance Grayscale      │
                │    - Optical RGB Texture Projection    │
                │    - GPU Slope Landslide Hazard Map    │
                │ 4. Offscreen Canvas HUD Telemetry      │
                │ 5. Multi-Format Exporter (.OBJ / .PNG) │
                └────────────────────────────────────────┘

---

🎯 Alignment with ISRO Problem Statement 26175

Requirement	ISRO PS 26175 Specification	DepthWizard Implementation
Non-Georeferenced RGB	Produce Relative Digital Surface Model (rDSM) for PNG/JPG	ZoeDepth / DepthAnythingV2 monocular depth extraction with WebGPU fallback
Georeferenced RGB / TIFF	Produce Absolute Digital Surface Model (DSM) for GeoTIFF	Native 1-band float32 GeoTIFF parsing with rasterio and metric preservation
Scale Calibration	Calibrate relative depth to absolute metric units ($m$)	Interactive GCP pin placement with least-squares affine fitting ($\text{Height} = \alpha \cdot d + \beta$)
3D Rendering Engine	Project optical image onto generated 3D mesh in WebGL	Three.js shader pipeline with high-fidelity displacement (512x512 / 1024x1024)
Interactive Navigation	First-person & aerial 3D terrain analysis	60 FPS orbit controls, Autonomous FPV Flythrough, and raycasting HUD
Disaster Management	Support structural height, slope, and terrain analysis	Real-time GPU slope hazard heatmap ($0^\circ-30^\circ$ safe, $30^\circ-45^\circ$ warning, $>45^\circ$ danger)
GIS Interoperability	Export to standard geospatial and 3D formats	1-click export to 32-bit float GeoTIFF (.tif), Wavefront 3D Mesh (.OBJ), and DEM (.PNG)

---

📦 3D Asset & Geospatial Export Suite

DepthWizard bridges the gap between web-based interactive exploration and production GIS pipelines by enabling 1-click downloads across three industry-standard formats:

Format	Extension	Target Use Case	Technical Pipeline
Wavefront 3D Mesh	.OBJ	3D Visualization, Blender, Unity, Unreal Engine	CPU vertex extraction with affine elevation scaling ($\alpha \cdot d + \beta$) and UV mapping
DEM Elevation Raster	.PNG	Fast 2D preview, raster heightmaps	8-bit high-contrast grayscale normalization matrix
Scientific GeoTIFF	.tif	QGIS, ArcGIS, Hydrological & Slope Modeling	32-bit floating-point raster encoded via rasterio with spatial GSD resolution tags

---

🏔️ Real-Time Slope Hazard & Landslide Risk Assessment

Designed specifically for the ISRO Disaster Management theme, DepthWizard features a real-time GPU-accelerated Slope Hazard classification engine:

• Physical Slope Extraction: Computes exact terrain gradients in real time using partial derivatives (dFdx, dFdy) on the world-space mesh.
• Z-Scale Neutralization: Automatically cancels artificial visual vertical exaggeration so physical slope angles remain true to reality.
• Tactical Classification:
  • 🟩 Safe Terrain ($0^\circ - 30^\circ$): Flat plains and gentle foothills.
  • 🟨 Warning Slope ($30^\circ - 45^\circ$): Moderate inclines requiring monitoring.
  • 🟥 High Landslide Risk ($> 45^\circ$): Dangerous steep ravines and escarpments prone to slope failure.
• Glassmorphic HUD Legend: Displays an automated tactical overlay card when Slope Hazard mode is activated.

---

📌 Interactive Ground Control Point (GCP) Metric Scale Calibration

For non-georeferenced optical images, DepthWizard allows field operators to calibrate relative depth into authentic metric heights ($m$):

1. Pinning Mode: Click anywhere on the 3D terrain to pin known reference landmarks (e.g. river level, valley floor, mountain peak).
2. Elevation Assignment: Enter known ground-truth elevations ($m$) for 3+ control points.
3. Least-Squares Solver: Solves the linear affine system $\text{Height} = \alpha \cdot d_{\text{rel}} + \beta$ and outputs live goodness-of-fit telemetry:
   • $R^2$ Score (Correlation coefficient)
   • RMSE (Root Mean Square Error in meters)
4. Live GPU Adaptation: Instantly updates shader uniforms and mesh geometry without reloading.

---

🛠️ Technology Stack

Backend Infrastructure

• Language: Python 3.10+
• API Framework: FastAPI (Uvicorn ASGI)
• ML Engines: PyTorch, HuggingFace Transformers (ZoeDepth, DepthAnythingV2)
• GIS Processing: Rasterio, GDAL C-Engine, OpenCV, NumPy

Frontend Infrastructure

• Core Framework: React 18 + Vite (TypeScript)
• Rendering Engine: Three.js / @react-three/fiber / @react-three/drei
• Styling: Vanilla CSS & Tailwind CSS (v4)
• State Management: Zustand

---

💻 Quick Start & Installation Guide

Option A: 🐳 Single-Command Docker Setup (Recommended for Evaluators)

Tip

Zero-Troubleshooting Deployment: Avoid CUDA, Python version, Node, and C-level GDAL dependency mismatches by running the entire stack in isolated containers with a single command:

git clone https://github.com/jhaabhijeet864/DepthWizard.git
cd DepthWizard

# Build and launch both Backend and Frontend instantly:
docker compose up --build

• Access the Frontend Web App: http://localhost:5173
• Access the Backend FastAPI Swagger Docs: http://localhost:8001/docs

---

Option B: 🛠️ Manual Local Setup

Prerequisites

• Python 3.10 or higher
• Node.js v18.0 or higher
• npm or yarn
• GDAL Binaries (for scientific GeoTIFF parsing via rasterio)

🌐 GIS & GDAL Dependency Setup

Important

rasterio depends on C-level GDAL binaries. Ensure system GDAL libraries are installed prior to running Python pip dependencies:

• Windows: Modern PyPI wheels package pre-compiled GDAL binaries (pip install rasterio). Alternatively, use Conda:

  conda install -c conda-forge rasterio

• Linux (Ubuntu / Debian):

  sudo apt-get update && sudo apt-get install -y gdal-bin libgdal-dev

• MacOS:

  brew install gdal

1. Clone Repository

git clone https://github.com/jhaabhijeet864/DepthWizard.git
cd DepthWizard

2. Backend Setup

Note

The backend server runs on Port 8001 (shifted from 8000 to prevent socket collisions with system services on Windows).

cd backend
python -m venv venv

# Windows Activation:
.\venv\Scripts\activate

# Linux / MacOS Activation:
source venv/bin/activate

pip install -r requirements.txt
uvicorn main:app --reload --port 8001

Backend API service will run at http://localhost:8001 (Swagger documentation available at http://localhost:8001/docs)

3. Frontend Setup

Open a new terminal window:

cd frontend
npm install
npm run dev

Frontend web application will run at http://localhost:5173

---

👥 Development Team & Acknowledgments

• Abhijeet Jha — Team Leader & AI/ML Engineer
• Rohan Jha — AI Engineer & Documentation Lead
• Aman Sharma — Backend & Code Maintenance Lead
• Ayush Saxena — Backend Lead
• Suruchi Kumari — Frontend & UI/UX Lead
• Shreyashi Dwivedi — Research and Presentation Lead