Educational skin monitoring app with agentic AI workflows
In Memory of N. K. Trinh (†2006), whose life was lost to skin cancer. This project is built with the hope that earlier awareness and better tools can help others seek care in time.
See DEDICATION.md for full dedication.
DermaCheck is an educational iOS app that combines 5-node LangGraph agentic workflows with temporal skin analysis to help users track skin spots over time and understand when to seek professional dermatological care.
Key Innovation: Production-ready multi-agent system orchestrating MedGemma 1.5 4B with conditional routing, tool orchestration, and educational knowledge synthesis.
See SUBMISSION.md for complete technical documentation.
- 5-node LangGraph agent with conditional routing
- Intelligent path selection based on image quality and detected changes
- Tool orchestration: MedGemma API, knowledge base, quality checks
- Observable AI: Progressive step visualization in UI
- Production deployment: FastAPI backend on Render.com
- Photo timeline with automatic change detection
- Side-by-side comparison with agent analysis
- Timeline narrative describing evolution over time
- 99.7% latency reduction through intelligent caching
- ABCDE dermatological framework with plain-language explanations
- Confidence scoring with abstention handling
- Three-level urgency guidance (monitor/schedule/seek-care)
- Comprehensive safety disclaimers
- First analysis: ~30 seconds (full agent workflow)
- Cached analysis: <100ms (instant from AsyncStorage)
- Visual indicator (💾 badge) for cached results
┌─────────────────────────────────────┐
│ React Native iOS App (TypeScript) │
│ - Agent workflow visualization │
│ - Markdown rendering │
│ - Comparison caching │
└──────────────┬──────────────────────┘
│ HTTPS
┌──────────────▼──────────────────────┐
│ FastAPI + LangGraph (Python 3.11) │
│ - 5-node StateGraph │
│ - Conditional routing │
│ - Deployed on Render.com │
└──────────────┬──────────────────────┘
│ gRPC
┌──────────────▼──────────────────────┐
│ Vertex AI Model Garden │
│ - Base MedGemma 1.5 4B │
│ - Google-managed endpoint │
└─────────────────────────────────────┘
User Photos → Router → Quality Check → Vision Analysis → Knowledge Base → Synthesis → Validation
↓ ↓ ↑
└───────────┴──────────────────────────────────────────────┘
(conditional routing: quality issues → skip to validation)
Routing Scenarios:
- Quality issues → Skip MedGemma, provide guidance (cost optimization)
- Change detected → Full pipeline with educational content
- No change → Skip knowledge retrieval (efficiency)
- macOS with Xcode 15+ installed
- Node.js 18+
- React Native CLI
- CocoaPods
- iOS Simulator or physical iOS device
# Clone repository
git clone https://github.com/dunktra/DermaCheck.git
cd DermaCheck
# Install dependencies
npm install
# Install iOS pods
cd ios && pod install && cd ..Option 1: Use deployed backend (default)
No configuration needed! The app connects to the production backend on Render.com automatically.
Option 2: Local development (optional)
For backend development, create backend/.env:
GOOGLE_CLOUD_PROJECT_ID=your-project-id
GOOGLE_CLOUD_LOCATION=us-central1
MEDGEMMA_ENDPOINT_ID=your-endpoint-id
MEDGEMMA_ENDPOINT_DNS=your-endpoint-dns
PORT=8000See backend/.env.example for full configuration options.
npm run iosThe app launches in iOS Simulator with pre-loaded sample data demonstrating:
- 3 realistic spots with photo timelines
- Agent workflow visualization
- Temporal comparison with caching
- All three urgency levels
- Framework: React Native 0.76.6
- Language: TypeScript 5.0+
- Storage: AsyncStorage (local persistence)
- Navigation: React Navigation (bottom tabs + nested stacks)
- Image Processing: react-native-image-picker, react-native-compressor
- API Client: Axios
- Framework: FastAPI 0.110 (async web framework)
- Agent: LangGraph StateGraph (5-node workflow)
- Language: Python 3.11
- Observability: LangChain + LangSmith (optional)
- Model: Vertex AI Model Garden (MedGemma 1.5 4B)
- Deployment: Render.com (free tier)
- Dataset: 900 synthetic temporal pairs from HAM10000
- Method: LoRA (Low-Rank Adaptation)
- Platform: Kaggle Notebooks (P100 GPU)
- Published:
dunktra/dermacheck-temporal-pairs(HuggingFace)
DermaCheck/
├── src/ # React Native frontend
│ ├── components/ # Reusable UI components
│ │ ├── AgentWorkflowAccordion.tsx
│ │ ├── MarkdownText.tsx
│ │ └── ...
│ ├── navigation/ # Navigation configuration
│ ├── screens/ # Screen components
│ │ ├── AnalyzeScreen.tsx
│ │ ├── SpotDetailScreen.tsx
│ │ ├── ComparisonScreen.tsx
│ │ └── ...
│ ├── services/ # API and storage
│ │ ├── agentApi.ts # Backend API client
│ │ └── storage.ts # AsyncStorage layer
│ ├── types/ # TypeScript definitions
│ │ ├── agent.ts # AgentState types
│ │ └── spot.ts # Data model types
│ └── utils/ # Utility functions
├── backend/ # FastAPI + LangGraph backend
│ ├── src/
│ │ ├── graph/ # LangGraph agent
│ │ │ ├── workflow.py # StateGraph definition
│ │ │ ├── state.py # AgentState schema
│ │ │ └── nodes/ # Node functions
│ │ ├── tools/ # Tool integrations
│ │ │ ├── medgemma.py # Vertex AI client
│ │ │ └── knowledge.py # ABCDE content
│ │ └── main.py # FastAPI app
│ ├── requirements.txt
│ └── COMPETITION_DEMO.ipynb # Demo notebook
├── ios/ # iOS native code
├── .planning/ # Development planning
│ └── phases/
│ ├── 05-fine-tuning-foundation/
│ ├── 06-agentic-backend/
│ ├── 08.1-ux-architecture-investigation/
│ └── 09-documentation-submission/
├── SUBMISSION.md # Competition submission
├── DEDICATION.md # Memorial dedication
└── README.md # This file
# Run iOS app
npm run ios
# Run TypeScript type checking
npx tsc --noEmit
# Clean iOS build
cd ios && xcodebuild clean && cd ..
# Reset Metro bundler cache
npm start -- --reset-cache
# Run backend locally (optional)
cd backend
uvicorn src.main:app --reload --port 80001. Analyze Tab:
- Take/upload photo
- Watch progressive step indicators
- See agent routing path (Quality → Vision → Knowledge → Synthesis)
- View ABCDE features with markdown rendering
2. Track Tab:
- Navigate to existing spot
- View photo timeline
- Click "Compare" between photos
- First time: 30s agent analysis
- Repeat: <100ms cached result (💾 badge)
3. Agent Visualization:
- Expandable accordion shows workflow steps
- Each node displays status (in_progress → completed)
- Routing path visible:
[router, quality_check, vision_analysis, ...]
Pre-loaded spots demonstrate all features:
- Left Shoulder Mole (6 weeks, 3 photos) - Diameter increase → "Schedule checkup"
- Back of Hand Freckle (8 weeks, 4 photos) - Stable → "Monitor"
- Upper Back Spot (2 weeks, 2 photos) - Border irregularity → "Seek care soon"
- Quality check: <100ms
- Vision analysis (MedGemma): 15-25s
- Knowledge retrieval: <50ms
- Synthesis: <100ms
- Total (full pipeline): 15-30 seconds
- First analysis: 30s (runs full agent)
- Cached analysis: <100ms (AsyncStorage lookup)
- Latency reduction: 99.7%
- Cache hit rate: ~90%
- Cold start (free tier): 30-60s (container spin-up)
- Warm requests: 15-30s (agent execution only)
- Memory usage: ~400MB (well within 512MB limit)
DermaCheck provides educational information only and cannot diagnose skin conditions or replace professional medical evaluation. Always consult healthcare professionals for proper diagnosis and treatment.
AI Limitations:
- Analysis accuracy varies based on image quality and skin tone
- Medical imaging datasets historically underrepresent darker skin tones
- Confidence scoring and abstention handling prevent misleading guidance
Data Privacy:
- All data stored locally on device (AsyncStorage)
- Photos transmitted to backend only for analysis (encrypted HTTPS)
- No cloud backup or sync to external servers
- No user accounts or authentication required
✅ Multi-agent orchestration - 5-node LangGraph with conditional routing ✅ Intelligent routing - Multi-factor decisions (quality, change detection, urgency) ✅ Tool integration - Vertex AI, knowledge base, quality checks ✅ State management - TypedDict with annotated reducers ✅ Observability - LangSmith tracing + progressive UI visualization ✅ Production deployment - FastAPI on Render.com with 99.7% cache optimization
✅ Novel task identification - Temporal skin lesion change detection ✅ Synthetic dataset creation - 900 pairs from HAM10000 (published on HuggingFace) ✅ Memory-efficient fine-tuning - LoRA on Kaggle P100 ✅ Baseline evaluation - Base MedGemma F1: 0.8797 validates approach ✅ Public artifacts - Dataset, training notebook, documentation
Note: Production uses base MedGemma 1.5 4B (sufficient performance). Fine-tuning research demonstrates novel task feasibility and contributes public dataset.
v1.0 MVP (January 19-21, 2026) - 3 days, 10 plans
- Foundation, photo capture, MedGemma integration
- Tracking system, educational content
- Safety features, sample data
v2.0 Prize Enhancement (January 22-31, 2026) - 9 days, 17 plans
- Fine-tuning research (dataset creation, LoRA training)
- Agentic backend (LangGraph implementation)
- UX enhancements (markdown, visualization, caching)
- Production deployment (Render.com integration)
Total: 2 weeks, 27 plans executed, ~8,000 lines of code
- SUBMISSION.md - Complete competition submission (technical details, demo script, FAQ)
- DEDICATION.md - Memorial dedication
- backend/COMPETITION_DEMO.ipynb - Interactive agent demo
- .planning/ - Development planning and phase summaries
MIT
Developer: D. Tran GitHub: https://github.com/dunktra/DermaCheck
- MedGemma Team at Google for the medical AI model
- LangChain/LangGraph for agentic workflow framework
- React Native Community for excellent documentation and libraries
- DermNet NZ and American Academy of Dermatology for educational content references
- N. K. Trinh (†2006) - In loving memory
Combining agentic AI workflows with temporal analysis to make skin cancer monitoring more accessible, educational, and anxiety-reducing.