This application transforms traditional OCR into an intelligent document analysis platform by combining AWS Textract with Amazon Bedrock's Claude AI. It demonstrates autonomous AI agent capabilities by not just extracting text, but understanding, categorizing, and structuring document content without human intervention.
- Autonomous Decision Making: The system automatically selects specialized analysis strategies based on document type
- Complex Task Execution: Combines multiple AWS services (Textract β Bedrock β S3) in a coordinated workflow
- Intelligent Reasoning: Uses Claude 3 Haiku to extract structured insights, identify entities, and understand context
- Self-Service Operation: Processes documents end-to-end without human intervention once configured
- General Summary: Extracts key points, entities (people, organizations, dates, locations), and document type
- Invoice/Receipt Analysis: Automatically extracts vendor, amounts, line items, dates, and tax information
- Contract Review: Identifies parties, key terms, obligations, important clauses, and dates
- Form Data Extraction: Structures form fields, checkboxes, and completeness status
- OCR with AWS Textract: High-accuracy text extraction from PDFs and images
- Real-time Status Tracking: Live updates during document processing
- CSV & JSON Export: Download raw text and structured analysis results
- Document History: Access previous analyses without reprocessing
- Free Tier: 5 documents/month, 2 LLM analyses, 3 pages max
- Pro Tier: 200 documents/month, 50 LLM analyses, 50 pages max ($10/month)
- Enterprise Tier: 1000 documents/month, 500 LLM analyses, API access ($99/month)
- Google OAuth 2.0 integration
- Secure S3 presigned URLs for downloads
- API key authentication for Enterprise users
graph TB
subgraph "Client Layer"
User[π€ User Browser]
end
subgraph "Vercel Platform"
subgraph "Flask Application"
WebApp[π Flask Web Server]
LLM[π§ LLM Analyzer<br/>Claude Integration]
Auth[π Authentication]
end
end
subgraph "AWS Cloud Services"
S3[π¦ Amazon S3<br/>Document Storage]
Textract[π AWS Textract<br/>OCR Processing]
Bedrock[π€ Amazon Bedrock<br/>Claude 3 Haiku]
end
subgraph "External Services"
Google[π Google OAuth]
Stripe[π³ Stripe Payments]
DB[(ποΈ PostgreSQL)]
end
User -->|Upload Document| WebApp
WebApp -->|Store| S3
WebApp -->|Extract Text| Textract
Textract -->|Read| S3
WebApp -->|Analyze| Bedrock
Bedrock -->|Structured Insights| WebApp
WebApp -->|Save Results| S3
WebApp -->|Track Usage| DB
User -->|Authenticate| Google
User -->|Subscribe| Stripe
classDef aws fill:#FF9900,stroke:#232F3E,stroke-width:2px
classDef vercel fill:#000000,stroke:#FFFFFF,stroke-width:2px,color:#FFFFFF
classDef external fill:#4285F4,stroke:#1967D2,stroke-width:2px,color:#FFFFFF
class S3,Textract,Bedrock aws
class WebApp,LLM,Auth vercel
class Google,Stripe,DB external
1. Document Upload
β
2. AWS Textract OCR
β
3. Text Extraction
β
4. [AI Agent Decision Point]
ββ Check user quota
ββ Determine analysis type
ββ Select specialized prompt
β
5. Amazon Bedrock (Claude 3 Haiku)
ββ Understand document context
ββ Extract structured data
ββ Generate insights
β
6. JSON Structure & Validation
β
7. Store Results (S3 + Database)
β
8. Present to User
- Purpose: OCR text extraction from documents
- Features Used:
StartDocumentTextDetectionfor async processingGetDocumentTextDetectionfor result retrieval- Multi-page document support
- Cost: ~$0.0015 per page
- Purpose: LLM-powered document intelligence
- Features Used:
- Anthropic Claude 3 Haiku model
- Structured JSON output generation
- Low-temperature inference for consistency
- Cost: ~$0.25 per 1M input tokens, ~$1.25 per 1M output tokens
- Why Claude 3 Haiku: Fast, cost-effective, excellent at structured extraction
- Purpose: Document and result storage
- Features Used:
- Document upload and storage
- CSV and JSON result hosting
- Presigned URLs for secure downloads
- 90-730 day retention policies
- Cost: ~$0.023 per GB per month
- Purpose: Security and access control
- Permissions Required:
s3:PutObject,s3:GetObjecttextract:StartDocumentTextDetection,textract:GetDocumentTextDetectionbedrock:InvokeModel
- Python 3.9 or higher
- AWS Account with Bedrock access (see setup steps below)
- Google Cloud Project (for OAuth)
- Stripe Account (for payments)
- PostgreSQL database (Neon recommended)
- AWS CLI (optional, for manual testing)
git clone <repository-url>
cd <project-directory># Windows
python -m venv venv
venv\Scripts\activate
# macOS/Linux
python3 -m venv venv
source venv/bin/activatepip install -r requirements.txtNote: All required dependencies are listed in requirements.txt, including:
boto3- AWS SDK for Python (includes Bedrock support)Flask- Web frameworkpsycopg2-binary- PostgreSQL adapterstripe- Payment processingcryptography- Secure key generation
- Go to AWS Console β Amazon Bedrock
- Navigate to "Model access" in the left sidebar
- Click "Manage model access"
- Enable "Anthropic Claude 3 Haiku"
- Wait for access to be granted (usually instant)
- Go to AWS Console β IAM
- Create new user with programmatic access
- Attach the following policy (also available in
.iampolicyaws.json):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::YOUR_BUCKET_NAME",
"arn:aws:s3:::YOUR_BUCKET_NAME/*"
]
},
{
"Effect": "Allow",
"Action": [
"textract:StartDocumentTextDetection",
"textract:GetDocumentTextDetection"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel"
],
"Resource": "arn:aws:bedrock:*::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
}
]
}Important: Replace YOUR_BUCKET_NAME with your actual S3 bucket name.
- Save the Access Key ID and Secret Access Key
Minimum Required Permissions:
s3:PutObject,s3:GetObject,s3:ListBucket- For document storagetextract:StartDocumentTextDetection,textract:GetDocumentTextDetection- For OCRbedrock:InvokeModel- For LLM analysis with Claude 3 Haiku
aws s3 mb s3://your-bucket-name --region us-east-1- Go to Google Cloud Console
- Create a new project or select existing
- Navigate to "APIs & Services" β "Credentials"
- Create OAuth 2.0 Client ID
- Add authorized redirect URIs:
http://127.0.0.1:5000/login/callback(local)https://your-app.vercel.app/login/callback(production)
- Save Client ID and Client Secret
- Sign up at Stripe
- Create products for Pro and Enterprise tiers
- Get API keys from Dashboard
- Set up webhook endpoint:
https://your-app.vercel.app/stripe-webhook
Create a .env file in the project root:
# AWS Configuration
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
S3_BUCKET=your-bucket-name
# Note: AWS_REGION is used for all AWS services (S3, Textract, and Bedrock)
# Ensure your region supports Amazon Bedrock with Claude 3 Haiku.
# Recommended regions for full feature support:
# - us-east-1 (N. Virginia) - RECOMMENDED
# - us-west-2 (Oregon)
# - ap-southeast-1 (Singapore)
# - eu-central-1 (Frankfurt)
# - eu-west-3 (Paris)
#
# Note: us-west-1 (N. California) has limited Bedrock model availability
# For full list, see: https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-regions.html
# Google OAuth
GOOGLE_CLIENT_ID=your_client_id.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=your_client_secret
GOOGLE_DISCOVERY_URL=https://accounts.google.com/.well-known/openid-configuration
# Stripe Payment Configuration
STRIPE_SECRET_KEY=sk_test_...
STRIPE_PUBLISHABLE_KEY=pk_test_...
STRIPE_WEBHOOK_SECRET=whsec_...
STRIPE_SUBSCRIPTION_PRICE_ID=price_... # Pro tier ($10/month)
STRIPE_ENTERPRISE_PRICE_ID=price_... # Enterprise tier ($99/month)
STRIPE_ONETIME_PRICE_ID=price_... # One-time payment option
STRIPE_MODE=payment # Default mode: 'payment' or 'subscription'
# Database
DATABASE_URL=postgresql://user:password@host:5432/dbname
# Flask Configuration
SECRET_KEY=your_secret_key_here
FLASK_ENV=development
# Admin Configuration (optional)
ADMIN_EMAIL=admin@example.com| Variable | Required | Description |
|---|---|---|
AWS_ACCESS_KEY_ID |
Yes | AWS IAM user access key with S3, Textract, and Bedrock permissions |
AWS_SECRET_ACCESS_KEY |
Yes | AWS IAM user secret key |
AWS_REGION |
Yes | AWS region (must support Bedrock, e.g., us-east-1, us-west-2) |
S3_BUCKET |
Yes | S3 bucket name for document storage |
GOOGLE_CLIENT_ID |
Yes | Google OAuth 2.0 client ID |
GOOGLE_CLIENT_SECRET |
Yes | Google OAuth 2.0 client secret |
GOOGLE_DISCOVERY_URL |
Yes | Google OpenID discovery URL (usually constant) |
STRIPE_SECRET_KEY |
Yes | Stripe secret API key (use test key for development) |
STRIPE_PUBLISHABLE_KEY |
Yes | Stripe publishable API key |
STRIPE_WEBHOOK_SECRET |
Yes | Stripe webhook signing secret |
STRIPE_SUBSCRIPTION_PRICE_ID |
Yes | Stripe price ID for Pro tier subscription |
STRIPE_ENTERPRISE_PRICE_ID |
Yes | Stripe price ID for Enterprise tier subscription |
STRIPE_ONETIME_PRICE_ID |
No | Stripe price ID for one-time payments |
STRIPE_MODE |
No | Payment mode: 'payment' or 'subscription' (default: payment) |
DATABASE_URL |
Yes | PostgreSQL connection string |
SECRET_KEY |
Yes | Flask secret key for session management (generate random string) |
FLASK_ENV |
No | Flask environment: 'development' or 'production' |
ADMIN_EMAIL |
No | Admin email for special privileges |
Run the provided test script to verify all AWS services:
python test_bedrock_connection.pyThis script will test:
- AWS credentials configuration
- S3 bucket access
- Amazon Bedrock API access
- Claude 3 Haiku model invocation
Expected Output: All tests should pass (β ). If the Bedrock Invoke test fails, you need to:
- Go to AWS Console β Amazon Bedrock β Model access
- Click "Manage model access"
- Enable "Anthropic Claude 3 Haiku"
- Wait 1-2 minutes for access to be granted
- Run the test script again
Test that your IAM user has the correct permissions:
# Test S3 access
aws s3 ls s3://your-bucket-name --region us-east-1
# Test Textract access (requires a test document in S3)
aws textract start-document-text-detection \
--document-location '{"S3Object":{"Bucket":"your-bucket-name","Name":"test.pdf"}}' \
--region us-east-1
# Test Bedrock access (list available models)
aws bedrock list-foundation-models --region us-east-1
# Verify Claude 3 Haiku is accessible
aws bedrock list-foundation-models \
--by-provider anthropic \
--region us-east-1 \
--query 'modelSummaries[?modelId==`anthropic.claude-3-haiku-20240307-v1:0`]'If any command fails, review your IAM policy and ensure:
- The IAM user has the policy attached (see
.iampolicyaws.json) - Amazon Bedrock model access is enabled in the AWS Console
- Your region supports Bedrock (us-east-1, us-west-2, etc.)
- Wait a few minutes after enabling model access for permissions to propagate
flask --app api.index:create_app init-dbflask --app api.index:create_app run --debugVisit http://127.0.0.1:5000
# Install Vercel CLI
npm install -g vercel
# Deploy
vercel
# Add environment variables in Vercel dashboard
# Settings β Environment Variables
# Copy all variables from your .env fileImportant: When deploying to Vercel, ensure:
- All environment variables are added to the Vercel project settings
- AWS_REGION is set to a Bedrock-supported region
- IAM credentials have all required permissions
- Stripe webhook endpoint is updated to your production URL
Enterprise tier users get API access for programmatic document processing.
- Log in to the application
- Upgrade to Enterprise tier
- Navigate to Settings β API Keys
- Click "Generate New Key"
- Copy and save the key (shown only once)
POST /api/v1/analyze
Authorization: Bearer YOUR_API_KEY
Content-Type: multipart/form-data
# Parameters:
# - file: Document file (PDF or image)
# - analysis_type: general|invoice|contract|form (optional)
# - enable_llm: true|false (optional, default: true)
# Example with curl:
curl -X POST https://your-app.vercel.app/api/v1/analyze \
-H "Authorization: Bearer YOUR_API_KEY" \
-F "file=@document.pdf" \
-F "analysis_type=invoice" \
-F "enable_llm=true"
# Response:
{
"job_id": "abc123",
"status": "processing",
"status_url": "https://your-app.vercel.app/api/v1/status/abc123"
}GET /api/v1/status/{job_id}
Authorization: Bearer YOUR_API_KEY
# Response:
{
"job_id": "abc123",
"status": "completed",
"result_url": "https://your-app.vercel.app/api/v1/result/abc123"
}GET /api/v1/result/{job_id}
Authorization: Bearer YOUR_API_KEY
# Response:
{
"job_id": "abc123",
"csv_url": "https://s3.amazonaws.com/...",
"json_url": "https://s3.amazonaws.com/...",
"analysis": {
"vendor": "Acme Corp",
"invoice_number": "INV-001",
"total_amount": "1234.56",
...
}
}- 100 requests per hour per API key
- Returns
429 Too Many Requestswhen exceeded
{
"summary": "Employment agreement between TechCorp and John Doe for Software Engineer position starting January 2024.",
"key_points": [
"Full-time employment as Software Engineer",
"Annual salary of $120,000",
"Standard benefits package included",
"90-day probation period"
],
"document_type": "employment_contract",
"entities": {
"people": ["John Doe"],
"organizations": ["TechCorp Inc."],
"dates": ["2024-01-15", "2024-04-15"],
"locations": ["San Francisco, CA"]
}
}{
"vendor": "Office Supplies Co.",
"invoice_number": "INV-2024-001",
"date": "2024-01-15",
"due_date": "2024-02-15",
"total_amount": "1,234.56",
"currency": "USD",
"line_items": [
{
"description": "Printer Paper (10 reams)",
"quantity": 10,
"unit_price": "25.00",
"total": "250.00"
},
{
"description": "Ink Cartridges",
"quantity": 5,
"unit_price": "45.00",
"total": "225.00"
}
],
"subtotal": "1,134.56",
"tax": "100.00"
}{
"contract_type": "Service Agreement",
"parties": ["ABC Corp", "XYZ Services LLC"],
"effective_date": "2024-01-01",
"expiration_date": "2025-01-01",
"key_terms": [
{
"term": "Payment Terms",
"details": "Net 30 days from invoice date"
},
{
"term": "Termination",
"details": "Either party may terminate with 30 days written notice"
}
],
"obligations": {
"ABC Corp": [
"Pay monthly service fee of $5,000",
"Provide access to necessary systems"
],
"XYZ Services LLC": [
"Deliver monthly reports",
"Maintain 99.9% uptime SLA"
]
},
"important_clauses": [
"Confidentiality agreement",
"Limitation of liability",
"Dispute resolution through arbitration"
]
}| Service | Cost per Document | Notes |
|---|---|---|
| AWS Textract | $0.0015 | 1 page average |
| Amazon Bedrock | $0.0001 | ~400 tokens |
| S3 Storage | $0.000023 | Per MB per month |
| Total | ~$0.0016 | Per LLM analysis |
| Tier | Monthly Price | Max LLM Analyses | AWS Cost | Margin |
|---|---|---|---|---|
| Free | $0 | 2 | $0.0032 | -100% (acquisition) |
| Pro | $10 | 50 | $0.08 | 99.2% |
| Enterprise | $99 | 500 | $0.80 | 99.2% |
- Clean, modern design with drag-and-drop support
- Real-time quota display
- Analysis type selector (General, Invoice, Contract, Form)
- Enable/disable AI analysis toggle
- Tier comparison table
- Tabbed interface: "Text" and "Analysis" views
- Formatted JSON display with syntax highlighting
- Download buttons for CSV and JSON
- Analysis type badge
- Processing time display
- Table view of all processed documents
- Sortable by date, filename, type
- Quick access to previous results
- Download links for CSV and JSON
- Pagination for large histories
Error: AccessDeniedException: User is not authorized to perform: bedrock:InvokeModel
Solutions:
- Verify IAM policy includes Bedrock permissions (see
.iampolicyaws.json) - Check that model access is enabled in AWS Console β Bedrock β Model access
- Ensure your AWS region supports Bedrock
- Wait a few minutes after enabling model access for permissions to propagate
Error: ValidationException: The provided model identifier is invalid
Solutions:
- Verify you're using the correct model ID:
anthropic.claude-3-haiku-20240307-v1:0 - Check that Claude 3 Haiku is available in your region
- Ensure model access is granted in the Bedrock console
Error: InvalidSignatureException or connection timeouts
Solutions:
- Change
AWS_REGIONto a Bedrock-supported region:- us-east-1 (N. Virginia) - Recommended
- us-west-2 (Oregon)
- ap-southeast-1 (Singapore)
- eu-central-1 (Frankfurt)
- Update your S3 bucket to the same region for optimal performance
Symptoms: Documents process but no JSON analysis appears
Solutions:
- Check browser console for JavaScript errors
- Verify "Enable AI Analysis" checkbox is selected during upload
- Check that you haven't exceeded your LLM quota
- Review Flask logs for Bedrock API errors
- Ensure
api/llm_service.pyexists and is properly imported
Error: OperationalError: no such table: document_history
Solutions:
- Run database initialization:
flask --app api.index:create_app init-db - If using Flask-Migrate:
flask db upgrade - Check DATABASE_URL is correct and accessible
- Upload document with LLM analysis enabled
- Verify both CSV and JSON results generated
- Test each analysis type (general, invoice, contract, form)
- Verify quota enforcement for Free tier
- Test monthly quota reset
- Upload document exceeding page limit
- Test history page display
- View historical document results
- Test download links from history
- Test Enterprise API authentication
- Verify API rate limiting
Include test documents in test_documents/ directory:
sample_letter.pdf- General documentsample_invoice.pdf- Invoice with line itemssample_contract.pdf- Service agreementsample_form.pdf- Application form
- API Keys: Cryptographically secure random generation
- S3 Access: Presigned URLs with 5-minute expiration
- Authentication: Google OAuth 2.0 with HTTPS required
- Rate Limiting: 100 requests/hour for API endpoints
- Input Validation: File type and size restrictions
- SQL Injection: SQLAlchemy ORM with parameterized queries
- XSS Protection: Jinja2 auto-escaping enabled
- Multi-language OCR support
- Batch document processing
- Webhook notifications for async results
- Custom LLM prompts for Enterprise users
- Document comparison and diff analysis
- Integration with Slack, email, and business tools
- Advanced analytics dashboard
- Multi-model support (GPT-4, Gemini)
MIT License
Copyright (c) 2024
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- AWS: For providing Textract, Bedrock, and S3 services
- Anthropic: For Claude 3 Haiku model
- Vercel: For seamless Python hosting
- Flask Community: For excellent web framework
- Open Source Community: For supporting libraries
For questions, issues, or feature requests:
- Open an issue on GitHub
- Email: support@example.com
- Documentation: https://docs.example.com
Built for the AWS AI Agent Global Hackathon 2024
Demonstrating autonomous AI agents that combine multiple AWS services to deliver intelligent document processing without human intervention.