Terraform Deployment Guide - ProductionDeployment

System: Medical Note Generation - AWS Deployment
Version: 1.0.0
Last Updated: November 1, 2025


Table of Contents

  1. Prerequisites
  2. IAM User Setup
  3. Terraform Deployment
  4. Docker Image Deployment
  5. Verification & Testing
  6. Troubleshooting
  7. Updating the Application
  8. Cleanup & Tear Down

Prerequisites

Required Tools

1. Terraform (v1.0+)

# macOS
brew install terraform

# Verify
terraform --version

2. AWS CLI (v2.0+)

# macOS
brew install awscli

# Verify
aws --version

3. Docker (v20.0+)

# Download Docker Desktop from docker.com

# Verify
docker --version
docker ps  # Should not error

AWS Account Requirements


IAM User Setup

Problem: AWS Managed Policy Character Limit

AWS has a 6,144 character limit for custom IAM policies. Our comprehensive policy exceeds this limit.

Solution: Use AWS Managed Policies

Instead of creating a custom policy, attach these 9 AWS managed policies to your IAM user:

Step 1: Create IAM User

  1. Open AWS ConsoleIAMUsersCreate user
  2. User name: Terraform_deployer (or your preferred name)
  3. Access type: ✅ Check "Programmatic access"
  4. Click "Next: Permissions"

Step 2: Attach Managed Policies

Select "Attach existing policies directly" and search for each policy:

  1. AmazonEC2FullAccess
  2. AmazonECS_FullAccess
  3. AmazonEC2ContainerRegistryFullAccess
  4. CloudWatchLogsFullAccess
  5. IAMFullAccess
  6. SecretsManagerReadWrite
  7. ElasticLoadBalancingFullAccess
  8. AmazonDynamoDBReadOnlyAccess
  9. ComprehendMedicalFullAccess

You should see: "9 policies selected"

Note: You cannot add AWSApplicationAutoscalingECSServicePolicy or PowerUserAccess due to AWS's 10-policy limit. Auto-scaling is disabled in the current deployment but can be added later.

Step 3: Create Access Keys

  1. Click "Next: Tags" (optional)
  2. Click "Next: Review"
  3. Click "Create user"
  4. ⚠️ CRITICAL: Copy both keys immediately:
  5. Access Key ID: AKIA...
  6. Secret Access Key: wJalr... (click "Show")
  7. Download .csv or save securely
  8. Click "Close"

Step 4: Configure AWS CLI

Option A - Interactive:

aws configure --profile Terraform_deployer

Enter: - Access Key ID: [paste] - Secret Access Key: [paste] - Region: ap-southeast-2 - Format: json

Option B - Manual:

# Edit ~/.aws/credentials
nano ~/.aws/credentials

Add:

[Terraform_deployer]
aws_access_key_id = AKIA...
aws_secret_access_key = wJalr...
# Edit ~/.aws/config
nano ~/.aws/config

Add:

[profile Terraform_deployer]
region = ap-southeast-2
output = json

Step 5: Test Credentials

AWS_PROFILE=Terraform_deployer aws sts get-caller-identity

Expected output:

{
    "UserId": "AIDA...",
    "Account": "767398078453",
    "Arn": "arn:aws:iam::767398078453:user/Terraform_deployer"
}

If you see this, credentials are configured correctly!


Terraform Deployment

Step 1: Navigate to Terraform Directory

cd /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment/terraform

Step 2: Review Configuration

Edit terraform.tfvars if needed:

cat terraform.tfvars

Key settings:

aws_region  = "ap-southeast-2"
environment = "dev"
project_name = "medical-notes"

# ECS Configuration
ecs_task_cpu       = "512"   # 0.5 vCPU
ecs_task_memory    = "1024"  # 1 GB
ecs_desired_count  = 1       # Number of tasks

# API Keys (pulled from your .env)
openai_api_key = "sk-proj-..."
groq_api_key   = "gsk_..."

Note: API keys are stored in AWS Secrets Manager, not in plain text in AWS.

Step 3: Initialize Terraform

export AWS_PROFILE=Terraform_deployer
terraform init

Expected output:

Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.100.0...

Terraform has been successfully initialized!

Step 4: Plan the Deployment

terraform plan -out=tfplan

What this does: - Shows all resources that will be created - Validates configuration - Saves plan to file

Expected output:

Plan: 37 to add, 0 to change, 0 to destroy.

Saved the plan to: tfplan

Review the plan carefully. You should see: - VPC and networking resources - ECS cluster, service, task definition - Application Load Balancer - Security groups - IAM roles - Secrets Manager

Step 5: Apply the Deployment

terraform apply tfplan

Or with auto-approve:

terraform apply -auto-approve

What happens:

Creating resources... (this takes ~10-12 minutes)

aws_vpc.main: Creating...
aws_ecs_cluster.main: Creating...
aws_secretsmanager_secret.openai_key: Creating...
...

aws_lb.main: Creation complete after 3m1s
aws_ecs_service.api: Creating...
aws_ecs_service.api: Creation complete after 1s

Apply complete! Resources: 35 added, 0 changed, 0 destroyed.

Outputs:

alb_dns_name = "medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com"
alb_url = "http://medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com"
ecr_repository_url = "767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api"
vpc_id = "vpc-0f02741fc7dcb8c35"

✅ Infrastructure is now deployed!

Step 6: Save Important Outputs

# Save ALB URL
export ALB_URL=$(terraform output -raw alb_dns_name)
echo $ALB_URL

# Save ECR URL
export ECR_URL=$(terraform output -raw ecr_repository_url)
echo $ECR_URL

Docker Image Deployment

Now that infrastructure is ready, deploy your application code.

Step 1: Navigate to Project Root

cd /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment

Step 2: Login to ECR

export AWS_PROFILE=Terraform_deployer
aws ecr get-login-password --region ap-southeast-2 | \
  docker login --username AWS --password-stdin \
  767398078453.dkr.ecr.ap-southeast-2.amazonaws.com

Expected: Login Succeeded

Step 3: Build Docker Image

Important: Build for AMD64 (AWS Fargate architecture), not ARM64 (your Mac).

docker build --platform linux/amd64 -t medical-notes-api:latest .

This takes ~5-10 minutes. You'll see:

[+] Building 180.2s (14/14) FINISHED
=> [1/9] FROM docker.io/library/python:3.11-slim
=> [2/9] WORKDIR /app
=> [3/9] RUN apt-get update && apt-get install...
=> [4/9] COPY requirements.txt .
=> [5/9] RUN pip install...
=> [6/9] COPY app/ ./app/
=> [7/9] COPY ui/ ./ui/
=> [8/9] COPY db/ ./db/
=> [9/9] RUN mkdir -p logs
=> exporting to image

Step 4: Tag the Image

docker tag medical-notes-api:latest \
  767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api:latest

Step 5: Push to ECR

docker push 767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api:latest

This takes ~3-5 minutes depending on your internet speed. You'll see:

The push refers to repository [767398078453.dkr.ecr...]
38513bd72563: Pushed
ee4a5e2d5517: Pushed
...
latest: digest: sha256:a0fd0165... size: 856

Step 6: Force ECS Deployment

aws ecs update-service \
  --cluster medical-notes-cluster \
  --service medical-notes-service \
  --force-new-deployment \
  --region ap-southeast-2

Expected:

{
  "service": {
    "serviceName": "medical-notes-service",
    "status": "ACTIVE",
    "runningCount": 1
  }
}

Step 7: Wait for Task to Start

# Wait 30 seconds for task to start
sleep 30

# Check status
aws ecs describe-services \
  --cluster medical-notes-cluster \
  --services medical-notes-service \
  --region ap-southeast-2 \
  | jq '.services[0] | {status, runningCount, desiredCount}'

Expected:

{
  "status": "ACTIVE",
  "runningCount": 1,
  "desiredCount": 1
}

When runningCount = desiredCount, deployment is complete!


Verification & Testing

Test 1: Health Check

curl http://medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com/health

Expected:

{"status":"healthy","version":"1.0.0","database":"sqlite"}

Test 2: API Documentation

open http://medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com/docs

Should open Swagger UI with API documentation.

Test 3: API 2 (Note Generation)

curl -X POST \
  http://medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com/api/generate-note \
  -H "Content-Type: application/json" \
  -d '{
    "note_type": "soap",
    "transcription": "Patient is a 45-year-old male with abdominal pain",
    "visiting_id": "visit-123",
    "user_email_address": "dr@hospital.com"
  }'

Should start streaming medical note generation.

Test 4: UI

open http://medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com/

Should open the web interface for audio upload and note generation.


Troubleshooting

Issue: Terraform Apply Fails with "AccessDenied"

Cause: IAM user doesn't have required permissions.

Solution: 1. Verify all 9 managed policies are attached 2. Check credentials: AWS_PROFILE=Terraform_deployer aws sts get-caller-identity 3. Re-export profile: export AWS_PROFILE=Terraform_deployer

Issue: "InvalidClientTokenId"

Cause: Access Key ID or Secret Access Key is incorrect.

Solution:

# Reconfigure credentials
aws configure --profile Terraform_deployer

# Or check current config
aws configure list --profile Terraform_deployer

# Verify in AWS Console: IAM → Users → Terraform_deployer → Security credentials

Issue: Docker Build Fails

Cause: Docker not running or insufficient memory.

Solution: 1. Start Docker Desktop 2. Increase memory: Docker Desktop → Settings → Resources → Memory (4GB+) 3. Clean up: docker system prune -a

Issue: ECS Task Won't Start (RunningCount: 0)

Cause: Task is failing health checks or crashing.

Solution:

# Check task logs
aws logs tail /ecs/medical-notes --follow --region ap-southeast-2

# Check task status
aws ecs list-tasks --cluster medical-notes-cluster --region ap-southeast-2

# Describe tasks
aws ecs describe-tasks \
  --cluster medical-notes-cluster \
  --tasks <task-arn> \
  --region ap-southeast-2

Common issues: - Missing environment variables - API keys not accessible (check IAM role) - Port 8000 not exposed

Issue: 502 Bad Gateway from ALB

Cause: ECS tasks are starting up or unhealthy.

Solution:

# Wait 1-2 minutes for task to fully start
sleep 60

# Check target health
aws elbv2 describe-target-health \
  --target-group-arn $(terraform output -raw alb_target_group_arn) \
  --region ap-southeast-2

Healthy state:

{
  "State": "healthy",
  "Reason": "Target.ResponseCodeMismatch",
  "Description": "Health checks passed"
}

Issue: "Policy Limit Exceeded" When Adding Policies

Cause: AWS limits IAM users to 10 managed policies.

Solution: - Remove unused policies from the user - Or use IAM roles instead of users - Or skip optional features (like auto-scaling)

Issue: Terraform State Locked

Cause: Previous terraform command didn't complete.

Solution:

terraform force-unlock <lock-id>

Updating the Application

When to Update

Update Process

Step 1: Make Code Changes Locally

# Edit files
# Test locally first: python -m uvicorn app.main:app --reload

Step 2: Build New Docker Image

cd /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment
docker build --platform linux/amd64 -t medical-notes-api:latest .

Step 3: Tag and Push

export AWS_PROFILE=Terraform_deployer

# Login to ECR
aws ecr get-login-password --region ap-southeast-2 | \
  docker login --username AWS --password-stdin \
  767398078453.dkr.ecr.ap-southeast-2.amazonaws.com

# Tag
docker tag medical-notes-api:latest \
  767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api:latest

# Push
docker push 767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api:latest

Step 4: Deploy to ECS

aws ecs update-service \
  --cluster medical-notes-cluster \
  --service medical-notes-service \
  --force-new-deployment \
  --region ap-southeast-2

Step 5: Monitor Deployment

# Watch service status
aws ecs describe-services \
  --cluster medical-notes-cluster \
  --services medical-notes-service \
  --region ap-southeast-2 \
  | jq '.services[0] | {status, runningCount, desiredCount}'

# Watch logs
aws logs tail /ecs/medical-notes --follow --region ap-southeast-2

Deployment takes 1-2 minutes. When runningCount matches desiredCount, update is complete.


Infrastructure Changes (Terraform)

When to Run Terraform

Update Process

cd /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment/terraform
export AWS_PROFILE=Terraform_deployer

# 1. Edit .tf files
nano variables.tf  # or ecs.tf, main.tf, etc.

# 2. Format
terraform fmt

# 3. Validate
terraform validate

# 4. Plan
terraform plan -out=tfplan

# 5. Review plan carefully
# Look for:
# - Resources being DESTROYED (red)
# - Resources being MODIFIED (yellow)
# - Resources being CREATED (green)

# 6. Apply
terraform apply tfplan

Common Infrastructure Changes

Scale ECS Service: Edit terraform.tfvars:

ecs_desired_count = 2  # Change from 1 to 2

Then:

terraform apply -auto-approve

Increase Instance Size: Edit terraform.tfvars:

ecs_task_cpu    = "1024"  # 1 vCPU (from 512)
ecs_task_memory = "2048"  # 2 GB (from 1024)

Then apply.


Cleanup & Tear Down

⚠️ WARNING: This Deletes Everything!

What gets deleted: - All 35 AWS resources - ECS tasks and services - Load balancer - VPC and networking - ECR repository (and images) - Secrets Manager secrets - CloudWatch logs

What is NOT deleted (manual cleanup required): - DynamoDB tables (medical_note_prompts, user_note_examples) - Data in those tables

Destroy Command

cd /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment/terraform
export AWS_PROFILE=Terraform_deployer

# Preview what will be destroyed
terraform plan -destroy

# Destroy (requires confirmation)
terraform destroy

# Or auto-approve
terraform destroy -auto-approve

Takes ~5-10 minutes to delete all resources.

Partial Cleanup (Stop Billing)

If you want to pause without deleting everything:

Stop ECS Tasks (stops most billing):

aws ecs update-service \
  --cluster medical-notes-cluster \
  --service medical-notes-service \
  --desired-count 0 \
  --region ap-southeast-2

This stops: - ECS Fargate charges (~$15-20/month) - Data transfer charges

This continues: - ALB charges (~$16/month) - NAT Gateway charges (~$32/month)

To restart:

aws ecs update-service \
  --cluster medical-notes-cluster \
  --service medical-notes-service \
  --desired-count 1 \
  --region ap-southeast-2

Cost Management

Monthly Cost Breakdown

Fixed Costs (always running): | Service | Cost/Month | |---------|------------| | Application Load Balancer | $16 | | NAT Gateway | $32 | | CloudWatch Logs (30-day retention) | $5 | | ECR Storage (~2GB) | $1 | | Subtotal | $54 |

Variable Costs (depends on usage): | Service | Cost/Month | |---------|------------| | ECS Fargate (1 task, 24/7) | $15-20 | | DynamoDB reads | $1-5 | | Comprehend Medical | $5-15 | | OpenAI API (Whisper + GPT) | $20-50 | | Groq API | Free tier, then $5-10 | | Data Transfer | $5-10 | | Subtotal | $51-110 |

Total: $105-164/month

Cost Optimization Tips

1. Stop When Not in Use:

# Stop tasks (saves $15-20/month)
aws ecs update-service --desired-count 0 ...

2. Use Spot Instances (future): - ECS Fargate Spot: 70% cheaper - Requires Terraform changes

3. Reduce Retention: - CloudWatch logs: 7 days instead of 30 (saves $3/month)

4. Use Caching: - Cache DynamoDB results (reduces reads) - Cache validation results (reduces API calls)


Monitoring & Logs

CloudWatch Logs

View in Console: 1. AWS Console → CloudWatch → Log Groups 2. Select /ecs/medical-notes 3. View log streams (one per task)

Via CLI:

# Tail logs (follow mode)
aws logs tail /ecs/medical-notes --follow --region ap-southeast-2

# Last 1 hour
aws logs tail /ecs/medical-notes --since 1h --region ap-southeast-2

# Filter for errors
aws logs tail /ecs/medical-notes --filter-pattern ERROR --region ap-southeast-2

ECS Metrics

Service Status:

aws ecs describe-services \
  --cluster medical-notes-cluster \
  --services medical-notes-service \
  --region ap-southeast-2 \
  | jq '.services[0] | {
      status,
      runningCount,
      desiredCount,
      pendingCount,
      deployments: .deployments | length
    }'

Task Details:

# List tasks
aws ecs list-tasks \
  --cluster medical-notes-cluster \
  --region ap-southeast-2

# Describe specific task
aws ecs describe-tasks \
  --cluster medical-notes-cluster \
  --tasks <task-arn> \
  --region ap-southeast-2

Load Balancer Health

Target Health:

# Get target group ARN
TG_ARN=$(aws elbv2 describe-target-groups \
  --names medical-notes-tg \
  --region ap-southeast-2 \
  --query 'TargetGroups[0].TargetGroupArn' \
  --output text)

# Check health
aws elbv2 describe-target-health \
  --target-group-arn $TG_ARN \
  --region ap-southeast-2

Healthy status:

{
  "TargetHealthDescriptions": [
    {
      "Target": {"Id": "10.0.100.123", "Port": 8000},
      "HealthCheckPort": "8000",
      "TargetHealth": {
        "State": "healthy",
        "Reason": "Target.ResponseCodeMismatch",
        "Description": "Health checks passed"
      }
    }
  ]
}

Advanced Configurations

Enable RDS (When You Have Permissions)

Step 1: Uncomment RDS in Terraform

cd terraform
nano rds.tf

Remove the /* and */ comments around all resources.

Step 2: Uncomment RDS Outputs

nano outputs.tf

Uncomment the RDS outputs.

Step 3: Update ECS Environment Variables

nano ecs.tf

Change:

{name = "DB_TYPE", value = "sqlite"}

To:

{name = "DB_TYPE", value = "rds"}
{name = "DB_HOST", value = aws_db_instance.clinical_notes.address}
{name = "DB_PORT", value = "3306"}
{name = "DB_NAME", value = var.db_name}
{name = "DB_USER", value = var.db_username}

Uncomment DB password secret.

Step 4: Apply

terraform apply

This creates RDS instance (~10 minutes).

Step 5: Initialize RDS Schema

# Get RDS endpoint
RDS_ENDPOINT=$(terraform output -raw rds_address)

# Connect and create table
mysql -h $RDS_ENDPOINT -u admin -p clinical_notes < db/init_rds.sql

Enable Auto-Scaling (Requires Additional Permissions)

Currently disabled due to IAM policy limits.

To enable: 1. Get PowerUserAccess policy attached 2. Uncomment auto-scaling resources in terraform/ecs.tf 3. Run terraform apply


Terraform State Management

Current State

Backend: Local file (terraform.tfstate)

Location: /Users/sushmamurthy/Documents/MedconnectAI/ProductionDeployment/terraform/terraform.tfstate

⚠️ Warning: This file contains sensitive data (API keys). Do NOT commit to Git!

Future: Remote State (Production)

For production, use S3 backend:

Step 1: Create S3 Bucket

aws s3 mb s3://medical-notes-terraform-state --region ap-southeast-2

Step 2: Enable Versioning

aws s3api put-bucket-versioning \
  --bucket medical-notes-terraform-state \
  --versioning-configuration Status=Enabled

Step 3: Uncomment S3 Backend in main.tf

backend "s3" {
  bucket = "medical-notes-terraform-state"
  key    = "production/terraform.tfstate"
  region = "ap-southeast-2"
  encrypt = true
}

Step 4: Migrate State

terraform init -migrate-state

Quick Reference

Essential Commands

# Set profile (run this in every new terminal)
export AWS_PROFILE=Terraform_deployer

# Terraform
terraform init                    # Initialize
terraform plan                    # Preview changes
terraform apply                   # Deploy changes
terraform destroy                 # Delete everything
terraform output                  # Show outputs
terraform state list              # List resources

# AWS ECS
aws ecs list-clusters --region ap-southeast-2
aws ecs list-services --cluster medical-notes-cluster --region ap-southeast-2
aws ecs update-service --desired-count N ...  # Scale

# Docker
docker build --platform linux/amd64 -t medical-notes-api .
docker push <ecr-url>/medical-notes-api:latest

# Logs
aws logs tail /ecs/medical-notes --follow --region ap-southeast-2

Environment Variables

Local Testing (.env):

# OpenAI
OPENAI_API_KEY=sk-proj-...

# Groq
GROQ_API_KEY=gsk_...

# AWS
AWS_REGION=ap-southeast-2
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...

# Database
DB_TYPE=sqlite

# DynamoDB
DYNAMODB_PROMPTS_TABLE=medical_note_prompts
DYNAMODB_EXAMPLES_TABLE=user_note_examples

AWS Deployment (stored in Secrets Manager + ECS Task Definition): - API keys: Secrets Manager - AWS region: Task definition environment - Database config: Task definition environment


Security Best Practices

Secrets Management

✅ DO: - Store API keys in AWS Secrets Manager - Use IAM roles for AWS service access - Rotate credentials regularly - Use .env for local, never commit

❌ DON'T: - Hardcode API keys in code - Commit .env or terraform.tfvars to Git - Share credentials in chat/email - Use same keys for dev and production

Network Security

Current Setup: - ✅ ECS tasks in private subnets - ✅ Only ALB can reach tasks - ✅ Security groups properly configured - ❌ No WAF (Web Application Firewall) - ❌ HTTP only (no HTTPS)

Production Recommendations: 1. Add HTTPS with ACM certificate 2. Enable AWS WAF 3. Add rate limiting 4. Implement API authentication 5. Use VPC endpoints for AWS services


Production Checklist

Before Going to Production


Terraform Files Reference

Key Files

main.tf: - Provider configuration - VPC, subnets, networking - Security groups - Secrets Manager

ecs.tf: - ECS cluster - Task definition (container config) - ECS service - CloudWatch log group - IAM roles

ecr.tf: - ECR repository - Lifecycle policy (keep last 10 images)

rds.tf (commented out): - RDS instance - DB subnet group - Parameter group - CloudWatch alarms

variables.tf: - All configurable variables - Defaults and descriptions

terraform.tfvars: - Actual values for variables - ⚠️ Contains API keys, don't commit!

outputs.tf: - ALB DNS name - ECR repository URL - VPC ID - Deployment instructions


Deployment Timeline

Full Deployment from Scratch

Step Duration Total
IAM user creation 5 min 5 min
Terraform init 1 min 6 min
Terraform apply 10-12 min 18 min
Docker build 5-10 min 28 min
Docker push 3-5 min 33 min
ECS deployment 1-2 min 35 min
Total ~35-40 minutes

Update Deployment (Code Changes Only)

Step Duration Total
Docker build 2-3 min 3 min
Docker push 2-3 min 6 min
ECS deployment 1-2 min 8 min
Total ~8-10 minutes

Contact & Support

Deployed System Info

Account: 767398078453
Region: ap-southeast-2
User: Terraform_deployer
Cluster: medical-notes-cluster
Service: medical-notes-service
ALB: medical-notes-alb-1274198089.ap-southeast-2.elb.amazonaws.com
ECR: 767398078453.dkr.ecr.ap-southeast-2.amazonaws.com/medical-notes-api

End of Terraform Deployment Guide

For architecture details, see 1_ARCHITECTURE_AND_FLOW.md.
For local usage, see 3_LOCAL_USAGE_GUIDE.md.