terraform-aws-lambda-monitored

AWS Lambda function module with built-in monitoring and alerting capabilities.

This module creates a Lambda function with CloudWatch Logs, configurable error monitoring, and SNS-based alerting. Designed to meet ISO27001 compliance requirements for error rate monitoring.

Why This Module?

Most Lambda Terraform modules ship a function and stop there, leaving you to bolt on logging, alarms, and an incident channel yourself. This module treats Lambda as a production workload from day one and wires up the full monitoring stack so you can pass a compliance audit without follow-up work:

• Monitoring is not optional. Every function comes with CloudWatch Logs (encrypted, with retention), plus alarms for errors, throttles, and execution duration — the baseline Vanta and ISO27001 auditors look for.
• Two alert strategies, one flag. Pick immediate (fire on any error) for low-traffic or critical paths, or threshold (error-rate metric math) for high-volume functions that expect occasional failures.
• Alerts go somewhere useful. SNS topic with email subscriptions is created for you; you can also fan out to existing PagerDuty or Slack topics via alarm_topic_arns.
• Reproducible packaging. Dependencies are installed against a manylinux platform ladder (manylinux_2_28 → manylinux_2_17/manylinux2014, newest glibc floor first), so pip picks the best Lambda-compatible wheel per package and your local machine's architecture and Python version don't leak into the deployment artifact.
• Smart change detection. The packager hashes your source files plus requirements.txt — not the entire .build/ directory — so re-creating .terraform doesn't trigger spurious re-uploads.
• Tight IAM by default. Execution role uses name_prefix, logging policy is scoped to the function's log group, and VPC access (when enabled) grants ENI permissions only to the subnets you specified.
• Real integration tests. The test suite applies and destroys real AWS infrastructure across the full matrix of supported AWS providers, architectures, and Python versions — no mocks.

If all you need is "give me a Lambda resource" this module is overkill. If you need a Lambda you can hand to a compliance auditor, this is the shortest path.

Features

• Multi-architecture support (x86_64, arm64) with automatic platform-specific dependency packaging
• Multi-Python version support (3.9, 3.10, 3.11, 3.12, 3.13)
• Intelligent dependency packaging with manylinux wheels for target architecture
• CloudWatch Logs with configurable retention and encryption
• Flexible IAM permissions (attach custom policies)
• S3-based deployment with secure bucket management
• Error monitoring and alerting with SNS email notifications
• Configurable alert strategies (immediate or threshold-based)
• Throttle monitoring and alerts
• Optional memory utilization alarm backed by Lambda Insights (opt-in)
• Automatic change detection (re-packages only when code or dependencies change)

Documentation

Full documentation is published to GitHub Pages: https://infrahouse.github.io/terraform-aws-lambda-monitored/

• Getting Started — prerequisites and first deployment
• Architecture — how the module is put together
• Configuration — every variable explained
• Examples — common usage patterns
• Troubleshooting — common issues and fixes

Prerequisites

The module's packaging script requires the following tools to be installed on the system where Terraform runs:

• Python 3 - For installing dependencies

  • Ubuntu/Debian: sudo apt-get install python3 python3-pip
  • macOS: brew install python3
  • Amazon Linux: sudo yum install python3 python3-pip

• pip3 - For managing Python packages

  • Ubuntu/Debian: sudo apt-get install python3-pip
  • macOS: python3 -m ensurepip
  • Amazon Linux: sudo yum install python3-pip

• jq - For parsing JSON responses

  • Ubuntu/Debian: sudo apt-get install jq
  • macOS: brew install jq
  • Amazon Linux: sudo yum install jq

The packaging and deployment scripts will check for these dependencies and provide installation instructions if any are missing.

Usage

module "lambda" {
  source  = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  version = "2.0.1"

  function_name     = "my-lambda-function"
  lambda_source_dir = "${path.module}/lambda"

  # Optional: Python version and architecture
  python_version = "python3.12"
  architecture   = "arm64"

  # Optional: Lambda configuration
  timeout     = 60
  memory_size = 256
  description = "My Lambda function"

  # Optional: Environment variables
  environment_variables = {
    ENV = "production"
  }

  # Optional: Additional IAM permissions
  additional_iam_policy_arns = [
    aws_iam_policy.lambda_custom_permissions.arn
  ]

  # Required: Email addresses for alarm notifications
  alarm_emails = ["team@example.com", "oncall@example.com"]

  # Optional: Alert strategy
  alert_strategy = "immediate"  # or "threshold"

  # Optional: For threshold strategy
  error_rate_threshold           = 5.0  # 5% error rate
  error_rate_evaluation_periods  = 2
  error_rate_datapoints_to_alarm = 2

  # Optional: Memory utilization alarm (enables Lambda Insights when set).
  # Leave unset (null) to skip Lambda Insights and avoid the extra cost.
  memory_utilization_threshold_percent = 80

  # Optional: CloudWatch Logs retention
  cloudwatch_log_retention_days = 365

  tags = {
    Environment = "production"
    Project     = "my-project"
  }
}

# Example: Custom IAM policy
resource "aws_iam_policy" "lambda_custom_permissions" {
  name = "my-lambda-permissions"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:PutObject"
        ]
        Resource = "arn:aws:s3:::my-bucket/*"
      }
    ]
  })
}

Dependencies and Packaging

The module uses an intelligent packaging system that automatically handles Python dependencies:

With Dependencies:

module "lambda" {
  source  = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  version = "2.0.1"

  function_name      = "my-function"
  lambda_source_dir  = "${path.module}/lambda"
  requirements_file  = "${path.module}/lambda/requirements.txt"  # Optional
  architecture       = "arm64"  # Dependencies installed for ARM64
  python_version     = "python3.12"
  alarm_emails       = ["team@example.com"]
}

How it works:

1. The module installs platform-specific manylinux wheels, trying the ladder manylinux_2_28_${ARCH} → manylinux_2_17_${ARCH} → manylinux2014_${ARCH} (newest glibc floor first) and letting pip pick the best match per package — so packages that only ship the older manylinux_2_17/manylinux2014 wheels still resolve
2. Dependencies are installed with --only-binary=:all: to ensure AWS Lambda compatibility
3. Only re-packages when source code, dependencies, architecture, or Python version changes
4. Automatically cleans up Python cache files (__pycache__, .pyc)

Tracking Source Code Changes:

The module tracks changes to your source code to determine when to rebuild. By default, it only tracks main.py to avoid hashing installed dependencies. You can customize this with the source_code_files variable:

# Default - tracks only main.py
module "lambda" {
  source            = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  lambda_source_dir = "${path.module}/lambda"
  # source_code_files defaults to ["main.py"]
  ...
}

# Track multiple specific files
module "lambda" {
  source            = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  lambda_source_dir = "${path.module}/lambda"
  source_code_files = ["main.py", "utils.py", "config.py"]
  ...
}

# Track all root-level .py files (useful if you have multiple source files)
module "lambda" {
  source            = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  lambda_source_dir = "${path.module}/lambda"
  source_code_files = ["*.py"]
  ...
}

Important: Dependencies are tracked separately via requirements_file hash. Only list your actual source code files in source_code_files, not installed packages. This prevents unnecessary rebuilds when .terraform is recreated.

Requirements:

• Python 3 must be installed locally (used by packaging script)
• The pip module must be available
• For cross-architecture builds, ensure pip can download the correct platform wheels

VPC Configuration

Lambda functions can be attached to a VPC to access resources in private subnets (databases, internal APIs, etc.).

With VPC Configuration:

module "lambda" {
  source  = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  version = "2.0.1"

  function_name     = "my-vpc-function"
  lambda_source_dir = "${path.module}/lambda"
  alarm_emails      = ["team@example.com"]

  # VPC configuration
  lambda_subnet_ids         = ["subnet-abc123", "subnet-def456"]
  lambda_security_group_ids = [aws_security_group.lambda.id]
}

Important VPC Considerations:

1. NAT Gateway Required: Subnets must have a NAT gateway or NAT instance for internet access (AWS API calls, external dependencies, etc.)

2. Security Groups: Configure security group rules to allow:

   • Outbound traffic to required services (databases, APIs, etc.)
   • Inbound traffic if Lambda needs to receive requests from within VPC

3. IAM Permissions: The module automatically adds required EC2 network interface permissions:

   • ec2:CreateNetworkInterface
   • ec2:DescribeNetworkInterfaces
   • ec2:DeleteNetworkInterface
   • ec2:AssignPrivateIpAddresses
   • ec2:UnassignPrivateIpAddresses

4. Cold Start: VPC-attached Lambda functions have longer cold start times (~1-3 seconds additional) due to ENI creation

5. ENI Limits: Each Lambda function in a VPC creates elastic network interfaces (ENIs). Monitor your ENI limits in the AWS account

When to Use VPC:

• ✅ Access RDS databases in private subnets
• ✅ Connect to internal APIs or services
• ✅ Use AWS PrivateLink endpoints
• ✅ Access resources that require private IP addresses

When NOT to Use VPC:

• ❌ Public API calls only (no VPC needed, better performance)
• ❌ Pure compute functions with no external connections
• ❌ Functions requiring minimal cold start latency

Memory Monitoring (Optional)

Memory utilization alerting is opt-in. It is disabled by default because it requires enabling AWS Lambda Insights, which ships an extension layer with the function and publishes CloudWatch metrics at an additional cost.

Enable it by setting memory_utilization_threshold_percent:

module "lambda" {
  source  = "registry.infrahouse.com/infrahouse/lambda-monitored/aws"
  version = "2.0.1"

  function_name     = "my-lambda-function"
  lambda_source_dir = "${path.module}/lambda"
  alarm_emails      = ["team@example.com"]

  # Alarm when memory usage exceeds 80% of memory_size for a single invocation.
  memory_utilization_threshold_percent = 80
}

What happens when it's enabled:

1. The module attaches the AWS-managed Lambda Insights extension layer to the function. The layer ARN is region- and architecture-specific; the module picks the correct one automatically from var.architecture. Override with lambda_insights_layer_arn if you need to pin a different version.
2. The CloudWatchLambdaInsightsExecutionRolePolicy managed policy is attached to the Lambda execution role so the extension can publish metrics.
3. A CloudWatch alarm (<function_name>-memory) is created on the LambdaInsights / memory_utilization metric (Maximum statistic, 1-minute period). Alarms fan out to the same SNS topics as the other alarms in this module.

What happens when it's disabled (default):

• No Lambda Insights layer is attached.
• No managed policy is attached.
• No memory alarm is created.
• No additional CloudWatch/Insights charges from this module.

Threshold semantics: memory_utilization_threshold_percent is a percentage of the function's allocated memory_size. For example, setting it to 80 on a function with memory_size = 256 triggers the alarm when any invocation uses more than ~205 MB.

Notes

IAM Role Naming

If your function_name exceeds 37 characters, the IAM role name will be truncated to comply with AWS's 38-character name_prefix limit. The full function name is always preserved in the IAM role's function_name tag for identification purposes.

Best Practice: Reference the IAM role using the module outputs (lambda_role_arn or lambda_role_name) rather than constructing the role name manually.

Email Subscription Confirmation

When you specify alarm_emails, AWS will send a confirmation email to each address. Recipients must click the confirmation link to receive alerts. The Terraform apply will complete successfully, but notifications won't be sent until subscriptions are confirmed.

Important: If you destroy and recreate the module, new confirmation emails will be sent even to previously confirmed addresses.

S3 Bucket Security

The module uses the InfraHouse S3 bucket module which automatically configures:

• Server-side encryption
• Versioning
• Public access blocking
• Secure bucket policies

Development and Testing

This module includes a comprehensive test suite using pytest. The Makefile provides convenient targets for running tests.

Running Tests

The module provides several test targets:

# Run all tests
make test

# Run specific test suites
make test-simple          # Test simple Lambda deployment
make test-deps            # Test Lambda with dependencies
make test-monitoring      # Test error monitoring (keeps resources)
make test-sns             # Test SNS integration

# Run architecture-specific tests
make test-x86             # Test x86_64 architecture only
make test-arm             # Test arm64 architecture only

Filtering Tests

You can filter tests using the TEST_SELECTOR variable:

# Run specific test
make test-simple TEST_SELECTOR="test_lambda_deployment"

# Run tests for specific provider version
make test-simple TEST_SELECTOR="provider-6.x"

# Run tests for specific Python version
make test-deps TEST_SELECTOR="py3.13"

# Combine filters (AND logic)
make test-simple TEST_SELECTOR="provider-6.x and py3.12"

Customizing Test Configuration

Override test configuration variables:

# Use different AWS region
make test-simple TEST_REGION="us-east-1"

# Use different IAM role
make test-simple TEST_ROLE="arn:aws:iam::123456789:role/my-test-role"

# Keep resources after test (for inspection/debugging)
make test-simple KEEP_AFTER=1

# Don't keep resources (destroy after test)
make test-monitoring KEEP_AFTER=

# Combine multiple overrides
make test-simple \
  TEST_SELECTOR="test_lambda_deployment" \
  TEST_REGION="eu-west-1" \
  TEST_ROLE="arn:aws:iam::123456789:role/my-role" \
  KEEP_AFTER=1

Default Test Configuration

The following defaults are used when variables are not specified:

• TEST_REGION: us-west-2
• TEST_ROLE: arn:aws:iam::303467602807:role/lambda-monitored-tester
• TEST_SELECTOR: test_ (runs all tests)
• KEEP_AFTER: empty (destroys resources after test)

Note: The test-monitoring target keeps resources by default for alarm observation.

Other Make Targets

make bootstrap            # Install development dependencies
make lint                 # Check code style
make format              # Format Terraform and Python files
make clean               # Clean temporary files and test data

Examples

Working examples live under examples/:

• examples/immediate-alerts — fire an SNS alarm on the first error
• examples/threshold-alerts — rate-based error alarm for high-traffic functions
• examples/custom-permissions — attach custom IAM policies for S3/DynamoDB access

Contributing

Contributions are welcome. Please read CONTRIBUTING.md for the workflow and SECURITY.md for how to report vulnerabilities.

License

Apache 2.0 — see LICENSE.

---

Requirements

Name	Version
terraform	~> 1.0
aws	>= 5.31, < 7.0
null	~> 3.0

Providers

Name	Version
aws	6.20.0
null	3.2.4

Modules

Name	Source	Version
lambda_bucket	registry.infrahouse.com/infrahouse/s3-bucket/aws	0.6.0

Resources

Name	Type
aws_cloudwatch_log_group.lambda	resource
aws_cloudwatch_metric_alarm.duration	resource
aws_cloudwatch_metric_alarm.errors_immediate	resource
aws_cloudwatch_metric_alarm.errors_threshold	resource
aws_cloudwatch_metric_alarm.memory	resource
aws_cloudwatch_metric_alarm.throttles	resource
aws_iam_role.lambda	resource
aws_iam_role_policy.lambda_logging	resource
aws_iam_role_policy.lambda_vpc_access	resource
aws_iam_role_policy_attachment.additional	resource
aws_iam_role_policy_attachment.lambda_insights	resource
aws_lambda_function.this	resource
aws_lambda_function_event_invoke_config.this	resource
aws_s3_object.lambda_package	resource
aws_sns_topic.alarms	resource
aws_sns_topic_subscription.alarm_emails	resource
null_resource.lambda_package	resource
aws_caller_identity.current	data source
aws_iam_policy_document.lambda_assume_role	data source
aws_iam_policy_document.lambda_logging	data source
aws_iam_policy_document.lambda_vpc_access	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
additional_iam_policy_arns	List of IAM policy ARNs to attach to the Lambda execution role	list(string)	[]	no
alarm_emails	List of email addresses to receive alarm notifications. AWS will send confirmation emails that must be accepted. At least one email is required.	list(string)	n/a	yes
alarm_topic_arns	List of existing SNS topic ARNs to send alarms to (for advanced integrations like PagerDuty, Slack, etc.)	list(string)	[]	no
alert_strategy	Alert strategy: 'immediate' (alert on any error) or 'threshold' (alert when error rate exceeds threshold)	string	"immediate"	no
architecture	Instruction set architecture for the Lambda function. Valid values: x86_64 or arm64	string	"x86_64"	no
cloudwatch_log_retention_days	Number of days to retain CloudWatch logs	number	365	no
description	Description of the Lambda function	string	null	no
duration_threshold_percent	Percentage of function timeout that triggers duration alarm (1-100). If not specified, duration alarm is disabled. For example, 80 means alarm when execution duration exceeds 80% of the configured timeout.	number	null	no
enable_error_alarms	Enable CloudWatch alarms for Lambda errors	bool	true	no
enable_throttle_alarms	Enable CloudWatch alarms for Lambda throttling	bool	true	no
environment_variables	Map of environment variables for the Lambda function	map(string)	{}	no
error_rate_datapoints_to_alarm	Number of datapoints that must breach threshold to trigger alarm	number	2	no
error_rate_evaluation_periods	Number of evaluation periods for error rate alarm	number	2	no
error_rate_threshold	Error rate percentage threshold for 'threshold' alert strategy (0-100)	number	5	no
function_name	Name of the Lambda function	string	n/a	yes
handler	Lambda function handler (format: file.function_name)	string	"main.lambda_handler"	no
kms_key_id	ARN of the KMS key for encrypting CloudWatch Logs and SNS topic. If not specified, AWS-managed encryption keys are used. The key must allow the CloudWatch Logs and SNS services to use it.	string	null	no
lambda_insights_layer_arn	Full ARN of the Lambda Insights extension layer version to attach when the memory alarm is enabled. Only used when memory_utilization_threshold_percent is set. If null, the module uses a sensible default matching var.architecture. Override this to pin a specific version or use a different region (see https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Lambda-Insights-extension-versions.html ).	string	null	no
lambda_security_group_ids	List of security group IDs for Lambda VPC configuration. Required if lambda_subnet_ids is specified.	list(string)	null	no
lambda_source_dir	Path to the directory containing Lambda function source code	string	n/a	yes
lambda_subnet_ids	List of subnet IDs for Lambda VPC configuration. The subnets must have NAT gateway for internet access. If not specified, Lambda will not be attached to a VPC.	list(string)	null	no
memory_size	Lambda function memory size in MB	number	128	no
memory_utilization_threshold_percent	Percentage of allocated memory that triggers the memory utilization alarm (1-100). If null (default), the memory alarm is disabled and Lambda Insights is NOT enabled. When set, the module attaches the AWS-managed Lambda Insights extension layer to the function, grants the CloudWatchLambdaInsightsExecutionRolePolicy managed policy to the execution role, and creates a CloudWatch alarm on the LambdaInsights memory_utilization metric. Enabling this incurs Lambda Insights charges.	number	null	no
python_version	Python runtime version. Must be an Amazon Linux 2023 runtime (python3.12 or python3.13): this module installs manylinux_2_28 wheels (glibc 2.28), which only AL2023 runtimes can load. Earlier runtimes (Amazon Linux 2, python <= 3.11) are not supported. See https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html	string	"python3.12"	no
requirements_file	Path to requirements.txt file for Python dependencies. Dependencies will be installed with platform-specific wheels for the target architecture. If not specified, the module will automatically look for requirements.txt in var.lambda_source_dir. Set to null to explicitly skip dependency installation.	string	null	no
sns_topic_name	Name for the SNS topic. If not provided, defaults to '<function_name>-alarms'	string	null	no
source_code_files	List of source code file patterns to track for changes (relative to lambda_source_dir). Only these files will trigger repackaging. Installed dependencies are tracked separately via requirements_file. Use glob patterns like "*.py" for root-level files or specific files like "main.py", "utils.py". Default tracks only main.py to avoid hashing installed dependencies.	list(string)	[ "main.py" ]	no
tags	Map of tags to assign to resources	map(string)	{}	no
timeout	Lambda function timeout in seconds	number	60	no

Outputs

Name	Description
cloudwatch_log_group_arn	ARN of the CloudWatch Log Group
cloudwatch_log_group_name	Name of the CloudWatch Log Group
duration_alarm_arn	ARN of the duration CloudWatch alarm (if enabled)
error_alarm_arn	ARN of the error CloudWatch alarm (if enabled)
kms_key_id	ARN of the KMS key used for encrypting CloudWatch Logs and SNS topic (null if using AWS-managed encryption)
lambda_function_arn	ARN of the Lambda function
lambda_function_invoke_arn	Invoke ARN of the Lambda function (for use with API Gateway, etc.)
lambda_function_name	Name of the Lambda function
lambda_function_qualified_arn	Qualified ARN of the Lambda function (includes version)
lambda_insights_layer_arn	ARN of the Lambda Insights extension layer attached to the function (null if memory alarm is disabled)
lambda_role_arn	ARN of the IAM role used by the Lambda function
lambda_role_name	Name of the IAM role used by the Lambda function
memory_alarm_arn	ARN of the memory utilization CloudWatch alarm (if enabled via memory_utilization_threshold_percent)
pending_email_confirmations	List of email addresses pending SNS subscription confirmation
requirements_file_used	Path to the requirements.txt file used for packaging (or 'none' if no dependencies)
s3_bucket_arn	ARN of the S3 bucket storing Lambda packages
s3_bucket_name	Name of the S3 bucket storing Lambda packages
sns_topic_arn	ARN of the SNS topic for alarm notifications
sns_topic_name	Name of the SNS topic for alarm notifications
throttle_alarm_arn	ARN of the throttle CloudWatch alarm (if enabled)
vpc_config_security_group_ids	List of security group IDs for Lambda VPC configuration (if configured)
vpc_config_subnet_ids	List of subnet IDs for Lambda VPC configuration (if configured)