Hi there 👋, I am Subhamay Bhattacharyya, 6x AWS, 1x Azure, 2x GitHub and 1x HashiCorp Terraform, 2 x Snowflake and 1 x Databricks Certified Data Engineer and a AWS Cloud Enthusiast
I'm a Senior Manager & Cloud Data Engineer with a strong focus on multi-cloud architecture, DevOps automation, and data platforms. I build production-grade cloud systems across AWS, Azure, and GCP using Terraform, GitHub Actions, Ansible, and serverless services, with a deep emphasis on security, observability, and cost-efficient design.
My work blends hands-on engineering with architectural leadership—from designing event-driven data pipelines and cloud-native analytics platforms to building enterprise-ready CI/CD pipelines using OIDC-based zero-trust authentication and infrastructure-as-code.
I’m currently developing a multi-cloud Cloud Resume Challenge platform, where the same production-grade architecture is deployed on:
- AWS — S3, CloudFront, Route 53, Lambda, DynamoDB, Kinesis, RDS, VPC, EC2, Athena, API Gateway, Glue, Step Functions, SNS, SQS, EventBridge, CloudWatch, IAM, KMS, Secrets Manager
- GCP — Cloud Storage, Cloud CDN, Cloud DNS, Workload Identity Federation
- Azure — Storage Static Website, Azure Front Door, Azure DNS
Everything is provisioned using Terraform modules, deployed via GitHub Actions with OIDC, and validated with Ansible-driven configuration management—mirroring how real-world cloud platforms are built and operated.
Alongside this, I build reusable GitHub Actions, Terraform modules, and data engineering pipelines for AWS Glue, Snowflake, BigQuery, and Azure Data Factory.
- Multi-cloud Infrastructure as Code (Terraform, HCP Terraform, GitHub OIDC)
- Serverless & Event-Driven Architectures (Lambda, Glue, Kinesis, Pub/Sub, Dataflow)
- Data Engineering Platforms (Snowflake, BigQuery, Athena, Glue, DMS)
- CI/CD & Platform Engineering (GitHub Actions, reusable workflows, DevContainers)
- Security & Observability (OIDC, IAM federation, Honeycomb, Splunk, lineage tracking)
I actively maintain a broad certification portfolio across AWS, Azure, GCP, Snowflake, GitHub, and Terraform, using them as a way to validate and deepen real-world engineering skills—not just theory.
To design and operate cloud platforms that are secure, automated, observable, and scalable by default—and to build public, open-source projects that demonstrate what modern cloud engineering really looks like in practice.
Legend: ⚪ Initialized 🟡 In Progress 🟢 Complete 🔴 Blocked
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-template |
📄🎯 GitHub Repository Template for Terraform as IaC. Not Available |
⚪ | |
| terraform-google-service-demo |
🏗️ A template repository to be used to create Google Services Demo Not Available |
🟡 | |
| terraform-snowflake-module-template |
✅ Terraform Snowflake module GitHub repository template Not Available |
🟢 | |
| terraform-aws-module-template |
✅ Terraform AWS Module Template Not Available |
🟢 | |
| terraform-google-module-template |
✅ A reusable, opinionated Terraform module template for provisioning and managin... Not Available |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| image-labeling |
This project demonstrates how to build a simple image labeling pipeline using AW... Current - Stack created. Next - Fix the S3 notification |
🟡 | |
| gcp-terraform-bootstrap |
✅ Bootstrap Google Cloud projects using Terraform with secure, keyless authentic... Not Available |
🟢 | |
| event-driven-etl-automation |
Event-driven, serverless ETL pipeline on AWS that automates data ingestion, tran... Current - Repository Initialized. Next - Create the entire stack. |
⚪ | |
| aws-snowflake-e2e-project |
End-to-end Snowflake data engineering project: layered stage/raw/curated design,... Need to progress further with the Udemy course resource. |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| aws-athena-iceberg-demo |
A hands-on demo showcasing Apache Iceberg tables on Amazon S3 queried using Amaz... Done - S3 bucket created. Next - Create Athena tables |
🟡 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| databricks-terraform-bootstrap |
Terraform-based bootstrap for provisioning a Databricks workspace, clusters, and... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| gcp-webhook-to-slack-function |
Provision a Google Cloud project and deploy a Python HTTP Cloud Function with Te... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| snowflake-de-environment-setup |
Environment setup demo for Snowflake Data Engineering labs, including roles, war... Not Available |
⚪ | |
| snowflake-de-semi-structured-json |
Demo project showcasing querying, flattening, and transforming semi-structured J... Not Available |
⚪ | |
| snowflake-de-iceberg-tables |
Hands-on demo for creating and querying Apache Iceberg tables in Snowflake, incl... Not Available |
⚪ | |
| snowflake-de-external-data-lake |
Demonstrates Snowflake data lake patterns using external stages, external tables... Not Available |
⚪ | |
| snowflake-de-snowpipe-ingestion |
End-to-end Snowpipe demo showing automated file ingestion from stages into Snowf... Not Available |
⚪ | |
| snowflake-de-json-loading |
Demo focused on loading semi-structured JSON files into Snowflake using stages, ... Not Available |
⚪ | |
| snowflake-de-batch-ingestion |
Batch ingestion and ELT demo covering file loading, JSON parsing, schema detecti... Not Available |
⚪ | |
| snowflake-de-scripting-procedures |
Snowflake Scripting demo showcasing stored procedures, loops, exception handling... Not Available |
⚪ | |
| snowflake-de-udf-extensibility |
Demonstrates Snowflake extensibility features including SQL UDFs, Python UDFs, a... Not Available |
⚪ | |
| snowflake-de-data-exchange |
Demo exploring Snowflake Marketplace and Data Exchange, including querying and d... Not Available |
⚪ | |
| snowflake-de-snowpark-notebooks |
Introduction to Snowpark using notebooks, showing data processing and transforma... Not Available |
⚪ | |
| snowflake-de-unstructured-data |
Demo for working with unstructured data using directory-enabled stages, scoped U... Not Available |
⚪ | |
| snowflake-de-llm-functions |
Hands-on demo using Snowflake Large Language Model functions for text analysis, ... Not Available |
⚪ | |
| snowflake-de-unstructured-pipelines |
Pipeline automation demo combining unstructured data processing with Snowflake t... Not Available |
⚪ | |
| snowflake-de-dynamic-tables |
Comprehensive demo of Snowflake Dynamic Tables including chaining, joins, refres... Not Available |
⚪ | |
| snowflake-de-lakehouse-pipeline |
End-to-end Snowflake Lakehouse pipeline demo covering bronze, silver, and gold l... Not Available |
⚪ | |
| snowflake-de-query-optimization |
Demo highlighting Snowflake SQL best practices for high-performance queries, pru... Not Available |
⚪ | |
| snowflake-de-performance-toolkit |
Performance analysis demo using Snowflake query history, tuning metrics, and wor... Not Available |
⚪ | |
| snowflake-de-performance-scenarios |
Scenario-based performance troubleshooting demo covering investigation, optimiza... Not Available |
⚪ | |
| snowflake-de-clustering-mv |
Demo showcasing table clustering strategies and materialized views for query per... Not Available |
⚪ | |
| snowflake-de-data-unload |
Demonstrates unloading structured Snowflake data into JSON files using internal ... Not Available |
⚪ | |
| snowflake-de-observability-alerts |
Management and observability demo using alerts, notifications, and monitoring pa... Not Available |
⚪ | |
| snowflake-de-alerting-framework |
Advanced metadata-driven alerting framework demo using tables, alerts, tasks, an... Not Available |
⚪ | |
| snowflake-de-data-metrics |
Demo for creating and using Data Metric Functions (DMFs) to measure data quality... Not Available |
⚪ | |
| snowflake-de-azure-iceberg-tables |
Terraform-based reference implementation for building Snowflake Data Engineering... Not Available |
⚪ | |
| snowflake-de-dynamic-table-basic |
🎯Dynamic Table Lab 1: Implement Basic Dynamic Table with Incremental Refresh Not Available |
⚪ | |
| snowflake-de-basic-snowpipe |
Terraform-first implementation of Snowflake Basic Snowpipe, demonstrating infras... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-gcp-lab-01-gce |
This repository is part of a hands-on Terraform learning series for Google Cloud... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| s3-storage-best-practice |
S3 Security Best Practices using Terraform. Terraform test not added to CI pipeline |
🟡 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| snowflake-dynamic-table-tutorial |
Hands-on tutorial demonstrating Snowflake Dynamic Tables with incremental refres... CI/CD Failing. |
🟠 | |
| snowflake-dynamic-table-pipeline |
🎯 End-to-end Snowflake Dynamic Table pipeline showcasing automated ingestion, tr... Not Available |
🟡 | |
| snowflake-iceberg-table-tutorial |
🎯 Hands-on tutorial repository demonstrating how to create and manage Snowflake ... Not Available |
⚪ | |
| snowflake-external-table-csv-tutorial |
🎯 This repository provides a step-by-step guide to implementing Snowflake Extern... S3 bucket,Database, Schama, File Format and External Stage created. |
🟡 | |
| snowflake-external-table-json-tutorial |
🎯 This tutorial demonstrates how to implement Snowflake External Tables on semi-... Ready to start working on this repo. |
🟡 | |
| snowflake-external-table-partition-tutorial |
🎯 Hands-on tutorial repository demonstrating how to create and manage Snowflake ... Not Available |
⚪ | |
| snowflake-external-table-parquet-tutorial |
🎯 This repository provides a step-by-step guide to implementing Snowflake Extern... Not Available |
⚪ | |
| snowflake-snowpipe-dynamic-table-pipeline |
🎯 End-to-end Snowflake data pipeline using External Stage and Snowpipe for autom... This is the pipeline of DE Simplified Dynamic Table masterclass - section 5 |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-aws-iceberg-table |
Terraform project to provision Apache Iceberg tables on AWS S3 using AWS Glue Da... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| cloud-champion-team-challenge |
🎯 Cloud Champion Bootcamp Team Challenge project by Team Cloud Catalyst demonstr... Not Available |
🟡 | |
| snowflake-dynamic-tables-declarative-pipelines |
Advanced implementation of declarative data pipelines in Snowflake using Dynamic... Not Available |
⚪ | |
| customer360-snowflake-pipeline |
🏗️ End-to-end Snowflake data lake for a fictional retail bank — ingests 25,000 n... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-gcp-practice |
Terraform with GCP practice Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-gcs-bucket-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Storage bucket on GCP wi... Basic and Lifecycle enabled. Need to create others. |
🟡 | |
| terraform-pubsub-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Pub/Sub topic and subscr... Not Available |
🟡 | |
| terraform-bigtable-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Bigtable instance, clust... Not Available |
🟡 | |
| terraform-bigquery-demo |
🏗️ ⌾Terraform demo project provisioning a Google Cloud BigQuery dataset and tabl... Not Available |
🟡 | |
| terraform-gce-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Compute Engine instance on GCP... Not Available |
🟡 | |
| terraform-cloud-function-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Function on GCP with con... Not Available |
🟡 | |
| terraform-cloud-sql-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud SQL instance on GCP with... Not Available |
🟡 | |
| terraform-cloud-spanner-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Spanner instance and dat... Not Available |
🟡 | |
| terraform-cloud-datastore-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Datastore database on GC... Not Available |
🟡 | |
| terraform-cloud-firestore-demo |
🏗️ ⌾Terraform demo project provisioning a Google Cloud Firestore database on GCP... Not Available |
🟡 | |
| terraform-dataflow-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud Dataflow job on GCP with... Not Available |
🟡 | |
| terraform-alloydb-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud AlloyDB cluster and inst... Not Available |
🟡 | |
| terraform-dataproc-demo |
🏗️⌾Terraform demo project provisioning a Google Cloud Dataproc cluster on GCP wi... Not Available |
🟡 | |
| terraform-vpc-subnet-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud VPC network and subnets ... Not Available |
🟡 | |
| terraform-kms-demo |
🏗️ ⌾ Terraform demo project provisioning a Google Cloud KMS key ring and crypto ... Not Available |
🟡 | |
| terraform-compute-mig-demo |
🏗️⌾Terraform demo project provisioning a Google Cloud Managed Instance Group on ... Not Available |
🟡 | |
| terraform-iam-demo |
🏗️ ⌾Terraform demo project provisioning Google Cloud IAM resources on GCP with s... Not Available |
🟡 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| checkov-report-action |
✅ GitHub JavaScript Action - Print Checkov Scan Report Not Available |
🟡 | |
| checkov-scan-action |
GitHub Composite Action - Checkov Scan Not Available |
🟡 | |
| tf-validate-action |
✅ GitHub Composite Action - Terraform Format Check and Validation. Not Available |
🟢 | |
| branch-issue-action |
✅ GitHub Composite Action - Check Branch Issue, to prevent pushing the code to a... Not Available |
🟢 | |
| tf-lint-action |
✅ GitHub Composite Action - Terraform Lint. Not Available |
🟢 | |
| tf-plan-action |
✅ GitHub Composite Action - Terraform Plan. Current - Working for AWS,Azure, GCP, Databricks. Next-Test for Snowflake |
🟢 | |
| tf-apply-action |
✅ GitHub Composite Action - Terraform Apply. Not Available |
🟢 | |
| tf-destroy-action |
GitHub Composite Action - Terraform Destroy. Not Available |
🟢 | |
| tf-yor-action |
GitHub Composite Action - Add Git metadata tags . Not Available |
🟡 | |
| create-release-action |
GitHub Composite Action - Create Draft Release with a clickable link to create t... Not Available |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| github-action-template |
✅ GitHub Composite Action: A Starter Template for GitHub Composite Action Not Available |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-destroy-wf |
GitHub Reusable Workflow to tear down a stack using Terraform Destroy Not Available |
🟢 | |
| tf-ci-reusable-wf |
GitHub Reusable Workflow: CI Pipeline using Terraform as IaC Current - Working for AWS,Azure, GCP, Databricks. Next-Test for Snowflake |
🟢 | |
| slack-notification-wf |
GitHub Reusable Workflow : Send Slack Notification on opening an issue of pull r... Not Available |
🟢 | |
| tf-release-wf |
GitHub Reusable Workflow: Terraform Release Not Available |
🟢 | |
| tf-deploy-multi-reusable-wf |
GitHub Reusable workflow to deploy a stack using Terraform multi-cloud environme... Not Available |
🔵 | |
| gcp-project-onboarding |
🏗️ Reusable GitHub Actions workflow for standardised GCP project onboarding. Pro... Not Available |
🟡 | |
| aws-project-onboarding |
✅ Reusable GitHub Actions workflow for standardized AWS project onboarding. Pro... Ready to release |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| checkov-custom-policies |
🏗️ A centralized repository for reusable custom Checkov policies, metadata-drive... Initial build successful |
⚪ | |
| terraform-checkov-scan-detector |
🏗️ GitHub Action to scan Terraform configurations, detect which cloud services a... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-snowflake-warehouse |
✅ Terraform module to manage Snowflake warehouses with configurable sizing, scal... Working as expected. |
🟢 | |
| terraform-snowflake-external-volume |
Terraform module to manage Snowflake external volume. Not Available |
⚪ | |
| terraform-snowflake-dynamic-table |
✅ Terraform module to manage Snowflake dynamic table. Working as expected. |
🟢 | |
| terraform-snowflake-database-schema |
✅ Terraform module to create and manage Snowflake databases and schemas using in... Working as expected. |
🟢 | |
| terraform-snowflake-file-format |
✅ Terraform module to manage Snowflake file format (CSV / JSON / Parquet) Working as expected. |
🟢 | |
| terraform-snowflake-storage-integration |
✅ Terraform module to manage Snowflake storage integration. Working as expected. |
🟢 | |
| terraform-snowflake-pipe |
✅ Terraform module to manage Snowflake pipe. Working as expected. |
🟢 | |
| terraform-snowflake-task |
✅ Terraform module to manage Snowflake task. Not Available |
🟡 | |
| terraform-snowflake-stage |
✅ Terraform module to manage Snowflake internal and external stage. Working as expected. |
🟢 | |
| terraform-snowflake-stream |
🎯 Terraform module to manage Snowflake stream. Need to start working on this repository. |
⚪ | |
| terraform-snowflake-table |
✅ Terraform module to manage Snowflake table. Working as expected. |
🟢 | |
| terraform-snowflake-seed-data |
✅ Terraform Snowflake module to execute controlled SQL (INSERT, UPDATE, MERGE) f... Working as expected. |
🟢 |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-aws-s3 |
✅ Terraform AWS Module for S3 Bucket Not Available |
🟢 | |
| terraform-aws-iam |
✅ Terraform AWS Module for IAM Working as expected |
🟢 | |
| terraform-aws-glue-database |
🎯 Reusable Terraform module to create and manage AWS Glue Data Catalog databases... Ready for merge |
🟢 | |
| terraform-aws-glue-catalog-table |
🎯 Terraform module to create and manage AWS Glue Data Catalog tables, supporting... Ready for merge to main |
🟡 | |
| terraform-aws-cloudwatch |
🎯 Reusable Terraform modules for provisioning and managing AWS CloudWatch resour... Not Available |
⚪ |
| Repository | Description | Status | Stats |
|---|---|---|---|
| terraform-google-project-hierarchy |
✅ Terraform module for managing GCP organization folder and project hierarchies ... Not Available |
🟢 | |
| terraform-google-gcs-bucket |
🏗️ Terraform module to provision and manage Cloud Storage buckets, lifecycle rul... Initial build successful |
🟡 | |
| terraform-google-dataflow |
🏗️ Terraform module to deploy and manage Dataflow jobs, templates, staging confi... Initial build successful |
🟡 | |
| terraform-google-spanner-instance |
🏗️ Terraform module to provision and manage Google Cloud Spanner instances with ... Working on Claude SKILL |
🟡 | |
| terraform-google-vpc |
🏗️ Terraform module to create and manage VPC networks, subnets, routes, firewall... Initial release successful |
🟡 | |
| terraform-google-spanner-database |
🏗️ Terraform module to create and manage databases within Google Cloud Spanner i... Working on Claude SKILL |
🟡 | |
| terraform-google-cloud-sql-instance |
🏗️ Terraform module to provision and manage Cloud SQL instances, including engin... Initial build successful |
🟡 | |
| terraform-google-cloud-sql-database |
🏗️ Terraform module to create and manage databases within Cloud SQL instances, i... Working on Claude SKILL |
🟡 | |
| terraform-google-alloydb-cluster |
🏗️ Terraform module to provision and manage AlloyDB clusters, including networki... Working on Claude SKILL |
🟡 | |
| terraform-google-alloydb-instance |
🏗️ Terraform module to create and manage AlloyDB instances (primary and read poo... Working on Claude SKILL |
🟡 | |
| terraform-google-firestore |
🏗️ Terraform module to configure Firestore databases, indexes, field settings, a... Working on Claude SKILL |
🟡 | |
| terraform-google-datastore |
🏗️ Terraform module to manage Datastore-mode database configuration, indexes, an... Working on Claude SKILL |
🟡 | |
| terraform-google-bigtable-instance |
🏗️ Terraform module to provision Bigtable instances, clusters, autoscaling, stor... Initial build successful |
🟡 | |
| terraform-google-bigtable-table |
🏗️ Terraform module to create and manage Bigtable tables, including column famil... Initial build successful |
🟡 | |
| terraform-google-bigquery-dataset |
🏗️ Terraform module to manage BigQuery datasets, including location, access cont... Initial build successful |
🟡 | |
| terraform-google-kms |
🏗️ Terraform module to manage KMS key rings, crypto keys, IAM bindings, rotation... Initial build successful |
🟡 | |
| terraform-google-dataproc-cluster |
🏗️ Terraform module to create and manage Dataproc clusters, batches (serverless)... Initial build successful |
🟡 | |
| terraform-google-pubsub-topic |
🏗️ Terraform module to provision Pub/Sub topics, including message retention, or... Initial build successful |
🟡 | |
| terraform-google-pubsub-subscription |
🏗️ Terraform module to create and manage Pub/Sub subscriptions, including push/p... Initial build successful |
🟡 | |
| terraform-google-compute-instance |
🏗️ Terraform module to create and manage standalone Compute Engine VM instances,... Initial build successful |
🟡 | |
| terraform-google-compute-mig |
🏗️ Terraform module to create and manage Managed Instance Groups (MIGs), includi... Working on Claude SKILL |
🟡 |
Last generated: 2026-04-19 12:11:01 EST
Total repositories: 129










