Skip to content

Repository files navigation

πŸš€ Navigant Education Consultants β€” Task Management & AI Engineering Workspace

A continuously evolving, dual-track internship portfolio built with Node.js, Express.js, PostgreSQL, Docker, Docker Compose, and Swagger UI, as part of the FlyRank AI Internship (2026).

This repository is maintained across two parallel internship tracks:

Track Focus Status
Track 1: Backend AI Engineering Workspace System architecture, Express.js servers, database layers, validation, containerization, and test evidence βœ… Active β€” Weeks 2–6 documented below
Track 2: AI Fluency Workspace Prompt engineering blueprints, model benchmarking, and AI-assisted development tooling 🟑 Scaffolded β€” content to be published as assignments are completed

πŸ“– Project Overview

This repository documents the continuous evolution of a backend application developed throughout the FlyRank Backend AI Engineering Internship (2026).

The project began as a simple RESTful CRUD API and is being enhanced incrementally through weekly backend engineering assignments. Each assignment introduces new technologies, architectural improvements, and engineering practices while building upon the previous implementation.

Instead of creating separate repositories for each assignment, this project demonstrates the complete engineering journey β€” from a beginner-friendly CRUD API toward a more structured, containerized, and production-oriented backend application.

The application is customized with realistic operational workflows from Navigant Education Consultants, illustrating how backend technologies can support education, career development, and future AI-powered digital solutions aligned with:

  • πŸŽ“ SDG 4 – Quality Education
  • πŸ‘©β€πŸ’Ό SDG 5 – Gender Equality

This repository showcases progressive backend engineering concepts including:

  • API development
  • CRUD operations
  • REST architecture
  • OpenAPI / Swagger documentation
  • Database integration and migration
  • SQL and parameterized queries
  • Repository-based database access
  • PostgreSQL
  • Environment configuration
  • Docker containerization
  • Docker Compose
  • Persistent Docker volumes
  • API testing
  • Debugging and troubleshooting
  • Git and GitHub workflow
  • AI-assisted development and engineering comparison
  • Progressive backend architecture

The repository is intentionally maintained as a single evolving project so that the codebase, Git history, documentation, and engineering decisions demonstrate how the application grows throughout the internship.


βš™οΈ Track 1: Backend AI Engineering Workspace

πŸ“‹ Assignment Feature & Skills Index

Assignment Week Status Key Features Built Skills & Tools Used
BE-01 β€” Build Your First CRUD API Week 2 βœ… Completed RESTful CRUD endpoints for tasks, JSON request/response handling, route parameters (:id), request validation, correct HTTP status codes Node.js, Express.js, REST API design, JSON, API validation, Postman & Swagger UI testing
BE-02 β€” Connecting CRUD API to the Database Week 3 βœ… Completed SQLite β†’ PostgreSQL migration, PostgreSQL repository / data-access layer, automatic table initialization & seeding, parameterized queries PostgreSQL, pg driver, SQL, parameterized queries, DB Browser for SQLite, repository/data-access pattern
BE-04 β€” Containerize Your Stack Week 3 βœ… Completed Dockerized API + PostgreSQL, Docker Compose orchestration, named persistent volume, verified container-restart persistence Docker, Dockerfile, Docker Compose, Docker volumes, environment-variable configuration
Authentication β€” Login & Protect (BE-03) Week 4 βœ… Completed Supabase-backed signup / login / logout endpoints, JWT access_token / refresh_token issuance, reusable auth middleware, Swagger bearer-auth integration Supabase Auth, JWT, Express middleware, OpenAPI securitySchemes
BE-05 β€” The Polite Scraper Week 5 βœ… Completed Rate-limited, polite scraping pipeline, Zod-validated structured data extraction, generated PDF summary report Node.js web scraping, Zod schema validation, PDF report generation, nested technical documentation
BE-07 β€” Put an LLM Behind Your API Week 6 βœ… Completed OpenAI SDK wired through OpenRouter's free routing pathway, Stage 0 connectivity script (src/llm/hello.js) OpenAI SDK, OpenRouter, Node.js module/environment debugging
Assignment A7 β€” Your First Background Job Week 6 βœ… Completed Out-of-process job orchestration, fast 202 release routing gateway, state polling tracking loops, autonomous chronological cron heartbeat engines Inngest SDK, Node.js Events, Inngest CLI Local Dev Servers, UUID Generation, Cron Schedules

πŸš€ Project Evolution

This repository is maintained as a single, continuously evolving backend engineering project throughout the FlyRank Backend AI Engineering Internship (2026).

Each assignment extends the existing application rather than creating a completely separate project. This approach demonstrates how the same backend evolves through progressively more advanced engineering concepts, technologies, architecture, testing, documentation, and AI integration.

Week Backend AI Engineering Assignment Status
Week 2 BE-01 β€” Build Your First CRUD API βœ… Completed
Week 3 BE-02 β€” Connecting CRUD API to the Database βœ… Completed
Week 3 BE-04 β€” Containerize Your Stack βœ… Completed
Week 4 Authentication β€” Login & Protect βœ… Completed
Week 5 The Polite Scraper βœ… Completed
Week 6 Your First Background Job βœ… Completed
Week 6 Connect to an AI API βœ… Completed
Week 7 Build an AI Decision Flow with React Flow + Inngest βœ… Completed
Week 7 PDF Report Generator ⏳ Planned
Week 7 Your first background job ⏳ Planned
Week 8 Backend Capstone Documentation & Case Study ⏳ Planned

10-Week Internship Roadmap: The internship is a 10-week learning journey. This table records the backend assignments currently identified in the project roadmap and is updated progressively as each assignment is completed. Future assignments and milestones will be added to the repository as the internship progresses.

Continuous Engineering Approach

Rather than treating each assignment as an isolated exercise, this repository preserves the application's engineering history across the internship.

The project progressively demonstrates:

CRUD API β†’ Database Integration β†’ PostgreSQL β†’ Repository/Data Access β†’ Docker β†’ Persistent Containers β†’ Authentication β†’ AI Integration β†’ Background Processing β†’ AI Workflows β†’ Reporting β†’ Capstone Engineering

This continuous evolution allows the repository to demonstrate not only individual technologies, but also the ability to extend, refactor, test, document, and improve an existing backend system as new engineering requirements are introduced.


πŸ† Engineering Skills Progression

Throughout this internship, this repository demonstrates progressive backend engineering experience across API development, databases, containerization, testing, documentation, version control, and AI-assisted engineering.

βœ… Completed Skills

  • βœ… REST API Design
  • βœ… CRUD Operations
  • βœ… Express.js
  • βœ… Node.js Backend Development
  • βœ… OpenAPI / Swagger Documentation
  • βœ… JSON Request & Response Handling
  • βœ… Route Parameters
  • βœ… API Validation
  • βœ… HTTP Status Codes
  • βœ… API Testing with Postman
  • βœ… Swagger UI Testing
  • βœ… Browser-based API Testing
  • βœ… API Debugging & Troubleshooting
  • βœ… SQL and Parameterized Queries
  • βœ… SQLite Database Integration
  • βœ… PostgreSQL Database Integration
  • βœ… PostgreSQL Repository / Data Access Layer
  • βœ… Environment Variables
  • βœ… .env / .env.example
  • βœ… Docker
  • βœ… Dockerfile
  • βœ… Docker Compose
  • βœ… Docker Volumes
  • βœ… Containerized PostgreSQL
  • βœ… Containerized Node.js / Express API
  • βœ… Persistent Data Across Container Restarts
  • βœ… Git Version Control
  • βœ… GitHub Repository Management
  • βœ… AI-Assisted Development
  • βœ… AI vs Me Implementation Compariso
  • βœ… Background Jobs & Event Orchestration
  • βœ… Chronological Cron Schedules

⏳ Upcoming Skills

As the internship progresses, the same project will be extended to develop additional backend and AI engineering capabilities, including:

  • ⏳ Authentication & Authorization
  • ⏳ AI API Integration
  • ⏳ Web Scraping
  • ⏳ Background Jobs
  • ⏳ AI Workflow Orchestration
  • ⏳ PDF Generation
  • ⏳ Production Deployment
  • ⏳ Additional backend engineering practices introduced through later internship assignments

Progressive Engineering Principle: Skills are marked as completed only when they have been implemented and verified within the project. Future capabilities remain marked as upcoming until the corresponding internship work is completed and evidenced.


✨ Features

Current Features

🌐 API & Backend

  • βœ… RESTful CRUD API
  • βœ… Node.js backend
  • βœ… Express.js server
  • βœ… JSON request and response handling
  • βœ… Route parameters (:id)
  • βœ… CRUD operations for tasks
  • βœ… Request validation
  • βœ… Appropriate HTTP status codes
  • βœ… API debugging and troubleshooting

πŸ—„ Database & Persistence

  • βœ… SQLite database integration during the initial database stage
  • βœ… PostgreSQL relational database integration
  • βœ… PostgreSQL repository / data-access layer
  • βœ… SQL queries
  • βœ… Parameterized SQL queries
  • βœ… Automatic PostgreSQL table initialization
  • βœ… Automatic default task seeding
  • βœ… Seeding only when the database is empty
  • βœ… Persistent PostgreSQL storage
  • βœ… PostgreSQL data preserved across container restarts

🐳 Docker & Infrastructure

  • βœ… Dockerized Node.js / Express API
  • βœ… Dockerized PostgreSQL database
  • βœ… Dockerfile
  • βœ… Docker Compose
  • βœ… API + PostgreSQL launched together with docker compose up
  • βœ… Named Docker volume for PostgreSQL persistence
  • βœ… Containerized development environment
  • βœ… Verified application and database container startup
  • βœ… Verified database persistence after stack restart

πŸ” Configuration & Security

  • βœ… Environment-variable based configuration
  • βœ… .env for local configuration
  • βœ… .env.example configuration template
  • βœ… .env excluded from version control
  • βœ… Database credentials kept outside committed source code

πŸ“š API Documentation & Testing

  • βœ… OpenAPI 3.0 specification
  • βœ… Interactive Swagger UI documentation
  • βœ… Postman API testing
  • βœ… Swagger UI (Try it out) testing
  • βœ… Browser testing for GET requests
  • βœ… CRUD endpoint verification
  • βœ… PostgreSQL persistence verification

🧰 Engineering Workflow

  • βœ… Git version control
  • βœ… GitHub repository management
  • βœ… Incremental Git commits
  • βœ… README documentation
  • βœ… Evidence-based development documentation
  • βœ… AI-assisted development
  • βœ… AI vs Me implementation comparison
  • βœ… Engineering decisions evaluated against an independent AI implementation

πŸŽ“ Real-World Project Context

  • βœ… Realistic Navigant Education Consultants task data
  • βœ… Educational and community-oriented workflow examples
  • βœ… Project evolution documented as part of the FlyRank Backend AI Engineering Internship
  • βœ… Single continuously evolving repository rather than isolated assignment repositories

πŸ›  Technology Stack

Technology Purpose
Node.js JavaScript runtime for the backend application
Express.js Backend web framework for building the REST API
JavaScript Primary programming language
REST API Backend API architecture and communication style
JSON Request and response data format
Swagger UI Interactive API documentation and endpoint testing
OpenAPI 3.0 API specification and documentation
Postman API endpoint testing and verification
SQLite Initial persistent database used during the earlier database integration stage
sqlite3 Node.js SQLite driver used during the earlier database implementation
DB Browser for SQLite Database inspection and SQL testing during the earlier SQLite stage
PostgreSQL Current relational database used for persistent application data
pg Node.js PostgreSQL client/driver
SQL Database querying and data manipulation
Parameterized SQL Queries Safer database query execution and input handling
Repository / Data Access Layer Separates database operations from the application/API layer
dotenv Loads environment variables from .env
.env Local database and application configuration
.env.example Safe configuration template for project setup
Docker Containerizes the backend application and database environment
Dockerfile Defines the container image for the Node.js API
Docker Compose Orchestrates the API and PostgreSQL services together
Docker Volume Provides persistent PostgreSQL data storage across container restarts
Git Source-code version control
GitHub Remote repository and project history
VS Code Development environment
AI Assistant / ChatGPT AI-assisted learning, development support, debugging, comparison, and engineering reflection

🧩 BE-01 β€” Build Your First CRUD API (Week 2)

πŸ”— API Endpoints

The Task Management API exposes RESTful CRUD endpoints for creating, reading, updating, and deleting task records stored in PostgreSQL.

Method Endpoint Description Success Response
GET /tasks Retrieve all tasks 200 OK
GET /tasks/:id Retrieve a specific task 200 OK
POST /tasks Create a new task 201 Created
PUT /tasks/:id Update an existing task 200 OK
DELETE /tasks/:id Delete an existing task 204 No Content
GET /docs Interactive Swagger UI documentation 200 OK

Validation & Error Responses

The API also handles common invalid requests and missing resources:

Scenario Response
Missing task title on POST 400 Bad Request
Requested task does not exist 404 Not Found
Unknown task ID during PUT 404 Not Found
Unknown task ID during DELETE 404 Not Found

API Base URL

When the Dockerized application is running:

http://localhost:3000

Example Endpoints

GET    http://localhost:3000/tasks
GET    http://localhost:3000/tasks/1
POST   http://localhost:3000/tasks
PUT    http://localhost:3000/tasks/1
DELETE http://localhost:3000/tasks/1

API Documentation

Interactive Swagger UI is available at:

http://localhost:3000/docs

The OpenAPI specification is maintained in:

openapi.json

The API can be tested through Postman, Swagger UI (Try it out), and browser GET requests.


πŸ—„οΈ BE-02 β€” Connecting CRUD API to the Database (Week 3)

Database Evolution

This project demonstrates the evolution of the application's data layer during the FlyRank Backend AI Engineering Internship (2026).

The application initially used SQLite as a lightweight persistent database during the early database integration stage. It was later migrated to PostgreSQL as the project progressed toward a more realistic, containerized backend architecture.

This progression demonstrates an important backend engineering principle: the application can evolve from a simple local database implementation toward a client-server relational database without changing the core purpose of the API.

Phase 1 β€” SQLite

SQLite was used during the initial database integration stage because it is lightweight, serverless, easy to configure, and well suited to learning relational database concepts in a local development environment.

The SQLite implementation demonstrated:

  • Persistent relational data storage
  • Automatic database creation
  • Automatic table creation
  • Default task seeding
  • SQL queries
  • Parameterized queries
  • CRUD operations
  • Database persistence across application restarts
  • Database inspection using DB Browser for SQLite

The earlier SQLite database was stored locally as:

flyrank-crud-api/
└── tasks.db

SQLite Initialization

During the SQLite stage, the application automatically:

  • Created tasks.db if it did not exist.
  • Created the tasks table if it did not exist.
  • Seeded three default tasks when the table was empty.

Phase 2 β€” PostgreSQL

As the project progressed, SQLite was replaced by PostgreSQL to introduce a more realistic client-server relational database architecture and prepare the application for containerized development.

The current application uses:

  • PostgreSQL as the persistent relational database
  • Node.js pg driver for database communication
  • Parameterized SQL queries
  • A PostgreSQL repository / data-access layer
  • Environment-based database configuration
  • Dockerized PostgreSQL
  • A persistent Docker volume

The PostgreSQL database runs as a separate Docker service from the Node.js API.

Current Architecture

Client
   β”‚
   β–Ό
Node.js / Express API
   β”‚
   β–Ό
PostgreSQL Repository / Data Access Layer
   β”‚
   β–Ό
PostgreSQL Database
   β”‚
   β–Ό
Persistent Docker Volume

This separation allows the API application and database to run independently while communicating through the database connection configured through environment variables.

PostgreSQL Initialization

When the containerized application starts:

  • PostgreSQL runs as a Docker service.
  • The API connects to PostgreSQL using environment-based configuration.
  • The application's tasks table is initialized when required.
  • Default Navigant Education Consultants tasks are seeded when the database is empty.
  • Existing database records are preserved when the stack is restarted.

The current PostgreSQL setup uses a Docker volume so database data is stored independently of the lifecycle of the PostgreSQL container.

Why PostgreSQL Replaced SQLite

The migration from SQLite to PostgreSQL was part of the project's progression from a learning-focused local database toward a more realistic backend architecture.

PostgreSQL provides a dedicated database server architecture that better represents the type of relational database environment commonly used in production backend systems.

The migration also introduced practical experience with:

  • Database connection configuration
  • PostgreSQL drivers
  • Repository / data-access patterns
  • Dockerized databases
  • Docker Compose
  • Persistent Docker volumes
  • Environment variables
  • Database service connectivity
  • Application and database lifecycle management

Database Persistence

Persistence was verified using the Dockerized PostgreSQL database.

The verification process involved:

  1. Creating task records.
  2. Running the API and PostgreSQL through Docker Compose.
  3. Stopping the application and database containers.
  4. Starting the stack again.
  5. Querying PostgreSQL to confirm that previously stored records remained available.

This demonstrated that the data was persisted through the Docker volume rather than being lost when the containers were stopped or recreated.

Engineering Lesson

The database evolution demonstrates that backend architecture is not static.

The project progressed from:

SQLite
   ↓
PostgreSQL
   ↓
Dockerized PostgreSQL
   ↓
Persistent PostgreSQL storage

This progression provided practical experience in replacing a local database implementation with a more production-oriented database while preserving the application's core CRUD functionality.

πŸ§ͺ Example SQL Query

SQL was used throughout the database stages of the project to inspect, verify, and work with task records.

Current PostgreSQL Query

The current application uses PostgreSQL as its persistent database.

The following query was used to inspect the records stored in the tasks table:

SELECT * FROM tasks;

The query can be executed through PostgreSQL using:

docker exec -it flyrank-crud-api-db-1 psql -U postgres -d tasks -c "SELECT * FROM tasks;"

This was used to verify that task records existed in the PostgreSQL database and remained available after restarting the Dockerized application stack.

Example Result

The PostgreSQL database was successfully queried and returned persisted task records, including realistic Navigant Education Consultants workflow data.

Example:

 id |                             title                              | done
----+-----------------------------------------------------------------+------
  1 | Prepare SDG 4 education workshop materials                     |   0
  2 | Schedule meeting with women mentors for the LeadHer initiative |   0
  3 | Publish Navigant Education Consultants website update          |   1

This provided direct evidence that the API was connected to PostgreSQL and that the database contained persistent application data.

Earlier SQLite Stage

During the earlier SQLite database stage, the same SQL concept was tested using DB Browser for SQLite:

SELECT * FROM tasks;

This allowed the database records to be inspected directly while learning database persistence and SQL operations.

The project therefore demonstrates the same core database operation across two stages:

SQLite
   ↓
SQL Query
   ↓
PostgreSQL
   ↓
SQL Query

The database technology changed as the architecture evolved, while the core task-management data model and CRUD purpose remained consistent.

πŸ“Έ Database Screenshot

The current application uses PostgreSQL as its persistent database and runs PostgreSQL inside Docker.

Database verification was performed by querying the running PostgreSQL container and confirming that task records were stored successfully.

The database verification evidence is maintained in the project's Notion documentation.

PostgreSQL Database Evidence

PostgreSQL Tasks Data

PostgreSQL Tasks Data

Dockerized PostgreSQL + API

Dockerized PostgreSQL Stack

The verification demonstrated:

  • βœ… PostgreSQL container running successfully
  • βœ… tasks table available
  • βœ… Task records stored in PostgreSQL
  • βœ… Realistic Navigant Education Consultants task data present
  • βœ… Data remained available after restarting the Dockerized stack
  • βœ… PostgreSQL persistence provided through the Docker volume

Example verification query:

SELECT * FROM tasks;

The project also retains evidence from the earlier SQLite stage, where DB Browser for SQLite was used to inspect and verify the database before the migration to PostgreSQL.

Evidence: Database screenshots and terminal verification outputs are documented in the project's Notion knowledge base.

Earlier SQLite Stage

The project previously used SQLite during BE-02. The following screenshot documents that earlier database stage before the migration to PostgreSQL.

SQLite Database


🐳 BE-04 β€” Containerize Your Stack (Week 3)

πŸ“‚ Project Structure

flyrank-crud-api/
β”‚
β”œβ”€β”€ .gitignore
β”œβ”€β”€ .env.example
β”œβ”€β”€ README.md
β”œβ”€β”€ Dockerfile
β”œβ”€β”€ compose.yaml
β”œβ”€β”€ server.js
β”œβ”€β”€ database.js
β”œβ”€β”€ openapi.json
β”œβ”€β”€ package.json
β”œβ”€β”€ package-lock.json
└── tasks.db

Key Files

File Purpose
server.js Main Node.js / Express application and API routes
database.js PostgreSQL database connection, table initialization, seeding, and database access logic
openapi.json OpenAPI 3.0 specification for the REST API
Dockerfile Defines the Docker image for the Node.js API
compose.yaml Defines and orchestrates the API and PostgreSQL services
.env.example Safe template showing required environment configuration without real credentials
.gitignore Prevents local files and sensitive configuration such as .env from being committed
README.md Project documentation, setup instructions, architecture notes, and engineering progression
package.json Project metadata, dependencies, and npm scripts
package-lock.json Locks the installed dependency versions
tasks.db Earlier SQLite database file from the previous database stage

Current Runtime Architecture

The current application runs as two Docker Compose services:

Docker Compose
β”‚
β”œβ”€β”€ API Container
β”‚   └── Node.js + Express
β”‚       └── server.js
β”‚
└── Database Container
    └── PostgreSQL
        └── Persistent Docker Volume

The earlier tasks.db file is retained in the repository as part of the project's database migration history. PostgreSQL is the current database used by the running application.

πŸš€ Getting Started

Prerequisites

Make sure the following are installed:

  • Node.js
  • Docker Desktop
  • Git

You do not need to install PostgreSQL separately because the current application runs PostgreSQL inside Docker.

Clone Repository

git clone https://github.com/SyedaMun/flyrank-crud-api.git
cd flyrank-crud-api

Install Dependencies

Install the Node.js dependencies defined in package.json:

npm install

This installs the packages required by the API for local development and ensures that the project's dependency versions are installed according to package-lock.json.

Note: The current Docker workflow also installs the application dependencies inside the API container when the image is built.

Configure Environment Variables

Create a local .env file in the project root using .env.example as the template.

The .env.example file documents the required configuration without exposing real credentials.

Example:

DATABASE_URL=your_postgresql_connection_string
PORT=3000

Security: .env is intentionally excluded from Git through .gitignore. Never commit real database credentials, passwords, or other secrets to the repository.

Start the Full Stack

The current application is containerized with Docker Compose.

Start the API and PostgreSQL database together with:

docker compose up

Docker Compose starts:

  • Node.js / Express API
  • PostgreSQL database
  • Persistent PostgreSQL Docker volume

The API will be available at:

http://localhost:3000

API

Task endpoints are available under:

http://localhost:3000/tasks

Swagger Documentation

Interactive API documentation is available at:

http://localhost:3000/docs

Run in Detached Mode

To start the stack in the background:

docker compose up -d

You can verify the running services with:

docker compose ps

The expected services are:

  • API container
  • PostgreSQL database container

Stop the Stack

To stop the containers:

docker compose down

The PostgreSQL data remains available because the database uses a persistent Docker volume. Stopping the containers does not remove the volume.

Verify PostgreSQL Data

The PostgreSQL database can be queried from inside the running database container:

docker exec -it flyrank-crud-api-db-1 psql -U postgres -d tasks -c "SELECT * FROM tasks;"

This can be used to verify that task records exist in the current PostgreSQL database.

Persistence Verification

To verify persistence:

  1. Start the stack with docker compose up.
  2. Create task records through the API.
  3. Stop the stack with docker compose down.
  4. Start the stack again with docker compose up -d.
  5. Verify the containers are running with docker compose ps.
  6. Query PostgreSQL and confirm that the previously created records remain.

This demonstrates that PostgreSQL data persists through application and container restarts because it is stored in the Docker volume.


πŸ” Authentication β€” Login & Protect (BE-03) (Week 4)

Authentication Infrastructure (Week 4 / BE-03 Update)

The Task Management Engine features a modular token-based authentication subsystem leveraging the Supabase identity platform layer. Secure endpoints are guarded utilizing custom interceptive route middleware patterns.

Expanded Security Features

  • User Provisioning Gateway (POST /auth/signup): Handles input format checking and routes records into encrypted offsite database tables.
  • Cryptographic Token Dispatch Desk (POST /auth/login): Authorizes user identity details and issues standard JWT access_token and refresh_token envelopes.
  • Session Deactivation Gate (POST /auth/logout): Destroys active validation strings globally across cloud registries.
  • Decoupled Verification Middleware (authenticateUser): A reusable Express interceptor function that dynamically extracts bearer strings from request headers and parses validation states securely.
  • Interactive Lock Integration: OpenAPI spec configurations extended via securitySchemes to implement interactive lock controls across the Swagger UI browser dashboard layer.

πŸ•ΈοΈ BE-05 β€” The Polite Scraper (Week 5)

A rate-limited, "polite" scraping pipeline was built on top of the existing backend to collect and structure book-catalog data end-to-end β€” from raw HTML collection through to a finished report artifact.

Key features built:

  • A staged Stage 0 β†’ Stage 6 scraping roadmap (check-before-you-collect, listing collection, detail-page extraction, cleaning/validation & storage, failure survival & reporting, and evidence publication)
  • Zod structural schemas enforcing strict data-typing across every scraped item
  • A generated PDF summary report of the scraped catalog data
  • A nested scraper/README.md documenting the pipeline architecture and evidence

Skills & tools used: Node.js-based web scraping, Zod schema validation, PDF report generation, resilient/rate-limited request handling, and structured technical documentation.

Full stage-by-stage evidence (screenshots, logs, and milestone checkpoints) for BE-05 is maintained in the project's Notion knowledge base.


πŸ€– AI Engine Integration (Week 6 & 7 / BE-07 Milestone Update)

The Task Management API features a fully completed, production-ready live LLM connectivity layout. The endpoint ingests unstructured text strings and processes them into strictly typed, validated JSON formats under a fault-tolerant backend layer.

πŸ“‹ Architectural Job Card (POST /tickets/classify)

  • Input Contract Payload: { "text": "string, 1-2000 characters" }
  • Output Schema Contract Layout:
    {
      "category": "billing | bug | feature | other",
      "urgency": "low | normal | high",
      "confidence": 0.0 - 1.0,
      "reason": "one short descriptive sentence"
    }
  • Strict Guardrails: The system never returns free-form conversational text filler, never hallucinates out-of-bounds enums, and handles unmapped inputs by safely reverting to "other" with low confidence scores.

πŸ—οΈ Production Resilience Infrastructure Built (Stage 4 Stability Core)

  1. Explicit Client Connection Ceiling: Configured a strict 30000ms (30s) connection ceiling to eliminate hanging thread vulnerabilities.
  2. Automated Exponential Backoff Loop: Built a custom 3-attempt backoff system featuring dynamic delay multiplier scaling and random jitter to handle transient 429 rate limits or 5xx cloud cluster crashes.
  3. Hardcoded Abort Drop Gates: Enforces early exit sequences to block redundant retry loops whenever encountering unrecoverable client errors (400, 401, or 403).
  4. Self-Healing Loop Optimization: Employs an internal fallback parsing retry sequence to automatically hand schema formatting failures back to the model for format remediation.
  5. Administrative Kill Switch: Reads an environmental property (LLM_ENABLED=false) allowing developers to immediately divert traffic to local deterministic safety objects during upstream platform outages.

πŸ“ˆ Automated Benchmark Evaluation Scores (Stage 5 Verification)

An automated suite executed validation tests sequentially against a static 8-case dataset matrix to calculate accuracy metrics natively under live conditions:

  • Total Execution Benchmark Score: 3 / 8 Test Inbound Cases Passed Successfully
  • Final Accuracy Metrics Rating: 37.5%
  • Evaluation Chronological Date: September 07, 2026
  • Assigned Model ID Asset: openrouter/free

Technical Observation Note:* Failed cases successfully triggered Stage 3 layout exception blocks and gracefully returned structured 422 Unprocessable Entity wrappers, protecting the system core from unclassified text strings or malformed data properties.

πŸ§ͺ Local Setup & Verification Commands

  1. Start the API Application Server:
    node server.js
  2. Run the Automated Evaluation Script (In a Separate Terminal Tab):
    node src/llm/evals/runEval.js
  3. Manual Verification via curl:
    curl -i -X POST http://localhost:3000/tickets/classify \
      -H "Content-Type: application/json" \
      -d '{"text": "Hey! Your app crashed when I tried to pay, and it took my money! Fix this now!"}'

⏱️ Assignment A7 β€” Your First Background Job (Week 6 Core)

An out-of-process distributed asynchronous workflow engine was mounted onto our Express backend stack using the Inngest framework lane. This architecture implements the high-velocity "accept-fast, process-decoupled, track-status" design pattern used for resource-heavy workloads.

πŸ—οΈ Workflow Engine Architecture Block

POST /api/reports GET /api/reports/:id Inngest Dashboard (Fast 202 Handshake) (Polling Status Gate) (Port :8288 Console) β”‚ β”‚ β”‚ β”œβ”€β”€β–Ί [Dispatches Event] β”œβ”€β”€β–Ί [Checks Memory Store] β”œβ”€β”€β–Ί [Tracks Execution Graph] β”‚ report/generate β”‚ pending or complete β”‚ Maps retry cascades β–Ό β–Ό β–Ό Responds in <20ms Returns live workflow state Automated 1-Min Cron Heartbeat

πŸ“‹ Decoupled Background Architectural Routes

  1. The Fast Door Gateway (POST /api/reports): Ingests execution requests, instantly seeds an in-memory transactional record mapping, dispatches an asynchronous event token to the local runner, and releases the connection chain in under 20 milliseconds by returning an HTTP 202 Accepted handshake wrapper.
  2. The Status Verification Gate (GET /api/reports/:id): Provides a clean data polling interface layer. It returns a "status": "pending" token string while the report bakes out-of-process, and transforms automatically to "status": "complete" appending compilation payload payloads once finalized.

πŸ› οΈ Configured Background Worker Functions (src/llm/backgroundJobs.js)

  • say-hello Worker Core: Our initial phase pipeline worker bound to trigger upon receiving incoming test/hello test events.
  • generate-report Heavy Baker: Listens to report/generate events, isolates an 8-second execution sleep step without tying up web request threads, and writes successful state records into the tracking store layer.
  • cron-heartbeat Scheduled Clock Engine: An autonomous chronological job configured using classic 5-field Unix cron mapping expressions (* * * * *). It boots every 60 seconds automatically on the clock, logging continuous pulses independent of user requests.
  • retry-simulation Fault Tolerant Worker: Programmed with restrictive retry parameters (retries: 3) to map error backoff recovery waterfall graphs visually inside our tracking dashboards upon transient failure conditions.

πŸ§ͺ Dual-Terminal Boot Setup & Verification Commands

  1. Spin up the Inngest Background Runner (Terminal Pane #2): npx inngest-cli@latest dev -u http://localhost:3000/api/inngest
  2. Launch the Core Express API Application Server (Terminal Pane #1): npm start
  3. Verify Asynchronous Processing Loop Via Postman:

πŸ§ͺ Testing

The project has been tested progressively throughout the internship as new backend capabilities were introduced.

API Testing

The REST API was tested using:

  • βœ… Postman
  • βœ… Swagger UI (Try it out)
  • βœ… Browser testing for GET requests

The CRUD operations were verified for:

  • GET /tasks
  • GET /tasks/:id
  • POST /tasks
  • PUT /tasks/:id
  • DELETE /tasks/:id

Validation & Error Testing

The API was also tested against invalid and missing data, including:

  • Missing task title
  • Non-existent task IDs
  • Invalid update requests
  • Invalid delete requests

Expected HTTP responses were verified, including:

  • 400 Bad Request
  • 404 Not Found
  • 201 Created
  • 200 OK
  • 204 No Content

Database Testing

Database behavior was verified during both the SQLite and PostgreSQL stages.

SQLite Stage

The earlier SQLite implementation was tested using:

  • DB Browser for SQLite
  • SQL queries
  • Application restarts
  • CRUD operations

PostgreSQL Stage

The current PostgreSQL implementation was verified using:

  • PostgreSQL running inside Docker
  • Docker Compose
  • PostgreSQL SQL queries
  • docker exec
  • Application logs
  • Database container logs

Example verification query:

SELECT * FROM tasks;

Docker Testing

The containerized stack was verified using:

docker compose up

and:

docker compose up -d

Running services were verified with:

docker compose ps

The expected architecture was confirmed:

Docker Compose
β”‚
β”œβ”€β”€ API Container
β”‚   └── Node.js + Express
β”‚
└── Database Container
    └── PostgreSQL

Persistence Testing

Database persistence was explicitly tested by:

  1. Creating task records.
  2. Confirming the records in PostgreSQL.
  3. Stopping the application and database containers.
  4. Starting the stack again.
  5. Querying PostgreSQL after restart.
  6. Confirming that the previously stored records remained available.

This verified that the PostgreSQL Docker volume was providing persistent storage rather than temporary container storage.

Configuration Testing

Environment configuration was verified using:

  • .env
  • .env.example
  • .gitignore

The actual .env file remains local and is not committed to GitHub.

AI-Assisted Testing & Comparison

During Stage 6, the implemented solution was also compared with an independently generated AI implementation.

The comparison examined areas including:

  • Architecture
  • Database handling
  • Docker configuration
  • Environment configuration
  • Repository/data-access design
  • Error handling
  • Code organization
  • Documentation
  • Engineering trade-offs

The purpose was not simply to determine which implementation was "better", but to evaluate the differences and use AI as an engineering review and learning tool.

Testing Evidence

Testing evidence is documented in the project's Notion knowledge base and includes relevant screenshots, terminal outputs, database verification, Docker status, Git status, and AI-assisted engineering comparison evidence.


🎯 Learning Outcomes

Through the progressive development of this project during the FlyRank Backend AI Engineering Internship (2026), I have developed practical experience in the following areas:

Backend Engineering

  • Backend application setup and project organization
  • Node.js runtime fundamentals
  • Express.js backend development
  • REST API architecture
  • CRUD operations
  • HTTP methods and status codes
  • Route parameters
  • JSON request and response handling
  • Input validation
  • API error handling
  • API debugging and troubleshooting

Database Engineering

  • Relational database concepts
  • SQL queries
  • Parameterized SQL queries
  • SQLite database integration
  • PostgreSQL database integration
  • Database initialization and table creation
  • Data seeding
  • Repository / data-access layer concepts
  • Persistent data storage
  • Database verification and inspection
  • Migration from a lightweight local database to PostgreSQL

Docker & Infrastructure

  • Docker fundamentals
  • Writing a Dockerfile
  • Containerizing a Node.js API
  • Running PostgreSQL in Docker
  • Docker Compose
  • Multi-container application architecture
  • Docker networking between services
  • Docker volumes
  • Persistent database storage
  • Starting and managing a complete backend stack with one command
  • Verifying application and database containers

API Documentation & Testing

  • OpenAPI 3.0
  • Swagger UI
  • Postman API testing
  • Browser-based API testing
  • CRUD endpoint verification
  • Database persistence verification
  • Testing expected error responses
  • Evidence-based technical documentation

Configuration & Security

  • Environment variables
  • .env configuration
  • .env.example templates
  • .gitignore
  • Keeping local credentials outside the public repository
  • Understanding the difference between configuration and source code

Version Control & Professional Workflow

  • Git version control
  • Git staging and commits
  • Meaningful commit messages
  • GitHub repository management
  • Git push / pull workflow
  • Maintaining a clean working tree
  • Incremental project development
  • README documentation
  • Portfolio-oriented project organization

AI-Assisted Engineering

  • Using AI as a development and learning assistant
  • Breaking backend tasks into smaller engineering stages
  • Using AI for debugging and troubleshooting
  • Comparing an independently implemented solution with an AI-generated solution
  • Evaluating AI-generated architecture and implementation choices
  • Identifying differences, strengths, weaknesses, and trade-offs
  • Improving prompts based on implementation results
  • Using AI to support engineering reasoning rather than simply copying generated code

Engineering Mindset

The most important outcome of this project is learning how to evolve an existing backend system rather than repeatedly building isolated examples.

The project demonstrates a progression from:

REST API
   ↓
Persistent Database
   ↓
PostgreSQL
   ↓
Repository / Data Access
   ↓
Dockerized Application
   ↓
Dockerized PostgreSQL
   ↓
Persistent Containerized Stack
   ↓
AI-Assisted Engineering Review

This experience has strengthened my understanding of how backend components work together and how engineering decisions affect maintainability, persistence, portability, testing, and future scalability.

Learning principle: The objective of this internship is not only to complete assignments, but to understand the concepts behind each implementation and build a portfolio that demonstrates practical engineering reasoning.


πŸ‘©β€πŸ’» Author

Syeda Munazza Bukhari

Founder β€” Navigant Education Consultants

Backend AI Engineering Intern β€” FlyRank AI Internship (2026)

Building practical backend and AI engineering solutions that support education, career development, and digital empowerment.


πŸ™ Acknowledgements

Developed as part of the FlyRank Backend AI Engineering Internship (2026) to strengthen practical backend engineering skills through hands-on project development.

About

Persistent Task Management REST API built with Node.js, Express.js, SQLite, and Swagger UI. Developed during the FlyRank Backend AI Engineering Internship using real-world educational workflows from Navigant Education Consultants.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages