🧠 Software Engineering Playbook

A complete methodology for building software projects: how to think and build from scratch to deployment, from CLI tools to full-stack apps.

📌 About This Playbook

Most developers don't get stuck because they lack tools. They get stuck because they lack a repeatable process. This playbook is that process, written down: a fixed sequence of steps that takes you from a blank folder to a deployed, tested, documented project.

Who it's for

Students and junior developers building their first portfolio projects
Self-taught developers who know the syntax but not the workflow
Anyone who starts projects with enthusiasm and abandons them at hour three

What's inside

🧠 The philosophy: why you get stuck and how to think before you code
🛠️ The 9-step process: understanding, pseudocode, design, code, tests, refactor, docs, deploy, practice
🔀 The technical workflow: Git branching, commits, essential files, CI/CD
🚀 A full deployment guide: Streamlit Cloud, Render, Neon/Supabase, Vercel, Docker
📊 Case studies: real projects broken down branch by branch
✅ Checklists: copy-paste checklists for the start and end of every project

Stack at a glance

Path	Stack	Best for
Classic	Python, Flask / Streamlit, SQLAlchemy, MySQL / PostgreSQL	Fast portfolio projects, dashboards
Modern full-stack	FastAPI, React, PostgreSQL, Alembic, Docker	Production-style apps with a separate frontend
CLI	Python stdlib, argparse, JSON	Small tools, learning fundamentals

How to use it

Read Part A once, since it changes how you approach projects.
Before every project, walk through Part B in order.
Use Part C as a reference while you work (Git, files, CI).
Use Section 10 when you're ready to ship.
Copy the checklists in Part E into each new project.

📖 Table of Contents

PART A — The Philosophy
- 1. Why You Get Stuck: The Real Problem
- 2. The 3 Phases of Thinking
PART B — The Process (9 Steps)
PART C — The Technical Workflow
PART D — Case Studies
PART E — Checklists & Templates

PART A — The Philosophy

1. Why You Get Stuck: The Real Problem

You say: "I don't know the process well enough yet."

That's the right starting point. Every junior developer feels the same. The difference is that most think the problem is:

❌ "I don't know enough technologies"
❌ "I haven't read enough tutorials"
❌ "I haven't built enough projects"

Wrong. The real problem is:

You haven't internalized the PROCESS. You know Python, Flask, FastAPI, SQL, React — but you don't know the sequence of steps that takes you from blank to finished.

It's like knowing how to use a hammer, nails, and a saw — but not knowing how to build a table. You know the tools, not the method.

What This Guide Will Change

Before:

Start a project → open the editor → start coding → 3 hours later you're stuck → don't know what's missing → give up.

After:

Start a project → think for 30 minutes on paper → write pseudocode → split into modules → write tests first → write code → refactor → deploy.

The difference: You think first, then you write.

2. The 3 Phases of Thinking

Every project goes through 3 phases of thinking:

🧠 Phase 1: Abstraction

"What do I want to do, in plain words?"

You think at a high level. You don't touch code.

Example: "I want a CLI that tracks expenses and shows me totals."

🔧 Phase 2: Decomposition

"What small pieces does it break into?"

You split the big thing into small pieces. Each piece becomes a module or a function.

Example:

Expense Tracker CLI:
├── Add expense
├── View expenses
├── Summary by category
└── Persistence (JSON)

💻 Phase 3: Implementation

"How do I turn this into code?"

You write actual code. The thinking is already done — now you execute.

Example: expense_tracker.py, storage.py, models.py

🔑 Rule: 80% thinking, 20% coding.

Juniors do the opposite: 5% thinking, 95% coding → they rewrite everything 3 times.

PART B — The Process (9 Steps)

3. Understanding & Decomposition

3.1 The 6 Questions Before Writing a Single Line

Write the answers in a NOTES.md before opening the editor.

#	Question	Example (Expense Tracker)
1	What problem am I solving? (1 sentence)	"I want to track expenses and see totals."
2	Who uses it and how?	Me, via terminal
3	Inputs?	`amount: float`, `category: str`, `description: str`
4	Outputs?	Expense list, totals, messages
5	Constraints?	amount > 0, category from a list, persistence
6	Edge cases?	amount="abc", empty list, huge number

💡 For web apps, also answer: What are the API contracts (endpoints, request/response shapes)? Who is authenticated, and what can each user see? Which secrets does the app need?

3.2 Decomposition — The Split

Rule: If you can't describe a sub-problem in 1-2 sentences, it's too big.

Example:

"Track and analyze expenses"
├── 1. Add an expense
│   ├── 1a. Validate amount
│   ├── 1b. Validate category
│   └── 1c. Save
├── 2. View expenses
├── 3. Summary by category
└── 4. Persistence (JSON)

Each leaf → one function or one class.

3.3 When You Know Decomposition Is Done

✅ Every leaf is clear on what it does
✅ Every leaf can be written independently
✅ You can estimate time for each leaf

If not → split more.

3.4 Decomposing a Full-Stack App

For a full-stack project, split first by layer, then by feature:

Full-Stack Expense Tracker
├── Backend (FastAPI + SQLAlchemy + Alembic)
│   ├── Auth (JWT / OAuth)
│   ├── Expenses CRUD & Caching
│   └── Database models & Migrations
└── Frontend (React + Tailwind + Vitest)
    ├── Authentication UI
    ├── Dashboard & Charts
    └── API Integration (Fetch/Axios)

4. Algorithmic Thinking

4.1 Pseudocode

Write the logic in plain English, using fixed keywords:

FUNCTION add_expense(amount, category):
    IF amount <= 0 THEN
        RETURN error
    
    IF category NOT IN valid_list THEN
        RETURN error
    
    CREATE new expense
    APPEND to list
    RETURN confirmation

Why: You think about the logic without worrying about Python syntax. When you write it in Python, the translation is nearly 1-to-1.

4.2 Dry Run — "Executing" on Paper

Take the pseudocode and "run" it by hand:

Input: amount = -5, category = "Food"
Step 1: -5 <= 0 → YES
Step 2: Returns error "Amount must be positive"
✅ Correct.

Rule: Dry run with 3 scenarios:

Happy path: amount=50, category="Food" → ✅ adds
Edge case: amount=0 → ✅ rejects
Invalid: amount="abc" → ✅ rejects

5. Design

5.1 UML Class Diagram

Shows:

Class name
Attributes (data)
Methods (behaviors)

┌─────────────────────┐
│      Expense        │
├─────────────────────┤
│ - amount: float     │
│ - category: str     │
│ - date: date        │
├─────────────────────┤
│ + to_dict() -> dict │
└─────────────────────┘

┌─────────────────────────────┐
│     ExpenseTracker          │
├─────────────────────────────┤
│ - expenses: list[Expense]   │
├─────────────────────────────┤
│ + add_expense(...)          │
│ + get_summary()             │
└─────────────────────────────┘

Questions:

What "things" (nouns) exist? → Classes
What data "belongs" to each thing?
What actions (verbs) does it do?
Do I need a class or a function? (If no state, a function is enough.)

5.2 Modular Architecture

Each file = one responsibility (SRP).

Classic layout (Flask / Streamlit / CLI):

project/
├── src/
│   ├── __init__.py
│   ├── main.py            # Entry point
│   ├── models/            # Domain models
│   ├── services/          # Business logic
│   ├── routes/            # HTTP endpoints
│   ├── storage/           # Persistence
│   └── ui/                # Presentation
├── tests/
├── requirements.txt
├── .env.example
├── .gitignore
└── README.md

Full-stack layout (FastAPI + React):

project/
├── backend/
│   ├── src/
│   │   ├── models/       # SQLAlchemy models
│   │   ├── schemas/      # Pydantic schemas
│   │   ├── services/     # Business logic
│   │   ├── routers/      # FastAPI endpoints
│   │   └── core/         # Config, database, security
│   ├── alembic/          # Database migrations
│   ├── tests/            # Pytest (unit & integration)
│   ├── Dockerfile
│   └── requirements.txt
├── frontend/
│   ├── src/              # React components & hooks
│   ├── tests/            # Vitest / E2E tests
│   └── package.json
├── docker-compose.yml
├── .env.example
├── .gitignore
└── README.md

Key Question: "If I change storage (e.g. JSON → DB), how many files do I touch?"

✅ 1 (storage/) → good design
❌ 5 → tight coupling

5.3 Data & API Design (Web Apps)

Before writing endpoints, write the contract:

Method	Path	Auth	Request	Response
`POST`	`/auth/register`	No	`{email, password}`	`201 {id, email}`
`POST`	`/auth/login`	No	`{email, password}`	`200 {access_token}`
`GET`	`/expenses`	JWT	—	`200 [Expense]`
`POST`	`/expenses`	JWT	`{amount, category}`	`201 Expense`

Also sketch an ERD (tables, primary keys, foreign keys) before creating any models.

6. Implementation

6.1 Rules for Functions

One function, one job (SRP)
Name = verb (add_expense, not expense_stuff)
Type hints always
Docstring on every non-obvious function

def add_expense(
    tracker: "ExpenseTracker",
    amount: float,
    category: str,
    description: str = "",
) -> "Expense":
    """Add a new expense after validating it.
    
    Args:
        tracker: The tracker to add to.
        amount: Amount > 0.
        category: Category from the allowed list.
        description: Optional.
    
    Returns:
        The new Expense.
    
    Raises:
        ValueError: If amount <= 0 or category is unknown.
    """
    if amount <= 0:
        raise ValueError("Amount must be positive")
    ...

6.2 Rules for Classes

Represents a "thing" (not just a bag of functions)
__init__ = initialization only (no heavy logic)
Encapsulation — underscore for private attributes
A class shouldn't know too much about other classes

6.3 When to Use OOP vs Functions

Need a class if...	Need a function if...
It has state (`self.x`)	No state
You want inheritance	No relation to others
You want many objects of the same type	Pure logic

Example:

ExpenseTracker → class (holds the list of expenses)
hash_password(pw) → function (no state)

6.4 Configuration & Secrets

Never hardcode secrets, URLs, or credentials. Read them from environment variables.
Load config in one place (core/config.py) and import it everywhere else.
Use different configs for development, testing, and production.
Initialize external connections lazily (on first use, not at import time) so tests and CI don't crash.

# core/config.py
import os
from dotenv import load_dotenv

load_dotenv()

class Settings:
    DATABASE_URL: str = os.getenv("DATABASE_URL", "sqlite:///./dev.db")
    SECRET_KEY: str = os.getenv("SECRET_KEY", "")
    DEBUG: bool = os.getenv("DEBUG", "false").lower() == "true"

settings = Settings()

6.5 Error Handling

Validate at the boundary. Check user input where it enters the system (routes, CLI args), not deep inside.
Raise specific exceptions (ValueError, custom NotFoundError), never bare Exception.
Never swallow errors silently (except: pass). Log them or re-raise.
In APIs, map exceptions to HTTP status codes (400 bad input, 401 unauthenticated, 403 forbidden, 404 missing, 422 validation, 500 unexpected).

6.6 Database Migrations

Never edit a production schema by hand. Use a migration tool (Alembic for SQLAlchemy).
Every schema change = one migration file, committed to Git.

alembic revision --autogenerate -m "add expenses table"
alembic upgrade head      # apply
alembic downgrade -1      # roll back one step

6.7 Security Basics

✅ Hash passwords with bcrypt or argon2 (never store plain text, never use MD5/SHA1)
✅ Scope every query by the current user (prevents users reading each other's data)
✅ Use parameterized queries / the ORM (prevents SQL injection)
✅ Keep JWT secrets long, random, and in env vars
✅ Restrict CORS to your real frontend origin in production
✅ Rotate any secret that was ever committed, even briefly

7. Testing & Debugging

7.1 Testing — 3 Tests per Function

For every function, write at least 3 tests:

def test_add_expense_happy_path():
    """Normal scenario."""
    tracker = ExpenseTracker()
    tracker.add_expense(50.0, "Food")
    assert len(tracker.expenses) == 1

def test_add_expense_rejects_negative():
    """Edge case: negative."""
    with pytest.raises(ValueError):
        ExpenseTracker().add_expense(-10, "Food")

def test_add_expense_rejects_unknown_category():
    """Invalid input."""
    with pytest.raises(ValueError):
        ExpenseTracker().add_expense(10, "Unknown")

Why 3:

Happy path → the basic works
Edge case → works at the boundaries
Invalid → rejects bad input

7.2 The Testing Pyramid

        /\
       /E2E\        few   — full user flows (slow)
      /------\
     /Integr. \     some  — API + database together
    /----------\
   /   Unit     \   many  — single functions (fast)
  /--------------\

Layer	Tool (backend)	Tool (frontend)	What it checks
Unit	`pytest`	`Vitest`	One function / component
Integration	`pytest` + test DB (SQLite in-memory)	React Testing Library	Layers working together
E2E	`pytest` + `httpx` / Playwright	Playwright / Cypress	Real user flows

Tips:

Use fixtures for repeated setup (database session, test client).
Mock external APIs so tests are fast and don't depend on the network.
Track coverage with pytest --cov=src (aim for meaningful coverage, not 100%).

7.3 Debugging — 5 Steps

Read the traceback — bottom to top
Isolate — comment out pieces
Print/Log intermediate values
Form a hypothesis — "I think it's X" → test it
20-25 minute rule — if stuck, ask for help

7.4 Logging vs Print

print() disappears. logging keeps history:

from loguru import logger

logger.info(f"Added expense: {amount}")
logger.warning(f"Invalid amount: {amount}")
logger.error(f"Failed to save: {e}")

Levels:

debug → dev details
info → normal events
warning → unusual
error → something failed

8. Refactoring

Refactoring = Change the STRUCTURE, not the BEHAVIOR.

⚠️ Do it AFTER the tests are green — tests are your safety net.

Questions:

Code that repeats 2+ times? → Function
Function > 20 lines? → Split it
Magic numbers? → Constants
Bad names? → Rename
PEP 8? → black, ruff

8.1 Automate Quality with Pre-commit

Catch formatting and lint issues before they reach Git:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.6.9
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

pip install pre-commit
pre-commit install

9. Documentation

9.1 Docstrings vs Comments

Docstrings → the "what":

def add_expense(...):
    """Adds a new expense."""

Comments → the "why":

# BAD: states the obvious
i += 1  # increments i

# GOOD: explains why
i = 1  # skip header row

9.2 README Structure

# Project Name
Short description.

## Features
- ...

## Tech Stack
- ...

## Installation
```bash
git clone ...
pip install -r requirements.txt
```

## Usage
```bash
python main.py
```

## Screenshots
![...](...)

## Testing
```bash
pytest tests/ -v
```

## License
MIT

9.3 API Documentation

FastAPI generates interactive docs automatically. Link them in your README:

Swagger UI: https://your-api.onrender.com/docs
ReDoc: https://your-api.onrender.com/redoc

For Flask, document endpoints in the README as a table (method, path, auth, description).

10. Deployment

Deployment is not the last-minute step. It's a skill with its own checklist. Follow the sections below in order.

10.1 Before Deploy — Checklist

Clean venv — does it work from scratch? (rm -rf .venv && python -m venv .venv && pip install -r requirements.txt)
Secrets in environment variables (not hardcoded)
.env in .gitignore
requirements.txt up to date (pip freeze or manually curated)
All tests pass locally and in CI
Error handling for production (no stack traces shown to users)
Debug mode off (DEBUG=false)
README up to date
Code pushed to GitHub (all hosts below deploy from a Git repo)

10.2 Deployment Options

Project Type	Where	Time	Free Tier
Streamlit	Streamlit Community Cloud	5 min	✅
Flask	Render / Railway	10 min	✅ (sleeps when idle)
FastAPI	Render / Railway	10 min	✅ (sleeps when idle)
CLI	GitHub only (or PyPI)	—	✅
Frontend (React/Vite)	Vercel / Netlify / GitHub Pages	5 min	✅
Database (PostgreSQL)	Neon / Supabase	5 min	✅
Full Stack	Vercel + Render + Neon	30 min	✅

⚠️ Free tiers change often. Check each provider's current limits and pricing before you rely on them.

10.3 The Deployment Order for Full-Stack Apps

Always deploy from the bottom up, because each layer needs the one below it:

1. Database   (Neon / Supabase)   → get DATABASE_URL
        │
        ▼
2. Backend    (Render / Railway)  → get API URL
        │
        ▼
3. Frontend   (Vercel / Netlify)  → set VITE_API_URL to the API URL
        │
        ▼
4. Back to backend → set CORS_ORIGINS to the frontend URL

10.4 Step 1 — Deploy the Database (Neon or Supabase)

Create an account at neon.tech or supabase.com.
Create a new project and pick the region closest to your backend.
Copy the connection string.
Make sure it matches your driver and requires SSL:

postgresql+psycopg2://USER:PASSWORD@HOST/DBNAME?sslmode=require

Save it as DATABASE_URL. You'll paste it into your backend host in the next step.
Run migrations against it once (from your machine, with the production URL exported temporarily):

export DATABASE_URL="postgresql+psycopg2://..."
alembic upgrade head

💡 Use connection pooling (Neon's pooled connection string, or Supabase's pooler) for serverless/free-tier setups to avoid exhausting connections.

10.5 Step 2 — Deploy the Backend (Render)

Prepare the repo:

requirements.txt includes your server (gunicorn for Flask, uvicorn[standard] for FastAPI).
Add a health check endpoint:

@app.get("/health")
def health():
    return {"status": "ok"}

On Render:

Go to render.com → New → Web Service → connect your GitHub repo.
Configure:

Setting	FastAPI	Flask
Root Directory	`backend` (if monorepo)	repo root or subfolder
Runtime	Python 3	Python 3
Build Command	`pip install -r requirements.txt`	`pip install -r requirements.txt`
Start Command	`uvicorn src.main:app --host 0.0.0.0 --port $PORT`	`gunicorn "src:create_app()" --bind 0.0.0.0:$PORT`
Health Check Path	`/health`	`/health`

Under Environment, add your variables:

DATABASE_URL=postgresql+psycopg2://...?sslmode=require
SECRET_KEY=<long random string>
DEBUG=false
CORS_ORIGINS=http://localhost:5173
PYTHON_VERSION=3.12.0

Generate a strong secret locally:

python -c "import secrets; print(secrets.token_urlsafe(48))"

Click Create Web Service. Wait for the build, then open https://your-service.onrender.com/docs (FastAPI) or /health.
Copy the service URL. The frontend needs it.

⚠️ Free tier note: free web services sleep after a period of inactivity, so the first request can take 30-60 seconds. Mention this in your README so recruiters aren't surprised.

Running migrations on deploy: if your plan supports a pre-deploy command, set it to alembic upgrade head. Otherwise, chain it into the start command:

alembic upgrade head && uvicorn src.main:app --host 0.0.0.0 --port $PORT

Railway alternative: New Project → Deploy from GitHub repo → add the same environment variables → set the start command. The steps are nearly identical.

10.6 Step 3 — Deploy the Frontend (Vercel)

Go to vercel.com → Add New → Project → import your GitHub repo.
Configure:

Setting	Value
Root Directory	`frontend`
Framework Preset	Vite (or Create React App / Next.js)
Build Command	`npm run build`
Output Directory	`dist` (Vite) or `build` (CRA)

Add the environment variable pointing at your backend:

VITE_API_URL=https://your-service.onrender.com

Click Deploy. Vercel gives you a URL like https://your-app.vercel.app.

In the frontend, read the URL from the environment, never hardcode it:

const API_URL = import.meta.env.VITE_API_URL;
const res = await fetch(`${API_URL}/expenses`);

💡 Vite variables must start with VITE_ to be exposed to the browser. Never put secrets in them; everything in the frontend is public.

10.7 Step 4 — Connect Them (CORS)

The browser blocks requests from your frontend to your backend unless the backend allows that origin. Go back to Render and set:

CORS_ORIGINS=https://your-app.vercel.app

And in FastAPI:

import os
from fastapi.middleware.cors import CORSMiddleware

origins = os.getenv("CORS_ORIGINS", "").split(",")

app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

Redeploy the backend. Never use allow_origins=["*"] together with credentials in production.

10.8 Deploying a Streamlit App

Push the repo to GitHub. The entry file (e.g. app.py) and requirements.txt must be in the repo.
Go to share.streamlit.io → New app.
Select the repository, branch (main), and main file path (app.py).
Open Advanced settings → Secrets and paste your secrets in TOML format:

DATABASE_URL = "postgresql://USER:PASSWORD@HOST/DBNAME?sslmode=require"
API_KEY = "your_api_key_here"

Click Deploy. Your app goes live at https://your-app.streamlit.app.

Read secrets lazily (inside functions, not at import time) so tests and CI don't fail with StreamlitSecretNotFoundError:

import streamlit as st

def get_database_url() -> str:
    return st.secrets["DATABASE_URL"]

10.9 Deploying with Docker (Optional but Recommended)

Docker guarantees "it works on my machine" also means "it works everywhere."

Backend Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["sh", "-c", "uvicorn src.main:app --host 0.0.0.0 --port ${PORT:-8000}"]

docker-compose.yml for local development:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: dev
      POSTGRES_DB: mydb
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data

  backend:
    build: ./backend
    ports: ["8000:8000"]
    env_file: .env
    depends_on: [db]

  frontend:
    build: ./frontend
    ports: ["5173:5173"]
    depends_on: [backend]

volumes:
  pgdata:

Verify locally before deploying:

docker compose up --build

Render and Railway can also deploy straight from your Dockerfile. Choose Docker as the runtime instead of Python.

10.10 After Deploy — Verification Checklist

/health returns 200
Swagger UI (/docs) loads
Register and login work in the production frontend
Data persists after a page refresh (database connected)
No CORS errors in the browser console (F12)
No secrets visible in the frontend source or network tab
Live URLs added to the README
README badge / "Live Demo" link works

10.11 Updating & Rolling Back

Update: merge to main → Render and Vercel redeploy automatically on push.

Rolling back:

Render / Vercel: open the dashboard → Deploys → pick the last good deploy → Rollback / Redeploy.
Database: alembic downgrade -1 (only if the migration is reversible, so test downgrades locally first).
Code: git revert <commit> (creates a new commit; never rewrite main history).

10.12 Deployment Troubleshooting

Symptom	Likely Cause	Fix
Build fails: `ModuleNotFoundError`	Dependency missing from `requirements.txt`	Add it and push
App crashes: `KeyError: DATABASE_URL`	Env var not set on the host	Add it in the dashboard, redeploy
`Address already in use` / port error	Not binding to `$PORT`	Use `--port $PORT --host 0.0.0.0`
`SSL connection required`	Missing `sslmode=require`	Append it to `DATABASE_URL`
Frontend shows CORS error	Backend doesn't allow the frontend origin	Set `CORS_ORIGINS`, redeploy backend
Frontend calls `localhost` in production	Hardcoded API URL	Use `VITE_API_URL`, redeploy frontend
`relation "x" does not exist`	Migrations not applied	Run `alembic upgrade head` on production
First request takes ~1 minute	Free-tier cold start	Expected; note it in the README
Works locally, fails on the host	Python version mismatch	Pin `PYTHON_VERSION`
Streamlit `StreamlitSecretNotFoundError`	Secrets read at import time	Read lazily; add secrets in app settings

11. Practice

The 3 Levels

Level 1 — Daily (15-20 min)

Exercism.org
CodeWars
Advent of Code

Level 2 — Weekly (small projects)

Rebuild projects without looking at previous solutions
Todo → Expense → Quiz → Inventory

Level 3 — Core projects

Your 4 phase projects
Full portfolio projects

Habits

Re-read old code after 1 month
Explain your code to someone (rubber duck)
Read others' code on GitHub
Keep a postmortem — "what would I change?"
Timed practice — 30 min/kata

🌱 The truth: Comfort comes around the 15th-20th project, not the 3rd. Don't get discouraged.

PART C — The Technical Workflow

12. Git Workflow: Branches & Commits

12.1 The Branch Rule

Every feature = new branch. NEVER commit directly to main.

12.2 Branch Naming

Prefix	When	Example
`chore/`	Maintenance	`chore/initial-setup`
`feat/`	New feature	`feat/storage-layer`
`fix/`	Bug fix	`fix/config-ssl`
`test/`	Tests	`test/unit-tests`
`docs/`	Documentation	`docs/readme`
`refactor/`	Refactor	`refactor/service-layer`

12.3 The Flow (for every feature)

# 1. Sync
git checkout main
git pull

# 2. New branch
git checkout -b feat/feature-name

# 3. Work + commits (atomic!)
git add <file>
git commit -m "feat(scope): description"

# 4. Merge back
git checkout main
git merge --no-ff feat/feature-name -m "merge: description"
git branch -d feat/feature-name

# 5. Push
git push

💡 Team / PR workflow: instead of step 4, push the branch and open a Pull Request on GitHub. CI runs, you review, then merge via the GitHub UI.

12.4 Why `--no-ff`

Without: Flat history → you lose the structure. With: Creates a merge commit → branches are visible in git log --graph.

12.5 Commit Messages — Conventional Commits

Format:

<type>(<scope>): <short description>

Types:

feat — new feature
fix — bug fix
docs — documentation
test — tests
refactor — refactor
chore — maintenance
style — formatting

Examples:

chore: add .gitignore for Python
feat(config): add env loader
feat(storage): add JSON file storage
test(storage): add unit tests
docs: rewrite README with screenshots
fix(config): correct indentation
merge: add storage layer

⛔ BAD:

"fixed stuff"
"wip"
"asdf"
"update"

✅ GOOD:

"feat(auth): add bcrypt password hashing"
"fix(api): handle 404 from GitHub API"
"test(tasks): add edge cases for empty input"

12.6 Atomic Commits — The Golden Rule

One commit = one logical change.

If you revert, you lose one thing — not five.

Example:

❌ feat: add everything → 500 lines, 10 concerns
✅ feat(storage): add JSON storage
✅ test(storage): add tests for JSON storage
✅ docs: document storage format

12.7 The Typical Branch Sequence for One Project

#	Branch	What You Do	Commits
1	`chore/initial-setup`	`.gitignore`, `.env.example`, `README.md`, folders	4-5
2	`chore/requirements-and-config`	`requirements.txt`, `config.py`	2-3
3	`feat/database-models`	SQLAlchemy models	2
4	`feat/database-connection`	Engine + session	1-2
5	`feat/services-layer`	Business logic	2-3
6	`feat/routes-and-templates`	HTTP + UI	2-3
7	`test/unit-tests`	Tests	2-3
8	`chore/ci`	GitHub Actions	1
9	`docs/final-readme`	README + screenshots	1-2
10	`chore/add-license`	LICENSE	1

Total: ~20-25 commits, 10 branches.

12.8 When to Commit

Every time you:

✅ Complete one logical change
✅ Tests pass
✅ Can describe the change in one line

Not:

❌ At end of day (batching 10 changes)
❌ When "it works" (without tests)
❌ Before merge (if no tests)

12.9 Handy Git Commands

git status                      # what changed?
git log --oneline --graph       # visual history
git diff --staged               # review before committing
git restore --staged <file>     # unstage a file
git stash / git stash pop       # park work temporarily
git revert <commit>             # safely undo a commit (new commit)

13. The Essential Files of Every Project

13.1 `.gitignore` — First file!

Create it BEFORE anything else.

# Virtual Environment
.venv/
venv/
env/

# Secrets
.env
*.env

# Python cache
__pycache__/
*.pyc
*.pyo
.pytest_cache/
.ruff_cache/
.coverage
htmlcov/
*.egg-info/
build/
dist/

# Node (full-stack projects)
node_modules/

# OS junk
.DS_Store
Thumbs.db

# IDE
.vscode/
.idea/

# App data
data/
*.log
*.db
tasks.json

Why first: Prevents accidental commits.

⚠️ *.env also ignores .env.example-style names only if they end in .env. .env.example is not matched, so it stays tracked. Verify with git status.

13.2 `.env.example` — Template

What others need, WITHOUT the real secrets.

# Flask
SECRET_KEY=your_secret_key_here
FLASK_DEBUG=True

# Database
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=your_password_here
DB_NAME=myapp
DATABASE_URL=mysql+pymysql://root:password@localhost:3306/myapp

# Web (full-stack)
CORS_ORIGINS=http://localhost:5173
VITE_API_URL=http://localhost:8000

Rule: Placeholders always. NEVER real credentials.

13.3 `.env` — Real Secrets

NOT pushed to Git (it's in .gitignore).

SECRET_KEY=<generated, long and random>
DB_PASSWORD=<your real password>
DATABASE_URL=mysql+pymysql://root:<your real password>@localhost:3306/myapp

Creation: Copy-Item .env.example .env (Windows PowerShell) or cp .env.example .env (macOS/Linux) → fill in.

13.4 `requirements.txt` — Runtime

Flask>=3.0.0
SQLAlchemy>=2.0.0
python-dotenv>=1.0.0

13.5 `requirements-dev.txt` — Development

-r requirements.txt
pytest>=7.4.0
pytest-mock>=3.11.0
pytest-cov>=5.0.0
black>=24.0.0
ruff>=0.3.0

Why separate: Users who pip install don't need pytest.

13.6 `.venv/` — Virtual Environment

Creation:

python -m venv .venv
.venv\Scripts\Activate.ps1   # Windows
source .venv/bin/activate    # macOS/Linux

Why: Isolation — each project has its own packages.

⚠️ NEVER in Git (it's in .gitignore).

13.7 `README.md` — The Showcase

Structure:

Title + tagline
Badges (Python, License, Tests, CI)
Live Demo link
Screenshots
About (why you built it)
Features (what it does)
Tech Stack (what you use)
Installation (how to run it)
Usage (examples)
Testing (how to run tests)
Project Structure
Deployment (where it runs, env vars needed)
License

If you count 30 seconds: The recruiter reads the README before the code.

13.8 `LICENSE` — Legal

MIT License (most common for portfolios):

MIT License

Copyright (c) 2026 Your Name

Permission is hereby granted, free of charge, ...

13.9 `pyproject.toml` — Modern Packaging

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "task-cli"
version = "0.1.0"
description = "A simple CLI task manager"
requires-python = ">=3.10"
license = { text = "MIT" }

[project.scripts]
task = "task_cli.main:main"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 88

Why: pip install -e . → command task available everywhere.

13.10 Optional: `Dockerfile` + `docker-compose.yml`

See Section 10.9 for complete, ready-to-use files.

13.11 Optional: `Makefile` — Shortcuts

install:
	pip install -r requirements-dev.txt

test:
	pytest tests/ -v --cov=src

lint:
	ruff check . && black --check .

format:
	black . && ruff check --fix .

run:
	uvicorn src.main:app --reload

Now make test replaces a long command you'd otherwise forget.

14. CI/CD Pipeline

14.1 What CI Is

Continuous Integration = automatic tests on every push.

If they pass → ✅ If they fail → ❌ notification

Continuous Deployment (CD) = passing code is deployed automatically. Render and Vercel do this for you when connected to GitHub: push to main → auto-deploy.

14.2 GitHub Actions — `tests.yml`

Location: .github/workflows/tests.yml

name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    name: Python ${{ matrix.python-version }}
    runs-on: ubuntu-latest
    
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest
      
      - name: Lint
        run: |
          pip install ruff
          ruff check .
      
      - name: Run tests
        run: pytest tests/ -v

14.3 Testing Against a Real Database in CI (Optional)

    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    env:
      DATABASE_URL: postgresql+psycopg2://postgres:test@localhost:5432/testdb

Run alembic upgrade head before pytest to test migrations too.

14.4 Frontend CI (Vitest)

  frontend:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: frontend
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
          cache-dependency-path: frontend/package-lock.json
      - run: npm ci
      - run: npm test -- --run
      - run: npm run build

14.5 Why Matrix 3 Versions

Shows it works on multiple Python versions → production-ready.

14.6 Badge in README

[![Tests](https://github.com/USER/REPO/actions/workflows/tests.yml/badge.svg)](https://github.com/USER/REPO/actions)

Will show:

🟢 passing → all good
🔴 failing → something broke

14.7 When to Add CI

After:

✅ Tests (branch test/unit-tests)
✅ Before deploy

Before:

✅ Final README
✅ Push to GitHub (if not done yet)

14.8 Troubleshooting — Common Issues

Error	Cause	Fix
`StreamlitSecretNotFoundError`	`st.secrets` at import	Lazy init
`ModuleNotFoundError: src`	Missing `pyproject.toml`	Add config
`pytest: not found`	Missing from requirements	`pip install pytest`
`KeyError: DATABASE_URL`	Tests want prod config	`TestConfig`

Lesson: CI reveals design flaws before deploy.

15. Screenshots & Presentation

15.1 When to Take Screenshots

AFTER the project works live and has no bugs.

Before you:

Finalize README
Build portfolio website
Post on LinkedIn

15.2 How Many and Where

Project Type	How Many	What
Web app	3-4	Home + 2-3 features
CLI tool	2-3	`--help` + commands + output
API	1-2	Swagger UI + 1 request

15.3 How (Windows)

Method	How
Win + Shift + S	Snipping tool
Win + PrtScn	Full screen → auto-save
Alt + PrtScn	Active window only

15.4 Where to Store Them

project/
├── docs/
│   └── screenshots/
│       ├── home.png
│       ├── feature1.png
│       └── feature2.png

In README:

## 📸 Screenshots

| Home | Feature 1 |
|---|---|
| ![Home](docs/screenshots/home.png) | ![Feature](docs/screenshots/feature1.png) |

⚠️ On GitHub: Relative paths (docs/...), NOT absolute.

15.5 Best Practices

✅ Zoom before screenshot (Ctrl +)
✅ Dark mode (modern look)
✅ Full screen (F11)
✅ Compress before commit (tinypng.com)
❌ No image over 500KB

15.6 Show the Live Link First

Put the live demo URL and one screenshot at the very top of the README, right under the title. Visitors decide in seconds whether to keep reading.

PART D — Case Studies

📊 Overview

#	Project	Stack	Branches	Commits	Tests	CI
1	Weather Dashboard	Streamlit + API	10	~25	13	✅
2	To-Do App	Streamlit + PostgreSQL	10	~30	38	✅
3	Flight Search	Flask + MySQL	10	~25	14	✅
4	Task CLI	Python CLI	7	~20	33	✅
5	Expense Tracker (full-stack)	FastAPI + React + PostgreSQL	12+	~35	pytest + Vitest	✅

🌤️ Weather Dashboard

Branches:

chore/initial-setup — .gitignore, .env.example, README.md, folders
chore/requirements-and-config — requirements.txt, config.py
feat/weather-fetcher — API client
feat/data-visualizer — Plotly charts
feat/history-tracker — JSON persistence
feat/streamlit-ui — UI
test/unit-tests — pytest
docs/final-readme — README + screenshots
chore/add-license — MIT
chore/ci — GitHub Actions

Skills: REST API, OOP, caching, testing, deployment

Deploy: Streamlit Community Cloud (Section 10.8)

✅ To-Do App

Branches:

chore/initial-setup — setup
chore/requirements-and-config — deps + config
feat/database-models — User, Task
feat/database-connection — Engine + session
feat/auth-service — bcrypt + register/login
feat/task-service — CRUD scoped per user
feat/streamlit-auth-ui — Login/Register pages
feat/streamlit-task-ui — Task list
test/unit-tests — 38 tests
docs/final-readme — README + screenshots
chore/add-license — MIT
chore/ci — GitHub Actions

Skills: Auth, PostgreSQL, per-user isolation, bcrypt, SQLAlchemy 2.0

Deploy: Streamlit Community Cloud + Neon / Supabase (Sections 10.4 and 10.8)

✈️ Flight Search

Branches:

chore/initial-setup — setup
chore/requirements-and-config — deps + config
feat/database-models — 6 models + 2 associations
feat/database-connection — lazy init engine
feat/services-layer — business logic
feat/routes-and-templates — Flask blueprints + Jinja2
test/unit-tests — pytest + SQLite in-memory
docs/final-readme — README + screenshots
chore/add-license — MIT
chore/ci — GitHub Actions

Skills: Complex SQL, multi-table JOINs, admin auth, Flask blueprints

Deploy: Render + managed database (Section 10.5)

📝 Task CLI

Branches:

chore/initial-setup — setup
chore/requirements-and-config — pytest only
feat/storage — JSON persistence
feat/task-logic — business logic
feat/display — output formatting
feat/cli — argparse entry point
docs/final-readme — README + screenshots
chore/ci — GitHub Actions

Skills: CLI, argparse, stdlib-only, atomic JSON writes

Deploy: GitHub only (optionally publish to PyPI)

💸 Expense Tracker (Full-Stack)

Stack: FastAPI, SQLAlchemy, Alembic, PostgreSQL, React, Tailwind, Docker, pytest, Vitest

Branches:

chore/initial-setup — monorepo folders, .gitignore, .env.example
chore/docker-compose — Postgres + backend + frontend containers
feat/database-models — User, Expense
chore/alembic-migrations — first migration
feat/auth — JWT register/login, bcrypt
feat/expenses-crud — endpoints scoped per user
feat/caching — connection pooling + caching strategy
feat/frontend-auth-ui — login/register pages
feat/frontend-dashboard — charts + expense list
test/backend-tests — pytest with DB fixtures
test/frontend-tests — Vitest + Testing Library
chore/ci — GitHub Actions (backend + frontend jobs)
docs/final-readme — architecture, screenshots, live links
chore/deploy — Neon + Render + Vercel

Skills: JWT/OAuth, migrations, Docker, full-stack integration, CORS, CI/CD, three-tier deployment

Deploy: Neon (DB) → Render (API) → Vercel (UI), exactly the order in Section 10.3

PART E — Checklists & Templates

16. Pre-Project Checklist

Before writing code:

I answered the 6 questions (NOTES.md)
I split the problem into sub-problems
I wrote pseudocode
I did a dry run with 3 scenarios
I designed UML/ERD (and API contracts for web apps)
I decided on the tech stack
I estimated time
I created the repo with .gitignore as the first commit

17. Post-Project Checklist

Before deploy:

18. The Mental Model — Summary

       IDEA
        │
        ▼
  ┌──────────────────┐
  │ 1. UNDERSTANDING │  ← 30 min of thinking
  │    (6 questions) │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 2. DECOMPOSITION │  ← Split into small pieces
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 3. PSEUDOCODE    │  ← Logic without syntax
  │    + DRY RUN     │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 4. DESIGN        │  ← UML, ERD, API contracts
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 5. IMPLEMENT     │  ← Write code
  │  (branch-based)  │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 6. TEST          │  ← 3 tests/function
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 7. REFACTOR      │  ← Clean up
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 8. DOCUMENT      │  ← README + docstrings
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 9. CI/CD         │  ← GitHub Actions
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 10. DEPLOY       │  ← DB → API → UI → CORS
  └────────┬─────────┘
           │
           ▼
      PORTFOLIO

19. The Golden Rule

Think 80%, write 20%.

One commit = one change.

One branch = one feature.

Tests first, code after.

Never commit secrets.

Deploy from the bottom up: database, backend, frontend.

The README is 50% of the project.

20. The Most Important Thing

Comfort comes from repetition, not from reading.

1st project: You think through every step explicitly
5th project: You start remembering the sequence
10th project: Decomposition becomes automatic
20th project: It's second nature

You're at #4. You're not a beginner — you're mid-way.

🤝 Contributing

Found a typo, a better tool, or a missing step? Open an issue or a pull request.

📄 License

MIT License — see LICENSE for details.

Built with ☕ + 🧠 — next time you start a project, read this file BEFORE touching the editor. 🚀

Latest commit

History

Folders and files

Repository files navigation

🧠 Software Engineering Playbook

📌 About This Playbook

📖 Table of Contents

PART A — The Philosophy

1. Why You Get Stuck: The Real Problem

What This Guide Will Change

2. The 3 Phases of Thinking

🧠 Phase 1: Abstraction

🔧 Phase 2: Decomposition

💻 Phase 3: Implementation

PART B — The Process (9 Steps)

3. Understanding & Decomposition

3.1 The 6 Questions Before Writing a Single Line

3.2 Decomposition — The Split

3.3 When You Know Decomposition Is Done

3.4 Decomposing a Full-Stack App

4. Algorithmic Thinking

4.1 Pseudocode

4.2 Dry Run — "Executing" on Paper

5. Design

5.1 UML Class Diagram

5.2 Modular Architecture

5.3 Data & API Design (Web Apps)

6. Implementation

6.1 Rules for Functions

6.2 Rules for Classes

6.3 When to Use OOP vs Functions

6.4 Configuration & Secrets

6.5 Error Handling

6.6 Database Migrations

6.7 Security Basics

7. Testing & Debugging

7.1 Testing — 3 Tests per Function

7.2 The Testing Pyramid

7.3 Debugging — 5 Steps

7.4 Logging vs Print

8. Refactoring

8.1 Automate Quality with Pre-commit

9. Documentation

9.1 Docstrings vs Comments

9.2 README Structure

9.3 API Documentation

10. Deployment

10.1 Before Deploy — Checklist

10.2 Deployment Options

10.3 The Deployment Order for Full-Stack Apps

10.4 Step 1 — Deploy the Database (Neon or Supabase)

10.5 Step 2 — Deploy the Backend (Render)

10.6 Step 3 — Deploy the Frontend (Vercel)

10.7 Step 4 — Connect Them (CORS)

10.8 Deploying a Streamlit App

10.9 Deploying with Docker (Optional but Recommended)

10.10 After Deploy — Verification Checklist

10.11 Updating & Rolling Back

10.12 Deployment Troubleshooting

11. Practice

The 3 Levels

Habits

PART C — The Technical Workflow

12. Git Workflow: Branches & Commits

12.1 The Branch Rule

12.2 Branch Naming

12.3 The Flow (for every feature)

12.4 Why --no-ff

12.5 Commit Messages — Conventional Commits

12.6 Atomic Commits — The Golden Rule

12.7 The Typical Branch Sequence for One Project

12.8 When to Commit

12.9 Handy Git Commands

13. The Essential Files of Every Project

13.1 .gitignore — First file!

13.2 .env.example — Template

13.3 .env — Real Secrets

13.4 requirements.txt — Runtime

13.5 requirements-dev.txt — Development

13.6 .venv/ — Virtual Environment

12.4 Why `--no-ff`

13.1 `.gitignore` — First file!

13.2 `.env.example` — Template

13.3 `.env` — Real Secrets

13.4 `requirements.txt` — Runtime

13.5 `requirements-dev.txt` — Development

13.6 `.venv/` — Virtual Environment

13.7 `README.md` — The Showcase

13.8 `LICENSE` — Legal

13.9 `pyproject.toml` — Modern Packaging

13.10 Optional: `Dockerfile` + `docker-compose.yml`

13.11 Optional: `Makefile` — Shortcuts

14.2 GitHub Actions — `tests.yml`