Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
34dce42
Update integration-tests.yml
wiIliu Mar 12, 2026
54bd4b3
Update docker_build.yml
wiIliu Mar 13, 2026
5c56e89
Merge pull request #23 from wiIliu/wiIliu-patch-1
wiIliu Mar 13, 2026
c47b590
Update pylint.yml
wiIliu Mar 13, 2026
4d7671c
Merge pull request #24 from wiIliu/wiIliu-patch-1
wiIliu Mar 13, 2026
49a3635
Update docker_build.yml
wiIliu Mar 13, 2026
d8bacde
Update README.md
wiIliu Mar 13, 2026
17fb8df
Update README.md
wiIliu Mar 13, 2026
af6a9de
Merge pull request #27 from wiIliu/wiIliu-update-docker-action
wiIliu Mar 13, 2026
53942db
Update README.md project structure
wiIliu Mar 14, 2026
ff8da48
Create LICENSE
wiIliu Mar 14, 2026
0861b80
Update README.md
wiIliu Mar 14, 2026
30f01da
Update and rename integration-tests.yml to tests.yml
wiIliu Mar 14, 2026
ba18cff
Update README.md
wiIliu Mar 14, 2026
3c9ee47
Update README.md
wiIliu Mar 14, 2026
5ca31cd
Update README.md
wiIliu Mar 14, 2026
6e9ed1e
Merge pull request #28 from wiIliu/main
wiIliu Mar 14, 2026
e925e47
Update README.md
wiIliu Mar 14, 2026
c585677
Update README.md
wiIliu Mar 14, 2026
bd051dd
Update README.md
wiIliu Mar 14, 2026
e5e3689
Update README.md
wiIliu Mar 14, 2026
19cdd2a
Update README.md
wiIliu Mar 14, 2026
a1923dd
Merge pull request #29 from wiIliu/wiIliu-update-documentation
wiIliu Mar 14, 2026
938717d
Create .pylintrc for linting
wiIliu Mar 14, 2026
ef322de
Update pylint.yml action
wiIliu Mar 14, 2026
46d25ca
Update tests.yml
wiIliu Mar 14, 2026
90247aa
Potential fix for code scanning alert no. 3: Workflow does not contai…
wiIliu Mar 14, 2026
c778923
Merge pull request #30 from wiIliu/wiIliu-patch-1
wiIliu Mar 14, 2026
c386279
fix for code scanning alert no. 1: Workflow does not contain permissions
wiIliu Mar 14, 2026
105b007
Merge pull request #31 from wiIliu/main
wiIliu Mar 14, 2026
2e448a7
Update docker_build.yml
wiIliu Mar 14, 2026
c55633b
Update README.md overview
wiIliu Mar 15, 2026
e9e4ec6
Merge pull request #32 from wiIliu/wiIliu-update-documentation
wiIliu Mar 15, 2026
a633eb4
Merge pull request #33 from wiIliu/dev
wiIliu Mar 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 18 additions & 10 deletions .github/workflows/docker_build.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,27 @@
name: Build and push Docker image
name: Docker

on:
push:
branches:
- main
# pull_request:

jobs:
# test:
# runs-on: ubuntu-latest

# steps:
# - uses: actions/checkout@v4

# - name: Run tests
# run: |
# docker compose --env-file .env.test --profile test up --abort-on-container-exit --exit-code-from orders_service_test

build_and_push:
runs-on: ubuntu-latest
permissions:
contents: read
# needs: test

steps:
- uses: actions/checkout@v4
Expand All @@ -20,18 +34,12 @@ jobs:

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# - name: Set lowercase repo name
# run: |
# # echo "REPO=${GITHUB_REPOSITORY@L}" >> "${GITHUB_ENV}"
# echo "REPO=$(echo "${GITHUB_REPOSITORY##*/}" | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_ENV"
# ${{ vars.DOCKER_USERNAME }}/${{ env.REPO }}/orders-service:${{ github.sha }}



- name: Build and push
uses: docker/build-push-action@v6
with:
context: ./orders_serivice
context: ./orders_service
target: prod
# context: .
push: true
tags: |
Expand Down
62 changes: 42 additions & 20 deletions .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,51 @@ on:
pull_request:
branches:
- main
push:
branches:
- dev

jobs:
build:
pylint:
name: pylint
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
strategy:
matrix:
python-version: ["3.11"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Analysing the code with pylint
run: |
pylint $(git ls-files '*.py') --exit-zero \
--reports=y \
--output-format=junit-xml:pylint.xml
- uses: actions/upload-artifact@v6
with:
name: pylint-report
path: pylint.xml
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

- uses: dciborow/action-pylint@0.1.0
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
glob_pattern: "**/*.py"
pylint_rc: .pylintrc
reporter: github-pr-review
level: warning

# - name: Generate pylint JUnit report
# run: |
# FILES=$(git ls-files '*.py')
# if [ -n "$FILES" ]; then
# pylint $FILES --output-format=junit-xml:pylint.xml
# fi

# - name: Upload artifact
# uses: actions/upload-artifact@v6
# with:
# name: pylint-report
# path: pylint.xml
18 changes: 18 additions & 0 deletions .pylintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[MASTER]
ignore=.venv,venv,migrations,alembic
jobs=0

[MESSAGES CONTROL]
# Disable "missing module docstring" message (C0114)
# Disable "missing class docstring" message (C0115)
# Disable "missing function docstring" message (C0116)
# Disable "too many public methods" message (R0904).
disable=C0114,C0115,C0116,R0904

[BASIC]
# variable names to always be accepted
good-names=i,j,k,e,DB,db

[FORMAT]
# Maximum number of characters on a single line.
max-line-length=120
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Willow Connelly

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
229 changes: 227 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,227 @@
# python-CICD-pipeline
Two Python microservices (Order + Analytics) that communicate via REST integrated with a postgresql relational database. Each service is Dockerized, unit-tested, and has a CI pipeline that runs tests and security scans.
# ♾️ python CI/CD pipeline
![Python](https://img.shields.io/badge/python-3.11-blue)
<a href="https://github.com/wiIliu/python-CICD-pipeline/actions"><img alt="Actions Status" src="https://github.com/wiIliu/python-CICD-pipeline/actions/workflows/tests.yml/badge.svg"></a>
[![Docker image](https://github.com/wiIliu/python-CICD-pipeline/actions/workflows/docker_build.yml/badge.svg)](https://github.com/wiIliu/python-CICD-pipeline/actions/workflows/docker_build.yml)
![Pylint](https://github.com/wiIliu/python-CICD-pipeline/actions/workflows/pylint.yml/badge.svg)
<a href="https://github.com/wiIliu/python-CICD-pipeline/main/LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-purple"></a>
<!-- coverage badge to add -->
![coverage](https://img.shields.io/badge/coverage-pending-lightgrey)


Two Python microservices (Order + Analytics) that communicate via REST integrated with a PostgreSQL relational database. Each service is Dockerized, unit-tested, and has a CI pipeline that runs tests and security scans.

---

## Table of Contents

1. [Overview](#overview)
2. [Project Structure](#project-structure)
3. [Installation](#installation)
- [Setting up Environment](#setting-up-environment)
- [Installing Dependencies](#installing-dependencies)
4. [Usage](#usage)
5. [Contributing](#contributing)
6. [License](#license)
7. [References](#references)

---
<a id="overview"></a>
## 🔎 Overview

This repository contains two containerized RESTful backend microservices—an Orders Service and an Analytics Service—built using FastAPI and Docker with PostgreSQL integration.
The project also includes a CI/CD pipeline implemented with GitHub Actions that automatically runs tests, builds Docker images, and performs security analysis.

Project Components:
- Orders Microservice
- Analytics Microservice
- PostgreSQL Database
- Alembic Migrations
- Multi-stage Docker Builds
- Docker Compose Orchestration
- GitHub Actions for CI

The **Orders Microservice** implements a basic RESTful API (FastAPI) that performs CRUD operations against an `orders` database. (_Future work will include adding user authentication for additional security_)

The **Analytics Microservice** will communicate via API calls with the orders service to retrieve order data and provide calculated statistical metrics. Some of these metrics include total orders, revenue statistics, average order value, and order distribution (_Future work will include creating a basic GUI to display these results in an interactive environment_)

The services and database are coordinated using Docker Compose, allowing the entire system to be started with a single command.

### CI Pipeline
The repository includes a CI pipeline built using **GitHub Actions** that run automatically to enforce validation, Docker image builds, and code quality.

The pipeline performs the following:
- Tests - runs unit and integration tests using Pytest on every push and PR.
- Pylint - Lints the code using Pylint on every push and PR.
- Docker Build - Builds and uploads the PROD docker image to DockerHub on every PR to `main`
- CodeQL - Uses built-in GitHub CodeQL for static security analysis

### Tech Stack
- FastAPI
- SQLAlchemy
- Pydantic
- Alembic
- PostgreSQL
- Docker
- GitHub Actions
- Pytest

### Learning Goals
This project was constructed in order to gain hands-on experience with building a CI/CD pipeline from the ground-up. It was used as a method to gain experience in connecting microservices together to build scalable architecture.

#### ✍️ Author

LinkedIn: [Willow Connelly](https://www.linkedin.com/in/willow-connelly-28b197263/)<br>
GitHub: [WiIliu](https://github.com/wiIliu)

---
<a id="project-structure"></a>
## 🏗️ Project Structure

#### General project structure per service
```
.
{name}_service/ # the service folder
├──── alembic/ # holds all database migrations
├──── app/ # code for api and supporting elements
| ├─── main.py
| ├──── api/
| | ├──── v1/
| | | ├─── {service_routes}.py
| | | ├─── health.py # healthcheck routes
| ├──── core/ # configuration and database settings
| ├──── db_logic/
| | | ├─── crud.py # service table crud logic
| ├──── dependencies/
| ├──── models/
| ├──── schemas/
├──── tests/
| ├──── factories/
| ├──── integration_tests/
| ├──── unit_tests/
| | ├──── crud_tests/
| | ├──── schema_tests/
# misc
├── alembic.ini
├── Dockerfile
├── docker-entrypoint.sh
└── requirements.txt / requirements-dev.txt
```

---
<a id="installation"></a>
## ⬇️ Installation

### Setting up Environment

1. **Create** a new environment (example name: `myproject`):
```bash
python -m venv myproject
```
2. **Activate** it:

Windows
```bash
myproject\Scripts\activate
```

MacOS / Linux
```sh
source myproject/bin/activate
```

### Installing Dependencies

Within your **activated** environment:

1. Update pip
```bash
python -m pip install --upgrade pip
```

2. Install dependencies via requirements file

Prod Requirements file
```bash
pip install -r requirements.txt
```

Dev Requirements file
```bash
pip install -r requirements-dev.txt
```

*(Make sure you are in the new python environment so that packages are installed there.)*

**Typical Requirements** (already in `requirements.txt`):
- Python 3.11 (or similar)
- FastAPI 0.128
- SQLAlchemy 2.0
- Pydantic 2.12
- Alembic 1.18

---
<a id="usage"></a>
## 🚀 Usage

#### Run services using Docker
In project root run:
```sh
docker compose up --build
```
This starts:
- Orders Service
- Analytics Service
- PostgreSQL database

> If developing locally, a `test` profile can be used to run the test containers.

Once running, each Service can be accessed locally via the generated links:

Order API: http://localhost:8000 <br>
Analytics API: http://localhost:8001

Interactive Swagger API docs:

Order API Docs: http://localhost:8000/docs<br>
Analytics API Docs: http://localhost:8001/docs

---
<a id="contributing"></a>
## 🤝 Contributing

1. **Fork** this repository.
2. **Create** a new branch for your feature/fix:
```bash
git checkout -b feature-my-improvement
```
3. **Commit** your changes and push to your fork:
```bash
git commit -m "Add my new feature"
git push origin feature-my-improvement
```
4. **Open a Pull Request** into the `dev` branch.

_NOTE:_ The `main` branch is protected and only updated through
approved pull requests from the `dev` branch.<br>

After review and testing, changes will be merged into `main` during the next release.

We welcome suggestions, bug reports, and community contributions!

---
<a id="license"></a>
## 📜 License

This project is licensed under the [MIT License](LICENSE). You’re free to use, modify, and distribute the code as allowed by that license.

---
<a id="references"></a>
## 📋 References


---

_Thank you for visiting **python-CICD-pipeline**! If you have any questions or issues, feel free to open an [issue](../../issues) or reach out._


---
Loading