Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
c999a64
Why: Add new env variables to build. Values are nonsense.
set-element Oct 4, 2024
25ded01
Why: Change commands from docker to podman. Simple find and replace.
set-element Oct 4, 2024
71f5e81
Why: Add env variables for the various container functions. These are…
set-element Oct 4, 2024
2e5b1fd
Why: Add some env variables to compose.override.production.yml as wel…
set-element Oct 4, 2024
06e92fd
Why: Modify compose.yml in ordr to try to get the boot up order corre…
set-element Oct 4, 2024
19bcbe9
Why: This installs some basic troubleshooting tools on the django ins…
set-element Oct 4, 2024
007cba3
Why: Force ssl to be disabled. This can be undone, but the risk is mi…
set-element Oct 4, 2024
e25126c
Why: Add additional testing to ensure ssl, if unwanted, does not find…
set-element Oct 4, 2024
d616efc
Why: This class has not been created yet, so comment out to remove ex…
set-element Oct 4, 2024
035f9a9
Why: Move production service into a known state
set-element Oct 4, 2024
5a56708
Why: Original docker makefile
set-element Oct 4, 2024
31e927e
Why: Modre documentation and scripts for install
set-element Oct 5, 2024
fafcdba
Why: Insert .envs/* files that have brain dead values to get things g…
set-element Oct 6, 2024
471f8d5
Why: Add django-anymail to the base build since transition to product…
set-element Oct 10, 2024
8e627cc
Why: Add a toy nginx.conf file. This is a toy - just useful enough to…
set-element Oct 10, 2024
30258c6
Why: Fix the mount so that a file is not a directory that is supposed…
set-element Oct 10, 2024
c0a98b6
Why: More notes on the fly
set-element Oct 10, 2024
a9350ff
Merge branch 'esnet-security:main' into Podman-0.1
set-element Oct 10, 2024
b61dae1
Why: Add more information about the postgres error and how to remove it
set-element Oct 17, 2024
8fde54d
Why: Reorder the startup to take into consideration dependencies betw…
set-element Oct 17, 2024
6ebe6af
Why: Install empty env files
set-element Nov 5, 2024
e100537
Why: Add more documentation
set-element Nov 5, 2024
4263499
feat(portable-certs): test variables for .enf file
set-element Nov 9, 2024
0a0884f
fix(hostname): grab the hostname from the environment first
samoehlert Oct 17, 2024
67751d3
ci: try running new tests since local is broken
crankynetman Oct 8, 2024
26abc9a
chore: update gitignore for more vscodery
crankynetman Oct 10, 2024
d8ca90d
tests: add negative tests for invalid ASNs
crankynetman Oct 16, 2024
81b5b3c
style(whitespace): remove extraneous whitespace
samoehlert Oct 16, 2024
14f0a3e
ci(postgres): enable the containerized postgres container in CI
samoehlert Oct 17, 2024
6427425
test: add debugger
crankynetman Oct 16, 2024
5497db9
test: add translator debugger to local compose
crankynetman Oct 16, 2024
b411e2e
feat: debug ALL the things!
crankynetman Oct 22, 2024
dd97d6e
chore: docs, stahp
crankynetman Oct 22, 2024
8421a2f
docs: add debugger docs for VSCode and PyCharm
crankynetman Oct 23, 2024
0e7ffe7
refactor: move from fatal error to warning log
crankynetman Oct 24, 2024
53cb586
ci: change ports to avoid CI conflict
crankynetman Oct 25, 2024
9c2ec5a
fix: enable postgres locally by default (and remove deprecated docker…
crankynetman Oct 25, 2024
bbfaa79
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
bde9319
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
7ce9b18
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
942da84
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
f765cb5
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
1e15db7
ci: collapse cleanup into test stage
crankynetman Oct 29, 2024
583f2c8
ci: try adding needs to stage
crankynetman Oct 29, 2024
55806a2
ci: try just adding cleanup to pytest.
crankynetman Oct 29, 2024
8ba411d
ci: try re-running after runner cleanup
crankynetman Oct 29, 2024
d6eab80
ci: try running another pipeline
crankynetman Oct 29, 2024
724f22f
ci: try running another pipeline
crankynetman Oct 29, 2024
b7f79d0
ci: try running another pipeline
crankynetman Oct 29, 2024
ebcd7d5
ci: try running yet another pipeline
crankynetman Oct 29, 2024
fcd461e
ci: try running yet another yet another pipeline
crankynetman Oct 29, 2024
7cf6e0e
ci: try again running yet another yet another pipeline
crankynetman Oct 29, 2024
e08a896
ci: try again running again yet another yet another pipeline
crankynetman Oct 29, 2024
fbc7595
ci: *slams down drink* ANOTHER
crankynetman Oct 29, 2024
fd9b48c
ci: *slams down drink* yet ANOTHER
crankynetman Oct 29, 2024
86fd3ed
ci: show running containers
crankynetman Oct 30, 2024
569853a
ci: show me MORE
crankynetman Oct 30, 2024
c22f886
ci: bypass make
crankynetman Oct 30, 2024
47ae148
show me even MOAR
crankynetman Oct 30, 2024
7c84fae
ci: let compose do its composey bits
crankynetman Oct 30, 2024
5e8db6d
ci: set the variable in ALL the places we need it.
crankynetman Oct 30, 2024
8f0efd0
ci: add variable in probably the last place that we need it
crankynetman Oct 30, 2024
f628531
ci: cleanup CI from debugging
crankynetman Oct 31, 2024
ffcd0d2
fix: stop leaving behind orphans
crankynetman Oct 31, 2024
489db35
ci: more cleanup of ci file
crankynetman Oct 31, 2024
580372f
Why: Add comment
set-element Nov 8, 2024
a1e2053
Why: Change commands from docker to podman. Simple find and replace.
set-element Oct 4, 2024
75d6694
Why: Add env variables for the various container functions. These are…
set-element Oct 4, 2024
618e928
Why: Add some env variables to compose.override.production.yml as wel…
set-element Oct 4, 2024
e84bf5c
Why: Remove container config elements from the basic yaml file
set-element Nov 12, 2024
be3867b
Why: Initial notes about networking
set-element Nov 14, 2024
a54590e
Why: Add note about env variables. This makes more sense than trying …
set-element Nov 15, 2024
e9fed4b
Why: update notes
set-element Nov 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -1 +1,9 @@
CI_PROJECT_DIR=.
SCRAM_PEERING_IFACE=eth1
SCRAM_V4_SUBNET=10.124.0.0/24
SCRAM_V4_GATEWAY=10.124.0.1
SCRAM_V4_ADDRESS=10.124.0.4
SCRAM_V6_SUBNET=2004:600:202:9::8/126
SCRAM_V6_GATEWAY=2004:600:202:9::9
SCRAM_V6_ADDRESS=2004:600:202:9::a
HOSTNAME=$(hostname)
35 changes: 35 additions & 0 deletions .envs/.production/.django
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# General
# ------------------------------------------------------------------------------
#DJANGO_READ_DOT_ENV_FILE=True
#DJANGO_SETTINGS_MODULE=config.settings.production
#DJANGO_SECRET_KEY=
#DJANGO_ADMIN_URL=admin/
#DJANGO_ALLOWED_HOSTS=
# Security
# ------------------------------------------------------------------------------
# TIP: better off using DNS, however, redirect is OK too
#DJANGO_SECURE_SSL_REDIRECT=False

# Email
# ------------------------------------------------------------------------------
#DJANGO_SERVER_EMAIL=test@test.com

# Gunicorn
# ------------------------------------------------------------------------------
#WEB_CONCURRENCY=4

# Redis
# ------------------------------------------------------------------------------
#REDIS_URL=redis://redis:6379/0

#SCRAM_AUTH_METHOD=oidc
#SCRAM_AUTH_METHOD=local
#OIDC_OP_JWKS_ENDPOINT=
#OIDC_OP_AUTHORIZATION_ENDPOINT=
#OIDC_OP_TOKEN_ENDPOINT=
#OIDC_OP_USER_ENDPOINT=
#OIDC_RP_SIGN_ALGO=RS256



#OIDC_RP_CLIENT_ID=
7 changes: 7 additions & 0 deletions .envs/.production/.postgres
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# PostgreSQL
# ------------------------------------------------------------------------------
#POSTGRES_HOST=postgres
#POSTGRES_PORT=5432
#POSTGRES_DB=scram
#POSTGRES_USER=scram
#POSTGRES_PASSWORD=scram
4 changes: 4 additions & 0 deletions .envs/.production/.translator
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Translator
# ------------------------------------------------------------------------------
#TRANSLATOR_HOSTNAME=localhost
#TRANSLATOR_URL="ws://django:5000/ws/route_manager/translator_block/"
18 changes: 12 additions & 6 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,6 @@ __pycache__/
# Ignore prod nginx files so git doesnt think we've locally added new files when deploying templates with ansible
compose/production/nginx

.idea/
.idea/*
.idea/scram.iml
*.iml

# C extensions
*.so

Expand Down Expand Up @@ -63,6 +58,9 @@ staticfiles/

# Sphinx documentation
docs/_build/
docs/api
docs/*.db


# PyBuilder
target/
Expand Down Expand Up @@ -162,6 +160,7 @@ typings/
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
*.code-workspace


# Provided default Pycharm Run/Debug Configurations should be tracked by git
Expand All @@ -172,7 +171,13 @@ typings/
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

.idea/*
# Allow Debugger Files
/.idea/runConfigurations/*.xml
/.idea/inspectionProfiles
!.idea/runConfigurations/*_Debugger.xml

.idea/scram.iml
*.iml

# User-specific stuff:
.idea/**/workspace.xml
Expand Down Expand Up @@ -334,6 +339,7 @@ scram/media/


.ipython/
# envs are now stored in compose file
.envs/*
!.envs/.local/
compose.override.yml
Expand Down
23 changes: 8 additions & 15 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ include:
stages:
- lint
- test
- cleanup

variables:
POSTGRES_USER: scram
Expand All @@ -28,6 +27,8 @@ pytest:
image: docker:24.0.6-dind
services:
- docker:dind
variables:
POSTGRES_ENABLED: 1
before_script:
- apk add make
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
Expand All @@ -38,12 +39,18 @@ pytest:
- make migrate
- make run
script:
- export COMPOSE_PROJECT_NAME=$CI_PIPELINE_ID
- make coverage.xml
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
after_script:
- export COMPOSE_PROJECT_NAME=$CI_PIPELINE_ID
- make stop
- make clean


gemnasium-dependency_scanning:
variables:
Expand All @@ -59,17 +66,3 @@ code_quality_html:

sast:
stage: test

final_clean:
image: docker:24.0.6-dind
services:
- docker:dind
before_script:
- apk add make
- export COMPOSE_PROJECT_NAME=$CI_PIPELINE_ID
stage: cleanup
rules:
- when: always # run even if something failed
script:
- make stop
- make clean
17 changes: 17 additions & 0 deletions .idea/runConfigurations/Django_Debugger.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions .idea/runConfigurations/Translator_Debugger.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Django Debugger",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "0.0.0.0",
"port": 56780
},
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/app"
}
]
},
{
"name": "Translator Debugger",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "0.0.0.0",
"port": 56781
},
"pathMappings": [
{
"localRoot": "${workspaceFolder}/translator",
"remoteRoot": "/app"
}
]
}
]
}
48 changes: 24 additions & 24 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,28 +22,28 @@ compose.override.yml:
## behave-all: runs behave inside the containers against all of your features
.Phony: behave-all
behave-all: compose.override.yml
@docker compose run django coverage run -a manage.py behave --no-input --simple
@podman-compose run django coverage run -a manage.py behave --no-input --simple

## behave: runs behave inside the containers against a specific feature (append FEATURE=feature_name_here)
.Phony: behave
behave: compose.override.yml
@docker compose run django python manage.py behave --no-input --simple -i $(FEATURE)
@podman-compose run django python manage.py behave --no-input --simple -i $(FEATURE)

## behave-translator
.Phony: behave-translator
behave-translator: compose.override.yml
@docker compose exec -T translator /usr/local/bin/behave /app/acceptance/features
@podman-compose exec -T translator /usr/local/bin/behave /app/acceptance/features

## build: rebuilds all your containers or a single one if CONTAINER is specified
.Phony: build
build: compose.override.yml
@docker compose up -d --no-deps --build $(CONTAINER)
@docker compose restart $(CONTAINER)
@podman-compose up -d --no-deps --build $(CONTAINER)
@podman-compose restart $(CONTAINER)

## coverage.xml: generate coverage from test runs
coverage.xml: pytest behave-all behave-translator
@docker compose run django coverage report
@docker compose run django coverage xml
@podman-compose run django coverage report
@podman-compose run django coverage xml

## ci-test: runs all tests just like Gitlab CI does
.Phony: ci-test
Expand All @@ -52,18 +52,18 @@ ci-test: | toggle-local build migrate run coverage.xml
## clean: remove local containers and volumes
.Phony: clean
clean: compose.override.yml
@docker compose rm -f -s
@docker volume prune -f
@podman-compose rm -f -s
@podman volume prune -f

## collect-static: run collect static admin command
.Phony: collectstatic
collectstatic: compose.override.yml
@docker compose run django python manage.py collectstatic
@podman-compose run django python manage.py collectstatic

## django-addr: get the IP and ephemeral port assigned to docker:8000
.Phony: django-addr
django-addr: compose.override.yml
@docker compose port django 8000
@podman-compose port django 8000

## django-url: get the URL based on http://$(make django-addr)
.Phony: django-url
Expand All @@ -75,15 +75,15 @@ django-url: compose.override.yml
django-open: compose.override.yml
@open http://$$(make django-addr)

## down: turn down docker compose stack
## down: turn down podman-compose stack
.Phony: down
down: compose.override.yml
@docker compose down
@podman-compose down

## exec: executes a given command on a given container (append CONTAINER=container_name_here and COMMAND=command_here)
.Phony: exec
exec: compose.override.yml
@docker compose exec $(CONTAINER) $(COMMAND)
@podman-compose exec $(CONTAINER) $(COMMAND)

# This automatically builds the help target based on commands prepended with a double hashbang
## help: print this help output
Expand All @@ -95,41 +95,41 @@ help: Makefile
## list-routes: list gobgp routes
.Phony: list-routes
list-routes: compose.override.yml
@docker compose exec gobgp gobgp global rib -a ipv4
@docker compose exec gobgp gobgp global rib -a ipv6
@podman-compose exec gobgp gobgp global rib -a ipv4
@podman-compose exec gobgp gobgp global rib -a ipv6

## migrate: makemigrations and then migrate
.Phony: migrate
migrate: compose.override.yml
@docker compose run django python manage.py makemigrations
@docker compose run django python manage.py migrate
@podman-compose run django python manage.py makemigrations
@podman-compose run django python manage.py migrate

## pass-reset: change admin's password
.Phony: pass-reset
pass-reset: compose.override.yml
@docker compose run django python manage.py changepassword admin
@podman-compose run django python manage.py changepassword admin

## pytest: runs pytest inside the containers
.Phony: pytest
pytest: compose.override.yml
@docker compose run django coverage run -m pytest
@podman-compose run django coverage run -m pytest

## run: brings up the containers as described in compose.override.yml
.Phony: run
run: compose.override.yml
@docker compose up -d
@podman-compose up -d

## stop: turns off running containers
.Phony: stop
stop: compose.override.yml
@docker compose stop
@podman-compose stop

## tail-log: tail a docker container's logs (append CONTAINER=container_name_here)
.Phony: tail-log
tail-log: compose.override.yml
@docker compose logs -f $(CONTAINER)
@podman-compose logs -f $(CONTAINER)

## type-check: static type checking
.Phony: type-check
type-check: compose.override.yml
@docker compose run django mypy scram
@podman-compose run django mypy scram
Loading