Workflow - initial steps

[ehmatthes]

For the initial steps, it's a bit more involved than what I'm used to when targeting PAAS providers. These notes are relevant to creating a dev environment that involves live droplets.

• Create a droplet I can use for configuration-only work.
• Kill it if I'm stepping away from this for more than a few days.
• Implement Initial server setup steps, noting divergences from the guide.
• Implement django on DO steps, noting divergences from the guide.

[ehmatthes]

droplet notes

• NYC1
• Ubuntu 24.10 x64
• Basic, SSD, 1GB/25GB, $6/mo
• password authentication

[ehmatthes]

development env

• Open a terminal session at droplet.
• Make a dsd-dev project, with -e dsd-digitalocean.
• Set env vars:
  • DSD_HOST_IPADDR, DSD_HOST_USERNAME, DSD_HOST_PW
  • export $(cat .env)
• Install, and require, paramiko.
• Run deploy, until it updates the server.
  • Probably want to stream output of _update_server().
  • For now, just include a message that this might take a while.

[ehmatthes]

Working notes

• Write output from do_utils.run_server_cmd_ssh().
  • Import plugin_utils directly.
  • Write output by default; core should handle suppressing if there's no logging.
  • Simplify main file.
• Refactor util function for getting client? (Not needed at this point.)
• Refactor check for reboot.

[ehmatthes]

Process notes

This is probably worth documenting for people who are developing a plugin. My development process is pretty routine now:

1. Build dev env with build_dev_env.py
2. In that repo, install editable plugin.
3. Run deploy.
4. Run git checkout . && git clean -fd && git status
5. Go back to step 3

This is quite efficient. Sometimes it's better to call deploy --unit-testing, to avoid some steps that require remote resources.

[ehmatthes]

Next steps

Build this out for the automate-all mode, then separate out what might make sense for the configuration-only mode.

• Allow connections on port 8000.
• Configure code Configure code #7
  • first pass
  • correct
• Push code.
• Build venv
• migrate
• collectstatic
• Start gunicorn process
• Start caddy
• Deployed?

[ehmatthes]

Write serve_project.sh

• Write to local project, as part of configuration.
• After push, make this file executable.
• After push, run this script.

[ehmatthes]

Migrate

• Need an active venv, or use uv path explicitly? (explicit path, for now at least)
• using git push --force during development, because of all my reset --hard on local.

[ehmatthes]

Refinement

• Super slow performance. Troubleshoot.
  • Run gunicorn as a service, with logging and specific workers.
  • Copy gunicorn.service to correct location.
  • Use caddy in front.
• Open in new browser tab.
• Run functionality test against deployment.
• Test against bare droplet.
• Make video of the process.

[ehmatthes]

Final gunicorn work

• Troubleshoot the final systemctl commands in serve_project.sh.
• Reduce password entries
  • scp calls

[ehmatthes]

Gunicorn, caddy

• Run against a fresh droplet.
  • fix issues below
• Troubleshoot final systemctl commands.
  • Troubleshoot this directly on server; modify sudoers.d until I can run systemctl gunicorn commands without entering pw.
• Set up caddy.

[ehmatthes]

Run against fresh droplet

...
Looking for gunicorn...
  Added gunicorn to requirements file.
  Modified file: /Users/eric/projects/dsd-dev-project_jxqty/blog/settings.py

  Looking in /Users/eric/projects/dsd-dev-project_jxqty for serve_project.sh...
    File serve_project.sh not found. Generating file...

    Wrote serve_project.sh to /Users/eric/projects/dsd-dev-project_jxqty/serve_project.sh
  Configuring gunicorn to run as a service.
Copying /Users/eric/projects/dsd-digitalocean/dsd_digitalocean/templates/gunicorn.socket to server.
Running server command over SSH...
  command: sudo mv /home/django_user/gunicorn.socket /etc/systemd/system/gunicorn.socket
Copying /var/folders/gk/y2n2jsfj23g864pdr38rv4ch0000gn/T/tmpfd2c795v to server.
Running server command over SSH...
  command: sudo mv /home/django_user/gunicorn.service /etc/systemd/system/gunicorn.service
  Committing changes...

[main 130aa41] Configured project for deployment.
 4 files changed, 56 insertions(+)
 create mode 100644 serve_project.sh
  Deploying to Digital Ocean...
  Pushing project code to server.
Running server command over SSH...
  command: chmod +x /home/django_user/blog/serve_project.sh
chmod: cannot access '/home/django_user/blog/serve_project.sh': No such file or directory
Running server command over SSH...
  command: /home/django_user/blog/serve_project.sh
sh: 1: /home/django_user/blog/serve_project.sh: not found


--- Your project should now be deployed on Digital Ocean ---

It should have opened up in a new browser tab. If you see a
  "server not available" message, wait a minute or two and
  refresh the tab. It sometimes takes a few minutes for the
  server to be ready.
- You can also visit your project at http://167.99.154.203:8000/

If you make further changes and want to push them to Digital Ocean,
commit your changes and then run `...`.

• Does not seem to have made git push. (/home/django_user/blog/ is there, but empty.) Needed to enter pw, set up key pair? Seemed to hang at that step longer than it should. May have timed out and just moved on? breakpoint there?
  • git remote was stale
• Update was disabled; was that an issue? (no, not this)

[ehmatthes]

Caddy

• Write Caddyfile to accessible place
• Copy to correct place
• Start service
• View site?

[ehmatthes]

Misc

• Show output after calling run_quick_command().
• Make sure no calls like _update_server() are commented.
• Set DEBUG to False.

[ehmatthes]

Caddy troubleshooting

Set up to serve as a proxy, but returning blank page.

• Run against a fresh droplet, with update reenabled.
• Failed after initial reboot; running again seems to have addressed it?
• Shows default Caddy page in browser.
• Start gunicorn service and then start caddy service? (Shows blank page, no longer shows caddy default page.)
• Troubleshoot proxying. (needed to listen to gunicorn socket, not port 8000.)
• Got to debug page! Disallowed host issue, but that should be easier from this point forward.
• ALLOWED_HOSTS is empty, which means it's not seeing the ON_DIGITALOCEAN env var. How see this env var when starting the service?
  • Set env vars in gunicorn.service.
• Add a reload before starting gunicorn and caddy service?

[ehmatthes]

Bare droplet

• Troubleshoot opening in new browser tab.
• Run against fresh droplet (remove current remote first).