Key-based e2e test

[ehmatthes]

Enough has changed since #15, it's worth a new issue. key-based automate-all workflow is working well enough to write a draft e2e tests. The deployment doesn't fully work yet, ie admin CSS is not working, but there's enough correct behavior that a test will be helpful in the refinement work.

Tasks

• Merge current state from Support key-based ssh for droplets #18. No new release.
• Write an e2e test that replicates the current level of success, and carries out full teardown.
• Merge.
• Focus on refactoring, refinement, and cleaning up remaining issues with deployment.
• Any reason I'm accruing ~$2 in charges from a week of work on a bunch of 10-minute deployments of what should be a $6/month server? Are there creation charges above the $6/30/24/6 ~ $0.0014? That should be about 6 ten minute deployments for a penny.

[ehmatthes]

Droplet pricing

• Documentation says $0.00893/hr. I've done a bunch of tests, only a handful were more than an hour. Maybe one was left for a day. Still, that would be about $0.20 for a day. I've done fewer than 100 tests. I shouldn't see charges more than $1.
• Everything looks correct. I just went to that pricing page, clicked on a 1G droplet that should be $6/month, and the --size s-1vcpu-1gb matches the command I get in the "create from CLI" dialog.
• "All Droplets are billed hourly up to a monthly cap of 672 hours (the number of hours in 4 weeks)." That reads like a minimum 1-hour charge for droplet creation. That's still under $0.01.
• During next test, look at all aspects of the droplet that's created, and see if there's any reason for the charge.
• Look at my next invoice for any issues.
• (Note for anyone else skimming this, I am aware I can reach out to DO for support in developing this tool. I will do that when it's a little closer to a 1.0 release.)
• Run a test against a higher-spec droplet, and see if the test completes in noticeably less time.
• Oof. I think I was just reading the DO estimate wrong. I have several long-running droplets for other projects. I thought those were included for the full month in the estimate. But the estimate starts at 0 each month, and builds up based on daily usage. So the main cost I'm seeing in the estimate is my long-running droplets.
• Check next invoice, it should be <$1 more than my usual invoice.

[ehmatthes]

Parking lot

• Make an e2e test CLI arg that lets you specify the size of the droplet used in any given test. Keep the $6/month as the default, but make it easy to run a test on various size droplets.
• Don't use ~/.ssh/config, use a temp location and tell Git to use that.
• Refactor utils.py destroy_project().

[ehmatthes]

e2e test

• Call to run_simple_deploy() needs to accept additional plugin-specific args.
  • Run Fly e2e test, make sure current version is working.
  • Update core to accept the args. (Already done, plugin_args_string.)
  • Run e2e for Fly, make sure that still passes. (not needed, no changes to core.)
• Troubleshoot admin CSS.
• Pass functionality test manually, after the deployment and before destroying test project.
• Pass functionality test in e2e test.
• path to key should be an env var.

[ehmatthes]

First run

• Set breakpoint after call to deploy, then parse output for required info for finishing the test and teardown. (test deployment works)
  • deployed URL
  • droplet ID
• Does next e2e test deployment droplet get destroyed correctly?
• Remove test block from ~/.ssh/config