Deployment smoke tests

We need some tests writing which will check that the hugo build was successful and the site deploys correctly. This will be used as part of a PR workflow before enabling merges to main once complete. For example if the tests fail for any reason, merging will be blocked until they pass.

I envisage this taking the form of checking the existence of critical files including configuration files and certain show pages. Then the site will be built (as part of an action most likely), spun up (again as part of the action) and then the smoketests will be performed to ensure the thing is running as expected.

@kd2flz

I imagine that we could have a simple hugo command (like how we do for deployment) run to try and just build the site. Which can test and give fast feedback on if it'll build at all. (Validating the config for hugo works)

Then, I think validating key shows or other web things (i.e. jb-live loads the iframe, media files can be accessed, host/guest pictures are loading, etc.) will go hand in hand with your ideas in matrix about deployments.

Although, personally I'm kind of more partial to #257 mainly because it doesn't require any type of deployment on JB infra. At least maybe for an initially opened PR, which will help with security because it won't be touching any JB infra until later (maybe after merging into a develop branch). While also give fast feedback to PRs authors as well.

So it could go:

PR opened against develop --> attempted deployment to netlfiy (which validates hugo config or else it won't deploy at all)

--> if tests don't pass then person knows right away they need to fix something
--> tests do pass, then can be merged into develop by JB team member
- --> tests run again against .dev.jb.com to make sure everything is still working, and is now deployed (as a test run) on JB infra (like how it'll be in prod)
  - --> JB team member deploys to main/prod & tests run again, so it's getting a last check after the merge to make sure nothing on the main deployed website was messed up from the PR & if it was you can quickly revert it

I think this'll give a nice incremental rollout over JB infrastructure while also allowing for fast feedback for PR authors, and not compromising the security of JB infra by quickly deploying potentially malicious code from an unknown person who opens PRs against the repo.

For actually doing the test I've actually run across a really nice tool for web-ui testing tool recently that's a lot easier to use than something like selenium but has a similar in function: https://playwright.dev/

They also have a python port and it uses pytest: https://playwright.dev/python/docs/intro

Since we're already using python for the scraper, and python is big in the linux community we can leverage the port to take advantage of everyone's familiarity already with python.

It's a good testing framework which even lets you do video recordings and screenshots while it goes through, which lets you get a quick high level review of the site for extremely important sections. This is a pretty good/cool YT video about playwright:

BTW, @kd2flz's comment in this thread is something we still need to consider as well. Although, if we run out of them it's not that big of a deal IMO, because it's just an extra nice-to-have thing for quick feedback.

Based on this issue happening, we need to validate the expected page isn't set to draft. When someone is doing a PR, they should include tests in the CI for ensuring this.

Just going to start making a list of different tests we'll eventually need to implement, so I don't forget 🙃

Episode layout (for each show) i.e. how there were two logos here: Revamp Episode Layout #400 (comment)

Test all pages formatting

404 page: Add 404 page #427

Live page

When "live": "JB is live!" indicator - on page load, call the script to highlight the #livebutton red if jupiter.tube is live. #378 (using playwright's intercept feature to mock going live, mentioned here)

example event:

{"total":1,"data":[{"id":50,"uuid":"55b2e9d1-ccf3-4914-9f3f-c5d8a59bd4c1","shortUUID":"bzMpxgG2rDKn9C99GSab92","url":"https://jupiter.tube/videos/watch/55b2e9d1-ccf3-4914-9f3f-c5d8a59bd4c1","name":"LINUX Unplugged 471 - We Broke Our Server","category":{"id":15,"label":"Science & Technology"},"licence":{"id":5,"label":"Attribution - Non Commercial - Share Alike"},"language":{"id":"en","label":"English"},"privacy":{"id":1,"label":"Public"},"nsfw":false,"description":"We broke our server, so lets see if we can fix it live before the show is over!","isLocal":true,"duration":0,"views":0,"viewers":19,"likes":0,"dislikes":0,"thumbnailPath":"/static/thumbnails/b610e5f5-bbb5-4ee1-ac24-421c50f0daf5.jpg","previewPath":"/lazy-static/previews/fcd582d8-68c5-4f00-8a4a-0a8ffde1a4a5.jpg","embedPath":"/videos/embed/55b2e9d1-ccf3-4914-9f3f-c5d8a59bd4c1","createdAt":"2022-08-14T16:26:27.938Z","updatedAt":"2022-08-14T18:53:15.544Z","publishedAt":"2022-08-14T18:53:15.543Z","originallyPublishedAt":null,"isLive":true,"account":{"id":3,"displayName":"JBLive Stream","name":"jblive","url":"https://jupiter.tube/accounts/jblive","host":"jupiter.tube","avatars":[{"width":48,"path":"/lazy-static/avatars/e4d17e7f-a144-4a4a-b5d8-b297b06b727f.png","createdAt":"2022-06-07T23:43:56.565Z","updatedAt":"2022-06-07T23:43:56.565Z"},{"width":120,"path":"/lazy-static/avatars/1340f3e9-d0a7-4bc4-bcbd-a8c865eaf1b8.png","createdAt":"2022-05-30T20:36:19.005Z","updatedAt":"2022-05-30T20:36:19.005Z"}],"avatar":{"width":48,"path":"/lazy-static/avatars/e4d17e7f-a144-4a4a-b5d8-b297b06b727f.png","createdAt":"2022-06-07T23:43:56.565Z","updatedAt":"2022-06-07T23:43:56.565Z"}},"channel":{"id":2,"name":"live","displayName":"live","url":"https://jupiter.tube/video-channels/live","host":"jupiter.tube","avatars":[{"width":48,"path":"/lazy-static/avatars/8acfd2a2-ab4e-48aa-990c-3156a2765d2e.png","createdAt":"2022-06-07T23:43:56.607Z","updatedAt":"2022-06-07T23:43:56.607Z"},{"width":120,"path":"/lazy-static/avatars/e9bcfdb7-90a2-479c-b1da-1ab0e5fc9442.png","createdAt":"2022-05-30T20:38:33.632Z","updatedAt":"2022-05-30T20:38:33.632Z"}],"avatar":{"width":48,"path":"/lazy-static/avatars/8acfd2a2-ab4e-48aa-990c-3156a2765d2e.png","createdAt":"2022-06-07T23:43:56.607Z","updatedAt":"2022-06-07T23:43:56.607Z"}}}]}

when not live

Test different devices emulations (mentioned here)
make sure all external links open a new tab
navigate different user journeys
check accessibility (i.e. Accessibility check #414)
other functionality?

More testing optimizations: https://podverse.fm/episode/1Bc04qHgk

@CGBassPlayer

@CGBassPlayer, we probably need to check to make sure image paths are right. I just messed up on validating that for #506 (here), but I don't know exactly how to do that 😅

Also, posting it here because #475's convo thread is getting long and this is the original issue it spawned from.

Not sure how I missed this comment! I will add image validation to the list and see if I can figure something out

Just ask chatgpt to do it

…

On Sat, Apr 1, 2023 at 21:45 Ryan Goodwin ***@***.***> wrote: Not sure how I missed this comment! I will add image validation to the list and see if I can figure something out — Reply to this email directly, view it on GitHub <#286 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVFAWGBI3RIDAPWT65IJ33W7DKZ7ANCNFSM56SZ3LOQ> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

Navigation Menu

Repository navigation

Deployment smoke tests #286

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Deployment smoke tests #286

Description

Activity

elreydetoda commented on Aug 15, 2022

elreydetoda commented on Aug 17, 2022

elreydetoda commented on Sep 27, 2022

elreydetoda commented on Feb 19, 2023

CGBassPlayer commented on Apr 2, 2023

ironicbadger commented on Apr 2, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions