introduce global anomalies, build queue length, + manual via CLI

[syphar]

( updated description).

introduce "abnormality" to warn users about currently active issues.

Sources are (first version):

1. automatic for a too long queue
2. manual abnormaly in the database for the rest.

This is not for notifications, but only this kind of system status information. Notifications / alerts will be cached in the same WarningsCache, passed into the templates the same way, but shown via popup & discardable.

We cache the value for 5 minutes, so we won't have too many added database queries.

It will look like this, with multiple abnormalities:

We also have the added detail status page with more explanation.

the queue warning is additionally shown on the queue page

The audit error is fixed in #3295

[GuillaumeGomez]

I think you're trying to fix two different problems here:

1. docs.rs issues
2. docs.rs changes

The first one needs to be there as long as the issue(s) are there. The second one being a notice, it should be discard-able by the user and attract attention a lot more. So your PR fits well for 1., but doesn't for 2. imo. 2. should most likely be a static element floating over the content.

What do you think?

[syphar]

I admit I didn't think much further than just making the old feature not hardcoded, so we can start showing something somewhere.

I don't think I can build either of the other approaches in the next week, but hopefully some time after that, depending on many other factors.

[syphar]

I think you're trying to fix two different problems here:

1. docs.rs issues

2. docs.rs changes

The first one needs to be there as long as the issue(s) are there. The second one being a notice, it should be discard-able by the user and attract attention a lot more. So your PR fits well for 1., but doesn't for 2. imo. 2. should most likely be a static element floating over the content.

What do you think?

Right now I don't have an opinion on that (too tired / sick), how does crates.io do it?

Generally what you propose feels like it makes sense.

[syphar]

general thought:

with my dayjob problems right now, I don't think I can spend time on any of the proposed other things in the next week or two (except when I'm lucky, not sure).

so I'm asking myself if we should use this first to make people aware of the new default-target change first?

[GuillaumeGomez]

I can take over this PR and implement the missing parts if you want?

[syphar]

I can take over this PR and implement the missing parts if you want?

thanks for the offer!

My hunch is that you're right generally, but I want to think it through first when I'm a little better.

[syphar]

( will set the compiled-in warning in a separate PR for the current change only)

[GuillaumeGomez]

Sounds good to me. :)

[syphar]

@GuillaumeGomez I had another idea about multiple alerts, the separate page looked so empty with the information that was there..

what do you think?

It would collapse (so no dropdown) when there is only one alert.

Also now there is an example how the webserver now collects metrics from the builder-lib.

( not sure if I was too much into it and overengineered 😅 )

[GuillaumeGomez]

Well, I was kinda hoping to provide more information for each anomaly on the page. 😆

Like since how long it's happening, what's the impact to users, etc.

[syphar]

Well, I was kinda hoping to provide more information for each anomaly on the page. 😆

Like since how long it's happening, what's the impact to users, etc.

Want to add a commit how you would imagine it?

[GuillaumeGomez]

I can try. :)

[syphar]

I just rebased to fix the conflicts, will leave it alone now @GuillaumeGomez

[GuillaumeGomez]

I added the draft for the status page of docs.rs. It looks like this:

[syphar]

valid points, good idea, we can do that.

I assume that we merge the Abnormality and GlobalAlert things?

the abnormality only has the explanation & start-time on top?

[GuillaumeGomez]

GlobalAlert is different: it's like the one you can see on crates.io currently. It's supposed to have a start and an end date so we know when to render it to users.

[syphar]

will finish up this PR and ping you for the final review.

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[syphar]

@GuillaumeGomez I just realized we have to adapt how this is done, and I might need help from you.

The current impl would break with caching. For example, when a rustdoc page is cached in the CDN, it would be cached including the abnormalities.

The easiest solution is probably to have the abnormalities be a small partial that is loaded into the topbar, where we could define different caching rules.

what do you think?

[GuillaumeGomez]

I'm not sure to understand what you suggest. Considering the topbar content is updated at every new release (and sometimes in-between), don't we already have this issue?

[syphar]

I'm not sure to understand what you suggest. Considering the topbar content is updated at every new release (and sometimes in-between), don't we already have this issue?

the topbar is part of the page html, so it's cached together with the rustdoc content.
We invalidate the pages on every release, but only by crate. That means when there are no releases for one specific crate, we don't invalidate the topbar, because the content of the topbar doesn't change.

[GuillaumeGomez]

But the content of the topbar does change. For example we recently added a lot of new links in the "docs.rs" dropdown menu.

[syphar]

That's rare, and in these cases I manually purge the whole cdn.

[GuillaumeGomez]

Then instead of a link only appearing in case there are anomalies. Should we have a "docs.rs status" link? Like that it never changes and we're all good.

[syphar]

Then instead of a link only appearing in case there are anomalies. Should we have a "docs.rs status" link? Like that it never changes and we're all good.

I think I would want alerts ( and anomalies) always visible, and not behind a click.

I'll (try to) update this PR, so the anomalies are loaded in a separate small partial, and then injected into the topbar, similar to how we load the version-list in the main dropdown.

[syphar]

this change might also make this PR much less complex, since I don't have to inject the anomalies into all pages any more

[GuillaumeGomez]

Then instead of a link only appearing in case there are anomalies. Should we have a "docs.rs status" link? Like that it never changes and we're all good.

I think I would want alerts ( and anomalies) always visible, and not behind a click.

I'll (try to) update this PR, so the anomalies are loaded in a separate small partial, and then injected into the topbar, similar to how we load the version-list in the main dropdown.

Hmmmmm. The whole caching thing is really getting in the way here. ^^'

[syphar]

Then instead of a link only appearing in case there are anomalies. Should we have a "docs.rs status" link? Like that it never changes and we're all good.

I think I would want alerts ( and anomalies) always visible, and not behind a click.
I'll (try to) update this PR, so the anomalies are loaded in a separate small partial, and then injected into the topbar, similar to how we load the version-list in the main dropdown.

Hmmmmm. The whole caching thing is really getting in the way here. ^^'

perhaps. But like this, with just some well designed pages, we can have super fast server-side generated pages :)

[syphar]

@GuillaumeGomez

this is how I would imagine this,

not knowing what the best way is to have this placeholder here.

View changes since the review

[syphar]

@GuillaumeGomez

this is just AI generated code, to show you what I meant with the partial.

I assume we should do it differently? not sure what's the best way.

View changes since the review

[syphar]

@GuillaumeGomez I updated the implementation to what I think we should do.

The template & JS change just as a proof of concept, need your help to finish this up.

I assume the alerts could also use a similar concept?

[GuillaumeGomez]

Not sure to agree with your approach. Why do you want them to be available in the topbar directly? It's one click in any case, whether we list abnormalities in the topbar or if we have a link to the docs.rs status page. I can add a JS part to add a ⚠️ symbol if there is a current abnormality but I don't think it's a good idea to list them directly in the topbar.