Add a manual retry for partial publications, and stop mislabelling rejections as an empty queue - #2
Merged
Merged
Conversation
A non-2xx answer was decoded as an ordinary payload, which left Items empty and made callers report "no items available". A 400 from content-alchemist therefore surfaced as an empty publication queue: the message cron logged "no items for language en" while the real cause was an invalid-language rejection. Rejections now return an error carrying the message content-alchemist sent, with a fallback to the raw body for the plain-text auth and rate-limit replies. The fallback on "no text available for language" is removed - content-alchemist has not sent that message since it started falling back to an available language. Endpoints and the bearer header are resolved per call instead of at import time, so they no longer depend on when the environment was populated, and request bodies are built with encoding/json rather than string interpolation.
… landed The cron history only held free-form output, so nothing could tell which repository a run published or which integrations missed it. That is exactly what is needed to finish a partial publication by hand. Adds a details column holding url, sent, failed and manual, exposed through GetCronHistory. LogCronExecution keeps its signature and delegates with no details, so the collect job is unchanged. The column is nullable: runs recorded before it existed read back without details.
A message run marks its repository as posted as soon as any integration succeeds, so the item leaves the publication queue and the connectors that failed can never recover it on a later run. RetryMessagePost publishes one repository to the named integrations, fetching the text per integration in its configured language and reporting a per-API outcome. When no url is given it uses the most recently published repository, which is the item a partial run just consumed. The request building shared with MessageJob is extracted into publishItem so both paths cannot drift. A retry is recorded in the cron history as a manual message run and sends no Pushover notification - whoever triggered it is already watching. It removes only its own image instead of clearing the shared directory, and it uses a short socialify retry budget via SocialifyWithConfig so a synchronous request cannot block for minutes.
Exposes the manual retry over HTTP so the dashboard can finish a partial publication. The response is 200 once the retry has run: per-integration outcomes carry the individual failures, which say more than a single status code. Input problems answer 400.
The URL revalidation loop reassigned the fetch response and read it after the loop, so a rejected re-fetch left it nil and the next read panicked. The job's deferred handler re-panics, and no gocron panic handler is registered, so this took the whole process down - and skipped the posted update, leaving the item to be published a second time by the next run. Turning alchemist rejections into errors made that path routine rather than rare, so the loop now reports its own outcome instead. Run details are assembled at exit as well, so an early return or a panic still records which item was consumed and which connectors missed it; a run with nothing to retry records no details at all instead of an empty object.
Three ways the retry and the publishing cron could damage each other: - the cron's cleanup removed every file in the shared image folder, including an image a retry was still uploading, while a file the retry deleted first made that cleanup fail and report a successful run as failed. Retry images now live in a subdirectory the cleanup skips, and a file that vanished under it is no longer an error. - an image was generated per integration rather than once per retry, so each connector triggered its own upstream fetch. - the socialify client had no timeout, so bounding the attempts did not bound the request: a hung upstream blocked the synchronous endpoint indefinitely. Retries are also serialised, so a double-clicked button cannot publish twice, and the request body is size-limited.
Member
Author
|
🎉 This PR is included in version 3.8.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
On 2026-08-17 the message cron reported:
Two separate problems are visible there.
The error message was wrong. The publication queue was not empty — it held 463 items. content-alchemist had answered
400(it was rejecting every request that carried atext_language, see think-root/content-alchemist#6), andmakeRepositoryRequestdecoded that rejection as an ordinary payload with no items. Any rejection from content-alchemist therefore looked like an exhausted queue, in the cron history and in Pushover alike.A partial run cannot be recovered.
MessageJobmarks the repository as posted as soon as any integration succeeds, soresemble-ai/chatterboxwas flaggedposted=trueat the exact moment Threads failed. It left the queue without ever reaching Threads, and there was no way to finish the job: this service had no manual trigger of any kind, and content-alchemist could not serve a specific item by url.What changed
Four commits, each independently reviewable:
fix(repository)— a non-2xx response, or astatus: "error"envelope, now returns an error carrying content-alchemist's own message, with a fallback to the raw body for the plain-text auth and rate-limit replies. The dead fallback on"no text available for language"is dropped. Endpoints and the bearer header are resolved per call instead of at import time, so they no longer depend on when the environment was populated, and request bodies are built withencoding/jsoninstead of string interpolation.feat(history)— a nullabledetailscolumn oncron_historyholdingurl,sent,failedandmanual, exposed through/api/cron-history.LogCronExecutionkeeps its signature and delegates with no details, so the collect job is untouched; rows written before the column existed read back without details.feat(schedule)—RetryMessagePostpublishes one repository to the named integrations, fetching the text per integration in its configured language and returning a per-API outcome. With nourlit falls back to the most recently published repository, which is the item a partial run just consumed. The request building shared withMessageJobmoves intopublishItem, so the cron and the retry cannot drift apart.feat(api)—POST /api/message/retry, same middleware chain as every other route.Deliberate choices worth a look during review:
urlis given, the fetched item's url is compared against it and a mismatch is an error. An older content-alchemist ignores the url filter and answers with the head of the queue, which would otherwise publish the wrong repository silently.messagerun withdetails.manual = true, so existing name filters keep working.RemoveAllFilesInFolder, because the cron shares that directory and serves its image over/images/while its own request is in flight. It also uses a short retry budget via the newSocialifyWithConfig, since the cron default (5 attempts, 20s apart) would block a synchronous request for over a minute.Testing
go build ./... && go vet ./... && go test ./...— clean.TestGetRepositoryByURL: returns the requested item, refuses a different one, reports an empty result, requires a url.TestGetRepositorygained cases for a400rejection, a plain-text429, and a200carrying an error envelope — the case that previously asserted a500produces no error now asserts the opposite.TestRetryMessagePost*: only the requested integration is contacted, the text is fetched in that integration's language, per-API failures report "disabled" / "not configured", an empty api list is rejected without touching the history, and the history record carriesmanual: true.400/401/405/preflight behave; the scheduled cron writesdetails; and against a content-alchemist stub that replies with the 429 rejection, both the cron and the retry report the real cause instead of "no items".api_docs.mddocuments the endpoint and the newdetailsfield.