Builds hang in "Building SRPM"

[TomasTomecek]

cockpit-project/cockpit-podman#649
rhinstaller/anaconda#3170

Sentry is silent, gonna check logs what went wrong.

[TomasTomecek]

so the anaconda PR is actually correct: there is only ELN in the list so I assume that after a rebase, rawhide was removed from the list

[TomasTomecek]

No longer reproducible :/ the cockpit PR does not have correct testing status - it should probably be failed, I'm assuming this is TFT shortcoming

There is another cockpit-podman PR: cockpit-project/cockpit-podman#676 and it has correct build statuses so far

[jkonecny12]

so the anaconda PR is actually correct: there is only ELN in the list so I assume that after a rebase, rawhide was removed from the list

I've updated the settings https://github.com/rhinstaller/anaconda/blob/f34-devel/.packit.yml#L16 to run f34-devel branch tests on fedora-development which should work for Rawhide and Fedora-34.

[lbarcziova]

For the rhinstaller/anaconda#3170, the related configuration from the PR:

jobs:
  - job: tests
    trigger: pull_request
    metadata:
      targets:
        - fedora-rawhide
  # enable also smoke tests on Fedora ELN but right now it's not supported by Cruncher
  - job: copr_build
    trigger: pull_request
    metadata:
      targets:
        - fedora-eln

What was set:

• packit/rpm-build-fedora-eln-x86_64 - RPMs failed to be built.
• packit/testing-farm-fedora-rawhide-x86_64 - Building SRPM...

We set Building SRPM... status for all build and test targets configured in packit config file, therefore it was set for rawhide as test check and eln as build check.
But we run copr builds only for configured copr build targets and then set the status for corresponding build and test checks. Since there is not defined rawhide for copr build, rawhide check remained at Building SRPM ... state and no copr build and TF test was run.

We should run the build for rawhide as well, since we want to run tests on it, WDYT? Reproduced here.

---

The second PR cockpit-project/cockpit-podman#649 - I tried to investigate that build triggered 14 days ago, but all I discovered:

• status was set to Building SRPM...
• there is no corresponding SRPM model for this date and no Copr build
• there is nothing reported in Sentry related to this PR from that date
• since there is no SRPM model, but status was set to Building SRPM..., probably something went wrong in _create_srpm, but we catch the exceptions when building SRPM and send it to sentry, so this is a mystery for me, will think about it more.

[lachmanfrantisek]

@lbarcziova thanks for the investigation!

Regarding the setup Anaconda is using:

• In general, we should run build for rawhide as well.
• Specifically, they probably want to change fedora-rawhide to fedora-eln in the test job definition. (You can send PR or just make a comment so they know about it.)

[jkonecny12]

I guess the same as in Anaconda is happening also here: rhinstaller/python-simpleline#83

Only ELN got status which is because it is not set for tests but only for COPR build.

[jkonecny12]

@lbarcziova thanks for the investigation!

Regarding the setup Anaconda is using:

• In general, we should run build for rawhide as well.
• Specifically, they probably want to change fedora-rawhide to fedora-eln in the test job definition. (You can send PR or just make a comment so they know about it.)

Fedora-ELN is not supported by TFT so it would get failure / not getting result. It's blocked by fedora-eln/eln#19 . That is the reason why ELN is not in the test job.

[lbarcziova]

I guess the same as in Anaconda is happening also here: rhinstaller/python-simpleline#83

Yes, right. I am working on a fix for this. Until it gets to production, unfortunately, the only hack to make the test work for the other targets is to add them also to the copr-build configuration.

[jkonecny12]

The Simpleline project is not under heavy development so no problem with waiting for the fix there.

[lbarcziova]

The bug about using wrong build targets is fixed and the fix is in production. Regarding the second PR, I haven't found more evidence and it has not happened since then. I am closing this now and if it occurs in the future, we can reopen this issue.