Skip to content

feat(github_ci): speed up test runs with one binary build#5067

Merged
GheisMohammadi merged 9 commits into
devfrom
feature/speed-up-test-runs-with-one-binary-build
Jun 26, 2026
Merged

feat(github_ci): speed up test runs with one binary build#5067
GheisMohammadi merged 9 commits into
devfrom
feature/speed-up-test-runs-with-one-binary-build

Conversation

@mur-me

@mur-me mur-me commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Issue

What was done

  • Reworked CI so Harmony binaries are built once in the build job and reused by integration test jobs.
  • Built static harmony and bootnode binaries with make linux_static.
  • Uploaded only required binaries as a short-lived GitHub Actions artifact:
    • harmony
    • bootnode
  • Updated rpc-checker and pyhmy-checker to download and reuse the prebuilt binaries.
  • Removed duplicate Harmony binary builds from checker jobs.
  • Restored executable permissions after artifact download.
  • Updated localnet test execution to use prebuilt binaries with -B.
  • Started to build arm binary as well
  • Run RPC tests against arm binary.
  • upgrade setup go step to the 6.5.0 version

Note: pyhmy and unit tests will be covered separate, there are dependencies on the proto image and arm pyhmy cli

Why this is better

  • Avoids rebuilding the same binaries multiple times.
  • Makes RPC and pyhmy tests run against the exact binary produced by the build job.
  • Reduces CI time and wasted runner CPU.
  • Makes the CI flow easier to debug and reason about.

Conntected PRs

Test

Tests performed

  • Verified harmony and bootnode are built and uploaded as artifacts.
  • Verified checker jobs download the binaries into the expected bin/ path.
  • Verified binaries are executable after download.
  • Verified mounted binaries are visible inside the localnet container.
  • Verified RPC and pyhmy flows run with the prebuilt binary mode.

Test/Run Logs

https://github.com/harmony-one/harmony/actions/runs/28190779640

Diagram

flowchart LR
    subgraph BEFORE["Before"]
        A1["Build job builds harmony"]
        A2["RPC job builds harmony again"]
        A3["pyhmy job builds harmony again"]
        A4["localnet Dockerfile may build harmony again"]
        A5["run.sh may build again"]
        A6["Slow, duplicated, unclear tested binary"]
        A1 --> A2 --> A3 --> A4 --> A5 --> A6
    end

    subgraph NOW["Now"]
        B1["Build job builds static harmony + bootnode once"]
        B2["Upload harmony-binaries artifact"]
        B3["RPC downloads artifact"]
        B4["pyhmy downloads artifact"]
        B5["localnet runs with -B"]
        B6["Tests use same prebuilt binaries"]
        B1 --> B2
        B2 --> B3 --> B5 --> B6
        B2 --> B4 --> B5
    end

    BEFORE --> NOW
Loading

Operational Checklist

  1. Does this PR introduce backward-incompatible changes to the on-disk data structure and/or the over-the-wire protocol?. (If no, skip to question 8.)

    YES|NO

  2. Describe the migration plan.. For each flag epoch, describe what changes take place at the flag epoch, the anticipated interactions between upgraded/non-upgraded nodes, and any special operational considerations for the migration.

  3. Describe how the plan was tested.

  4. How much minimum baking period after the last flag epoch should we allow on Pangaea before promotion onto mainnet?

  5. What are the planned flag epoch numbers and their ETAs on Pangaea?

  6. What are the planned flag epoch numbers and their ETAs on mainnet?

    Note that this must be enough to cover baking period on Pangaea.

  7. What should node operators know about this planned change?

  8. Does this PR introduce backward-incompatible changes NOT related to on-disk data structure and/or over-the-wire protocol? (If no, continue to question 11.)

    YES|NO

  9. Does the existing node.sh continue to work with this change?

  10. What should node operators know about this change?

  11. Does this PR introduce significant changes to the operational requirements of the node software, such as >20% increase in CPU, memory, and/or disk usage?

TODO

@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch 3 times, most recently from 2b63cca to 2bcd724 Compare June 25, 2026 13:37
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch 2 times, most recently from 8249cb3 to 2e6fc42 Compare June 25, 2026 14:11
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch from 5bc0834 to 947a217 Compare June 25, 2026 15:08
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch from 947a217 to e199694 Compare June 25, 2026 15:08
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch from a0197e8 to c85a470 Compare June 25, 2026 16:51
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch 3 times, most recently from 4d0f62f to fbf001c Compare June 25, 2026 17:39
@mur-me mur-me force-pushed the feature/speed-up-test-runs-with-one-binary-build branch from fbf001c to b31336a Compare June 25, 2026 18:11
@mur-me mur-me changed the title feat(github_ci): add upload and download artifacts feat(github_ci): speed up test runs with one binary build Jun 26, 2026
@mur-me mur-me requested review from Frozen and GheisMohammadi June 26, 2026 08:36
@mur-me mur-me marked this pull request as ready for review June 26, 2026 08:39
@mergify

mergify Bot commented Jun 26, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@GheisMohammadi GheisMohammadi merged commit d8a1904 into dev Jun 26, 2026
9 checks passed
@mur-me mur-me deleted the feature/speed-up-test-runs-with-one-binary-build branch June 26, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants