cardano-diffusion: label ledger peer usage

[dancewithheart]

Description

Fixes #4683

Issue reference comment for ledgerPeers in genNodeArgs :

ouroboros-network/cardano-diffusion/tests/lib/Test/Cardano/Network/Diffusion/Testnet/Simulation.hs

Lines 450 to 451 in d842a23

	ledgerPeers :: [[NonEmpty LedgerRelayAccessPoint]]
	<- listOf1 (listOf1 (sublistOf1 (NonEmpty.fromList $ makeRelayAccessPoint <$> ledgerPeersRelays)))

genNodeArgs is used in generator for DiffusionScript:

ouroboros-network/cardano-diffusion/tests/lib/Test/Cardano/Network/Diffusion/Testnet/Simulation.hs

Line 677 in d842a23

nodeArgs <- genNodeArgs relays' minConnected localRts relay txs

It is used in many properties, but most likely target seems prop_peer_selection_trace_coverage that already tabulate TracePeerSelection events. I added another tabulate - over events with Set peeraddr (and not on single peeraddr):

cabal run cardano-diffusion:test:cardano-diffusion-sim-tests -- -p '/peer selection trace coverage/'

now shows:

cardano-diffusion-sim-tests
  Ouroboros.Network.Testnet
    IOSim
      coverage
        peer selection trace coverage: OK (35.13s)
          +++ OK, passed 100 tests.                                                                                                                                                         
          
          distribution of used ledger peers (8350 in total):
          63.15% TraceBigLedgerPeersResults 0
          26.49% TracePromoteColdBigLedgerPeers 1
           8.90% TraceBigLedgerPeersResults 1
           1.20% TraceForgetBigLedgerPeers 1
           0.18% TracePromoteWarmBigLedgerPeers 1
           0.08% TraceBigLedgerPeersResults 2
          
          peer selection trace (65219 in total):
          16.330% TraceGovernorWakeup
          12.394% TracePromoteColdFailed

Checklist

Quality

• Commit sequence makes sense and have useful messages, see ref.
• New tests are added and existing tests are updated.
• Self-reviewed the PR.

Maintenance

• Linked an issue or added the PR to the current sprint of ouroboros-network project.
• Added labels.
• Updated changelog files.
• The documentation has been properly updated, see ref.

[coot]

Thanks. Note that this only adds statistics for big ledger peers. For ledger peers you'd need to use

• TracePublicRootsResults
• for promotions of warm ledger peers, we don't have a separate event (like we do for ledger peers), so one would need to use a combination of TracePromoteWarmPeers and DebugPeerSelectionState (PeerSelectionState contains publicRootPeers field, which has access to getLedgerPeers - set of ledger peers).
• probably a similar approach is needed for other ledger traces.

We're getting 63% of time TraceBigLedgerPeersResults 0 - that's something to investigate.

[dancewithheart]

Thank you for the review and remarks.

I updated this to split the output into two labels:

• ledger peer result sizes
• ledger peers selected by actions

cardano-diffusion-sim-tests
  Ouroboros.Network.Testnet
    IOSim
      coverage
        peer selection trace coverage: OK (36.13s)
          +++ OK, passed 100 tests.                                                                                                                                                                                                 
          
          ledger peer result sizes (9350 in total):
          57.47% TraceBigLedgerPeersResults 0
          34.46% TracePublicRootsResults ledger peers 0
           6.73% TraceBigLedgerPeersResults 1
           1.30% TracePublicRootsResults ledger peers 1
           0.04% TraceBigLedgerPeersResults 2
          
          ledger peers selected by actions (1755 in total):
          93.45% TracePromoteColdBigLedgerPeers 1
           3.87% TraceForgetBigLedgerPeers 1
           2.68% TracePromoteWarmBigLedgerPeers 1
          
          peer selection trace (65943 in total):
          15.433% TraceGovernorWakeup
          14.466% TraceUseBootstrapPeersChanged
          11.684% TracePromoteColdFailed
           9.713% TracePromoteColdLocalPeers
           9.120% TraceBigLedgerPeersRequest
           9.108% TraceBigLedgerPeersResults
           5.083% TracePublicRootsRequest
           5.071% TracePublicRootsResults
           3.313% TraceTargetsChanged
           2.514% TracePromoteColdPeers
           2.487% TracePromoteColdBigLedgerPeers
           2.425% TraceLocalRootPeersChanged
           2.264% TracePromoteColdBigLedgerPeerFailed
           2.068% TraceChurnWait
...

TracePublicRootsResults now contributes normal ledger-peer result sizes via PublicRootPeers.getLedgerPeers.

For generic peer actions such as TracePromoteWarmPeers, the selected peer set is classified using the ledger-peer set from the most recent TraceDebugState / dpssPublicRootPeers.

Before the first debug-state snapshot, generic actions are left unclassified rather than counted as selecting zero ledger peers. This is done in last commit - without this it looked like so:

          ledger peer result sizes (8964 in total):
          59.86% TraceBigLedgerPeersResults 0
          31.05% TracePublicRootsResults ledger peers 0
           7.74% TraceBigLedgerPeersResults 1
           1.32% TracePublicRootsResults ledger peers 1
           0.03% TraceBigLedgerPeersResults 2

          ledger peers selected by actions (4336 in total):
          50.62% TracePromoteColdBigLedgerPeers 1
          41.21% TracePromoteColdPeers ledger peers 0
           4.68% TraceForgetColdPeers ledger peers 0
           1.66% TracePromoteWarmBigLedgerPeers 1
           1.45% TraceForgetBigLedgerPeers 1
           0.32% TracePromoteWarmPeers ledger peers 0
           0.02% TraceDemoteHotBigLedgerPeers 1
           0.02% TraceDemoteWarmBigLedgerPeers 1

We're getting 63% of time TraceBigLedgerPeersResults 0 - that's something to investigate.

Do you want the investigation of this to be part of this PR?

[coot]

LGTM

[coot]

This is not tail recursive, but surprisingly this doesn't inflate heap usage.