Skip to content

Fix IllegalArgumentException in markExhausted for template-based instance configurations#563

Open
sombraSoft wants to merge 1 commit into
jenkinsci:developfrom
sombraSoft:fix-mark-exhausted-template
Open

Fix IllegalArgumentException in markExhausted for template-based instance configurations#563
sombraSoft wants to merge 1 commit into
jenkinsci:developfrom
sombraSoft:fix-mark-exhausted-template

Conversation

@sombraSoft

Copy link
Copy Markdown

When a GCE capacity error (ZONE_RESOURCE_POOL_EXHAUSTED) hits an instance-template-based configuration, markExhausted() throws while formatting its log message:

java.lang.IllegalArgumentException
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:127)
    at com.google.cloud.graphite.platforms.plugin.client.util.ClientUtil.nameFromSelfLink(ClientUtil.java:84)
    at com.google.jenkins.plugins.computeengine.InstanceConfiguration.markExhausted(InstanceConfiguration.java:501)
    at com.google.jenkins.plugins.computeengine.ComputeEngineComputerLauncher.launch(ComputeEngineComputerLauncher.java:209)
    at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)

Template-based configurations leave machineType empty (it is only read when no template is set), and nameFromSelfLink("") rejects empty input. Consequences (hit in production during a real us-central1-a stockout):

The fix guards the log message: use machineType when set, fall back to the template name for template-based configurations.

Resolves #561

Testing done

  • New unit test InstanceConfigurationTest#testMarkExhaustedDoesNotThrowForTemplateBasedConfiguration builds a configuration with an empty machine type and a template, calls markExhausted, and asserts the zone is marked. It fails before the fix (IllegalArgumentException) and passes after.
  • Full InstanceConfigurationTest class passes (28/28).
  • Verified in a production Jenkins instance (public OpenROAD CI): with machineType populated as a workaround, markExhausted succeeds and the fallback-zone logic provisioned correctly in another zone during a subsequent real stockout.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests that demonstrate the feature works or the issue is fixed

…ance configurations

Template-based instance configurations leave machineType empty (it is only
used when no template is set), but markExhausted's log message called
nameFromSelfLink(machineType), which rejects empty input. The exception
propagated out of ComputeEngineComputerLauncher.launch() before
terminateNode() ran, so the zone was never marked exhausted (defeating the
fallback-zone feature) and the dead node was relaunched in a loop by the
retention strategy.
@sombraSoft
sombraSoft requested a review from a team as a code owner July 8, 2026 04:47

@das7pad das7pad left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

(We also see this in our logs at work.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

markExhausted throws IllegalArgumentException for template-based instance configurations, breaking zone fallback

2 participants