Skip to content

Chore/reconcile upgrade#1410

Open
nss10 wants to merge 31 commits into
calypr:developfrom
uc-cdis:chore/reconcile_upgrade
Open

Chore/reconcile upgrade#1410
nss10 wants to merge 31 commits into
calypr:developfrom
uc-cdis:chore/reconcile_upgrade

Conversation

@nss10

@nss10 nss10 commented May 11, 2026

Copy link
Copy Markdown
Contributor

Description

  • Modularize reconcile function such that it can be accessible to be invoked by a cron job, rather than just an internal ticker.
  • Add support to intelligently determine whether a job is supposed to be cleaned up when backoff Limit is set to a non zero value.

Motivation and Context

How Has This Been Tested?

Types of Changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have updated the documentation accordingly (link here).
  • I have tested that this feature locally.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • Reviewer has tested this feature locally

@lbeckman314 lbeckman314 added the safe to test Allow for external PR's to run integration tests in the Funnel repo label May 26, 2026
@nss10 nss10 marked this pull request as ready for review June 21, 2026 05:20
@lbeckman314 lbeckman314 self-requested a review June 22, 2026 02:41
@lbeckman314 lbeckman314 self-assigned this Jun 22, 2026
@lbeckman314 lbeckman314 added bug blocker blocks user interactions labels Jun 22, 2026
lbeckman314 added a commit that referenced this pull request Jun 22, 2026
Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

@lbeckman314 lbeckman314 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved in PR #1438 to allow Integration Tests to run!

matthewpeterkort pushed a commit that referenced this pull request Jul 8, 2026
* Add reconcile_modular draft code

* Improve reconciler modular code

* switch to reconclie_modular

* Add Debug logs for PVC deletiojn

* handle reconciler when job restarts happen

* cause exec_error to maje a sys_error

* Revert exec_err to sys_err, exec pod crash is no longer sys_err if job backof limit is not exhausted

* Remove lastAttempt check

* Remove TTLSecondsAfterFinished from worker pod.

* Set quay to cdis

* Remove old code and WIP comments

* TTLSecondsAfterFinished to worker job removed. Jobcleanup takes care of this.

* Print SQL logs to identify bug

* Add state filter in ListTasks

* Add state in the ListTasks tests

* Remove sql debug logs

* Add updated reconcile_monolith and modular reconciler

* Retry pods when job backoff limit is not reached yet

* Fix compiler error in reconciler_monolith

* Replace inner function to class level function and mark completed task in Running as System error

* Remove tests linked to TTLSecondsAfterFinished

* Update backofflimit check when task is in terminal state.

* task: Revert cancel response coe back to 200

* Update integration_tests_on_kind.yaml

* Fix isJobMarkedAsFailed for tasks that are completed.

* Add debug logs to understand the jobConditions for successful tasks

* isJobDone

* Final commits and comment improvements before making PR.

* debug: Integration tests

* ci: re-trigger integration tests

* ci: pin gen3-code-vigil to correct green commit 69c620a

891ea9e was the funnel merge commit, not gen3-code-vigil, so the
checkout failed with 'not our ref'. 69c620a (Gen3SDK Tests #558) is
the gen3-code-vigil commit from the last green run, and its conftest.py
predates the get_navigation_url() call that crashes on master.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Assisted-by: Claude Code:claude [Claude Code]

* debug: CI

* fix: Cleanup code

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

* task: Revert changes to better match PR #1410

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>

* chore: Unpin integration tests action

---------

Signed-off-by: Liam Beckman <lbeckman314@gmail.com>
Co-authored-by: Sai Shanmukha <nss10@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocker blocks user interactions bug safe to test Allow for external PR's to run integration tests in the Funnel repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants