Skip to content

chore: Run CI on changed files; add retries & gating #1614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
edwinjosechittilappilly wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: Run CI on changed files; add retries & gating #1614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 48 additions & 6 deletions .github/workflows/lint-backend.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,48 @@ name: Lint Backend

on:
pull_request:
paths:
- 'src/**/*.py'
- 'tests/**/*.py'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/lint-backend.yml'
merge_group:

# Cancel in-flight runs on the same PR when a new push lands.
concurrency:
group: lint-backend-${{ github.ref }}
cancel-in-progress: true

jobs:
changes:
runs-on: ubuntu-latest
outputs:
run: ${{ steps.decision.outputs.run }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
Comment on lines +18 to +19
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".github/workflows/lint-backend.yml"
if [ ! -f "$FILE" ]; then
  echo "Missing $FILE"
  exit 1
fi

echo "== File: $FILE =="
nl -ba "$FILE" | sed -n '1,120p'

echo "== Relevant search in repo =="
rg -n "persist-credentials" .github/workflows || true
rg -n "actions/checkout@" .github/workflows || true
rg -n "dorny/paths-filter@" .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 172

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".github/workflows/lint-backend.yml"
if [ ! -f "$FILE" ]; then
  echo "Missing $FILE"
  exit 1
fi

echo "== File: $FILE =="
nl -ba "$FILE" | sed -n '1,120p'

echo "== Relevant search in repo =="
rg -n "persist-credentials" .github/workflows || true
rg -n "actions/checkout@" .github/workflows || true
rg -n "dorny/paths-filter@" .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 172

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".github/workflows/lint-backend.yml"
if [ ! -f "$FILE" ]; then
  echo "Missing $FILE"
  exit 1
fi

echo "== File: $FILE (first 160 lines) =="
cat -n "$FILE" | sed -n '1,160p'

echo "== Relevant searches in .github/workflows =="
rg -n "persist-credentials" .github/workflows || true
rg -n "actions/checkout@" .github/workflows || true
rg -n "dorny/paths-filter@" .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 8349

Harden .github/workflows/lint-backend.yml changes job actions (pin + disable checkout credential persistence).

The changes job uses tag-based actions (actions/checkout@v4, dorny/paths-filter@v3) and checkout doesn’t set persist-credentials, increasing token/credential exposure risk.

Proposed hardening patch 
-      - uses: actions/checkout@v4
-      - uses: dorny/paths-filter@v3
+      - uses: actions/checkout@<pinned-sha>
+        with:
+          persist-credentials: false
+      - uses: dorny/paths-filter@<pinned-sha>
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 18-18: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 18-18: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 19-19: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/lint-backend.yml around lines 18 - 19, The workflow
currently references tag-based actions which increases supply-chain risk and the
checkout step does not disable credential persistence; update the uses entries
for actions/checkout and dorny/paths-filter to pinned commit SHAs (replace
actions/checkout@v4 and dorny/paths-filter@v3 with their corresponding full
commit SHA pins) and add a with: persist-credentials: false to the
actions/checkout step (so the checkout action referenced by the pinned SHA does
not persist the GITHUB_TOKEN). Ensure the unique symbols/actions mentioned
(actions/checkout and dorny/paths-filter) are the ones you modify.

id: filter
if: github.event_name == 'pull_request'
with:
filters: |
relevant:
- 'src/**/*.py'
- 'tests/**/*.py'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/lint-backend.yml'
- id: decision
run: |
# Skip on merge_group: lint runs on changed files only, so
# re-running on the queued ref would just re-check the same set
# the PR already passed. The required job below still reports
# green so the merge queue is not blocked.
if [ "${{ github.event_name }}" = "merge_group" ]; then
echo "run=false" >> "$GITHUB_OUTPUT"
Comment on lines +32 to +37
elif [ "${{ steps.filter.outputs.relevant }}" = "true" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi

lint:
needs: changes
if: needs.changes.outputs.run == 'true'
name: Ruff and mypy on changed files
runs-on: ubuntu-latest

Expand Down Expand Up @@ -87,3 +115,17 @@ jobs:
if [ "${#test_files[@]}" -gt 0 ]; then
uv run mypy "${test_files[@]}"
fi

required:
needs: lint
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify lint result
run: |
result="${{ needs.lint.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "lint job did not succeed: $result"
exit 1
fi
echo "lint result: $result (success or skipped — passing)"
Comment on lines +119 to +131
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Include changes result in the required gate.

required only validates needs.lint.result. If changes fails, lint is skipped and this job still passes, which can hide a real CI failure.

Proposed fix 
 required:
-  needs: lint
+  needs: [changes, lint]
   if: always()
   runs-on: ubuntu-latest
   steps:
     - name: Verify lint result
       run: |
-        result="${{ needs.lint.result }}"
-        if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
-          echo "lint job did not succeed: $result"
+        changes_result="${{ needs.changes.result }}"
+        lint_result="${{ needs.lint.result }}"
+        if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
+          echo "changes job did not succeed: $changes_result"
+          exit 1
+        fi
+        if [ "$lint_result" = "failure" ] || [ "$lint_result" = "cancelled" ]; then
+          echo "lint job did not succeed: $lint_result"
           exit 1
         fi
-        echo "lint result: $result (success or skipped — passing)"
+        echo "changes=$changes_result, lint=$lint_result (success or skipped — passing)"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
required:
needs: lint
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify lint result
run: |
result="${{ needs.lint.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "lint job did not succeed: $result"
exit 1
fi
echo "lint result: $result (success or skipped — passing)"
required:
needs: [changes, lint]
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify lint result
run: |
changes_result="${{ needs.changes.result }}"
lint_result="${{ needs.lint.result }}"
if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
echo "changes job did not succeed: $changes_result"
exit 1
fi
if [ "$lint_result" = "failure" ] || [ "$lint_result" = "cancelled" ]; then
echo "lint job did not succeed: $lint_result"
exit 1
fi
echo "changes=$changes_result, lint=$lint_result (success or skipped — passing)"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/lint-backend.yml around lines 119 - 131, The Verify lint
result step currently only checks ${{ needs.lint.result }} so a skipped lint due
to a failing changes job can be overlooked; update that step to also inspect ${{
needs.changes.result }} (from the 'changes' job) by capturing both into
variables (e.g. result_lint and result_changes) and failing the step (exit 1) if
either equals "failure" or "cancelled", otherwise print a passing message; keep
the step name "Verify lint result" and update the conditional logic accordingly.

43 changes: 40 additions & 3 deletions .github/workflows/lint-frontend.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,36 @@ name: Frontend Lint

on:
pull_request:
paths:
- 'frontend/**'
- '.github/workflows/lint-frontend.yml'
merge_group:

jobs:
changes:
runs-on: ubuntu-latest
outputs:
run: ${{ steps.decision.outputs.run }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
Comment on lines +13 to +14
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show the relevant workflow section with line numbers
FILE=".github/workflows/lint-frontend.yml"
if [ ! -f "$FILE" ]; then
  echo "Missing $FILE"
  exit 1
fi

echo "---- $FILE (lines 1-80) ----"
nl -ba "$FILE" | sed -n '1,80p'

echo "---- Find all checkout and paths-filter usages ----"
rg -n "uses:\s*(actions/checkout|dorny/paths-filter)@" "$FILE" || true
rg -n "persist-credentials" "$FILE" || true

Repository: langflow-ai/openrag

Length of output: 184

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE=".github/workflows/lint-frontend.yml"
if [ ! -f "$FILE" ]; then
  echo "Missing $FILE"
  exit 1
fi

echo "---- $FILE (first 120 lines with numbers) ----"
# Use cat -n since nl is missing
cat -n "$FILE" | sed -n '1,120p'

echo "---- Find all checkout and paths-filter usages ----"
if command -v rg >/dev/null 2>&1; then
  rg -n "uses:\s*(actions/checkout|dorny/paths-filter)@" "$FILE" || true
  rg -n "persist-credentials" "$FILE" || true
else
  grep -nE "uses:\s*(actions/checkout|dorny/paths-filter)@" "$FILE" || true
  grep -n "persist-credentials" "$FILE" || true
fi

Repository: langflow-ai/openrag

Length of output: 3197

🌐 Web query:

actions/checkout persist-credentials default true

💡 Result:

The actions/checkout GitHub Action has a default value of true for the persist-credentials input [1][2][3]. When persist-credentials is set to its default (true), the action configures the authentication token (such as GITHUB_TOKEN) or SSH key in the local git configuration of the checked-out repository [1][4][3]. This allows subsequent scripts within the same job to execute authenticated git commands directly [1][2]. The action is designed to remove these credentials from the local git configuration during the post-job cleanup phase [1][3]. However, security concerns have been raised regarding this behavior [5][6]. Because the credentials are written to the repository's.git/config file, they become accessible to any steps or processes running in the job [7][6]. In scenarios where artifacts are created and uploaded from the workspace, these credentials can potentially be inadvertently leaked [6]. To improve security and adhere to the principle of least privilege, it is recommended to explicitly set persist-credentials: false for any workflow steps that do not require authenticated git push capabilities [7][6]. Example usage to disable this behavior: - uses: actions/checkout@v4 with: persist-credentials: false

Citations:

🌐 Web query:

GitHub Actions security best practice pin actions to commit SHA instead of tag

💡 Result:

Pinning GitHub Actions to a full-length commit SHA is a recommended security best practice to ensure workflow immutability and protect against supply-chain attacks [1][2][3]. Why Pinning to SHA is Required GitHub Actions tags (e.g., @v4) and branches (e.g., @main) are mutable references [1][4][5]. This means a maintainer—or a malicious actor who compromises a maintainer's account—can update a tag or push new code to a branch, causing your workflow to run different, potentially malicious, code without any changes to your repository [1][4][6]. Pinning to a full-length commit SHA ensures that your workflow consistently runs the exact same, verified code [2][5]. Recommended Implementation Practices 1. Pin to full-length SHAs: Use the full-length commit SHA to ensure absolute immutability [2][7]. 2. Use version comments: Because SHAs are not human-readable, append the version tag as a comment on the same line to maintain clarity [7][4]. Example: - uses: actions/checkout@b4ffde6 # v4.1.1 3. Automate updates: Manual maintenance of SHAs is error-prone. Use tools like Renovate (with the helpers:pinGitHubActionDigests preset) or Dependabot to automatically generate pull requests that update your pinned SHAs when new versions are released [4][6]. 4. Enforce via Policy: GitHub allows organization and repository administrators to mandate SHA pinning [2][3]. When this policy is enabled, workflows using tags will fail, preventing the accidental use of unpinned actions [4][5]. Important Considerations While SHA pinning is a critical defense, it is not a complete security solution [8]. It ensures that you run the code you expect, but it does not inherently validate the security of the action itself [8]. Furthermore, it does not automatically pin transitive dependencies—if an action you pin references other actions internally via tags, those dependencies remain mutable [8]. Consequently, you should still practice least-privilege for workflow permissions, use OpenID Connect (OIDC) for cloud authentication instead of long-lived secrets, and audit high-risk workflows [9][10][11].

Citations:

Pin workflow actions and set persist-credentials: false for checkout.

  • .github/workflows/lint-frontend.yml uses mutable tags in the changes job (actions/checkout@v4, dorny/paths-filter@v3); pin both to commit SHAs.
  • actions/checkout is run without with: persist-credentials: false; default behavior keeps credentials in the job’s git config, increasing blast radius—apply persist-credentials: false to all actions/checkout steps (also present in biome and typecheck).
Proposed hardening patch 
-      - uses: actions/checkout@v4
-      - uses: dorny/paths-filter@v3
+      - uses: actions/checkout@<pinned-sha>
+        with:
+          persist-credentials: false
+      - uses: dorny/paths-filter@<pinned-sha>
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 13-13: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 13-13: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 14-14: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/lint-frontend.yml around lines 13 - 14, Replace mutable
action tags with pinned commit SHAs for the used actions (replace
actions/checkout@v4 and dorny/paths-filter@v3 with their respective commit SHA
pins) and add persist-credentials: false to every actions/checkout step (the
checkout invocation used in the changes job and also the biome and typecheck
jobs); ensure the checkout steps maintain any existing with: entries but include
persist-credentials: false to avoid leaving repo credentials in the job, and
update the action versions to the exact commit SHAs for actions/checkout and
dorny/paths-filter.

id: filter
if: github.event_name == 'pull_request'
with:
filters: |
relevant:
- 'frontend/**'
- '.github/workflows/lint-frontend.yml'
- id: decision
run: |
if [ "${{ github.event_name }}" = "merge_group" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
elif [ "${{ steps.filter.outputs.relevant }}" = "true" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi

biome:
needs: changes
if: needs.changes.outputs.run == 'true'
name: Biome
runs-on: ubuntu-latest
steps:
Expand Down Expand Up @@ -48,3 +72,16 @@ jobs:
- name: TypeScript type check
working-directory: frontend
run: npm run typecheck
required:
needs: biome
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify biome result
run: |
result="${{ needs.biome.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "biome job did not succeed: $result"
exit 1
fi
echo "biome result: $result (success or skipped — passing)"
Comment on lines +75 to +87
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fail the required job when changes fails.

Current guard only inspects needs.biome.result. A failed changes job can still yield a green required result via skipped biome.

Proposed fix 
 required:
-  needs: biome
+  needs: [changes, biome]
   if: always()
   runs-on: ubuntu-latest
   steps:
     - name: Verify biome result
       run: |
-        result="${{ needs.biome.result }}"
-        if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
-          echo "biome job did not succeed: $result"
+        changes_result="${{ needs.changes.result }}"
+        biome_result="${{ needs.biome.result }}"
+        if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
+          echo "changes job did not succeed: $changes_result"
+          exit 1
+        fi
+        if [ "$biome_result" = "failure" ] || [ "$biome_result" = "cancelled" ]; then
+          echo "biome job did not succeed: $biome_result"
           exit 1
         fi
-        echo "biome result: $result (success or skipped — passing)"
+        echo "changes=$changes_result, biome=$biome_result (success or skipped — passing)"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
required:
needs: biome
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify biome result
run: |
result="${{ needs.biome.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "biome job did not succeed: $result"
exit 1
fi
echo "biome result: $result (success or skipped — passing)"
required:
needs: [changes, biome]
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify biome result
run: |
changes_result="${{ needs.changes.result }}"
biome_result="${{ needs.biome.result }}"
if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
echo "changes job did not succeed: $changes_result"
exit 1
fi
if [ "$biome_result" = "failure" ] || [ "$biome_result" = "cancelled" ]; then
echo "biome job did not succeed: $biome_result"
exit 1
fi
echo "changes=$changes_result, biome=$biome_result (success or skipped — passing)"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/lint-frontend.yml around lines 55 - 67, The guard in the
required job only checks needs.biome.result and misses failures from the changes
job; update the run script in the required job (the steps under the job named
"required" that currently reads needs.biome.result) to also inspect
needs.changes.result (or the appropriate needs.<job>.result for your changes
job) and treat "failure" or "cancelled" from either job as a failing
condition—i.e., fetch both results, if either equals "failure" or "cancelled"
echo a descriptive message and exit 1, otherwise continue and print a passing
message.

80 changes: 68 additions & 12 deletions .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,16 @@ name: E2E Tests

on:
pull_request:
paths:
- 'src/**'
- 'frontend/**'
- 'tests/**'
- 'scripts/**'
- 'flows/**'
- 'docker-compose.yml'
- 'Dockerfile*'
- 'Makefile'
- '.github/workflows/test-e2e.yml'
merge_group:
workflow_dispatch:

# Serialize E2E and Integration on the same PR/queue ref. They share runner
# pool and Docker resources; running them concurrently inflates the flake
# rate. Different PRs have different refs and run in parallel as usual.
concurrency:
group: heavy-tests-${{ github.ref }}
cancel-in-progress: false

env:
NODE_VERSION: "22"
PYTHON_VERSION: "3.13"
Expand All @@ -23,7 +21,40 @@ env:
PLAYWRIGHT_VERSION: "1.57.0"

jobs:
changes:
runs-on: ubuntu-latest
outputs:
run: ${{ steps.decision.outputs.run }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
Comment on lines +29 to +30
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the target workflow with line numbers around the reported locations
echo "=== test-e2e.yml lines 1-200 (for context) ==="
nl -ba .github/workflows/test-e2e.yml | sed -n '1,200p'

echo "=== test-e2e.yml: uses: lines ==="
rg -n "^\s*- uses: " .github/workflows/test-e2e.yml

echo "=== Search other workflows for similar action usage patterns ==="
rg -n --glob ".github/workflows/*.yml" -S "actions/checkout@|dorny/paths-filter@|nick-fields/retry@" .github/workflows || true

echo "=== Search for any pinned SHAs for these actions across workflows ==="
rg -n --glob ".github/workflows/*.yml" -S "actions/checkout@([0-9a-f]{40})|dorny/paths-filter@([0-9a-f]{40})|nick-fields/retry@([0-9a-f]{40})" .github/workflows || true

echo "=== Check if repository has any workflow hardening policy docs ==="
ls -la .github || true
ls -la .github/workflows || true
rg -n "pinn?ed|immutable SHAs|supply chain|workflow policy" .github || true

Repository: langflow-ai/openrag

Length of output: 171

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== test-e2e.yml: first 200 lines (with line numbers) ==="
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/test-e2e.yml | sed -n '1,200p'

echo "=== test-e2e.yml: all lines with '- uses:' ==="
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/test-e2e.yml | rg -n "^\s*\d+\t-\s+uses:"

echo "=== Search other workflows for actions/checkout@, dorny/paths-filter@, nick-fields/retry@ ==="
rg -n --glob ".github/workflows/*.yml" -S "actions/checkout@|dorny/paths-filter@|nick-fields/retry@" .github/workflows || true

echo "=== Search for pinned SHAs for these actions across workflows ==="
rg -n --glob ".github/workflows/*.yml" -S "actions/checkout@[0-9a-f]{40}|dorny/paths-filter@[0-9a-f]{40}|nick-fields/retry@[0-9a-f]{40}" .github/workflows || true

echo "=== Check for any existing workflow hardening policy docs ==="
rg -n "pinn?ed|immutable SHAs|supply chain|workflow policy" .github || true
ls -la .github || true
ls -la .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 8366

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== test-e2e.yml: all '- uses:' lines ==="
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/test-e2e.yml | rg '^\s*[0-9]+\t-\s+uses:' || true

echo "=== Other workflows: occurrences of checkout/paths-filter/retry ==="
rg -n --glob ".github/workflows/*.yml" -S "(actions/checkout@|dorny/paths-filter@|nick-fields/retry@)" .github/workflows || true

echo "=== Other workflows: pinned SHA occurrences for those actions ==="
rg -n --glob ".github/workflows/*.yml" -S "(actions/checkout@[0-9a-f]{40}|dorny/paths-filter@[0-9a-f]{40}|nick-fields/retry@[0-9a-f]{40})" .github/workflows || true

echo "=== Search for supply-chain pinning policy/docs ==="
rg -n "pinn?ed|immutable SHAs|supply chain|workflow policy|dependabot|renovate.*workflow|actions/checkout@.*[0-9a-f]{40}" .github || true

Repository: langflow-ai/openrag

Length of output: 3174

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== Any pinned SHAs for actions/checkout/dorny/paths-filter/nick-fields/retry across workflows? ==="
rg -n --glob ".github/workflows/*.yml" -S "actions/checkout@[0-9a-f]{40}|dorny/paths-filter@[0-9a-f]{40}|nick-fields/retry@[0-9a-f]{40}" .github/workflows || true

echo "=== Any mention of pinning/immutable SHAs/workflow policy/dependabot/renovate in .github ==="
rg -n "pinn?ed|immutable SHAs|supply chain|workflow policy|dependabot|renovate" .github || true

echo "=== Check test-e2e.yml for submodules/persist-credentials/token usage ==="
rg -n "submodule|persist-credentials|token:|GITHUB_TOKEN" .github/workflows/test-e2e.yml || true

echo "=== Check test-e2e.yml for git commands needing auth ==="
rg -n "\bgit (clone|fetch|checkout|submodule|push|tag|config)\b" .github/workflows/test-e2e.yml || true

Repository: langflow-ai/openrag

Length of output: 387

Pin CI action refs to immutable commit SHAs (actions/checkout, dorny/paths-filter, nick-fields/retry)

In .github/workflows/test-e2e.yml, these steps use moving version tags (@v4, @v3), keeping the supply chain mutable:

  • actions/checkout@v4 (lines 29-30, and “Checkout” step)
  • dorny/paths-filter@v3 (lines 29-30)
  • nick-fields/retry@v3 (line 133)
Proposed hardening patch 
-      - uses: actions/checkout@v4
-      - uses: dorny/paths-filter@v3
+      - uses: actions/checkout@<pinned-sha>
+        with:
+          persist-credentials: false
+      - uses: dorny/paths-filter@<pinned-sha>
...
-      - name: Run Playwright tests
-        uses: nick-fields/retry@v3
+      - name: Run Playwright tests
+        uses: nick-fields/retry@<pinned-sha>
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 29-29: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 29-29: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 30-30: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-e2e.yml around lines 29 - 30, Update the GitHub
Actions steps that use mutable tags to use immutable commit SHAs: replace uses:
actions/checkout@v4, uses: dorny/paths-filter@v3, and uses: nick-fields/retry@v3
with the corresponding pinned commit SHAs for each action; locate the steps
referencing actions/checkout, dorny/paths-filter, and nick-fields/retry in
test-e2e.yml and substitute the tag with the verified commit SHA (and update any
references in the "Checkout" step) so the workflow uses fixed, auditable action
revisions.

id: filter
if: github.event_name == 'pull_request'
with:
filters: |
relevant:
- 'src/**'
- 'frontend/**'
- 'tests/**'
- 'scripts/**'
- 'flows/**'
- 'docker-compose.yml'
- 'Dockerfile*'
- 'Makefile'
- '.github/workflows/test-e2e.yml'
- id: decision
run: |
if [ "${{ github.event_name }}" = "merge_group" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
elif [ "${{ steps.filter.outputs.relevant }}" = "true" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi

e2e:
needs: changes
if: needs.changes.outputs.run == 'true'
runs-on:
labels: ["self-hosted", "linux", "ARM64", "langflow-ai-arm64-40gb-ephemeral-sudo"]
env:
Expand Down Expand Up @@ -99,7 +130,7 @@ jobs:


- name: Run Playwright tests
working-directory: frontend
uses: nick-fields/retry@v3
env:
CI: "true"
OPENSEARCH_HOST: localhost
Expand All @@ -108,7 +139,18 @@ jobs:
OPENSEARCH_PASSWORD: ${{ env.OPENSEARCH_PASSWORD }}
GOOGLE_OAUTH_CLIENT_ID: ""
GOOGLE_OAUTH_CLIENT_SECRET: ""
run: npx playwright test
with:
timeout_minutes: 60
max_attempts: 2
retry_on: error
command: |
cd frontend
npx playwright test
on_retry_command: |
# Services are still up — Playwright handles its own fixture
# lifecycle. Just clear stale artifacts so the retry's report
# isn't contaminated by the first attempt's output.
rm -rf frontend/test-results frontend/playwright-report || true

- name: Collect service logs on failure
if: failure()
Expand Down Expand Up @@ -150,3 +192,17 @@ jobs:
docker rm -f openrag-backend-proxy 2>/dev/null || true
make clean || true
docker system prune -f || true

required:
needs: e2e
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify e2e result
run: |
result="${{ needs.e2e.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "e2e job did not succeed: $result"
exit 1
fi
echo "e2e result: $result (success or skipped — passing)"
Comment on lines +196 to +208
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

required should validate both changes and e2e outcomes.

Right now a failed changes job can still surface as green because only needs.e2e.result is evaluated.

Proposed fix 
 required:
-  needs: e2e
+  needs: [changes, e2e]
   if: always()
   runs-on: ubuntu-latest
   steps:
     - name: Verify e2e result
       run: |
-        result="${{ needs.e2e.result }}"
-        if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
-          echo "e2e job did not succeed: $result"
+        changes_result="${{ needs.changes.result }}"
+        e2e_result="${{ needs.e2e.result }}"
+        if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
+          echo "changes job did not succeed: $changes_result"
+          exit 1
+        fi
+        if [ "$e2e_result" = "failure" ] || [ "$e2e_result" = "cancelled" ]; then
+          echo "e2e job did not succeed: $e2e_result"
           exit 1
         fi
-        echo "e2e result: $result (success or skipped — passing)"
+        echo "changes=$changes_result, e2e=$e2e_result (success or skipped — passing)"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-e2e.yml around lines 196 - 208, The verification step
in the "required" job only checks needs.e2e.result and ignores the outcome of
the needs.changes job, allowing a failed changes job to still show green; update
the Verify e2e result step (the script in the "required" job) to also read and
validate needs.changes.result (in addition to needs.e2e.result) and fail if
either result is "failure" or "cancelled" — ensure the script references both
needs.e2e.result and needs.changes.result and returns non‑zero when any of those
indicate failure/cancellation.

98 changes: 78 additions & 20 deletions .github/workflows/test-integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,7 @@ name: Integration Tests

on:
pull_request:
paths:
- 'src/**.py'
- 'tests/**.py'
- 'pyproject.toml'
- 'uv.lock'
- 'sdks/**'
- 'flows/**'
- '.github/workflows/test-integration.yml'
merge_group:
workflow_dispatch:
inputs:
use_local_images:
Expand All @@ -18,8 +11,46 @@ on:
type: boolean
default: true

# Serialize E2E and Integration on the same PR/queue ref. They share runner
# pool and Docker resources; running them concurrently inflates the flake
# rate. Different PRs have different refs and run in parallel as usual.
concurrency:
group: heavy-tests-${{ github.ref }}
cancel-in-progress: false

jobs:
changes:
runs-on: ubuntu-latest
outputs:
run: ${{ steps.decision.outputs.run }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
Comment on lines +27 to +28
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

# Show the workflow with line numbers (file should be small enough)
echo "=== test-integration.yml (with line numbers) ==="
nl -ba .github/workflows/test-integration.yml | sed -n '1,200p'

echo "=== Search for action uses across workflows ==="
rg -n "uses:\s*(actions/checkout|dorny/paths-filter|nick-fields/retry)@" .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 173

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "=== test-integration.yml (with line numbers) ==="
cat -n .github/workflows/test-integration.yml | sed -n '1,200p'

echo "=== Search for action uses across workflows ==="
rg -n "uses:\s*(actions/checkout|dorny/paths-filter|nick-fields/retry)@" .github/workflows || true

Repository: langflow-ai/openrag

Length of output: 10086

Pin checkout / paths-filter / retry action refs to SHAs and disable checkout credential persistence

Current workflow uses mutable tags (actions/checkout@v4, dorny/paths-filter@v3, nick-fields/retry@v3). Pin these to commit SHAs, and set with: persist-credentials: false on the actions/checkout steps (it appears at both the early checkout and the later “Checkout” step).

File: .github/workflows/test-integration.yml
Lines: 27-28 (also actions/checkout@v4 at 86-88, nick-fields/retry@v3 at 111-112)

      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
Proposed hardening patch 
-      - uses: actions/checkout@v4
-      - uses: dorny/paths-filter@v3
+      - uses: actions/checkout@<pinned-sha>
+        with:
+          persist-credentials: false
+      - uses: dorny/paths-filter@<pinned-sha>
...
-      - name: Run integration tests
-        uses: nick-fields/retry@v3
+      - name: Run integration tests
+        uses: nick-fields/retry@<pinned-sha>
🧰 Tools
🪛 zizmor (1.25.2)

[warning] 27-27: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

[error] 27-27: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

[error] 28-28: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)

(unpinned-uses)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-integration.yml around lines 27 - 28, Replace mutable
action refs with pinned commit SHAs for actions/checkout, dorny/paths-filter,
and nick-fields/retry across the workflow; also set with: persist-credentials:
false on both actions/checkout steps (the early checkout and the later
"Checkout" step). Locate occurrences of "uses: actions/checkout@v4", "uses:
dorny/paths-filter@v3", and "uses: nick-fields/retry@v3" and update each to the
corresponding commit SHA for that release, and add the persist-credentials:
false key under the actions/checkout steps so credentials are not persisted.

id: filter
if: github.event_name == 'pull_request'
with:
filters: |
relevant:
- 'src/**.py'
- 'tests/**.py'
Comment on lines +34 to +35
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

In picomatch (used by dorny/paths-filter), does src/.pymatch nested paths likesrc/a/b.py, or is src//*.py required for recursive matching?

💡 Result:

In picomatch, src/.py does NOT match nested paths like src/a/b.py; you need src//*.py for recursive matching. Reason (per picomatch glob semantics): * matches any characters excluding path separators, while is the special “globstar” that can match across path separators. The picomatch docs explicitly note that `*` won’t match `a/b.js` (it returns false for `a/b.js`) and that is what matches path separators when used appropriately (nested directories require ``). [1][2] So, src/.py is effectively “`src/` + any non-separator chars + `.py`”, not “`src/` + any number of nested directories + `.py`”. [1] For dorny/paths-filter: since it evaluates path expressions using picomatch, the same matching rules apply there. [3]

Citations:

Use recursive Python globs that match nested files.

src/**.py and tests/**.py do not match nested files like src/a/b.py with picomatch semantics (used by dorny/paths-filter). This causes integration tests to be skipped when changes occur in nested directories. Use src/**/*.py and tests/**/*.py for proper recursive matching.

Proposed fix 
           filters: |
             relevant:
-              - 'src/**.py'
-              - 'tests/**.py'
+              - 'src/**/*.py'
+              - 'tests/**/*.py'
               - 'pyproject.toml'
               - 'uv.lock'
               - 'sdks/**'
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- 'src/**.py'
- 'tests/**.py'
- 'src/**/*.py'
- 'tests/**/*.py'
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-integration.yml around lines 34 - 35, The globs
'src/**.py' and 'tests/**.py' do not match nested files; update the workflow to
use recursive globs by replacing those patterns with 'src/**/*.py' and
'tests/**/*.py' so changes in nested directories (e.g. src/a/b.py) trigger the
integration tests; modify the patterns wherever 'src/**.py' and 'tests/**.py'
appear in the test-integration.yml workflow.

Comment on lines +34 to +35
- 'pyproject.toml'
- 'uv.lock'
- 'sdks/**'
- 'flows/**'
- '.github/workflows/test-integration.yml'
- id: decision
run: |
if [ "${{ github.event_name }}" = "merge_group" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
elif [ "${{ steps.filter.outputs.relevant }}" = "true" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
else
echo "run=false" >> "$GITHUB_OUTPUT"
fi

tests:
needs: changes
if: needs.changes.outputs.run == 'true'
runs-on:
labels: ["self-hosted", "linux", "ARM64", "langflow-ai-arm64-40gb-ephemeral-sudo"]
env:
Expand Down Expand Up @@ -54,7 +85,7 @@ jobs:

- name: Checkout
uses: actions/checkout@v4

- name: Verify workspace
run: |
echo "Current directory: $(pwd)"
Expand All @@ -78,6 +109,7 @@ jobs:
run: uv sync

- name: Run integration tests
uses: nick-fields/retry@v3
env:
OPENSEARCH_HOST: localhost
OPENSEARCH_PORT: 9200
Expand All @@ -89,16 +121,42 @@ jobs:
GOOGLE_OAUTH_CLIENT_SECRET: ""
# Disable startup ingest noise unless a test enables it
DISABLE_STARTUP_INGEST: "true"
with:
timeout_minutes: 45
max_attempts: 2
retry_on: error
command: |
# For PRs and merge_group, always build locally since we're
# testing new code. For workflow_dispatch, use the input
# (defaults to true).
USE_LOCAL="${{ inputs.use_local_images }}"
if [ "${{ github.event_name }}" != "workflow_dispatch" ] || [ "$USE_LOCAL" != "false" ]; then
echo "Running tests with locally built images..."
make test-ci-local
else
echo "Running tests with DockerHub images..."
make test-ci
fi
echo "Keys directory after tests:"
ls -la keys/ || echo "No keys directory"
on_retry_command: |
# Heavyweight reset between attempts — the make target brings
# up the full docker-compose stack, and stale containers /
# volumes from the first attempt will poison the retry.
echo "Integration test attempt failed; tearing down before retry..."
docker compose -f docker-compose.yml down -v --remove-orphans || true
docker run --rm -v $(pwd):/work alpine sh -c "rm -rf /work/opensearch-data /work/config /work/langflow-data /work/keys /work/data /work/flows /work/openrag-documents" || true

required:
needs: tests
if: always()
runs-on: ubuntu-latest
steps:
- name: Verify tests result
run: |
# For PRs, always build locally since we're testing new code
# For workflow_dispatch, use the input (defaults to true)
USE_LOCAL="${{ inputs.use_local_images }}"
if [ "${{ github.event_name }}" == "pull_request" ] || [ "$USE_LOCAL" != "false" ]; then
echo "Running tests with locally built images..."
make test-ci-local
else
echo "Running tests with DockerHub images..."
make test-ci
result="${{ needs.tests.result }}"
if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
echo "tests job did not succeed: $result"
exit 1
fi
echo "Keys directory after tests:"
ls -la keys/ || echo "No keys directory"
echo "tests result: $result (success or skipped — passing)"
Comment on lines +150 to +162
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Prevent required from passing when changes fails.

required only checks needs.tests.result; if changes fails, tests can be skipped and this job still passes.

Proposed fix 
 required:
-  needs: tests
+  needs: [changes, tests]
   if: always()
   runs-on: ubuntu-latest
   steps:
     - name: Verify tests result
       run: |
-        result="${{ needs.tests.result }}"
-        if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then
-          echo "tests job did not succeed: $result"
+        changes_result="${{ needs.changes.result }}"
+        tests_result="${{ needs.tests.result }}"
+        if [ "$changes_result" = "failure" ] || [ "$changes_result" = "cancelled" ]; then
+          echo "changes job did not succeed: $changes_result"
+          exit 1
+        fi
+        if [ "$tests_result" = "failure" ] || [ "$tests_result" = "cancelled" ]; then
+          echo "tests job did not succeed: $tests_result"
           exit 1
         fi
-        echo "tests result: $result (success or skipped — passing)"
+        echo "changes=$changes_result, tests=$tests_result (success or skipped — passing)"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/test-integration.yml around lines 150 - 162, The
'required' job only inspects needs.tests.result so it can pass if 'changes'
failed and caused 'tests' to be skipped; update the job named "required" to
depend on the changes job and/or explicitly check its result: add changes to the
job's needs (e.g., needs: [tests, changes]) and modify the shell check to
consider both needs.tests.result and needs.changes.result (treat failure or
cancelled for either as a failing condition) so the job correctly fails when
changes fails.

Loading