Skip to content

Add CI test pipeline, split unit/integration tests, and enforce tests in release workflows#10

Merged
jwfing merged 4 commits into
InsForge:mainfrom
junaiddshaukat:feat/ci-test-pipeline
Mar 20, 2026
Merged

Add CI test pipeline, split unit/integration tests, and enforce tests in release workflows#10
jwfing merged 4 commits into
InsForge:mainfrom
junaiddshaukat:feat/ci-test-pipeline

Conversation

@junaiddshaukat
Copy link
Copy Markdown
Contributor

Summary

This PR implements a proper CI test pipeline and separating unit tests from integration tests.

What Changed

  1. Added CI workflow for PRs and pushes to main
  • New workflow: .github/workflows/ci.yml
  • Runs: ./gradlew build test
  1. Separated unit and integration tests
  • Added JUnit tag-based split:
    • test task excludes integration
    • new integrationTest task includes integration
  • Marked external-service tests with @Tag("integration") across integration test classes
  1. Removed skipped-test flags from release workflows
  • Updated .github/workflows/publish.yml
  • Updated .github/workflows/publish-maven-central.yml
  • Removed -x test and ensured build/tests run before publish
  1. Updated docs
  • README now includes:
    • ./gradlew test (unit tests)
    • ./gradlew integrationTest (integration tests)

Validation

  • ./gradlew clean build test
  • ./gradlew integrationTest --dry-run task wiring verified
  • Full integrationTest currently has environment-related failures (e.g., realtime token expiry / backend service availability), not introduced by this PR.

Issue

Fixes #2

@junaiddshaukat
Copy link
Copy Markdown
Contributor Author

@jwfing, please have a look on this pr when you get a chance. Since this PR is from a fork, GitHub Actions may require maintainer approval to run workflows. Could you approve and trigger the checks? Thanks!

@jwfing
Copy link
Copy Markdown
Member

jwfing commented Mar 18, 2026

Great job! Since we need to run integration tests in workflow, I think it's necessary to make TestConfig.kt configurable, for example, read BASE_URL/ANON_KEY from env vars.
Could you mind posting a screenshot for the result of all tests, running on local is okay. @junaiddshaukat

@junaiddshaukat
Copy link
Copy Markdown
Contributor Author

While implementing "remove -x test", and testing locally I discovered that the Napier dependency (2.7.1) was compiled with JDK 17 (class file version 61.0). Tests crash with UnsupportedClassVersionError on JDK 11 — the old -x test flag was masking this. I've bumped all workflows to JDK 17 so tests can actually run. The build.gradle.kts still targets JDK 11 for source/target compatibility, so the published artifact is unchanged.

@jwfing could you confirm you're okay with the JDK 17 bump in the CI and publish workflows?

…ump JDK to 17

- TestConfig now reads INSFORGE_BASE_URL, INSFORGE_ANON_KEY, and
  INSFORGE_TEST_JWT_TOKEN from environment variables, falling back
  to hardcoded defaults for local development.
- ApiResponseTest and CaptureApiResponsesTest now use TestConfig
  instead of duplicating credentials.
- ci.yml gains a dedicated integration-tests job that injects
  secrets via env vars.
- All workflows bumped from JDK 11 to 17 because the Napier
  dependency (2.7.1) requires class file version 61 (JDK 17).
@junaiddshaukat
Copy link
Copy Markdown
Contributor Author

@jwfing TestConfig is now env-var configurable. TestConfig.kt reads INSFORGE_BASE_URL, INSFORGE_ANON_KEY, and INSFORGE_TEST_JWT_TOKEN from environment variables, falling back to the existing hardcoded defaults for local development. I also updated ApiResponseTest and CaptureApiResponsesTest to use TestConfig instead of duplicating credentials.

Local test results (JDK 17, local Docker backend):

  • Unit tests: 2/2 passed (BUILD SUCCESSFUL)
  • Integration tests: 228/247 passed, 19 failed — all pre-existing (15 RealtimeLowLevelAPI tests need a valid local JWT, 2 StorageTest local file uploads hit OOM, 1 FunctionsTest hello function not deployed, 1 AuthTest assertion logic bug)
image image

@jwfing
Copy link
Copy Markdown
Member

jwfing commented Mar 18, 2026

Except of RealtimeLowLevelAPI, there are still 6 tests failed:

229 tests completed, 6 failed

> Task :integrationTest FAILED

FAILURE: Build failed with an exception.

Can you fix them? By the way, for JWT token, maybe we can generate it via auth#login with specified credentials(email + password), instead of passing by env. @junaiddshaukat

@junaiddshaukat
Copy link
Copy Markdown
Contributor Author

junaiddshaukat commented Mar 19, 2026

@jwfing I addressed your feedback:

Fixed 6 failing tests (excluding RealtimeLowLevelAPI):

  • AuthTest.handleAuthCallback — assertion checked for exchange_code but actual error says insforge_code
  • AuthTest.getCurrentUser — handled deserialization mismatch gracefully
  • FunctionsTest.invokeRaw — AssertionError extends Error, not Exception, so catch block never caught it
  • RealtimeTest (2 tests) — child coroutines not cancelled on failure → UncompletedCoroutinesError
  • RealtimeConnectionLifecycleTest — removed @tag("integration") since it uses fake sockets (unit test)
  • JWT via auth#signIn — replaced static INSFORGE_TEST_JWT_TOKEN env var with dynamic sign-in using TestConfig.TEST_EMAIL / TestConfig.TEST_PASSWORD. Token and userId are obtained at runtime via auth.signUp() + auth.signIn() and cached for the test run.

Updated ci.yml env vars accordingly (INSFORGE_TEST_EMAIL, INSFORGE_TEST_PASSWORD).

Local test results (Docker backend, no S3 configured):

Unit tests: 6/6 passed
Integration tests: 235/243 passed —> 6 RealtimeLowLevelAPI + 2 StorageTest local uploads fail due to local backend (no S3, anon role lacks admin access for bucket creation).

image

@jwfing jwfing self-requested a review March 19, 2026 18:41
Comment thread .github/workflows/ci.yml
- name: Run integration tests
run: ./gradlew integrationTest
env:
INSFORGE_BASE_URL: ${{ secrets.INSFORGE_BASE_URL }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is indeed the case that secrets are unavailable for PRs originating from forks. Consequently, the current job still executes, passing empty strings to the integration tests.
Could you modify the workflow to first check for the presence of secrets; if they are missing, the integration tests will be skipped entirely to prevent fork PRs from failing due to missing configuration.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

working on it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed now, please have a look

@jwfing jwfing merged commit 8bba342 into InsForge:main Mar 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[3 points] CI Test Pipeline

2 participants