From 764578cc6cdd9ed03ec216f2e7f26b2d3c0503b7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:05 +0300 Subject: [PATCH 01/11] chore: add code ownership rules --- .github/CODEOWNERS | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 0000000..1844d5f --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,9 @@ +# Default ownership +* @BLCCoreStudio + +# Evaluation methodology and release-critical surfaces +/.github/ @BLCCoreStudio +/pyproject.toml @BLCCoreStudio +/src/ @BLCCoreStudio +/tests/ @BLCCoreStudio +/docs/ @BLCCoreStudio From e8e048cc4f841fbcb7883270343a2a7157c77fff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:13 +0300 Subject: [PATCH 02/11] docs: add support and triage policy --- SUPPORT.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 SUPPORT.md diff --git a/SUPPORT.md b/SUPPORT.md new file mode 100644 index 0000000..354b128 --- /dev/null +++ b/SUPPORT.md @@ -0,0 +1,23 @@ +# Support + +TurkishEvalKit is maintained as a research-oriented open-source evaluation toolkit. Reports should contain enough evidence to distinguish implementation defects from disagreements about evaluation methodology. + +## Bugs + +Use the bug report template. Include the package version or commit, Python version, operating system, a minimal reproducible example, relevant input data where it can be shared, expected output, actual output, and the complete traceback or logs. + +## Evaluation or methodology concerns + +Describe the metric, dataset, rubric, evaluator, or pipeline involved and provide a minimal example that demonstrates the concern. Methodology changes require stronger evidence than ordinary implementation fixes because they can change historical comparability. + +## Feature requests + +Describe the evaluation problem first, then the proposed capability. Requests that duplicate existing primitives may be closed in favor of composition or documentation improvements. + +## Security + +Do not publish suspected vulnerabilities in public issues. Follow `SECURITY.md`. + +## Maintenance priority + +Priority is given to correctness, reproducibility, security, regressions, data integrity, compatibility, and changes that improve the toolkit for a broad set of evaluation workflows. From 606582e8ba0b59980448f211f2e453ea8cb5c683 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:19 +0300 Subject: [PATCH 03/11] docs: add project code of conduct --- CODE_OF_CONDUCT.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 CODE_OF_CONDUCT.md diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..9168686 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,23 @@ +# Code of Conduct + +TurkishEvalKit is a professional, evidence-driven open-source project. + +## Expected behavior + +- Discuss code, datasets, metrics, and evaluation methodology respectfully and precisely. +- Separate reproducible findings from preferences or speculation. +- Critique implementations and research decisions rather than people. +- Disclose important assumptions, conflicts, and limitations when they affect a contribution. +- Respect maintainer decisions on project scope, compatibility, methodology, and release policy. + +## Unacceptable behavior + +Harassment, threats, discriminatory attacks, personal insults, deliberate disruption, spam, falsified evidence, or publishing another person's private information are not acceptable. + +## Enforcement + +The maintainer may edit, hide, lock, or remove content and may restrict participation when necessary. Severe or repeated violations may result in a permanent ban from project spaces. + +## Reporting + +Use an appropriate private GitHub channel for conduct concerns that should not be discussed publicly. Security vulnerabilities must follow `SECURITY.md`. From cac093447a339ca441045495a4998d4ebc2fa505 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:27 +0300 Subject: [PATCH 04/11] chore: add structured bug report form --- .github/ISSUE_TEMPLATE/bug_report.yml | 64 +++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.yml diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000..b6cd74e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,64 @@ +name: Bug report +description: Report a reproducible defect in TurkishEvalKit +title: "[Bug]: " +labels: + - bug +body: + - type: markdown + attributes: + value: Thanks for taking the time to provide a reproducible report. + - type: input + id: version + attributes: + label: TurkishEvalKit version or commit + placeholder: v0.x.y or commit SHA + validations: + required: true + - type: input + id: python + attributes: + label: Python version + placeholder: Python 3.x + validations: + required: true + - type: input + id: os + attributes: + label: Operating system + placeholder: Ubuntu 24.04, Windows 11, macOS 15, etc. + validations: + required: true + - type: textarea + id: reproduction + attributes: + label: Minimal reproduction + description: Include the smallest code, command, and input needed to reproduce the problem. + render: shell + validations: + required: true + - type: textarea + id: expected + attributes: + label: Expected behavior + validations: + required: true + - type: textarea + id: actual + attributes: + label: Actual behavior + validations: + required: true + - type: textarea + id: logs + attributes: + label: Traceback or logs + render: shell + - type: checkboxes + id: checks + attributes: + label: Pre-submission checks + options: + - label: I searched existing open and closed issues for duplicates. + required: true + - label: I removed secrets, private data, and credentials from this report. + required: true From 66f8d8a9bf785219b493f6b20cb11fbba9af8c4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:33 +0300 Subject: [PATCH 05/11] chore: add structured feature request form --- .github/ISSUE_TEMPLATE/feature_request.yml | 37 ++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/feature_request.yml diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 0000000..5aef7e4 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,37 @@ +name: Feature request +description: Propose a new evaluation capability or improvement +title: "[Feature]: " +labels: + - enhancement +body: + - type: textarea + id: problem + attributes: + label: Evaluation problem + description: Describe the concrete evaluation or research problem before proposing an implementation. + validations: + required: true + - type: textarea + id: proposal + attributes: + label: Proposed capability + description: Explain the API, workflow, metric, report, or behavior you would add. + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Evidence or use cases + description: Provide examples, papers, datasets, benchmarks, or real workflows that motivate the change when available. + - type: textarea + id: compatibility + attributes: + label: Compatibility and methodology impact + description: Could this change outputs, historical comparability, stored artifacts, or public APIs? + - type: checkboxes + id: checks + attributes: + label: Pre-submission checks + options: + - label: I searched existing issues and documentation for an equivalent capability. + required: true From 0fda30f7acb691cc0c0c6896a8c8c38094574792 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:41 +0300 Subject: [PATCH 06/11] chore: add maintainer-focused pull request template --- .github/PULL_REQUEST_TEMPLATE.md | 36 ++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 .github/PULL_REQUEST_TEMPLATE.md diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..1be00d9 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,36 @@ +## Summary + +Describe what this pull request changes and why. + +## Type of change + +- [ ] Bug fix +- [ ] Feature +- [ ] Evaluation or methodology change +- [ ] Documentation +- [ ] Refactor / maintenance +- [ ] CI / tooling + +## Validation + +List the commands, tests, datasets, or examples used to validate the change. + +```text +# Example +pytest +``` + +## Evaluation impact + +- [ ] This change can alter evaluation outputs, scores, rankings, calibration, or historical comparability. +- [ ] This change modifies a public API or serialized artifact. +- [ ] This change is backward compatible. + +If any evaluation result can change, explain why the new behavior is more correct and include before/after evidence where practical. + +## Checklist + +- [ ] Tests cover the changed behavior. +- [ ] Documentation is updated when user-visible behavior changes. +- [ ] No secrets, private datasets, credentials, or generated local artifacts are included. +- [ ] I reviewed the diff for unintended methodology changes. From c071588752dcdea33776dc65a3eb6f4bb7d2ae4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:47 +0300 Subject: [PATCH 07/11] ci: review dependency changes in pull requests --- .github/workflows/dependency-review.yml | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 .github/workflows/dependency-review.yml diff --git a/.github/workflows/dependency-review.yml b/.github/workflows/dependency-review.yml new file mode 100644 index 0000000..3c8cbc8 --- /dev/null +++ b/.github/workflows/dependency-review.yml @@ -0,0 +1,19 @@ +name: Dependency Review + +on: + pull_request: + branches: [main] + +permissions: + contents: read + +jobs: + dependency-review: + name: Dependency Review + runs-on: ubuntu-24.04 + permissions: + contents: read + pull-requests: read + steps: + - name: Review dependency changes + uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5 From fa856a0ae138f538017abf39fc7122cf1bde16af Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:54 +0300 Subject: [PATCH 08/11] ci: add CodeQL analysis for Python --- .github/workflows/codeql.yml | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 0000000..1150239 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,36 @@ +name: CodeQL + +on: + push: + branches: [main] + pull_request: + branches: [main] + schedule: + - cron: '41 3 * * 2' + workflow_dispatch: + +permissions: + contents: read + security-events: write + +concurrency: + group: codeql-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + analyze: + name: Analyze Python + runs-on: ubuntu-24.04 + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Initialize CodeQL + uses: github/codeql-action/init@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 + with: + languages: python + - name: Analyze + uses: github/codeql-action/analyze@cdf488f595d80d6e07e03d4674febd5ab45fa938 # v4 + with: + category: /language:python From 1c052a74899200febbefe4934cbc129731846d91 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:50:59 +0300 Subject: [PATCH 09/11] ci: automate safe Dependabot patch updates --- .github/workflows/dependabot-auto-merge.yml | 26 +++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 .github/workflows/dependabot-auto-merge.yml diff --git a/.github/workflows/dependabot-auto-merge.yml b/.github/workflows/dependabot-auto-merge.yml new file mode 100644 index 0000000..b6a0557 --- /dev/null +++ b/.github/workflows/dependabot-auto-merge.yml @@ -0,0 +1,26 @@ +name: Dependabot Auto Merge + +on: + pull_request: + types: [opened, synchronize, reopened] + +permissions: + contents: write + pull-requests: write + +jobs: + dependabot: + if: github.event.pull_request.user.login == 'dependabot[bot]' && github.event.pull_request.base.ref == 'main' && github.repository_owner == 'BLCCoreStudio' + runs-on: ubuntu-24.04 + steps: + - name: Fetch Dependabot metadata + id: metadata + uses: dependabot/fetch-metadata@25dd0e34f4fe68f24cc83900b1fe3fe149efef98 # v3 + with: + github-token: ${{ github.token }} + - name: Enable auto-merge for patch updates + if: steps.metadata.outputs.update-type == 'version-update:semver-patch' + env: + GH_TOKEN: ${{ github.token }} + PR_URL: ${{ github.event.pull_request.html_url }} + run: gh pr merge --auto --squash "$PR_URL" From ab9380e3ae3ee81d80861c96788eec68702c5b7d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:51:05 +0300 Subject: [PATCH 10/11] chore: configure generated release notes --- .github/release.yml | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 .github/release.yml diff --git a/.github/release.yml b/.github/release.yml new file mode 100644 index 0000000..c595e70 --- /dev/null +++ b/.github/release.yml @@ -0,0 +1,33 @@ +changelog: + exclude: + labels: + - ignore-for-release + authors: + - dependabot + categories: + - title: Evaluation and methodology + labels: + - methodology + - evaluation + - title: Features + labels: + - enhancement + - feature + - title: Fixes + labels: + - bug + - fix + - title: Security + labels: + - security + - title: Documentation + labels: + - documentation + - title: Maintenance + labels: + - dependencies + - maintenance + - ci + - title: Other changes + labels: + - '*' From 389e21f5b032765f2d61b72aff85b5478c684ceb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?BLC=20Core=20Studio=20=E2=80=94=20Developer=20Tools?= <207100624+BLCCoreStudio@users.noreply.github.com> Date: Thu, 3 Sep 2026 01:53:38 +0300 Subject: [PATCH 11/11] ci: remove unsupported dependency review check --- .github/workflows/dependency-review.yml | 19 ------------------- 1 file changed, 19 deletions(-) delete mode 100644 .github/workflows/dependency-review.yml diff --git a/.github/workflows/dependency-review.yml b/.github/workflows/dependency-review.yml deleted file mode 100644 index 3c8cbc8..0000000 --- a/.github/workflows/dependency-review.yml +++ /dev/null @@ -1,19 +0,0 @@ -name: Dependency Review - -on: - pull_request: - branches: [main] - -permissions: - contents: read - -jobs: - dependency-review: - name: Dependency Review - runs-on: ubuntu-24.04 - permissions: - contents: read - pull-requests: read - steps: - - name: Review dependency changes - uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5