chore: Setup sdk infra by adnanrhussain · Pull Request #11 · learning-commons-org/evaluators

adnanrhussain · 2026-02-06T00:19:44Z

Summary

This PR sets up the basic infra for a TypeScript SDK.

Key Changes:

Setup TypeScript / NodeJS package + infra files
Setup a sdks/typescript/scripts/copy-prompts.js to copy the prompt text files from evals/prompts to the src & dist folders

Documentation

TODO: Update README in a downstream PR

Testing

Included new CI workflow should build successfully

SDK Feature PR Index

Copilot

Pull request overview

This PR establishes the foundational infrastructure for a TypeScript SDK that will provide educational text complexity evaluators. It's part of a larger effort (PRs #10-14) to build a complete SDK, with PR #10 having extracted prompts to text files and subsequent PRs implementing specific evaluators.

Changes:

Set up TypeScript/Node.js package with build tooling (tsup, vitest, eslint)
Created a dynamic prompt copying script that syncs .txt files from evals/prompts to SDK directories
Configured comprehensive testing infrastructure with unit and integration test support
Added CI workflow for automated testing, linting, and coverage reporting

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
package.json	Package configuration with dependencies, scripts, and metadata
tsconfig.json	TypeScript compiler configuration for ES2022/ESNext
tsup.config.ts	Build configuration with hardcoded prompt file copying
vitest.config.ts	Unit test configuration with v8 coverage
vitest.ci.config.ts	CI test configuration with hardcoded evaluator aliases
scripts/copy-prompts.js	Dynamic script to copy prompt .txt files for development
tests/unit/scripts/copy-prompts.test.ts	Unit tests for the copy-prompts script
.eslintrc.json	ESLint configuration for TypeScript
.gitignore	Ignore patterns for build artifacts and dependencies
.env.example	Example environment variables for API keys
LICENSE	Dual licensing (MIT for code, CC BY 4.0 for content)
README.md	Minimal placeholder documentation
src/index.ts	Empty entry point placeholder
.github/workflows/test-sdk-typescript.yml	CI pipeline with testing and coverage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdks/typescript/tsconfig.json

sdks/typescript/vitest.ci.config.ts

sdks/typescript/scripts/copy-prompts.js

.github/workflows/test-sdk-typescript.yml

sdks/typescript/package.json

sdks/typescript/tsup.config.ts

sdks/typescript/package.json

.github/workflows/test-sdk-typescript.yml

alexgb · 2026-02-19T22:50:44Z

sdks/typescript/LICENSE

@@ -0,0 +1,12 @@
+The Evaluator code is licensed under [MIT](https://opensource.org/license/mit).


This is the same license as the monorepo? It's unfortunate that we have to copy it into each SDK.

Yes and Yes. But I will still have it double-checked by Trust, just to make sure this is how it should be for sub-projects

sdks/typescript/package.json

alexgb · 2026-02-19T22:55:24Z

sdks/typescript/tsup.config.ts

+    const sourcePromptsDir = join(__dirname, '../../evals/prompts');
+    const destPromptsDir = join(__dirname, 'dist/prompts');
+
+    // Create directory structure


Why are we creating this organizational structure in the SDK and not in the parent project? Won't other SDKs need to recreate this logic?

Agreed its not ideal. The source prompts in PR#10 are organized above the SDKs, but each SDK needs prompts as part of its own build/dist. Moving the logic to a shared location is a worthwhile optimization but scoped to a future PR.

sdks/typescript/tests/integration/scripts/copy-prompts.test.ts

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdks/typescript/src/index.ts

czi-fsisenda

Looks good! You might have already addressed some of the comments from our conversations.

czi-fsisenda · 2026-02-20T00:52:52Z

sdks/typescript/scripts/copy-prompts.js

We'll copy new files and overwrite existing files, but we won't delete files that are removed from prompts.

We should add a CI step that ensures that src/prompts matches ../../evals/prompts

As I look through this, this seems like part of a build step, but it is used as a separate action.
Would it be easier to just have build step include rm -f src/prompts && cp -R ../../evals/prompts src/prompts or something like that?

@czi-fsisenda - I removed the whole copy-prompts logic, and instead just import the text files are typed resources

czi-fsisenda · 2026-02-20T01:29:21Z

sdks/typescript/.env.example

This makes me think we should use the package as the end consumer would use it rather than test source directly. So test the packaged SDK with some test app and see that required config is loaded as expected and we get the expected result.
We could test the packaged version separately, but then testing source would be redundant?

the .env file is only used for local integration tests. I'll think through this and handle it in a downstream PR

Copilot

Pull request overview

Copilot reviewed 12 out of 14 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdks/typescript/README.md

Copilot · 2026-02-20T03:09:06Z

sdks/typescript/LICENSE

+The Evaluator code is licensed under [MIT](https://opensource.org/license/mit).
+
+Evaluators content (including the prompt and settings information) is provided by Learning Commons under the CC BY 4.0 International license ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)).
+
+Annotated CLEAR Corpus is provided by Learning Commons (including annotations and enhancements) under CC BY-NC-SA 4.0 ([CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en)). The original dataset from CLEAR Corpus can be found at [The CLEAR Corpus by CommonLit](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) licensed under CC BY-NC-SA 4.0.
+
+**How to Cite the Evaluator Code:**
+Learning Commons (2025). Evaluators. GitHub. [https://github.com/learning-commons-org/evaluators](https://github.com/learning-commons-org/evaluators).
+Licensed under MIT.


package.json sets license to "MIT", but this LICENSE file is a custom notice with links and citation text rather than the standard MIT license text. This can create licensing/compliance ambiguity for package consumers and tooling. Either include the full MIT license text here (and move notices/citation to a separate NOTICE/ATTRIBUTION file), or adjust package.json to reflect the actual licensing of the published artifact.

Suggested change

The Evaluator code is licensed under [MIT](https://opensource.org/license/mit).

Evaluators content (including the prompt and settings information) is provided by Learning Commons under the CC BY 4.0 International license ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)).

Annotated CLEAR Corpus is provided by Learning Commons (including annotations and enhancements) under CC BY-NC-SA 4.0 ([CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en)). The original dataset from CLEAR Corpus can be found at [The CLEAR Corpus by CommonLit](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) licensed under CC BY-NC-SA 4.0.

**How to Cite the Evaluator Code:**

Learning Commons (2025). Evaluators. GitHub. [https://github.com/learning-commons-org/evaluators](https://github.com/learning-commons-org/evaluators).

Licensed under MIT.

MIT License

Copyright (c) 2025 Learning Commons

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

---

Additional Notices

==================

The Evaluator code is licensed under the MIT License set out above.

Evaluators content (including the prompt and settings information) is provided by Learning Commons under the CC BY 4.0 International license ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)).

Annotated CLEAR Corpus is provided by Learning Commons (including annotations and enhancements) under CC BY-NC-SA 4.0 ([CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en)). The original dataset from CLEAR Corpus can be found at [The CLEAR Corpus by CommonLit](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) licensed under CC BY-NC-SA 4.0.

**How to Cite the Evaluator Code:**

Learning Commons (2025). Evaluators. GitHub. [https://github.com/learning-commons-org/evaluators](https://github.com/learning-commons-org/evaluators).

Licensed under the MIT License.

sdks/typescript/package-lock.json

sdks/typescript/package.json

.github/workflows/test-sdk-typescript.yml

czi-fsisenda

🚀

adnanrhussain marked this pull request as ready for review February 19, 2026 04:46

adnanrhussain requested review from alexgb, czi-fsisenda and georgemelvin February 19, 2026 04:46

czi-fsisenda requested a review from Copilot February 19, 2026 22:34

Copilot started reviewing on behalf of czi-fsisenda February 19, 2026 22:34 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

alexgb approved these changes Feb 19, 2026

View reviewed changes

adnanrhussain requested a review from Copilot February 20, 2026 00:27

Copilot started reviewing on behalf of adnanrhussain February 20, 2026 00:27 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes

sdks/typescript/src/index.ts Show resolved Hide resolved

czi-fsisenda reviewed Feb 20, 2026

View reviewed changes

adnanrhussain requested a review from Copilot February 20, 2026 03:04

Copilot started reviewing on behalf of adnanrhussain February 20, 2026 03:05 View session

Copilot AI reviewed Feb 20, 2026

View reviewed changes

czi-fsisenda approved these changes Feb 23, 2026

View reviewed changes

adnanrhussain force-pushed the ahussain/sdk_1_prompts branch from ad3614d to 2b4ef17 Compare March 2, 2026 18:38

Base automatically changed from ahussain/sdk_1_prompts to ahussain/sdk_typescript March 2, 2026 18:39

adnanrhussain force-pushed the ahussain/sdk_2_infra branch from 3f727ff to 181d7dc Compare March 2, 2026 18:42

adnanrhussain added 6 commits March 2, 2026 11:12

chore: Setup sdk infra

98bf33a

re: feedback

4bfd96b

make ci workflow consistent with existing patterns

4e191c0

simplify ci workflow

c57712b

remove prompt copy

03be57d

fix to load actual text

6366f25

adnanrhussain added 2 commits March 2, 2026 11:12

fix package.lock

45808c9

fix node versions

ab29225

adnanrhussain force-pushed the ahussain/sdk_2_infra branch from 15a4c50 to ab29225 Compare March 2, 2026 19:13

adnanrhussain added 2 commits March 2, 2026 11:27

add codecov token

5c46345

remove dotenv

4736a44

adnanrhussain merged commit cad8ca4 into ahussain/sdk_typescript Mar 2, 2026
9 checks passed

adnanrhussain deleted the ahussain/sdk_2_infra branch March 2, 2026 23:20

adnanrhussain mentioned this pull request Mar 11, 2026

feat: Release TypeScript SDK 0.1.0 #20

Open

		@@ -0,0 +1,12 @@
		The Evaluator code is licensed under [MIT](https://opensource.org/license/mit).

-The Evaluator code is licensed under [MIT](https://opensource.org/license/mit).
-Evaluators content (including the prompt and settings information) is provided by Learning Commons under the CC BY 4.0 International license ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)).
-Annotated CLEAR Corpus is provided by Learning Commons (including annotations and enhancements) under CC BY-NC-SA 4.0 ([CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en)). The original dataset from CLEAR Corpus can be found at [The CLEAR Corpus by CommonLit](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) licensed under CC BY-NC-SA 4.0.
-**How to Cite the Evaluator Code:**
-Learning Commons (2025). Evaluators. GitHub. [https://github.com/learning-commons-org/evaluators](https://github.com/learning-commons-org/evaluators).
-Licensed under MIT.
+MIT License
+Copyright (c) 2025 Learning Commons
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+---
+Additional Notices
+==================
+The Evaluator code is licensed under the MIT License set out above.
+Evaluators content (including the prompt and settings information) is provided by Learning Commons under the CC BY 4.0 International license ([CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)).
+Annotated CLEAR Corpus is provided by Learning Commons (including annotations and enhancements) under CC BY-NC-SA 4.0 ([CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en)). The original dataset from CLEAR Corpus can be found at [The CLEAR Corpus by CommonLit](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) licensed under CC BY-NC-SA 4.0.
+**How to Cite the Evaluator Code:**
+Learning Commons (2025). Evaluators. GitHub. [https://github.com/learning-commons-org/evaluators](https://github.com/learning-commons-org/evaluators).
+Licensed under the MIT License.

Conversation

adnanrhussain commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes:

Documentation

Testing

SDK Feature PR Index

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

czi-fsisenda left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

czi-fsisenda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

adnanrhussain commented Feb 6, 2026 •

edited

Loading