Upgrade DotPrompt.Sql to full feature parity with DotPrompt 0.0.6.1

[azurecoder]

Why

DotPrompt.Sql was pinned to DotPrompt 0.0.5.1 and was missing several features that had since been added to the core library: few-shot examples, structured output schemas, temperature configuration, and prompt versioning. There was also a bug in SqlTablePromptStore.Save() that caused it to silently ignore the prompt file passed to it, and the manual YAML parsing in FromPromptFile(string) duplicated fragile logic already handled by DotPrompt itself.

What changed

Bug fix

• SqlTablePromptStore.Save() was using _promptFile (a path passed to the constructor) instead of the promptFile parameter — meaning every save silently wrote the wrong prompt. Fixed to use the parameter. The now-redundant string promptFile constructor argument and _promptFile field have been removed.

New fields — SqlPromptEntity

Added Version, Temperature (float?), OutputSchema (JSON string), and FewShots (JSON array string) to mirror all fields on PromptFile and PromptConfig. MaxTokens corrected from int to int? to match the nullable property in PromptConfig.

ToPromptFile() / FromPromptFile() rewrite

• ToPromptFile() now maps all new fields and no longer returns null when parameters or defaults are empty — it always returns a valid PromptFile.
• FromPromptFile(string) no longer manually re-implements YAML deserialization. It delegates to PromptFile.FromFile() (picking up all DotPrompt validation for free) then maps through the new FromPromptFile(PromptFile) overload.
• A new FromPromptFile(PromptFile) overload enables direct mapping from a parsed object, useful in SqlTablePromptStore.Save().

Database schema

• PromptFile table: added Temperature FLOAT NULL, OutputSchema NVARCHAR(MAX) NULL, FewShots NVARCHAR(MAX) NULL; MaxTokens changed to nullable.
• All additions use IF NOT EXISTS guards so existing databases are migrated non-destructively.
• sp_AddSqlPrompt stored procedure updated with new parameters and NULL-safe change-detection logic for the new columns.
• LoadPrompts.sql and GetLatestPromptByName.sql updated to SELECT the new columns.

Package & docs

• DotPrompt package bumped from 0.0.5.1 → 0.0.6.1.
• readme.md rewritten to reflect full feature parity, with an accurate feature table, installation snippet, code usage example, and schema documentation. The "few-shot not yet supported" note has been removed.

Migration notes

Existing SQL Server databases will be migrated automatically on next startup — no manual DDL required. The stored procedure is replaced with CREATE OR ALTER, which is idempotent.

Areas worth a close look

• The NULL-safe comparison in sp_AddSqlPrompt for the new nullable columns (Temperature, OutputSchema, FewShots) uses the pattern (col = @val OR (col IS NULL AND @val IS NULL)) — please verify this covers all edge cases for your environment.
• FewShots and OutputSchema are stored as raw JSON strings in NVARCHAR(MAX). If query-time JSON introspection is ever needed, a computed column or JSON index could be added later.

[Copilot]

Pull request overview

This PR updates DotPrompt.Sql to align with newer DotPrompt prompt-file capabilities (v0.0.6.1), extending SQL persistence to cover additional prompt config fields and improving prompt-file parsing/serialization.

Changes:

• Bumped DotPrompt dependency to 0.0.6.1 and expanded SqlPromptEntity to include Version/Temperature/OutputSchema/FewShots (+ nullable MaxTokens).
• Updated SQL schema + queries + stored procedure to store/load the new fields and perform NULL-safe change detection.
• Reworked prompt-file mapping to delegate parsing to PromptFile.FromFile() and fixed SqlTablePromptStore.Save() to use the provided PromptFile.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
readme.md	Rewritten docs: installation, CLI usage, supported-features table, and schema notes.
DotPrompt.Sql/SqlPromptStore.cs	Removes ctor prompt-file path; Save() now maps directly from PromptFile.
DotPrompt.Sql/SqlPromptRepository.cs	Passes new parameters (OutputSchema/Temperature/FewShots) into sp_AddSqlPrompt.
DotPrompt.Sql/SqlPromptEntity.cs	Adds new fields + rewrites To/From PromptFile mapping (JSON serialization for schema/few-shots).
DotPrompt.Sql/Resources/SqlQueries/LoadPrompts.sql	Loads latest prompt versions and now selects new columns.
DotPrompt.Sql/Resources/SqlQueries/InsertPromptFile.sql	Inserts new columns into PromptFile.
DotPrompt.Sql/Resources/SqlQueries/GetLatestPromptByName.sql	Retrieves latest prompt by name including new columns.
DotPrompt.Sql/Resources/SqlQueries/CreateDefaultPromptTables.sql	Adds/migrates columns (OutputSchema/Temperature/FewShots) + makes MaxTokens nullable.
DotPrompt.Sql/Resources/SqlQueries/AddSqlPrompt.sql	Updates stored proc signature, change detection, and insert for new columns.
DotPrompt.Sql/DotPrompt.Sql.csproj	Updates DotPrompt package reference.
DotPrompt.Sql.Test/TestSqlPromptEntity.cs	Updates tests to reflect new entity fields (adds Temperature in some cases).

Comments suppressed due to low confidence (2)

DotPrompt.Sql.Test/TestSqlPromptEntity.cs:103

• New versioning fields/logic were added (Temperature, OutputSchema, FewShots, nullable MaxTokens), but the tests don’t currently assert that changes to these new fields trigger a new version (or that they round-trip on retrieval). Adding coverage for at least one of these fields would help prevent regressions in the stored procedure change-detection logic.

    [Fact]
    public async Task AddSqlPrompt_ValidPrompt_InsertsSuccessfully()
    {
        // Arrange
        var entity = new SqlPromptEntity
        {
            PromptName = "myprompt",
            Model = "gpt4",
            OutputFormat = "json",
            MaxTokens = 500,
            Temperature = 0.7f,
            SystemPrompt = "Optimize SQL queries.",
            UserPrompt = "Suggest indexing improvements.",
            Parameters = new Dictionary<string, string>
            {
                { "query", "string" },
                { "topP", "number" }
            },
            Default = new Dictionary<string, object>
            {
                { "topP", "0.9" }
            }
        };

        // Act
        bool result = await _repository.AddSqlPrompt(entity);

        // Assert
        Assert.True(result, "Expected new prompt version to be inserted.");
    }

DotPrompt.Sql/Resources/SqlQueries/AddSqlPrompt.sql:57

• Change detection for parameters only checks for params present in @Parameters that are missing/changed in PromptParameters. If a parameter is removed (exists in PromptParameters but not in @Parameters), this won’t detect it and no new version will be inserted, even though prompt content changed. Add the inverse check (rows in PromptParameters for the existing version that don’t exist in @Parameters).

    OR EXISTS (
        -- Parameters changed?
        SELECT 1 FROM @Parameters p
        WHERE NOT EXISTS (
            SELECT 1 FROM PromptParameters pp
            WHERE pp.PromptId = @ExistingPromptId
            AND pp.VersionNumber = @ExistingVersion
            AND pp.ParameterName = p.ParameterName
            AND pp.ParameterValue = p.ParameterValue
        )
    )

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

VersionNumber is selected but not aliased to Version, so Dapper won’t populate SqlPromptEntity.Version (it will stay at the default 1). Since ToPromptFile() uses Version, loaded prompt files will report the wrong version. Consider aliasing VersionNumber AS Version in both the CTE and final SELECT (or renaming the C# property to VersionNumber).

[Copilot]

VersionNumber is returned without an alias, so Dapper won’t map it onto SqlPromptEntity.Version and callers will see PromptFile.Version as 1 even when the DB has higher versions. Alias lp.VersionNumber AS Version (or rename the C# property to VersionNumber) to keep versions consistent.

[Copilot]

FromPromptFile(PromptFile) copies promptFile.Version into SqlPromptEntity.Version, but SqlPromptRepository.AddSqlPrompt / sp_AddSqlPrompt never use this value (the proc always auto-increments VersionNumber). This makes the mapping misleading and breaks round-tripping if a prompt file carries an explicit version. Either persist the provided version (add a proc parameter and insert it) or stop mapping promptFile.Version here and treat DB versioning as the source of truth.

[Copilot]

The README says “New columns added in v0.2.3”, but the package version in DotPrompt.Sql.csproj is still 0.2.2. Please align the documented version with the actual package version (or bump the package version if 0.2.3 is intended).

Suggested change

	New columns added in v0.2.3:
	New columns added in v0.2.2:

[Copilot]

Similarly, default-value change detection only checks for defaults present in @Defaults that are missing/changed in the DB. If a default is removed (exists in ParameterDefaults for the existing version but not in @Defaults), this won’t trigger a new version. Consider adding an inverse EXISTS check for DB defaults not present in @Defaults.

Suggested change

	)
	)
	OR EXISTS (
	-- Defaults removed?
	SELECT 1
	FROM ParameterDefaults pd
	JOIN PromptParameters pp ON pd.ParameterId = pp.ParameterId
	WHERE pp.PromptId = @ExistingPromptId
	AND pp.VersionNumber = @ExistingVersion
	AND pd.VersionNumber = @ExistingVersion
	AND NOT EXISTS (
	SELECT 1
	FROM @Defaults d
	WHERE d.ParameterName = pp.ParameterName
	AND d.DefaultValue = pd.DefaultValue
	)
	)