Skip to content

Ability to truely use pyformat style#33

Open
vgvoleg wants to merge 1 commit intomainfrom
pyparam_usage
Open

Ability to truely use pyformat style#33
vgvoleg wants to merge 1 commit intomainfrom
pyparam_usage

Conversation

@vgvoleg
Copy link
Copy Markdown
Collaborator

@vgvoleg vgvoleg commented Mar 31, 2026

Adds opt-in support for standard Python DB-API parameter syntax via pyformat=True on connect() / async_connect(). By default behaviour is unchanged (raw YQL queries with $name placeholders and ydb.TypedValue).

When pyformat=True:

  • Named placeholders %(name)s and positional %s are supported
  • Python types are mapped to YDB types automatically (bool → Bool, int → Int64, str → Utf8, datetime → Timestamp, etc.)
  • %% is treated as a literal %
  • ydb.TypedValue values are passed through as-is for cases where explicit typing is needed

The old native YDB mode is now considered deprecated and will be removed in a future release.

Also fixes connect() / async_connect() signatures — previously they accepted *args, **kwargs, so IDEs provided no hints.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in support for Python DB-API “pyformat” parameter style (%(name)s / %s) by converting queries and parameters to YDB’s $name format and inferring YDB types from common Python values, while keeping the existing native YDB behavior as the default.

Changes:

  • Introduces convert_query_parameters() with automatic Python→YDB type wrapping (ydb.TypedValue) and % placeholder conversion.
  • Plumbs a pyformat: bool = False flag from connect() / async_connect() through connections to cursors to enable conversion in execute().
  • Adds unit tests for parameter conversion and documents both standard (pyformat=True) and native (default, deprecated) modes in the README.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
ydb_dbapi/utils.py Adds placeholder conversion + type inference/wrapping helpers.
ydb_dbapi/cursors.py Applies conversion in sync/async execute() when pyformat is enabled; updates parameter typing.
ydb_dbapi/connections.py Adds pyformat option to connection constructors and connect()/async_connect() signatures; passes flag to cursors.
tests/test_convert_parameters.py Adds unit tests for query/parameter conversion and type inference behavior.
README.md Documents pyformat=True usage, type mapping, and deprecates native mode in docs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ydb_dbapi/utils.py Outdated
Comment on lines +239 to +253
positional_index = 0

def replace(m: re.Match) -> str:
nonlocal positional_index
full = m.group(0)
if full == "%%":
return "%"
if full.startswith("%("):
return f"${m.group(1)}"
# %s — positional
positional_index += 1
return f"$p{positional_index}"

converted_query = re.sub(r"%%|%\((\w+)\)s|%s", replace, query)

Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

convert_query_parameters() currently converts placeholders without validating that the placeholder style in query matches the parameters container type (mapping vs sequence), and without checking that the number of positional placeholders (or the set of named placeholders) matches the provided parameters. This can lead to confusing downstream YDB errors (e.g., %s in query with a dict, missing keys, extra args). Consider detecting whether the query uses named vs positional placeholders (and disallow mixing), verifying counts/keys match, and raising ProgrammingError with a clear message when they don’t.

Copilot uses AI. Check for mistakes.
Comment thread ydb_dbapi/utils.py
Comment thread ydb_dbapi/cursors.py
Comment thread ydb_dbapi/cursors.py
Comment thread ydb_dbapi/utils.py
if value is None:
return value
ydb_type = _infer_ydb_type(value)
if ydb_type is not None:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Why if we can't infer the type we pass it through? Let's raise TypeError

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have some transformation logic on sdk type, but it would be more clear to have an error here without relying on sdk. Fixed

Comment thread ydb_dbapi/utils.py Outdated
positional_index += 1
return f"$p{positional_index}"

converted_query = re.sub(r"%%|%\((\w+)\)s|%s", replace, query)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: It's better to compile the regexp only once and store it on module level:

_REGEXP = re.compile(r"%%|%\((\w+)\)s|%s")

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread ydb_dbapi/utils.py Outdated
positional_index += 1
return f"$p{positional_index}"

converted_query = re.sub(r"%%|%\((\w+)\)s|%s", replace, query)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: How does it handle "%%%"?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not so good. it will be an error, added test to check

@vgvoleg vgvoleg force-pushed the pyparam_usage branch 2 times, most recently from a6f5d37 to 3a08242 Compare April 22, 2026 19:27
@vgvoleg vgvoleg requested review from LuckySting and Copilot April 22, 2026 19:29
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
Comment thread ydb_dbapi/utils.py Outdated
Comment thread ydb_dbapi/connections.py
Comment thread ydb_dbapi/connections.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants