FEAT: Support for Complex Data Type- sql_variant

[gargsaumya]

Work Item / Issue Reference

AB#42724

GitHub Issue: #<ISSUE_NUMBER>

---

Summary

This pull request adds support for the sql_variant SQL Server type to the Python MSSQL driver, ensuring that sql_variant columns are fetched and mapped correctly to their underlying types. The changes introduce new constants, update type mappings, and enhance the fetch logic to detect and process sql_variant columns using their native types, improving compatibility and correctness when handling complex data.

Support for sql_variant type

• Added the SQL_SS_VARIANT constant and related attribute constants in both mssql_python/constants.py and the C++ binding layer to enable recognition and handling of the sql_variant type. [1] [2]
• Included SQL_SS_VARIANT in the set of valid types for type validation logic.

Type mapping and fetch logic

• Updated the C type mapping in cursor.py so that SQL_SS_VARIANT is mapped to SQL_C_BINARY, allowing binary transfer of the variant data.
• Implemented a helper function in the C++ layer to map a sql_variant's underlying C type to the appropriate SQL data type, enabling the fetch logic to reuse existing code for each possible underlying type.
• Enhanced the fetch routines (SQLGetData_wrap, FetchMany_wrap, and FetchAll_wrap) to detect sql_variant columns, determine their true data types at runtime, and handle them with the correct logic. This includes always using the streaming fetch path for sql_variant columns to preserve native type fidelity. [1] [2] [3]

Other improvements

• Improved error logging and debug output for easier troubleshooting and visibility into how sql_variant columns are processed. [1] [2] [3] [4]

These changes collectively ensure that the driver can now fully support reading sql_variant columns, mapping them to their actual types, and handling them efficiently.

[Copilot]

Pull request overview

Adds sql_variant read support to the MSSQL Python driver by introducing the SQL Server-specific type constant and enhancing the C++ fetch path to resolve and deserialize sql_variant values as their underlying base types, with a new test suite to validate behavior across fetch methods.

Changes:

• Added SQL_SS_VARIANT (-150) to Python constants and allowed it in SQLTypes.get_valid_types().
• Updated the C++ fetch path to detect sql_variant, map its underlying type, and route decoding through existing type handlers; adjusted LOB detection to force streaming for sql_variant.
• Added a dedicated pytest suite covering many base types and fetch patterns for sql_variant.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
mssql_python/constants.py	Introduces SQL_SS_VARIANT and includes it in valid SQL types for setinputsizes.
mssql_python/cursor.py	Adds a SQL→C type mapping entry for SQL_SS_VARIANT.
mssql_python/pybind/ddbc_bindings.cpp	Implements sql_variant type detection/mapping in SQLGetData_wrap and routes sql_variant through the streaming/LOB path for fetchmany/fetchall.
tests/test_019_sql_variant.py	New pytest coverage for fetching sql_variant across base types and fetch APIs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 86%

🎯 Overall Coverage 77%

📈 Total Lines Covered: 5756 out of 7423 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/constants.py (100%)
• mssql_python/pybind/ddbc_bindings.cpp (86.1%): Missing lines 2955-2956,2968,2972,2978,3041-3045,3055-3058,4343

Summary

• Total: 109 lines
• Missing: 15 lines
• Coverage: 86%

mssql_python/pybind/ddbc_bindings.cpp

Lines 2951-2960

  2951             return SQL_WVARCHAR;
  2952         case SQL_C_DATE:
  2953         case SQL_C_TYPE_DATE:
  2954             return SQL_TYPE_DATE;
! 2955         case SQL_C_TIME:
! 2956         case SQL_C_TYPE_TIME:
  2957         case SQL_SS_VARIANT_TIME:
  2958             return SQL_TYPE_TIME;
  2959         case SQL_C_TIMESTAMP:
  2960         case SQL_C_TYPE_TIMESTAMP:

---

Lines 2964-2976

  2964         case SQL_C_GUID:
  2965             return SQL_GUID;
  2966         case SQL_C_NUMERIC:
  2967             return SQL_NUMERIC;
! 2968         case SQL_C_TINYINT:
  2969         case SQL_C_UTINYINT:
  2970         case SQL_C_STINYINT:
  2971             return SQL_TINYINT;
! 2972         default:
  2973             // Unknown C type code - fallback to WVARCHAR for string conversion
  2974             // Note: SQL Server enforces sql_variant restrictions at INSERT time, preventing
  2975             // invalid types (text, ntext, image, timestamp, xml, MAX types, nested variants,
  2976             // spatial types, hierarchyid, UDTs) from being stored. By the time we fetch data,

---

Lines 2974-2982

  2974             // Note: SQL Server enforces sql_variant restrictions at INSERT time, preventing
  2975             // invalid types (text, ntext, image, timestamp, xml, MAX types, nested variants,
  2976             // spatial types, hierarchyid, UDTs) from being stored. By the time we fetch data,
  2977             // only valid base types exist. This default handles unmapped/future type codes.
! 2978             return SQL_WVARCHAR;
  2979     }
  2980 }
  2981 
  2982 // Helper function to check if a column requires SQLGetData streaming (LOB or sql_variant)

---

Lines 3037-3049

  3037             //    Without this probe call, SQLColAttribute returns incorrect type codes.
  3038             SQLLEN indicator;
  3039             ret = SQLGetData_ptr(hStmt, i, SQL_C_BINARY, NULL, 0, &indicator);
  3040             if (!SQL_SUCCEEDED(ret)) {
! 3041                 LOG_ERROR("SQLGetData: Failed to probe sql_variant column %d - SQLRETURN=%d", i,
! 3042                           ret);
! 3043                 row.append(py::none());
! 3044                 continue;
! 3045             }
  3046             if (indicator == SQL_NULL_DATA) {
  3047                 row.append(py::none());
  3048                 continue;
  3049             }

---

Lines 3051-3062

  3051             SQLLEN variantCType = 0;
  3052             ret =
  3053                 SQLColAttribute_ptr(hStmt, i, SQL_CA_SS_VARIANT_TYPE, NULL, 0, NULL, &variantCType);
  3054             if (!SQL_SUCCEEDED(ret)) {
! 3055                 LOG_ERROR("SQLGetData: Failed to get sql_variant underlying type for column %d", i);
! 3056                 row.append(py::none());
! 3057                 continue;
! 3058             }
  3059             effectiveDataType = MapVariantCTypeToSQLType(variantCType);
  3060             LOG("SQLGetData: sql_variant column %d has variantCType=%ld, mapped to SQL type %d", i,
  3061                 (long)variantCType, effectiveDataType);
  3062         }

---

Lines 4339-4347

  4339             ret = SQLFetch_ptr(hStmt);
  4340             if (ret == SQL_NO_DATA)
  4341                 break;
  4342             if (!SQL_SUCCEEDED(ret))
! 4343                 return ret;
  4344 
  4345             py::list row;
  4346             SQLGetData_wrap(StatementHandle, numCols, row, charEncoding,
  4347                             wcharEncoding);  // <-- streams LOBs correctly

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 67.8%
mssql_python.pybind.ddbc_bindings.cpp: 70.4%
mssql_python.row.py: 70.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.__init__.py: 77.1%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.connection.py: 85.2%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sumitmsft]

@gargsaumya This is reviewed. Please make sure that we document a performance aspect of using sql variant types on the queries so that customer are aware of the fact that they may see slight degradation.