Expose vector dimensions in PostgreSQL catalogs

[fulghum]

Populate pg_attribute.atttypmod from stored column types and extend format_type() to render extension and user-defined types with PostgreSQL-compatible typmods, array spelling, quoting, and schema qualification.

Adds catalog coverage for bounded and unbounded pgvector types, dimensioned arrays, views, quoted names, and shadowed schemas. The new queries were also validated unchanged against PostgreSQL 16 with pgvector.

Fixes #3230

[github-actions]

	Main	PR
Total	42090	42090
Successful	19271	19271
Failures	22819	22819
Partial Successes1	5459	5459

	Main	PR
Successful	45.7852%	45.7852%
Failures	54.2148%	54.2148%

Footnotes

1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct. ↩

[itoqa]

Commit: e66c6de: 14 test cases ran, 13 passed ✅, 1 additional finding ⚠️.

Summary

Coverage spans PostgreSQL-compatible type and column metadata across built-in, extension, custom, bounded, unbounded, array, renamed, shadowed, and unknown types, including edge cases around modifiers and search-path changes. It also exercises metadata consistency through table changes, views, derived columns, and concurrent catalog reads, with the underlying formatting and catalog behavior broadly healthy.

Safe to merge — the sole observed issue is a medium-severity, pre-existing view metadata defect that is explicitly not attributable to this PR, with no regressions or new PR-related failures identified. It is a flag for later rather than a merge blocker.

Tests run by Ito

Result	Severity	Type	Description
✅	—	General	The formatter leaves negative modifiers off, renders zero as vector(0), keeps positive modifiers, and preserves the [] suffix for arrays.
✅	—	General	The column kept the correct type and modifier before and after its definition changed. Independent checks confirmed that each pair described a valid vector type.
✅	—	General	Changing the search path updates how same-named types are shown. The type first on the path is shown without its schema, while the shadowed type keeps its schema name.
✅	—	General	Concurrent readers saw either the old vector(1024) definition or the new vector(2048) definition. No reader saw mixed or invalid column details.
✅	—	Builtin	Integer, text, date, and timestamp with time zone kept their standard PostgreSQL names. Integer also kept the same name when given the negative typmod value.
✅	—	Builtin	An unknown type ID returns exactly ??? instead of an error.
✅	—	Format	The database returned the expected names for bounded, unbounded, and array extension types.
✅	—	Format	A custom type with no modifier formatter is shown with its numeric modifier in parentheses, as expected.
✅	—	Name	The first type is shown as "Mixed Type", while the same name in the second schema is shown with its schema name.
✅	—	Name	A type outside the session search path was shown with its schema and quoted type name, so users can identify the correct type.
✅	—	Rev	The derived view returned all three columns in the expected order. Direct and cast vector values kept dimensions 3 and 5, while the unbounded expression correctly reported plain vector metadata.
✅	—	Type	Catalog queries show the correct sizes for bounded columns, keep the unbounded column at its default, and preserve the same details in the view.
✅	—	Type	A local text column stayed visible with catalog type OID 25 and the default typmod of -1.
⚠️		Rev	The table shows four correct columns, but the view shows one blank column with position 0 and no type information.

Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 View columns are missing from metadata results

• Severity: Medium
• Description: The table shows four correct columns, but the view shows one blank column with position 0 and no type information.
• Impact: Clients that inspect views through the standard information schema may see a blank column with no type instead of the view's real columns. This can break schema discovery and tools that rely on accurate view metadata, while the underlying view data remains intact.
• Steps to Reproduce:
  1. Create a table with vector(3), halfvec(7), vector, and text columns.
  2. Create a view that selects all four table columns.
  3. Query information_schema.columns for the table and the view.
  4. Compare the view rows with the columns returned by pg_catalog.pg_attribute.
• Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
• Code Analysis: columnsRowIter in server/tables/information_schema/columns_table.go:102-124 appends rows from getRowsFromDatabase and then separately appends rows from getRowsFromViews. The table path at getRowsFromTable (lines 201-216) calls information_schema.SchemaForTable and getRowFromColumn, which computes a one-based ordinal position and derives data_type and UDT metadata. In contrast, getRowsFromViews at lines 219-279 calls information_schema.ViewsInDatabase and, for every view, appends exactly one literal sql.Row. That row sets column_name to an empty string, ordinal_position to int32(0), data_type to nil, and all type metadata including udt_schema and udt_name to nil. It never reads the view definition, analyzes its SELECT output, or loops over output columns. The neighboring pg_attribute implementation in server/tables/pgcatalog/pg_attribute.go:140-191 demonstrates the required production path: it obtains the view SELECT body, calls engine.AnalyzeQuery, loops over analyzed.Schema(ctx), and assigns attnum i+1 plus the analyzed type OID and typmod. The smallest practical fix is to make getRowsFromViews use the same analyzed output schema, then pass each output column through the existing row-building/type-formatting logic with its actual ordinal position; if analysis fails, it should return an explicit error or consistently omit the view rather than emitting a fabricated ordinal-zero row.

Evidence Package

• text/REV-7/REV-7-catalog-readback.txt — 3 KB
• text/REV-7/REV-7-source-path.txt — 938 B

Tip

Reply with @itoqa to send us feedback on this test run.

[coffeegoddd]

@fulghum DOLT

read_tests	from_latency	to_latency	percent_change
covering_index_scan_postgres	2.52	2.52	0.0
groupby_scan_postgres	77.19	78.6	1.83
index_join_postgres	2.26	2.26	0.0
index_join_scan_postgres	1.61	1.61	0.0
index_scan_postgres	475.79	467.3	-1.78
oltp_point_select	0.37	0.36	-2.7
oltp_read_only	6.55	6.43	-1.83
select_random_points	0.72	0.72	0.0
select_random_ranges	1.01	1.01	0.0
table_scan_postgres	475.79	467.3	-1.78
types_table_scan_postgres	1191.92	1170.65	-1.78

write_tests	from_latency	to_latency	percent_change
oltp_delete_insert_postgres	6.67	6.67	0.0
oltp_insert	3.36	3.36	0.0
oltp_read_write	13.46	13.46	0.0
oltp_update_index	3.62	3.62	0.0
oltp_update_non_index	3.3	3.3	0.0
oltp_write_only	7.04	7.04	0.0
types_delete_insert_postgres	7.17	7.17	0.0

[itoqa]

Ito Diff Report — e66c6de → 29db9c5: 8 test cases ran, 1 fixed ✅, 6 passing ✅, 1 additional finding ⚠️.

Diff Summary

The run covers database type formatting and catalog metadata across normal columns and derived views, including bounded and unbounded types, modifier changes, type-name resolution, schema search order, and consistency after updates. It exercises both expected behavior and edge cases involving quoted names, shadowed types, and view column discovery.

Safe to merge — the tested PR behavior shows no attributable regressions or new failures. A medium-severity view metadata issue was observed, but it is an existing, unrelated application finding and is a flag for later rather than a merge blocker.

Tests run by Ito

Result	State	Severity	Type	Description
✅	❌->✅ Fixed	—	Rev	The catalog query completed and returned all three columns with the expected PostgreSQL type names. The bounded text column kept its length, while the unbounded text column stayed unbounded.
✅	Passing	—	Attribute	The catalog query returned integer for the id column and character varying(255) for the bounded text column.
✅	Passing	—	General	Changing a bounded text column updated its displayed length without mixing old and new metadata. The integer column stayed unchanged.
✅	Passing	—	General	The same type was shown without its schema when its schema came first, with its schema when another matching type came first, and without its schema again after restoring the original order.
✅	Passing	—	Format	A varchar(255) column kept its type limit when the catalog metadata was formatted. The result was character varying(255), as expected.
✅	Passing	—	Search	A mixed-case type that is first on the search path is returned with the correct quotes and without an unnecessary schema name.
✅	Passing	—	Type	The catalog reports the integer column as integer and the bounded text column as character varying(255).
⏸️	Skipped	—	General	The formatter leaves negative modifiers off, renders zero as vector(0), keeps positive modifiers, and preserves the [] suffix for arrays.
⏸️	Skipped	—	General	The column kept the correct type and modifier before and after its definition changed. Independent checks confirmed that each pair described a valid vector type.
⏸️	Skipped	—	General	Changing the search path updates how same-named types are shown. The type first on the path is shown without its schema, while the shadowed type keeps its schema name.
⏸️	Skipped	—	General	Concurrent readers saw either the old vector(1024) definition or the new vector(2048) definition. No reader saw mixed or invalid column details.
⏸️	Skipped	—	Builtin	Integer, text, date, and timestamp with time zone kept their standard PostgreSQL names. Integer also kept the same name when given the negative typmod value.
⏸️	Skipped	—	Builtin	An unknown type ID returns exactly ??? instead of an error.
⏸️	Skipped	—	Format	The database returned the expected names for bounded, unbounded, and array extension types.
⏸️	Skipped	—	Format	A custom type with no modifier formatter is shown with its numeric modifier in parentheses, as expected.
⏸️	Skipped	—	Name	The first type is shown as "Mixed Type", while the same name in the second schema is shown with its schema name.
⏸️	Skipped	—	Name	A type outside the session search path was shown with its schema and quoted type name, so users can identify the correct type.
⏸️	Skipped	—	Rev	The derived view returned all three columns in the expected order. Direct and cast vector values kept dimensions 3 and 5, while the unbounded expression correctly reported plain vector metadata.
⏸️	Skipped	—	Type	Catalog queries show the correct sizes for bounded columns, keep the unbounded column at its default, and preserve the same details in the view.
⏸️	Skipped	—	Type	A local text column stayed visible with catalog type OID 25 and the default typmod of -1.
⚠️	Additional Finding		General	The view has no catalog rows for its output columns. The information schema shows one blank row with position 0 instead of the integer, bounded text, and aliased columns.

Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 View columns disappear from catalog metadata

• Severity: Medium
• Description: The view has no catalog rows for its output columns. The information schema shows one blank row with position 0 instead of the integer, bounded text, and aliased columns.
• Impact: Tools and users that inspect database views may see missing or incorrect column names, types, and positions. The views can still be queried, but standard database introspection and tools that depend on it may not work correctly.
• Steps to Reproduce:
  1. Create a table with an integer column and a varchar(120) column.
  2. Create a view that selects both columns and adds a varchar(120) expression with an alias.
  3. Query pg_class and pg_attribute for the view, then query information_schema.columns for the same view.
  4. Compare the view rows with the three output columns and their types.
• Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
• Code Analysis: The runtime result is supported by three independent production paths. In server/tables/pgcatalog/pg_class.go:484, pgClassToRow always writes int16(0) for relnatts, so even a relation classified as a view cannot report its output-column count. In server/tables/pgcatalog/pg_attribute.go:140-191, view attributes are built only when a query analyzer is available and AnalyzeQuery succeeds; the engine-nil branch at lines 141-143 and the analyze-error branch at lines 163-166 return successfully without adding any attributes. When analysis succeeds, the loop at lines 168-190 does create one row per output column and preserves a DoltgresType typmod at lines 171-174, but that path is not reached in the failing case. Separately, server/tables/information_schema/columns_table.go:219-279 does not derive view columns at all: getRowsFromViews appends exactly one hard-coded row per view, with an empty column_name at line 235, ordinal_position 0 at line 236, and nil type metadata at lines 239-259. These implementations directly explain the observed relkind v with relnatts 0, zero pg_attribute rows, and the malformed information_schema row. The smallest practical fix is to make pg_class report the actual attribute count and make both catalog projections use the analyzed view output schema, while retaining the existing typmod extraction for each output column; analysis failures should be handled explicitly rather than silently presenting a successful empty schema.

Evidence Package

• text/BF-CROSS-3/BF-CROSS-3-view-metadata.txt — 4 KB

Tip

Reply with @itoqa to send us feedback on this test run.

[fulghum]

The Ito QA additional finding about view metadata is legit. Opened issue #3244 to track.

[Hydrocharged]

LGTM!