Skip to content

Conversation

@kyleconroy
Copy link
Collaborator

@kyleconroy kyleconroy commented Nov 30, 2025

Summary

Adds a new expander package under internal/x/expander/ that expands * expressions in SQL queries to explicit column names by preparing the query against a database.

Features

  • Expands SELECT * to explicit column list
  • Preserves table prefix for qualified stars (e.g., authors.*authors.id, authors.name, authors.bio)
  • Handles RETURNING * in INSERT/UPDATE/DELETE statements (PostgreSQL)
  • Recursively expands CTEs, including dependent CTEs where later CTEs reference earlier ones
  • Supports subqueries in FROM clause
  • Database-agnostic design via ColumnGetter interface

Supported Databases

  • PostgreSQL - Uses pgxpool connection to get column metadata via Prepare
  • MySQL - Uses forked driver (github.com/sqlc-dev/mysql) with StmtMetadata interface
  • SQLite - Uses native ncruces/go-sqlite3 API for column metadata

Example

-- Input
SELECT * FROM authors

-- Output
SELECT id, name, bio FROM authors
-- Input (PostgreSQL)
WITH a AS (SELECT * FROM authors), b AS (SELECT * FROM a) SELECT * FROM b

-- Output
WITH a AS (SELECT id, name, bio FROM authors), b AS (SELECT id, name, bio FROM a) SELECT id, name, bio FROM b

Test plan

  • PostgreSQL tests (14 test cases) - CTEs, RETURNING *, qualified stars
  • MySQL tests (8 test cases) - qualified stars, table-scoped expansion
  • SQLite tests (8 test cases) - basic expansion, qualified stars
  • COUNT(*) correctly left unexpanded across all databases
  • All tests pass with proper SQL formatting (spaces after commas)

🤖 Generated with Claude Code

kyleconroy and others added 2 commits November 30, 2025 17:49
Adds a new expander package that expands * expressions in SQL queries
to explicit column names by preparing the query against a PostgreSQL
database.

Features:
- Expands SELECT * to explicit column list
- Preserves table prefix for qualified stars (e.g., table.*)
- Handles RETURNING * in INSERT/UPDATE/DELETE statements
- Recursively expands CTEs, including dependent CTEs
- Supports subqueries in FROM clause
- Works with both cgo (pganalyze/pg_query_go) and non-cgo
  (wasilibs/go-pgquery) builds

Example:
  SELECT * FROM authors
  → SELECT id, name, bio FROM authors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Move expander from internal/engine/postgresql/expander to internal/x/expander
and port it to use the internal AST types instead of pg_query nodes.

Key changes:
- Use internal AST types (*ast.SelectStmt, *ast.InsertStmt, etc.)
- Use astutils.Search for star detection
- Use ast.Format instead of pg_query deparse
- Add Parser interface for dependency injection
- Add test cases for COUNT(*) (should not be expanded)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kyleconroy kyleconroy force-pushed the claude/postgresql-star-expander branch from 3ae2d2f to 475cfcf Compare December 1, 2025 02:53
kyleconroy and others added 6 commits November 30, 2025 18:59
- Rename TestExpand to TestExpandPostgreSQL
- Add TestExpandMySQL for MySQL database support
- Replace pgxpool.Pool with ColumnGetter interface for database-agnostic column resolution
- Add PostgreSQLColumnGetter and MySQLColumnGetter implementations
- MySQL tests skip edge cases (double star, star in middle) due to intermediate query formatting issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
MySQL doesn't support unqualified `*` mixed with other columns (e.g.,
`SELECT *, *` or `SELECT id, *, name`). These are valid PostgreSQL
but invalid MySQL syntax.

Update MySQL tests to use table-qualified stars which are valid:
- `SELECT authors.*, authors.*` instead of `SELECT *, *`
- `SELECT id, authors.*, name` instead of `SELECT id, *, name`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add TestExpandSQLite with 8 test cases using in-memory SQLite database
- Rename MySQLColumnGetter to SQLColumnGetter since both MySQL and SQLite
  use the same database/sql-based implementation
- SQLite supports the same star syntax as PostgreSQL (including `SELECT *, *`
  and `SELECT id, *, name`)

Test results:
- PostgreSQL: 14 tests
- MySQL: 8 tests
- SQLite: 8 tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Use PrepareContext to validate the query before executing it to get
column metadata. While database/sql doesn't expose column names from
prepared statements directly (unlike pgx), this at least validates
the SQL syntax before execution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Use the native sqlite3.Conn.Prepare and stmt.ColumnName/ColumnCount APIs
to get column names without executing the query. This is more efficient
and consistent with how PostgreSQL handles it.

Changes:
- Add SQLiteColumnGetter using native sqlite3.Conn
- Rename SQLColumnGetter to MySQLColumnGetter (MySQL still needs to execute)
- SQLite test now uses sqlite3.Open instead of sql.Open

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Update MySQLColumnGetter to use github.com/sqlc-dev/mysql fork with
  StmtMetadata interface for getting column names via prepare
- Add replace directive to go.mod for the forked MySQL driver
- Fix list formatting to use ", " separator instead of "," for proper
  SQL spacing (e.g., "SELECT id, name, bio" instead of "SELECT id,name,bio")
- Update test expectations to reflect proper spacing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kyleconroy kyleconroy changed the title feat(postgresql): Add star expander for SELECT * and RETURNING * feat(expander): Add star expander for SELECT * and RETURNING * (PostgreSQL, MySQL, SQLite) Dec 1, 2025
@kyleconroy kyleconroy marked this pull request as ready for review December 1, 2025 03:35
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. 🔧 golang labels Dec 1, 2025
@kyleconroy kyleconroy merged commit 21e6557 into main Dec 1, 2025
13 checks passed
@kyleconroy kyleconroy deleted the claude/postgresql-star-expander branch December 1, 2025 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files. 🔧 golang

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants