Skip to content

BUG: improve error message for non-string dtype with DB-API connection (GH#61385)#65893

Open
adebayopeter wants to merge 5 commits into
pandas-dev:mainfrom
adebayopeter:fix-dtype-error-message
Open

BUG: improve error message for non-string dtype with DB-API connection (GH#61385)#65893
adebayopeter wants to merge 5 commits into
pandas-dev:mainfrom
adebayopeter:fix-dtype-error-message

Conversation

@adebayopeter

Copy link
Copy Markdown

Closes #61385

Problem

When using a raw DB-API connection (e.g. sqlite3 without SQLAlchemy),
to_sql(dtype=...) requires SQL type strings. Passing a SQLAlchemy type
object (e.g. DOUBLE()) raised an unhelpful error:

ValueError: A (<sqlalchemy.sql.sqltypes.DOUBLE object at ...>) not a string

This gave no indication of why a string was required or what to do
about it.

Fix

Per @rhshadrach's guidance on the issue (no automatic fallback
conversion — that logic shouldn't be owned by pandas), this PR only
improves the error message to explain the constraint and point users
toward using a SQLAlchemy engine if they need SQLAlchemy types:

ValueError: Column 'A' has dtype '...' which is not a string. When 
using a DB-API connection (e.g. sqlite3), dtype values must be SQL 
type strings (e.g. 'TEXT', 'FLOAT'). To use SQLAlchemy types, use a 
SQLAlchemy engine instead.

Tests

Added test_sqlite_dtype_not_string_raises covering the new message
on a raw sqlite3 connection.

…n (GH#61385)

When using a raw DB-API connection (e.g. sqlite3 without SQLAlchemy),
to_sql(dtype=...) requires SQL type strings. Passing a SQLAlchemy
type object previously raised an unhelpful error showing only the
column name and repr of the type.

This adds a clearer message explaining the constraint and suggesting
the use of a SQLAlchemy engine for SQLAlchemy types, per maintainer
guidance on the issue.

No fallback conversion is added, consistent with the maintainer's
stated preference that pandas not own such conversion logic.
The existing test asserted on the old error message format
('B (<class 'bool'>) not a string'). Updated the expected regex
to match the new, more descriptive message introduced in this PR.
Comment thread pandas/io/sql.py Outdated
Comment thread pandas/io/sql.py Outdated
Comment thread pandas/tests/io/test_sql.py Outdated
Per @rhshadrach's review:
- Changed 'dtype' to 'type' in the error message, since the offending
  value is not necessarily a pandas/numpy dtype (it may be any
  non-string object, e.g. a SQLAlchemy type).
- Removed the SQLAlchemy-specific suggestion from the message, since
  it's a non-sequitur when my_type isn't actually a SQLAlchemy type.
- Removed test_sqlite_dtype_not_string_raises as redundant: the
  existing test_sqlite_test_dtype already exercises this exact
  ValueError path and asserts against the updated message.

Verified: 616 sqlite tests passing, pre-commit clean.
@adebayopeter

Copy link
Copy Markdown
Author

Thanks for the thorough review @rhshadrach — all addressed:

  1. Removed the SQLAlchemy-specific suggestion from the message, since it's a non-sequitur when my_type isn't actually an SQLAlchemy type.
  2. Changed "dtype" to "type" in the column-level message, per your suggested wording — agreed that "dtype" implies a pandas/numpy dtype specifically, which this isn't.
  3. Removed test_sqlite_dtype_not_string_raises — you're right that it was redundant with test_sqlite_test_dtype, which already covers this exact path and now asserts against the updated message.

Pushed the latest commit. 616 sqlite tests passing, pre-commit clean.

@adebayopeter adebayopeter requested a review from rhshadrach June 20, 2026 13:53
Comment thread pandas/io/sql.py Outdated
Per @jbrockmendel's feedback: the previous phrasing 'Column has type X
which is not a string' could be misread as describing the column's
actual type in the table, rather than the invalid dtype value passed
by the user.

Reworded to 'Invalid type X for dtype of column Y: expected a string'
to make clear it's describing the dtype argument the user passed, not
the table column's type.

Verified: 616 sqlite tests passing, pre-commit clean.
Comment thread doc/source/whatsnew/v3.1.0.rst Outdated
Per @jbrockmendel's question, the whatsnew entry referenced
SQLAlchemy/engine guidance that was removed from the actual error
message in an earlier review round. Updated to match current
behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: to_sql works only for strings

3 participants