BUG: improve error message for non-string dtype with DB-API connection (GH#61385)#65893
Open
adebayopeter wants to merge 5 commits into
Open
BUG: improve error message for non-string dtype with DB-API connection (GH#61385)#65893adebayopeter wants to merge 5 commits into
adebayopeter wants to merge 5 commits into
Conversation
…n (GH#61385) When using a raw DB-API connection (e.g. sqlite3 without SQLAlchemy), to_sql(dtype=...) requires SQL type strings. Passing a SQLAlchemy type object previously raised an unhelpful error showing only the column name and repr of the type. This adds a clearer message explaining the constraint and suggesting the use of a SQLAlchemy engine for SQLAlchemy types, per maintainer guidance on the issue. No fallback conversion is added, consistent with the maintainer's stated preference that pandas not own such conversion logic.
The existing test asserted on the old error message format
('B (<class 'bool'>) not a string'). Updated the expected regex
to match the new, more descriptive message introduced in this PR.
rhshadrach
requested changes
Jun 20, 2026
Per @rhshadrach's review: - Changed 'dtype' to 'type' in the error message, since the offending value is not necessarily a pandas/numpy dtype (it may be any non-string object, e.g. a SQLAlchemy type). - Removed the SQLAlchemy-specific suggestion from the message, since it's a non-sequitur when my_type isn't actually a SQLAlchemy type. - Removed test_sqlite_dtype_not_string_raises as redundant: the existing test_sqlite_test_dtype already exercises this exact ValueError path and asserts against the updated message. Verified: 616 sqlite tests passing, pre-commit clean.
Author
|
Thanks for the thorough review @rhshadrach — all addressed:
Pushed the latest commit. 616 sqlite tests passing, pre-commit clean. |
Per @jbrockmendel's feedback: the previous phrasing 'Column has type X which is not a string' could be misread as describing the column's actual type in the table, rather than the invalid dtype value passed by the user. Reworded to 'Invalid type X for dtype of column Y: expected a string' to make clear it's describing the dtype argument the user passed, not the table column's type. Verified: 616 sqlite tests passing, pre-commit clean.
Per @jbrockmendel's question, the whatsnew entry referenced SQLAlchemy/engine guidance that was removed from the actual error message in an earlier review round. Updated to match current behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #61385
Problem
When using a raw DB-API connection (e.g.
sqlite3without SQLAlchemy),to_sql(dtype=...)requires SQL type strings. Passing a SQLAlchemy typeobject (e.g.
DOUBLE()) raised an unhelpful error:This gave no indication of why a string was required or what to do
about it.
Fix
Per @rhshadrach's guidance on the issue (no automatic fallback
conversion — that logic shouldn't be owned by pandas), this PR only
improves the error message to explain the constraint and point users
toward using a SQLAlchemy engine if they need SQLAlchemy types:
Tests
Added
test_sqlite_dtype_not_string_raisescovering the new messageon a raw sqlite3 connection.