Skip to content

BUG: read_csv pyarrow engine ignored tuple names for MultiIndex columns#65925

Open
jbrockmendel wants to merge 2 commits into
pandas-dev:mainfrom
jbrockmendel:tst-pyarrow-csv-tuple-names
Open

BUG: read_csv pyarrow engine ignored tuple names for MultiIndex columns#65925
jbrockmendel wants to merge 2 commits into
pandas-dev:mainfrom
jbrockmendel:tst-pyarrow-csv-tuple-names

Conversation

@jbrockmendel

Copy link
Copy Markdown
Member

Splits the tuple-namesMultiIndex columns fix out of GH-65862 into a standalone, bite-size change.

With engine="pyarrow", passing tuples in names produced flat columns (an Index of tuples) instead of MultiIndex columns like the C and python engines:

>>> pd.read_csv(StringIO("1,2\n3,4"), names=[("a", "x"), ("b", "y")], engine="pyarrow").columns
Index([('a', 'x'), ('b', 'y')], dtype='object')   # was flat; other engines give a MultiIndex

The fix routes the result columns through the same _maybe_make_multi_index_columns helper the C/python engines already use, so all engines now agree.

  • 4-line fix in arrow_parser_wrapper.py (no-op when columns aren't tuples)
  • un-xfails the names-tuple parametrizations of test_header_multi_index_common_format1/2/3 — their blanket @xfail_pyarrow only applied because the separate header=[0, 1] parametrization raises; those now assert the correct MultiIndex, with the list-header variant kept as a conditional xfail (that's a separate concern handled in BUG: raise helpful errors for unsupported pyarrow read_csv usage #65862)
  • adds a dedicated regression test

🤖 Generated with Claude Code

Passing tuples in ``names`` to ``read_csv`` with ``engine="pyarrow"``
produced flat columns instead of MultiIndex columns like the other
engines. Route the result columns through ``_maybe_make_multi_index_columns``
as the C/python engines do.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jbrockmendel jbrockmendel added Bug IO CSV read_csv, to_csv labels Jun 21, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug IO CSV read_csv, to_csv

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant