fix(client): discard truncated JSON document on mid-stream EOF#97
Open
krynju wants to merge 1 commit into
Open
fix(client): discard truncated JSON document on mid-stream EOF#97krynju wants to merge 1 commit into
krynju wants to merge 1 commit into
Conversation
JSONChunkReader's `_read_json_chunk` brace-counts one JSON document off the response BufferStream, but its `while !eof(io)` loop also terminates on end-of-stream. When a streaming response closed mid-document (a dropped or cancelled connection), the partial bytes were handed to the JSON parser and threw `Unexpected end of input`, surfacing as a logged "exception reading chunk" error in the streaming consumer. Track completeness in the object/array and string branches: only the balanced close (`depth == 0`, or the terminating quote) marks the document complete. If the loop exits via EOF while incomplete, return empty so `iterate` treats it as a clean end of stream instead of parsing a truncated document. Complete documents are unaffected. Updates the existing jsonchunk4 test (it asserted the old throw-on-truncation behavior) and adds `_read_json_chunk` unit tests for truncated object / nested object / array / string, and complete-then-truncated. Bumps to 0.2.4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
JSONChunkReader._read_json_chunkbrace-counts one JSON document off the responseBufferStream, but itswhile !eof(io)loop also terminates on end-of-stream. When a streaming response closes mid-document (a dropped/cancelled connection, or a server that ends the stream partway through an object), the partial bytes are handed toJSON.parse, which throwsUnexpected end of input. In the streaming client this surfaces as a logged error:Fix
Track completeness in the object/array and string branches of
_read_json_chunk: a document is onlycompletewhen it balances (depth == 0) or the string's terminating quote is read. If the loop exits via EOF while still incomplete, returnUInt8[]soiterate's existingisempty(chunk_bytes) && return nothingtreats it as a clean end of stream rather than parsing a truncated document. Complete documents (always ≥2 bytes) are unaffected, including}/]inside string values and nested structures.Tests
jsonchunk4— it previously asserted the buggy@test_throws TaskFailedExceptionon a truncated trailing object; it now asserts the document is discarded, iteration ends without throwing, and the complete documents read before it are still yielded._read_json_chunkunit tests: truncated object ({"statuses":), truncated nested object, truncated array ([1, 2), truncated string ("dev/termina), and complete-then-truncated.Full chunk-reader suite (62 tests) passes.
Bumps version to
0.2.4.🤖 Generated with Claude Code