Skip to content

design(config): define ownership of typed operation inputs #121

Description

@keli-wen

Context

PR #120 exposed an unresolved design boundary in the operation input and configuration layer. quantmind/configs/paper.py currently combines paper identifiers such as ArxivIdentifier and DoiIdentifier, transport locators such as HttpUrl and LocalFilePath, and content inputs such as RawText. Some are structurally duplicated in other flow modules, while some variants validate as PaperInput even though Paper Flow V1 rejects them at runtime.

This issue intentionally separates the cross-flow Pydantic placement decision from #119 and PR #120. The Paper Flow refactor should not silently establish a repository-wide convention while implementing one vertical slice.

Questions to Resolve

  • When two inputs have the same fields and discriminator, should they share one transport-level Pydantic type or remain domain-specific nominal types?
  • Should operation input unions contain only currently executable variants, or may they include reserved variants that fail at runtime?
  • Should reusable source locators live under quantmind.configs, quantmind.preprocess, a dedicated input package, or beside the public operation that consumes them?
  • Should a flow configuration remain flat, or compose stage-specific models such as ingest, parsing, chunking, summarization, persistence, and runtime settings?
  • Which values should be closed enums or literals, and which extension points must remain open strings?
  • How should any move preserve Pydantic JSON schema, YAML and magic resolution, discriminator stability, imports, and migration behavior?

Design Options

Shared structural types

Define reusable values such as HttpSource, LocalFileSource, and RawTextSource, then compose them into operation-specific discriminated unions.

This minimizes duplication but can erase domain semantics when the same shape has different content guarantees or runtime behavior.

Flow-owned nominal types

Keep each input model beside its operation and give it a domain-specific name such as PaperPdfUrl or EarningsReleaseUrl, even when fields match.

This preserves semantic intent but duplicates validation and transport behavior.

Hybrid boundary

Share transport primitives and validators while exposing operation-owned semantic wrappers or aliases. Operation unions contain only supported variants; future variants are introduced when their resolver exists. Stage-specific configuration models are composed by a small operation or pipeline config.

This is the initial candidate, not a decision. The discussion should test it against current paper, earnings, news, batch, magic, YAML, and schema-generation use cases.

Acceptance Criteria

  • Document the ownership rule for reusable transport values, domain identifiers, operation inputs, and stage configuration.
  • Decide whether unsupported reserved variants belong in public discriminated unions.
  • Define naming and import conventions that avoid same-named models with different semantics.
  • Show the intended Pydantic JSON schema for at least Paper and Earnings inputs.
  • Specify compatibility and migration behavior for existing Python, YAML, and magic callers.
  • Implement focused unittest.TestCase coverage and one concise example after the design is accepted.

Related Work

Metadata

Metadata

Assignees

Labels

area: flowsPublic operation implementations under quantmind/flows/type: featureAdds a new capability or observable behavior

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions