Skip to content

[dagster-dbt] Surface isolated dbt-fusion models in asset selection#33938

Open
Vamsi-klu wants to merge 2 commits into
dagster-io:masterfrom
Vamsi-klu:dbt-fusion-isolated-models
Open

[dagster-dbt] Surface isolated dbt-fusion models in asset selection#33938
Vamsi-klu wants to merge 2 commits into
dagster-io:masterfrom
Vamsi-klu:dbt-fusion-isolated-models

Conversation

@Vamsi-klu

Copy link
Copy Markdown

Summary

DbtProjectComponent (and any selection that goes through select_unique_ids) silently drops dbt models that contain no source()/ref() calls when the project is compiled with dbt-fusion (manifest schema v2+). The model is present and enabled in manifest["nodes"], yet it never appears as a Dagster asset and no error is raised.

Root cause

_select_unique_ids_from_manifest in dagster_dbt/utils.py builds the selection graph solely from manifest["child_map"]:

graph = graph_selector.Graph(DiGraph(incoming_graph_data=child_map))

dbt-core keys child_map by every node, but dbt-fusion omits any node that has neither parents nor children. A model with no source()/ref() calls (and nothing referencing it) is therefore absent from the DiGraph. NodeSelector intersects matched nodes with the graph's node set, so a node that is missing from the graph can never be selected — it is silently dropped.

Fix

After constructing the graph from child_map, add any selectable unique_ids from the manifest that are missing from the graph. This is a no-op for well-formed dbt-core manifests (where child_map already contains every node) and only repairs fusion manifests.

Testing

Added test_select_unique_ids_includes_isolated_fusion_models, which builds a manifest where an isolated model is absent from child_map (the exact dbt-fusion shape from the issue) and asserts the model is still selected. Verified the test fails without the fix and passes with it. ruff clean.

closes: #33801


Was generative AI tooling used to author this PR?

Yes — Claude Code (Opus 4.8). Generated-by: Claude Code (Opus 4.8)

dbt-fusion manifests omit nodes with no parents or children from
child_map, so a model with no source()/ref() calls was silently
dropped from the asset graph. Add missing selectable node ids to the
selection graph before running NodeSelector; this is a no-op for
dbt-core manifests. Adds a regression test.

closes dagster-io#33801
@Vamsi-klu Vamsi-klu marked this pull request as ready for review June 19, 2026 23:31
@greptile-apps

greptile-apps Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a silent asset-drop in _select_unique_ids_from_manifest when a project is compiled with dbt-fusion: isolated models (no source()/ref() and no dependents) are absent from child_map in dbt-fusion manifests, so they were never added to the NetworkX DiGraph and were silently filtered out by NodeSelector.

  • utils.py: After building the DiGraph from child_map, collect all selectable unique_ids from the eight known manifest collections and add any that are absent from the graph — a no-op for dbt-core manifests where child_map is already exhaustive.
  • test_asset_selection.py: Two new test functions verify both point selection (isolated) and broad selection (fqn:*) on a minimal fusion-shaped manifest, and a second test validates a mixed graph that has a source-connected model alongside an isolated one.

Confidence Score: 5/5

Safe to merge — the change is additive and explicitly a no-op for well-formed dbt-core manifests.

The graph-backfill step only adds nodes; it never removes edges or rewrites existing graph data. For dbt-core manifests, child_map already contains every node, so the set difference is always empty and add_nodes_from is never called. For dbt-fusion manifests the fix restores the expected behavior. Tests confirm both the isolated-model regression and the mixed graph, and the production code path is narrow and well-understood.

No files require special attention.

Important Files Changed

Filename Overview
python_modules/libraries/dagster-dbt/dagster_dbt/utils.py Adds isolated-node backfill to the DiGraph before passing it to graph_selector.Graph; well-commented, no-op for dbt-core manifests, and covers all known selectable manifest collections.
python_modules/libraries/dagster-dbt/dagster_dbt_tests/core/test_asset_selection.py Two new parameterized tests cover the core regression (isolated model absent from child_map) and a mixed graph with both connected and isolated nodes; helper builders keep the fixtures readable.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[manifest_json] --> B[Build DiGraph from child_map]
    B --> C{Any selectable unique_ids\nmissing from graph?}
    C -- Yes\ndbt-fusion isolated nodes --> D[add_nodes_from missing ids]
    C -- No\nnormal dbt-core manifest --> E[No-op]
    D --> F[graph_selector.Graph]
    E --> F
    F --> G[NodeSelector]
    A --> H[Manifest object\nnodes / sources / metrics / etc.]
    H --> G
    G --> I[Apply select / exclude / selector string]
    I --> J[Return matched unique_ids]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[manifest_json] --> B[Build DiGraph from child_map]
    B --> C{Any selectable unique_ids\nmissing from graph?}
    C -- Yes\ndbt-fusion isolated nodes --> D[add_nodes_from missing ids]
    C -- No\nnormal dbt-core manifest --> E[No-op]
    D --> F[graph_selector.Graph]
    E --> F
    F --> G[NodeSelector]
    A --> H[Manifest object\nnodes / sources / metrics / etc.]
    H --> G
    G --> I[Apply select / exclude / selector string]
    I --> J[Return matched unique_ids]
Loading

Reviews (2): Last reviewed commit: "[dagster-dbt] Expand dbt fusion isolated..." | Re-trigger Greptile

@Vamsi-klu

Copy link
Copy Markdown
Author

Just marked this ready for review. It's a small dagster-dbt change that surfaces isolated dbt-fusion models in asset selection, with a focused test added. @smackesey when you get a chance, could you take a look or route it to the right dbt owner? CI is gated for fork PRs, so it'll need a maintainer to unblock it. cc @cmpadden for triage. Thanks!

@Vamsi-klu Vamsi-klu force-pushed the dbt-fusion-isolated-models branch from fb9a6ee to 9f7f404 Compare June 20, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[dagster-dbt] DbtProjectComponent silently drops isolated dbt models (no source()/ref()) when using dbt-fusion manifests

1 participant