Skip to content

[#692] Restore partial import semantics for include/exclude branches#689

Open
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:features/import-include-branch
Open

[#692] Restore partial import semantics for include/exclude branches#689
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:features/import-include-branch

Conversation

@vharseko

@vharseko vharseko commented Jul 4, 2026

Copy link
Copy Markdown
Member

Fixes #692

Problem

Since the import was rewritten around OnDiskMergeImporter (DJ 3.x), a partial import — import-ldif with --includeBranch and/or --excludeBranchsilently destroys all existing entries outside of the imported scope: the importer deletes the whole entry container (beforePhaseOneentryContainer.delete()) before rebuilding it from the filtered LDIF content, and then aborts on the orphan check because the parent of the included branch is gone:

The import has been aborted because the entry '...ou=people' does not have a parent entry

The 2.x importer used to migrate those entries into the new container (MigrateExistingTask / MigrateExcludedTask in the old jeb/Importer.java, removed in OPENDJ-2337): --includeBranch defines the scope of the replacement, everything else in the base DN is preserved.

Found by running the slow TestNG group (excluded from CI), where TestImportAndExport.testImportExport fails on the include-branch: ou=people,dc=example,dc=com case with STOPPED_BY_ERROR.

Fix

Restore the 2.x behaviour in a way compatible with the on-disk-merge architecture (trees are rebuilt wholesale, so preserved entries must go through the import pipeline):

  • Before the import starts, exportEntriesToPreserve() snapshots the entries which must survive into temporary LDIF files using the regular ExportJob machinery (the storage is opened READ_ONLY and closed again before startImport()): for each affected base DN, the entries outside of the include branches (unless the branches cover the whole base DN), plus the entries below exclude branches located inside the replaced scope.
  • A new CompositeSource chains an unfiltered, non-schema-validating LDIFReaderSource over the snapshot files with the main LDIF source, so preserved entries are re-imported through the same phase-1/phase-2 pipeline.
  • Containers not touched by the import (no include branch below them, or an entirely excluded base DN) are left alone, as before.
  • Temporary files are deleted when the import finishes.

Verification

  • TestImportAndExport: 12/12 pass — the include-branch case completes successfully and the base entry and sibling branches survive (no orphan abort). The test is moved out of the slow group into the default build.
  • OnDiskMergeImporterTest: 29/29 pass.
  • Backend regressions: JETestCase 32/32, PDBTestCase 32/32.

Since the import was rewritten around OnDiskMergeImporter, a partial
import (import-ldif with --includeBranch and/or --excludeBranch)
silently destroyed all existing entries outside of the imported scope:
the importer deletes the whole entry container before rebuilding it
from the filtered LDIF content, and then aborts on the orphan check
because the parent of the included branch is gone ("The import has
been aborted because the entry ... does not have a parent entry").
The 2.x importer used to migrate those entries into the new container
(MigrateExistingTask/MigrateExcludedTask).

Restore that behaviour in an architecture-compatible way: before the
import starts, snapshot the entries which must survive - entries
outside of the include branches, plus entries below exclude branches
of an imported branch - into temporary LDIF files using the regular
export machinery, then feed them through the import pipeline together
with the user-provided LDIF via a composite source. Containers not
touched by the import are left alone, as before.

Move TestImportAndExport.testImportExport out of the "slow" group into
the default build.
@vharseko vharseko changed the title Restore partial import semantics for include/exclude branches [#692] Restore partial import semantics for include/exclude branches Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Partial import-ldif with --includeBranch/--excludeBranch destroys all entries outside the imported scope

2 participants