Skip to content

Avoid full attribute walk for types without subordinate types#679

Open
vharseko wants to merge 2 commits into
OpenIdentityPlatform:masterfrom
vharseko:features/compare-3
Open

Avoid full attribute walk for types without subordinate types#679
vharseko wants to merge 2 commits into
OpenIdentityPlatform:masterfrom
vharseko:features/compare-3

Conversation

@vharseko

@vharseko vharseko commented Jul 3, 2026

Copy link
Copy Markdown
Member

Summary

Second result of the COMPARE hot-path profiling (after #674). CPU profiling
shows ~4% of server CPU in Entry.hasAttributeOrSubType /
AttributeType.isSuperTypeOf: subtype-inclusive attribute lookups
(getAllAttributes, hasAttribute) iterate every attribute of the
entry
, calling isSuperTypeOf() for each, because LDAP semantics require
matching subordinate attribute types (a compare on name must match cn,
sn, …). These lookups run for every entry read — compare assertions,
search filter evaluation, password policy attribute reads on bind — yet for
the vast majority of attribute types (uid, l, userPassword, …)
nothing in the schema extends them, and the walk degenerates to an exact
map lookup.

Change

  • Entry caches, per schema instance, the set of attribute types that
    have at least one subordinate type (one pass over
    schema.getAttributeTypes() walking the superior chains; rebuilt when
    the schema reference changes; the schema and its set are held in one
    immutable holder swapped atomically — see the review follow-up below).
  • The three subtype walkers (addAttributeTypeOrSubTypeValue ×2,
    hasAttributeOrSubType) take an exact-map-lookup fast path when the
    requested type has no subordinates — isSuperTypeOf() can only match
    the type itself in that case, so behaviour is identical.
  • When no server schema is available (offline tools, early startup) the
    code conservatively falls back to the full walk.
  • Public methods and their contracts are unchanged; +92 lines, nothing
    removed.

Benchmark

Same harness as #674: packaged server, je backend, 5,000 users, JDK 11,
8-core host, 200 persistent connections bound as regular users issuing
COMPAREs on random users, caffeinate, warm-up run after every restart,
interleaved order old→new→new→old→old→new:

adjacent pair old new Δ
1 42,031 ops/s / 4.75 ms 45,262 / 4.41 ms +7.7%
2 44,743 / 4.47 ms 45,213 / 4.42 ms +1.1%
3 (both in thermal fade of the shared host) 28,359 / 7.05 ms / p99.9 127 ms 33,612 / 5.95 ms / p99.9 40 ms +18.5%

Median +7.6%, consistently better tail latency; 0 errors in all runs. BIND
and search filter evaluation take the same fast path (password attribute
reads and filter matching go through the same walkers).

Review follow-up (52df338)

  • The cached schema and its set were two independent volatile fields.
    The publication order (set before schema, read in the reverse order)
    made the pairing safe for a single rebuilder, but two threads
    rebuilding concurrently for different schema instances could interleave
    the four writes and skew the pair — and the invariant lived entirely in
    the ordering of two assignments. They are now one immutable holder
    swapped atomically, which also halves the volatile reads on the hot
    path.
  • The fast path is skipped for place-holder attribute types (types not
    defined in the schema): isSuperTypeOf() matches place-holders against
    schema-defined types by name, which an exact hash lookup keyed on
    the numeric OID cannot honor. Without the guard, a lookup using a
    place-holder type could miss a same-named attribute held by an Entry
    decoded under an older schema that still defined the type. (For
    schema-defined types the equivalence holds as before:
    AttributeType.equals()/hashCode() compare the numeric OID only.)

Testing

  • mvn -P precommit -pl opendj-server-legacy verify for TestEntry,
    EntrySchemaCheckingTestCase, SearchFilterTests (10,468 — the full
    filter matching matrix), SearchOperationTestCase,
    CompareOperationTestCase, PasswordPolicyTestCase:
    10,769 tests, 0 failures.
  • Subtype semantics verified against a live server: compare on uid
    (fast path) → COMPARE_TRUE; compare on name with a cn value
    (subtype walk) → COMPARE_TRUE; wrong value → COMPARE_FALSE.

Files

  • opendj-server-legacy/src/main/java/org/opends/server/types/Entry.java

Subtype-inclusive attribute lookups (getAllAttributes, hasAttribute)
iterate every attribute of the entry calling isSuperTypeOf() for each,
because LDAP semantics require matching subordinate attribute types.
These lookups run for every entry read: compare assertions, search
filter evaluation, password policy attributes. But for the vast
majority of attribute types nothing in the schema extends them, and the
walk degenerates to an exact map lookup.

Cache, per schema instance, the set of attribute types that have at
least one subordinate type (rebuilt on schema change, published through
volatile references), and take an exact-lookup fast path in the three
subtype walkers when the requested type has no subordinates. Falls back
to the full walk when no server schema is available.

Interleaved A/B under the COMPARE benchmark scenario: +1..+18%
throughput (median +7.6%) with a visibly better p99.9; 10,769 tests
pass including the full SearchFilterTests matrix, and subtype matching
(compare on "name" matching a cn value) verified against a live server.
@vharseko vharseko added this to the 5.2.0 milestone Jul 3, 2026
@vharseko vharseko requested a review from maximthomas July 3, 2026 11:04
Hold the schema and the set computed for it in one immutable object
swapped atomically, so a reader can never pair a set with the wrong
schema even when two threads rebuild the cache concurrently (previously
the invariant relied on the field write/read order); this also halves
the volatile reads on the hot path. Skip the exact-lookup fast path for
place-holder attribute types: isSuperTypeOf() matches them against
schema-defined types by name, which a hash lookup keyed on the numeric
OID cannot honor.
@vharseko vharseko added the performance Performance / concurrency / lock-contention work label Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark enhancement performance Performance / concurrency / lock-contention work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants