Fix InMemoryRegistrar performance: O(1) lookups, ConcurrentHashMap, cache clearing#2
Conversation
…ntHashMap, cache clearing - Add parent-child index (childrenByParent) to eliminate O(n) scan in getChildTargets() - Add count-by-type index (targetCountByType, labelCount) for O(1) getTargetCount()/getLabelCount() - Replace HashMap/HashSet with ConcurrentHashMap for thread-safe reads without coarse locking - Remove synchronized from read-only methods hasTarget(), isTargetReferenced(), isIdentifierReferenced() - Add ValidationTarget.clearCache() and call it from ValidateLauncher after each validation to prevent OOM Fixes #1569 Co-Authored-By: jordan.h.padams <jordan.h.padams@jpl.nasa.gov>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Fixes potential ConcurrentModificationException when isIdentifierReferenced() reads inner Sets without synchronization while addIdentifierReference() or setTargetIdentifier() writes to them concurrently. Co-Authored-By: jordan.h.padams <jordan.h.padams@jpl.nasa.gov>
…ype on duplicate addTarget calls Guard index updates with an isNew check so that re-registering the same location does not append duplicate children or over-count target types. The old code derived counts/children from the targets Map (unique keys), so duplicates were implicitly handled. The new indexes must be explicitly guarded.
jordanpadams
left a comment
There was a problem hiding this comment.
Review: InMemoryRegistrar Performance Optimization
Good performance work overall — the O(1) child lookup and count caching address real scalability problems for large bundles. A few issues need to be resolved before merge.
Critical: Semantic Change in getChildTargets() Without Test Coverage
The old implementation found children by filesystem path-prefix matching:
targetLocation.startsWith(parentLocation + File.separator)
&& !targetLocation.substring(parentLocation.length()).contains(File.separator)
The new implementation relies entirely on the parentLocation argument passed to addTarget(). These are equivalent only if every caller passes the correct direct parent. If any caller passes a grandparent, null, or an unrelated location, children will be silently omitted — breaking referential integrity checks.
Please: (1) audit all addTarget() call sites to confirm parentLocation is always the immediate parent; (2) add a test that exercises getChildTargets() on a multi-level hierarchy.
Bug: Mutable ArrayList Inside ConcurrentHashMap Is Misleading
childrenByParent.computeIfAbsent(parentLocation, k -> new ArrayList<>()).add(location);
The ArrayList is not thread-safe. It happens to be safe here because both addTarget() and getChildTargets() are synchronized, so the ArrayList is only accessed under the object monitor. But the ConcurrentHashMap wrapper implies lock-free safety, which is misleading. Either document clearly that the ArrayList is only accessed under synchronized, or use a truly concurrent list type.
Concern: clearCache() Timing
validator.validate(monitor, target);
monitor.endValidation();
ValidationTarget.clearCache(); // called immediately after
If validator.validate() spawns background threads that still hold references to cachedTargets entries (e.g., async content validation), those threads will see nulls after clearCache(). Please confirm monitor.endValidation() guarantees all spawned work is complete before returning.
Minor: labelCount Is a Plain int, Inconsistent with ConcurrentHashMap Approach
labelCount is only modified in synchronized methods so visibility is correct today. But it is inconsistent with the rest of the PR — if synchronized is later removed from setTargetIsLabel()/getLabelCount(), this silently becomes a data race. Consider AtomicInteger.
Minor: Redundant synchronized on Methods That Only Read ConcurrentHashMap
getTargetCount() and getLabelCount() are still synchronized but only read from concurrent structures. The synchronization is unnecessary and contradicts the PR's goal of reducing lock contention.
Required Before Merge: Full Test Suite
Only mvn compile was verified locally per the PR description. Please run at minimum:
mvn test -Dtest=\!ReferenceIntegrityTest* -Dcucumber.filter.tags='@v3.7.x'
The getChildTargets() semantic change makes this especially important.
…chronized, document ArrayList thread-safety, add InMemoryRegistrarTest Co-Authored-By: jordan.h.padams <jordan.h.padams@jpl.nasa.gov>
|
All six points from this review have been addressed in commit 1. Semantic Change in
|
| this.collections.put(location, target); | ||
| } | ||
|
|
||
| boolean isNew = !this.targets.containsKey(location); | ||
| this.targets.put(location, target); | ||
|
|
||
| // Only update indexes for genuinely new targets to avoid duplicates | ||
| if (isNew) { | ||
| // Index parent-child relationship for O(1) child lookups | ||
| if (parentLocation != null) { | ||
| childrenByParent.computeIfAbsent(parentLocation, k -> new ArrayList<>()).add(location); | ||
| } | ||
|
|
||
| // Increment count-by-type index | ||
| targetCountByType.merge(type, 1, Integer::sum); | ||
| } |
There was a problem hiding this comment.
🟡 targetCountByType becomes stale when target is re-added with a different TargetType
In addTarget(), the isNew check at line 71 prevents updating targetCountByType when the same location is added again. However, the target object in the targets map IS replaced unconditionally (line 72), and bundles/collections maps are also updated unconditionally (lines 65-69). If the same location is re-added with a different TargetType, the count for the old type remains inflated and the count for the new type is missing the entry — yet the targets map now has the new target.
While the primary caller RegisterTargets guards against re-addition (InMemoryRegistrar.java:44 via registrar.hasTarget()), addTarget() itself is a public interface method with no such contract, and the bundles/collections maps are still unconditionally updated even when isNew is false — creating an inconsistency where a target can appear in bundles but not be reflected in targetCountByType for BUNDLE.
Was this helpful? React with 👍 or 👎 to provide feedback.
jordanpadams
left a comment
There was a problem hiding this comment.
Re-Review: InMemoryRegistrar Performance Optimization
All five issues from the initial review have been addressed. The AtomicInteger for labelCount, removal of redundant synchronized on the count getters, the multi-level hierarchy test, the addTarget() call-site audit, and the clearCache() timing analysis are all solid additions. The test suite passes.
One Remaining Concern: Re-registration With a Different TargetType
The isNew guard correctly prevents duplicate children and inflated counts when addTarget() is called multiple times for the same location. However, it only checks whether the location is new — it does not handle the case where the same location is re-registered with a different TargetType:
boolean isNew = !this.targets.containsKey(location);
this.targets.put(location, target); // overwrites old target (new type)
if (isNew) {
targetCountByType.merge(type, 1, Integer::sum); // only runs for first registration
}If this ever happens, targets will reflect the new type, but targetCountByType will still count the old type. The PR's own checklist flags this as an open item. The original code was also undefined for this case, so this is pre-existing behavior, but worth confirming: can addTarget() ever be called for the same location with different types? The RegisterTargets.java audit presumably rules this out for the two known call sites, but a brief comment in the code (or an assert type == existingTarget.getType() under a debug flag) would make the assumption explicit and catch future regressions.
Minor: getTargetCount() Reads Boxed Integer From ConcurrentHashMap Without Synchronization
public int getTargetCount(TargetType type) {
return targetCountByType.getOrDefault(type, 0);
}ConcurrentHashMap.merge() and getOrDefault() are each individually atomic, but the boxed Integer values mean each write via merge() allocates a new object. This is fine for correctness (ConcurrentHashMap's internal locking ensures visibility), but worth noting if GC pressure ever becomes a concern on very large bundles. LongAdder per type (or ConcurrentHashMap<TargetType, AtomicInteger>) would eliminate the allocation, but this is a minor optimization and not a blocker.
Confirmed Fixed
getChildTargets()semantic change audited: bothRegisterTargets.javacall sites pass the correct direct parent ✓- Multi-level hierarchy test (
testGetChildTargetsMultiLevelHierarchy) added and passes ✓ ArrayListinsideConcurrentHashMapdocumented as safe undersynchronized✓clearCache()timing confirmed safe:BlockingTaskManager.submit()is synchronous ✓labelCount→AtomicInteger✓- Redundant
synchronizedremoved fromgetTargetCount()andgetLabelCount()✓ - Full test suite passes ✓
Summary
This is ready to merge once the re-registration concern is either confirmed impossible (with a comment in the code) or handled defensively. Everything else looks good.
Remaining Issue 1: Re-registration of the same location with a different
|
…to AtomicInteger to avoid boxing Co-Authored-By: jordan.h.padams <jordan.h.padams@jpl.nasa.gov>
|
Both issues addressed in commit Issue 1: Re-registration with different TargetTypeAdded a defensive check in if (!isNew) {
ValidationTarget existing = this.targets.get(location);
if (existing != null && !type.equals(existing.getType())) {
LOG.warn("addTarget(): location {} re-registered with type {} (was {})",
location, type, existing.getType());
}
}I chose Issue 2: Boxed Integer allocationsSwitched // Before: allocates a new Integer on every merge()
targetCountByType.merge(type, 1, Integer::sum);
// After: in-place increment, no boxing
targetCountByType.computeIfAbsent(type, k -> new AtomicInteger(0)).incrementAndGet();
All 6 |
There was a problem hiding this comment.
🟡 setTargets() bypasses new indexes, causing stale getChildTargets(), getTargetCount(), and getLabelCount() results
The PR changes getChildTargets(), getTargetCount(), and getLabelCount() to use precomputed indexes (childrenByParent, targetCountByType, labelCount) instead of scanning the targets map. However, the existing setTargets() method at InMemoryRegistrar.java:338 replaces the targets map without rebuilding these indexes. After a setTargets() call, the indexes become stale: getChildTargets() returns children from the old map, getTargetCount() returns old counts, and getLabelCount() reflects old label state. Before this PR, all three methods scanned the live targets map, so setTargets() was inherently consistent. The same issue applies to setBundles() and setCollections(), though those only affect their own maps. While there are no current callers of setTargets() in the codebase, it is a public method on the TargetRegistrar interface (TargetRegistrar.java:191).
(Refers to lines 338-340)
Prompt for agents
The setTargets() method (and similarly setBundles/setCollections) replaces the targets map but does not rebuild the childrenByParent, targetCountByType, or labelCount indexes that were introduced in this PR. Before the PR, getChildTargets/getTargetCount/getLabelCount scanned the live targets map, so setTargets was inherently consistent. Now they use precomputed indexes that become stale.
Possible approaches:
1. Rebuild all indexes inside setTargets() by iterating the new map (but parent-child info is lost since it's not stored on ValidationTarget).
2. Throw UnsupportedOperationException in setTargets() if there are no callers, or deprecate it.
3. Document that setTargets() must not be called after addTarget() has been used, or that callers must also manually rebuild indexes.
The same issue applies to setCollections() and setBundles(), though those don't affect the child/count indexes directly.
Files: InMemoryRegistrar.java (setTargets at line 338, setBundles at line 328, setCollections at line 318), TargetRegistrar.java interface.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
Addresses performance bottlenecks in
InMemoryRegistrarthat cause OOM and slow validation on large bundles (23k+ products). Four files changed:InMemoryRegistrar.javachildrenByParentindex populated inaddTarget()sogetChildTargets()is O(1) lookup instead of O(n) full scan.targetCountByTypemap (ConcurrentHashMap<TargetType, AtomicInteger>) andlabelCount(AtomicInteger) counter sogetTargetCount()/getLabelCount()return cached values instead of iterating all targets. UsesAtomicIntegervalues to avoid boxedIntegerallocations on every increment.isNewcheck to prevent duplicate children and inflated counts whenaddTarget()is called multiple times for the same location. Logs a warning if a location is ever re-registered with a differentTargetType(documents the invariant that this should not happen).HashMap/HashSetwithConcurrentHashMap/ConcurrentHashMap.newKeySet()— including inner Sets inidentifierDefinitionsByLidandreferencedIdentifiersByLid— and removessynchronizedfrom five read-only methods:hasTarget(),isTargetReferenced(),isIdentifierReferenced(),getTargetCount(),getLabelCount().ValidationTarget.javaclearCache()static method to clear the unboundedcachedTargetsstatic HashMap.ValidateLauncher.javaValidationTarget.clearCache()after each target validation completes to free memory between runs.InMemoryRegistrarTest.java(new)addTarget()deduplication, count-by-type, count inflation prevention, and label counting.Updates Since Initial Review
Addresses all six points from @jordanpadams' first review, plus two follow-up items:
addTarget()call sites audited: Both call sites inRegisterTargets.javapass the correct direct parent (getParentTarget()at line 41 andtargetLocationat line 59).testGetChildTargetsMultiLevelHierarchy()builds bundle → collections → products and verifies only direct children are returned at each level.childrenByParentfield explains that inner ArrayLists are safe because bothaddTarget()andgetChildTargets()aresynchronized.clearCache()timing confirmed safe:BlockingTaskManager.submit()runs tasks synchronously, somonitor.endValidation()guarantees all work is complete beforeclearCache()is called.labelCount→AtomicInteger: Future-proofed against removal ofsynchronizedfromsetTargetIsLabel().synchronizedremoved fromgetTargetCount()andgetLabelCount()— they now only read fromConcurrentHashMapandAtomicInteger.addTarget()now reads the existing target's type before overwriting and emitsLOG.warnif the type differs. The invariant is documented inline.targetCountByType→ConcurrentHashMap<TargetType, AtomicInteger>(follow-up): Eliminates short-lived boxedIntegerallocations frommerge()on everyaddTarget()call. UsescomputeIfAbsent(...).incrementAndGet()instead.mvn test -Dtest=\!ReferenceIntegrityTest* -Dcucumber.filter.tags='@v3.7.x'→ 38 run, 0 failures, BUILD SUCCESS.Review & Testing Checklist for Human
getChildTargets(): The old implementation discovered children by filesystem path-prefix matching (parentLocation + File.separator). The new implementation relies on theparentLocationargument passed toaddTarget(). The twoRegisterTargets.javacall sites were audited and a unit test was added, but other code paths or plugins that calladdTarget()may exist — grep foraddTargetacross the full codebase to confirm.addTarget()is called for the same location with a differentparentLocation, thechildrenByParentindex retains only the original parent's entry. The type-change case now logs a warning, but the parent-change case is silent. Verify this never happens in practice.clearCache()in non-CLI contexts: TheBlockingTaskManageranalysis confirms safety for the CLI path. Ifvalidateis used as a library with an asyncTaskManager,clearCache()could race with in-flight work. Verify thatValidateLauncheris the only entry point that matters.Notes
setTargets()/setCollections()/setBundles()public setters can replace theConcurrentHashMapinstances with arbitraryMapimplementations — pre-existing issue, not introduced here.identifierDefinitionsByLidandreferencedIdentifiersByLidnow useConcurrentHashMap.newKeySet()instead ofHashSetto avoidConcurrentModificationExceptionfrom unsynchronized reads inisIdentifierReferenced().Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/085fab5912d94d948b9826a7dd8d7b71