This offers an optimization that allows loading larger
inventories of various structure by improving the
scaling laws involved for adding hosts and groups.
The primary speed benefit is the elimination of duplicate
recusion from traversing converging paths.
(cherry picked from commit 153c9bd539)
