Problem 2527

We have found a possible performance improvement for sequential runs without changed geometry.

In our use case (https://allpix-squared.docs.cern.ch/) we create many runs with mostly one particle while not touching the geometry. We noticed that when adding many volumes (imagine e.g. five 800x800 pixel detectors) the RunInitialization() method in BeamOn() takes a significant amount of time. After profiling the culprit was pinned down to G4RegionStore::UpdateMaterialList().
The full call tree:

G4RunManager::BeamOn()
G4RunManager::RunInitialization()
G4RunManagerKernel::RunInitialization
G4RunManagerKernel::UpdateRegion()
G4RegionStore::UpdateMaterialList()

In Allpix Squared we worked around this using implementing a custom Run Manager based on the WorkerRunManager. A minimal working implementation of it can be found here: https://github.com/stephanlachnit/Geant4MWE/tree/runManagerTest

This works because G4RunManagerKernel::UpdateRegion() explicitly checks if runManagerKernelType is workerRMK and skips G4RegionStore::UpdateMaterialList() in that case. It is not clear to me why this is implemented in such a way.

With this setup, we can significantly improve performance:

# 1x BeamOn(100)
## customWorkerMTRunManager
Result Prep:  49600 ms
Result Bench: 0 ms
## G4MTRunManager
Result Prep:  49638 ms
Result Bench: 372 ms
## G4RunManager
Result Prep:  48381 ms
Result Bench: 350 ms
# 10x BeamOn(10)
## customWorkerMTRunManager
Result Prep:  47786 ms
Result Bench: 1 ms
## G4MTRunManager
Result Prep:  47768 ms
Result Bench: 2689 ms
## G4RunManager
Result Prep:  48917 ms
Result Bench: 2682 ms
# 100x BeamOn(1)
## customWorkerMTRunManager
Result Prep:  49147 ms
Result Bench: 12 ms
## G4MTRunManager
Result Prep:  50295 ms
Result Bench: 26965 ms
## G4RunManager
Result Prep:  50427 ms
Result Bench: 26690 ms


All benchmarks were run on one thread only. You can run these benchmarks yourself from the code in the repo.

It's clear that the actual benchmark ("Result Bench") scales linearly with the amount of calls to BeamOn() and not the actual number of simulation runs which is identical in all three cases.

From my limited understand I think there is a check to GeometryHasChanged() missing, and if this function is insufficient to test if G4RegionStore::UpdateMaterialList() needs to be called then another check should be implemented. I assume complex geometry does not only apply to Allpix Squared but ATLAS, CMS, etc simulations as well.