| Summary: | Slow Performance of simulation high energy e-m particles due to G4NavigationHistoryPool | ||
|---|---|---|---|
| Product: | Geant4 | Reporter: | vitali choutko <vitali.choutko> |
| Component: | geometry/management | Assignee: | Gabriele Cosmo <Gabriele.Cosmo> |
| Status: | RESOLVED FIXED | ||
| Severity: | enhancement | ||
| Priority: | P5 | ||
| Version: | 10.1 | ||
| Hardware: | All | ||
| OS: | All | ||
| Attachments: |
G4NavigationHistory.hh
Revised version of G4NavigationHistoryPool |
||
Can you please provide a test case we can use to verify this? G4NavigationHistoryPool has been introduced in release 10.1 to overcome issues of memory churn due to excessive copy of vectors for the navigation-history levels, and we measured considerable benefits in several test cases, compared to previous implementations. Pasting reply sent by email... ---------------- Hi Gabriele, I will try to do that, however what we dealt with was our real simulation for AMS. I think one can easily reproduce this by setting the e+-,gamma production threshold low enough eg @ 100 mkm or less and run a few TeV electron in em calorimeter. Note also that in 9.6.x we reported a similar problem with G4EnhancedVecAllocator.hh. Just removing it and using a standard stl vector one allocator provides factor of 4 increase of speed for high energy particles, so I am not sure are the benefits you mentioned w/r with original 9.6.x memory management or with the version without G4EnhancedVecAllocator. I would like also mention that we also have quite simple and fast implementation of garbage collection for G4Allocators, which allowing us to efficiently run multi threaded jobs. Please let me know if you may be interested in that. Attached is the comparison of dynamics of memory usage per thread for 40 thread job with or without such mechanism. Best regards, Vitali Hi Vitali, the motivation which lead us to get rid of G4EnhancedVecAllocator in favour of an implementation based on a pool for memory reuse was mainly the following: - It was found G4EnhancedVecAllocator could induce a significant penalty (of the order of 30%, depending on the application) when in the user's code a particular treatment of tracks and particles is made, like stopping and copying them when reaching determined geometrical boundaries, or similar manipulations made by the experiment's software framework (cases in ATLAS and ALICE). We found that in this case G4EnhancedVecAllocator was ineffective, as allocations and deallocations of the entire pool was triggered many times, therefore invalidating any benefit it could bring. In all other normal uses, G4EnhancedVecAllocator allowed similar performance than using the standard STL allocator for vector collections, with the advantage of reducing the memory churn and allow proper scalability in MT when large number of threads are involved (as its purpose was mainly to -reduce- the number of internal calls to malloc/free, which in MT mode can cause intrinsic data-race conditions). The implementation based on a pool for memory reuse (G4NavigationHistoryPool) overcomes these limitations, yet not inducing run-time penalty, at least, according to all tests we could make and confirmation we could get from ATLAS colleagues. So, if you could provide us a relatively simple setup which allows us to reproduce the problem you see, it would definitely help! Concerning garbage collection in G4AllocatorPool, I would be interested to understand exactly what you are doing in the AMS software. I saw the presentation just made at CHEP, and looking from the plots presented, to me it looks like mostly a problem of memory leak (which we know exists in Geant4 for some hadronic models, misusing G4Allocator for the allocation of reaction-products, leading to a constant growth of the pool in long runs); we hope to fix this in one of the next patches. Now, G4AllocatorPool provides all tools to clear its content, so, if in a simulation run one periodically clears up a pool, this will certainly help in reducing the overall memory allocated, but at the expense of run-time, as the benefits of using an allocator get again, at least partially, invalidated! Cheers, Gabriele Please, reply directly in this bug-report through Bugzilla! Created attachment 332 [details]
Revised version of G4NavigationHistoryPool
I'm attaching a revised version of G4NavigationHistoryPool, where some optimisations are implemented and there's no longer dependence on vectors size in accessing elements.
Can you please try it out and let us know if that improves the situation? Thanks.
Hi Gabriele,
I confirm the the new G4NavigationhistoryPool solves the problem:
I have for E [2,10000] GeV E^{-1} electrons:
Original G4NavigationHistoryPool 123.7 sec/evt
Removing G4HistoryPool 32.6 (+-0.15) sec/evt
Modified G4NavigationHistoryPool 34.1 (+-0.15) sec/evt
Concerning G4allocators I will prepare some small note and create a new entry in a Bugzilla.
Best Regards, Vitali
Very good. Thanks for feedback Vitali. This confirms the penalty was totally due to the search logic in the pool, which becomes ineffective when the size of the pool grows due to multiplicity of an event. We'll include this revision in the next patch. Revised G4NavigationHistoryPool will be included in the next patch to release 10.1. |
Created attachment 328 [details] G4NavigationHistory.hh We have noticed significant slowdown of geant4.10.xx for high energy particles. Typically, we found that the simulation of 1 TeV electron in a calorimeter requires factor of 100 times more then 50 gev one instead of 20 as one can naively expected. We have traced this performance to an introduction of G4NavigationHistoryPool.hh in geant 4.10.xx version. Removing the reference to G4NavigationHistoryPool.hh in G4NavigationHistory.hh solves this problems and restore the essentially linear dependence of electromagnetic showers with energy.