Problem 1731 - G4PAIModel is broken in MT mode with G4 10.0+
Summary: G4PAIModel is broken in MT mode with G4 10.0+
Status: RESOLVED FIXED
Alias: None
Product: Geant4
Classification: Unclassified
Component: processes/electromagnetic (show other problems)
Version: 10.1
Hardware: All All
: P5 normal
Assignee: Vladimir.Grichine
URL:
Depends on:
Blocks:
 
Reported: 2015-04-09 10:10 CEST by Bastian Beischer
Modified: 2015-06-09 00:46 CEST (History)
2 users (show)

See Also:


Attachments
proposed fix for the problem (1.58 KB, patch)
2015-04-09 10:10 CEST, Bastian Beischer
Details | Diff
single thread: correct distribution of deposited energy (92.02 KB, image/png)
2015-04-22 14:00 CEST, Bastian Beischer
Details
multiple threads: broken distribution of deposited energy (85.51 KB, image/png)
2015-04-22 14:00 CEST, Bastian Beischer
Details

Note You need to log in before you can comment on or make changes to this problem.
Description Bastian Beischer 2015-04-09 10:10:30 CEST
Created attachment 329 [details]
proposed fix for the problem

The G4PAIModel is broken when using G4 10.0+ in MT mode.

The reason is that the worker thread instances of the process do not fill their fMaterialCutsCoupleVector. Then, in the functions which calculate energy loss fluctuations (et al) the FindCoupleIndex() function always returns -1 because the vector is empty.

This leads to very sharply peaked energy deposits which look non physical when using G4PAIModel in G4 10 MT applications. This already happens with just one worker thread.

Single threaded applications are not affected by this issue.

Attached is a proposed patch which fixes the issue.
Comment 1 Vladimir.Ivantchenko 2015-04-09 13:31:28 CEST
Hello Vladimir,

this problem may take place - our physics tests were sequential, so we may miss the problem during release validation. If I understand correctly we need to be sure that G4PAIModel, G4PAIPhotModel are properly initialized in worker threads. For that two extra access methods to material cut couple vector an drange vector should be added and in InitialiseLocal these vectors should be initialized.

Vladimir
Comment 2 Vladimir.Grichine 2015-04-22 11:19:41 CEST
Hi Bastian,
TestEm8 runs properly with PAI macro and MT number 1-4. It shows reasonable dE/dx distributions. Can you kindly send your application including files with proposed fix? 
Thanks.
Comment 3 Bastian Beischer 2015-04-22 14:00:28 CEST
Created attachment 335 [details]
single thread: correct distribution of deposited energy
Comment 4 Bastian Beischer 2015-04-22 14:00:49 CEST
Created attachment 336 [details]
multiple threads: broken distribution of deposited energy
Comment 5 Bastian Beischer 2015-04-22 14:08:00 CEST
Dear Vladimir,

attaching the application is rather difficult for me, because it's a very large full detector simulation of our experiment (AMS)...

I attached two pictures, which show the original problem with G4 10.01 p01.

The single_thread.png shows the correct (expected) distribution which was generated with just one thread. The multi_thread.png shows the broken distribution with the current G4PAIModel enabled in MT mode.

The problem goes away completely with the patch I attached in my original report.

Our debugging process went like this:

- we identified that the problem lies in G4PAIModel, because switching to the standard EM fluctuation model fixes the issue also in MT mode.

- I noticed that in the Initialise() function, the G4PAIModelData and the fMaterialCutsCoupleVector are only filled for the Master thread (it's in the if (IsMaster()) part).

- The pointer to the model data is copied over to the worker thread in InitialiseLocal, but the fMaterialCutsCoupleVector is still empty on the worker threads.

I don't know why you can't reproduce the issue. Maybe the fMaterialCutsCoupleVector isn't required in your case?! Or maybe, since 25% of the events are ok, the issue is not as obvious as in our case (where we used 32 threads)?

Can you print out the size of the fMaterialCutsCoupleVector on the 4 threads?

I did the same with a parallel debugger and found that master thread had reasonable size and the worker vectors had size 0.
Comment 6 Bastian Beischer 2015-04-22 14:10:24 CEST
Also, I'd be interested to know what Vladimir Ivantchenko ment by "drange vector" - I can't find any reference for that in G4PAIModel.

I never used G4PAIPhotModel so far, maybe it's required there?!
Comment 7 Vladimir.Ivantchenko 2015-05-25 15:28:34 CEST
Dear Bastian,

thank you very much for pointing to the bug. When we have introduced the fix, which you proposed, a next problem was discovered which we also have fixed. Fixes will be publicly available in June beta release of Geant4.

VI
Comment 8 Vladimir.Grichine 2015-05-26 08:09:06 CEST
Samplings of both energy transfers and cut vectors were directed to the tables in the master thread. TestEm8 works similarly for both PAI and PAIPhoton models.
Comment 9 Bastian Beischer 2015-06-09 00:46:17 CEST
Hello all,

thanks for fixing the problem. Could you please kindly provide me with the full patch you applied to both G4PAIModel and G4PAIPhotModel? I'd like to try them!

A few words on the additional issue you found and fixed would be appreciated as well.

Thanks very much
Bastian