Hi, After fixing compiling errors with migration from Geant4 10.06 to 10.07 our simple test cases discovered a bug with the analysis manager. The analysis manager complains that nTuple columns do not exist that were properly registered. The errors are erratic and change as the number of branches and tuples are changed. The code uses only the `G4RootAnalysisManager`. I increased the verbosity to 10 (1 didn't seem to do anything) and I get the following messages. The columns are not found for nTupleId 1 columnId 3,4,5, but they are clearly created above. The same code worked without issue for Geant4 10.05 and 10.06. ``` Run 0 - 2 Events G4WT0 > ... going to create ntuple booking : PrimaryParticle G4WT0 > ... create ntuple booking : PrimaryParticle ntupleId 0 - done G4WT0 > ... going to create ntuple T column : eventNumber ntupleId 0 G4WT0 > ... create ntuple T column : eventNumber ntupleId 0 - done G4WT0 > ... going to create ntuple T column : x_mm ntupleId 0 G4WT0 > ... create ntuple T column : x_mm ntupleId 0 - done G4WT0 > ... going to create ntuple T column : y_mm ntupleId 0 G4WT0 > ... create ntuple T column : y_mm ntupleId 0 - done G4WT0 > ... going to create ntuple T column : z_mm ntupleId 0 G4WT0 > ... create ntuple T column : z_mm ntupleId 0 - done G4WT0 > ... going to create ntuple T column : E_MeV ntupleId 0 G4WT0 > ... create ntuple T column : E_MeV ntupleId 0 - done G4WT0 > ... going to create ntuple T column : px_MeV ntupleId 0 G4WT0 > ... create ntuple T column : px_MeV ntupleId 0 - done G4WT0 > ... going to create ntuple T column : py_MeV ntupleId 0 G4WT0 > ... create ntuple T column : py_MeV ntupleId 0 - done G4WT0 > ... going to create ntuple T column : pz_MeV ntupleId 0 G4WT0 > ... create ntuple T column : pz_MeV ntupleId 0 - done G4WT0 > ... going to create ntuple T column : pdgParticleId ntupleId 0 G4WT0 > ... create ntuple T column : pdgParticleId ntupleId 0 - done G4WT0 > ... going to create from booking ntuple : PrimaryParticle G4WT0 > ... done create from booking ntuple : PrimaryParticle G4WT0 > ... going to create ntuple booking : PV_siDetector G4WT0 > ... create ntuple booking : PV_siDetector ntupleId 1 - done G4WT0 > ... going to create ntuple booking : LV_siDetectorVolume G4WT0 > ... create ntuple booking : LV_siDetectorVolume ntupleId 2 - done G4WT0 > ... going to create ntuple T column : eventNumber ntupleId 1 G4WT0 > ... create ntuple T column : eventNumber ntupleId 1 - done G4WT0 > ... going to create ntuple T column : numHits ntupleId 1 G4WT0 > ... create ntuple T column : numHits ntupleId 1 - done G4WT0 > ... going to create ntuple T column : numSecondaries ntupleId 1 G4WT0 > ... create ntuple T column : numSecondaries ntupleId 1 - done G4WT0 > ... going to create ntuple T column : depositedEnergy_MeV ntupleId 1 G4WT0 > ... create ntuple T column : depositedEnergy_MeV ntupleId 1 - done G4WT0 > ... going to create ntuple T column : preInteractionEnergy_MeV ntupleId 1 G4WT0 > ... create ntuple T column : preInteractionEnergy_MeV ntupleId 1 - done G4WT0 > ... going to create ntuple T column : postInteractionEnergy_MeV ntupleId 1 G4WT0 > ... create ntuple T column : postInteractionEnergy_MeV ntupleId 1 - done G4WT0 > ... going to create ntuple T column : eventNumber ntupleId 2 G4WT0 > ... create ntuple T column : eventNumber ntupleId 2 - done G4WT0 > ... going to create ntuple T column : numHits ntupleId 2 G4WT0 > ... create ntuple T column : numHits ntupleId 2 - done G4WT0 > ... going to create ntuple T column : numSecondaries ntupleId 2 G4WT0 > ... create ntuple T column : numSecondaries ntupleId 2 - done G4WT0 > ... going to create ntuple T column : depositedEnergy_MeV ntupleId 2 G4WT0 > ... create ntuple T column : depositedEnergy_MeV ntupleId 2 - done G4WT0 > ... going to create ntuple T column : preInteractionEnergy_MeV ntupleId 2 G4WT0 > ... create ntuple T column : preInteractionEnergy_MeV ntupleId 2 - done G4WT0 > ... going to create ntuple T column : postInteractionEnergy_MeV ntupleId 2 G4WT0 > ... create ntuple T column : postInteractionEnergy_MeV ntupleId 2 - done G4WT0 > ... going to create ntuple booking : TrackingSD G4WT0 > ... create ntuple booking : TrackingSD ntupleId 3 - done G4WT0 > ... going to create ntuple T column : eventNumber ntupleId 3 G4WT0 > ... create ntuple T column : eventNumber ntupleId 3 - done G4WT0 > ... going to create ntuple T column : trackID ntupleId 3 G4WT0 > ... create ntuple T column : trackID ntupleId 3 - done G4WT0 > ... going to create from booking ntuple : TrackingSD G4WT0 > ... done create from booking ntuple : TrackingSD G4WT0 > ... going to create from booking ntuple : PV_siDetector G4WT0 > ... done create from booking ntuple : PV_siDetector G4WT0 > ... going to create from booking ntuple : LV_siDetectorVolume G4WT0 > ... done create from booking ntuple : LV_siDetectorVolume G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 0 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 1 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 2 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 3 value 1000 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 4 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 5 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 6 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 7 value -43.3306 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 8 value 2212 G4WT0 > ... going to add ntuple row : ntupleId 0 G4WT0 > ... going to add ntuple row : ntupleId 0 G4WT0 > ... going to fill ntuple T column : ntupleId 1 columnId 0 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 1 columnId 1 value 1 G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 3 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 4 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 5 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 0 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 1 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 3 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 4 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 5 value -1 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 0 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 1 value 1 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 1 G4WT0 > ... going to add ntuple row : ntupleId 1 G4WT0 > ... going to add ntuple row : ntupleId 2 G4WT0 > ... going to add ntuple row : ntupleId 2 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 1 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 2 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 3 value 1000 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 4 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 5 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 6 value 0 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 7 value -43.3306 G4WT0 > ... going to fill ntuple T column : ntupleId 0 columnId 8 value 2212 G4WT0 > ... going to add ntuple row : ntupleId 0 G4WT0 > ... going to add ntuple row : ntupleId 0 G4WT0 > ... going to fill ntuple T column : ntupleId 1 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 1 columnId 1 value 3 G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 3 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 4 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > -------- WWWW ------- G4Exception-START -------- WWWW ------- *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 5 does not exist. *** This is just a warning message. *** -------- WWWW -------- G4Exception-END --------- WWWW ------- G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 1 value 3 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 3 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 4 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 2 columnId 5 value -1 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 1 value 1 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 1 value 1 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 0 value 1 G4WT0 > ... going to fill ntuple T column : ntupleId 3 columnId 1 value 1 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 3 G4WT0 > ... going to add ntuple row : ntupleId 1 G4WT0 > ... going to add ntuple row : ntupleId 1 G4WT0 > ... going to add ntuple row : ntupleId 2 G4WT0 > ... going to add ntuple row : ntupleId 2 G4WT0 > ... going to write files G4WT0 > ... going to write file : out_r0_t0.root G4WT0 > ... write file : out_r0_t0.root - done G4WT0 > ... write files - done G4WT0 > ... going to close files G4WT0 > ... going to close file : out_r0_t0.root G4WT0 > ... close file : out_r0_t0.root - done G4WT0 > ... close files - done 100.0% [==================================================] Evt 2 0.1 sec / 0.0 sec rem Run Time: 0.1 sec / 50.3 ms/evt ``` Thanks!
Hello, Can you, please, try to update to 10.7.p1 and check if the problem persists? Thank you,
I have tried this with both 10.07 and 10.07.p01.
Thank you, it is indeed strange that only certain columns are not found. Would you mind to upload also the output from the beginning of the program which precedes the printing that you posted? Then, can you let me know in which class & function you do the following steps: - Create ntuples - Fill ntuple - OpenFile - Write/CloseFile And, if possible, can you post the code where you create the ntuples?
The output prior is not relevant, physics lists info and such. This is part of a larger project with >25k lines of code. I will attempt to make a Minimum Working Example that demonstrates the problem. A wrapper was developed around the G4RootAnalysisManager to provide tracking of the ntuple and column ids. Ntuples are created in multiple places, but the one that fails is created in RunAction::BeginOfRunAction Ntuples are filled in G4Run::RecordEvent The relevant output file is opened in RunAction::BeginOfRunAction These files are then closed in RunAction::EndOfRunAction.
I was interested in the output from analysis; the lines starting with ... It will be indeed helpful if you could provide an example where I can reproduce the problem.
The lines starting with `...` came from the Analysis manger verbose mode, I have not omitted anything there. I will work to produce a MWE that you can run, but it might take me a little bit of time to find a situation which reproduces the problem. I'll work on this today.
I was able to make something nice and short, but this complains about not being able to find the ntuple instead of the branch. CMakeLists.txt: ``` cmake_minimum_required(VERSION 3.17) project(MWE) find_package(Geant4) add_executable(mwe mwe.cxx) target_include_directories(mwe PRIVATE ${Geant4_INCLUDE_DIRS}) target_link_libraries(mwe ${Geant4_LIBRARIES}) ``` mwe.cxx: ``` #include <G4RootAnalysisManager.hh> int main(int argc, char** argv) { auto rootManager = G4RootAnalysisManager::Instance(); rootManager->SetVerboseLevel(10); rootManager->OpenFile("test.root"); auto ntuple1 = rootManager->CreateNtuple("test", "This is a test tree"); auto branch1 = rootManager->CreateNtupleIColumn(ntuple1, "branch1"); rootManager->FillNtupleIColumn(ntuple1, branch1, 50); rootManager->CloseFile("test.root"); } ``` Output: ``` ... going to create ntuple manager ... done create ntuple manager ... going to create fileInformation : test.root ... going to create file : test.root ... create file : test.root - done ... going to open analysis file : test.root ... open analysis file : test.root - done ... going to create ntuple booking : test ... create ntuple booking : test ntupleId 0 - done ... going to create ntuple T column : branch1 ntupleId 0 ... create ntuple T column : branch1 ntupleId 0 - done -------- WWWW ------- G4Exception-START -------- WWWW ------- *** ExceptionHandler is not defined *** *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn ntuple 0 does not exist. *** This is just a warning message. *** -------- WWWW ------- G4Exception-END -------- WWWW ------- ... going to close files ... going to close file : test.root ... close file : test.root - done ```
Never mind, I forgot to mark the tuple finished. I will continue with this to try and create an example that fails as above.
(In reply to ksmith from comment #6) > The lines starting with `...` came from the Analysis manger verbose mode, I > have not omitted anything there. > > I will work to produce a MWE that you can run, but it might take me a little > bit of time to find a situation which reproduces the problem. I'll work on > this today. There should be information about opening a file; this must be always done before calling FillNtupleColumn
Here is MWE it looks like the issue is due to ntuple3 being marked finsihed before ntuple2. mwe.cxx: ``` #include <G4RootAnalysisManager.hh> int main(int argc, char** argv) { auto rootManager = G4RootAnalysisManager::Instance(); rootManager->SetVerboseLevel(10); rootManager->OpenFile("test.root"); auto ntuple1 = rootManager->CreateNtuple("tree1", "This is a test tree"); auto branchX = rootManager->CreateNtupleDColumn(ntuple1, "x"); auto branchY = rootManager->CreateNtupleDColumn(ntuple1, "y"); auto branchZ = rootManager->CreateNtupleDColumn(ntuple1, "z"); rootManager->FinishNtuple(ntuple1); auto ntuple2 = rootManager->CreateNtuple("tree2", "This is a second test tree"); auto branch1 = rootManager->CreateNtupleIColumn(ntuple2, "branch1"); auto branch2 = rootManager->CreateNtupleIColumn(ntuple2, "branch2"); auto ntuple3 = rootManager->CreateNtuple("tree3", "This is a third test tree"); auto branchAlpha = rootManager->CreateNtupleIColumn(ntuple3, "alpha"); // This is likely the cause finishing ntuple3, before ntuple2! rootManager->FinishNtuple(ntuple3); rootManager->FinishNtuple(ntuple2); rootManager->FillNtupleDColumn(ntuple1, branchX, 1.0); rootManager->FillNtupleDColumn(ntuple1, branchY, 2.0); rootManager->FillNtupleDColumn(ntuple1, branchZ, 3.0); rootManager->FillNtupleIColumn(ntuple2, branch1, 1); rootManager->FillNtupleIColumn(ntuple2, branch2, 20); rootManager->FillNtupleIColumn(ntuple3, branchAlpha, 100); rootManager->CloseFile("test.root"); } ``` Output with Geant4 10.07.p01 ``` ... going to create ntuple manager ... done create ntuple manager ... going to create fileInformation : test.root ... going to create file : test.root ... create file : test.root - done ... going to open analysis file : test.root ... open analysis file : test.root - done ... going to create ntuple booking : tree1 ... create ntuple booking : tree1 ntupleId 0 - done ... going to create ntuple T column : x ntupleId 0 ... create ntuple T column : x ntupleId 0 - done ... going to create ntuple T column : y ntupleId 0 ... create ntuple T column : y ntupleId 0 - done ... going to create ntuple T column : z ntupleId 0 ... create ntuple T column : z ntupleId 0 - done ... going to create from booking ntuple : tree1 ... done create from booking ntuple : tree1 ... going to create ntuple booking : tree2 ... create ntuple booking : tree2 ntupleId 1 - done ... going to create ntuple T column : branch1 ntupleId 1 ... create ntuple T column : branch1 ntupleId 1 - done ... going to create ntuple T column : branch2 ntupleId 1 ... create ntuple T column : branch2 ntupleId 1 - done ... going to create ntuple booking : tree3 ... create ntuple booking : tree3 ntupleId 2 - done ... going to create ntuple T column : alpha ntupleId 2 ... create ntuple T column : alpha ntupleId 2 - done ... going to create from booking ntuple : tree3 ... done create from booking ntuple : tree3 ... going to create from booking ntuple : tree2 ... done create from booking ntuple : tree2 ... going to fill ntuple T column : ntupleId 0 columnId 0 value 1 ... going to fill ntuple T column : ntupleId 0 columnId 1 value 2 ... going to fill ntuple T column : ntupleId 0 columnId 2 value 3 ... going to fill ntuple T column : ntupleId 1 columnId 0 value 1 -------- WWWW ------- G4Exception-START -------- WWWW ------- *** ExceptionHandler is not defined *** *** G4Exception : Analysis_W011 issued by : G4TNtupleManager::FillNtupleTColumn() ntupleId 1 columnId 1 does not exist. *** This is just a warning message. *** -------- WWWW ------- G4Exception-END -------- WWWW ------- ... going to fill ntuple T column : ntupleId 2 columnId 0 value 100 ... going to close files ... going to close file : test.root ... close file : test.root - done ```
(In reply to Ivana Hrivnacova from comment #9) > (In reply to ksmith from comment #6) > > The lines starting with `...` came from the Analysis manger verbose mode, I > > have not omitted anything there. > > > > I will work to produce a MWE that you can run, but it might take me a little > > bit of time to find a situation which reproduces the problem. I'll work on > > this today. > > There should be information about opening a file; this must be always done > before calling FillNtupleColumn Yes this is odd that the create commands don't appear. I see the close files command at the end. Perhaps the lines about opening were overwritten? I think we should focus on the simple MWE example I posted in comment 10 before trying to debug the complex program any more.
Thank you for providing the example. I was able to reproduce the problem. I will provide a fix in the development version and for the patch02 to 10.7. When ready, I will attach the patch also to this bug report. There are two possible work around's: - either call FinishNtuple in the same order as the ntuples are created - or call OpenFile() after all ntuples are booked (and finished) Best regards,
(In reply to Ivana Hrivnacova from comment #12) > Thank you for providing the example. I was able to reproduce the problem. > I will provide a fix in the development version and for the patch02 to 10.7. > When ready, I will attach the patch also to this bug report. > > There are two possible work around's: > - either call FinishNtuple in the same order as the ntuples are created > - or call OpenFile() after all ntuples are booked (and finished) > > Best regards, Thanks for the quick response. I'll be happy to see this fixed. As an aside are vectors of strings supported yet?
Created attachment 678 [details] Patch to be applied to 10.7.p01 Hi, I've attached a patch with a fix, based on Geant4 10.7.p01. Best regards,
The fix is now available in 10.7.p02. I noted your requirement for vectors of strings for the next Geant4 version.
