|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
|
!!!!!!!!!!!!!!
!!!!!!!!!!!!!!
Changes were made to add option to use also only ALCARECO collection:
# if = 0 -> do not use digis at all
flagToUseDigiCollectionsORNot = cms.int32(1),
!!!!!!!!!!!!!!
Warning: before use, correct everywhere below the letters:
...qq8/CMSSW_12_3_0_pre4... for needed dirs.
!!!!!!!!!!!!!!
setenv SCRAM_ARCH slc7_amd64_gcc10
cmsrel CMSSW_12_3_0_pre4
cd CMSSW_12_3_0_pre4/src
cmsenv
git cms-addpkg DPGAnalysis/HcalTools
git cms-addpkg Calibration/HcalAlCaRecoProducers
git cms-addpkg Calibration/HcalCalibAlgos
scram b
!!!!!!!!!!!!!!
I.SELECTION OF RUNS FOR EACH ERA
==============================
1. fill into files (ztmpHcalNZS_A,...E...HI) all list of runs for each
dataset era (...Run2018A-v1,..E....HI..) using commands:
dasgoclient --query="run dataset=/HcalNZS/Run2018A-v1/RAW" --limit=0 > ztmpHcalNZS_A
or for ALCARECO collection:
dasgoclient --query="run dataset=/HcalNZS/Commissioning2021-HcalCalMinBias-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalNZS_ALCARECO
dasgoclient --query="run dataset=/MinimumBias/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB_ALCARECO
dasgoclient --query="run dataset=/MinimumBias0/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB0_ALCARECO
dasgoclient --query="run dataset=/MinimumBias1/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB1_ALCARECO
dasgoclient --query="run dataset=/MinimumBias2/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB2_ALCARECO
2. specify in file rundate.sh the line with ...dataset=...
and correct the line:
mv index_selection.html index_selectionA(B...).html
and use obtained txt file ztmpHcalNZS_A(B...)
and execute one of the commands:
./rundate.sh ztmpHcalNZS_ALCARECO > ztmpNZSALCARECO
./rundate.sh ztmpHcalNZS_A > ztmpA
./rundate.sh ztmpHcalNZS_B > ztmpB
./rundate.sh ztmpHcalNZS_C > ztmpC
./rundate.sh ztmpHcalNZS_D > ztmpD
./rundate.sh ztmpHcalNZS_E > ztmpE
./rundate.sh ztmpHcalNZS_HI > ztmpHI
(file index_selectionA(B...).html is created )
3. use file index_selectionA(B...).html to select runs
of needed size and only 1 run per day(last one),
with comparison with
https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18
/13TeV/PromptReco/Cert_314472-325175_13TeV_PromptReco_Collisions18_JSON.txt
, create txt files:
runtime_runA(B...)
runlist_runA(B...)
II. running of main cc-kode (creation of root files)
===================================================
1) interactively
(w/o config.Data.lumiMask =
'https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18
/13TeV/PromptReco/Cert_314472-325175_13TeV_PromptReco_Collisions18_JSON.txt'
)
- necessary corrections:
1. correct file "file_lists.csh" or "file_ALCARECO.csh" or "file_IMPSM.csh" for dataset used
2. correct file "runlist_runA(B...)" to specify run list
- running:
./mkcfg_new120.csh runlist_runA(B...)
./mkcfg_newALCARECO.csh runlist_2ALCARECO
- see, how many py-files in PYTHON_runlist_run dir..
- run ~ 15 jobs per PC: 1-15, 15-30, ...nn1- NN using commands:
./run_interactive.csh runlist_run nn1 NN
2) using crab:
see https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18/13TeV/PromptReco/
- correct: PSM_Global_2018_cfg.py
- for each run correct and use scripts:
crab_NZS2018C_1(2...).py
- running:
source /cvmfs/cms.cern.ch/crab3/crab.csh
source /cvmfs/cms.cern.ch/crab3/crab_standalone.csh
setenv SCRAM_ARCH slc7_amd64_gcc10
cmsenv
crab submit -c crab_NZS2018C_1(2...).py
to check(kill) use:
crab status
crab status -d ./crab_20200212_101405
crab tasks
crab tasks --days
crab kill -d ./crab_20200212_101405
III. copy and merge root files
==============================
1) from eos, merge pices of root files into one big run-root file in your_dir on afs:
(using script cp_runs_fromeos_into_longruns.sh )
./cp_runs_fromeos_into_longruns.sh runlist_runA /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
./cp_runs_fromeos_into_longruns.sh runlist_runB /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
./cp_runs_fromeos_into_longruns.sh runlist_runC /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
(input dir is /eos/cms/store/user/zhokin/PSM/HcalNZS/2018 ; see: ll /eos/cms/store/user/zhokin/PSM/HcalNZS/ )
[1.1 (DOESN'T WORK YET) in principle, you can use merging directly within eos dir.
For that, use script glu_runs_oneos.csh directly from eos:
cd /eos/cms/store/user/zhokin/PSM/HcalNZS
./glu_runs_oneos.csh /eos/cms/store/user/zhokin/PSM/HcalNZS/2018 runlist_eostest1
]
2) in use of interactive running use the next :
cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
2.1 check all root-files availability:
cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
./checkoutput120.csh PYTHON_runlist_run longruns
2.2 merging run-root-piecies into one big run-root file:
cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
(crashed jobs to be re-runned interactively and copied into dir longruns )
./glu_runs_fromfile.csh longruns runlist_run
3) copy into /eos:
cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
copy all files Global_"run".root from your_dir into eos:
cp Global_*.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos/.
cp Global_*.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM/histos/.
or each run only::
eos cp /your_dir/Global_345821.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos
check:
eos ls -l /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos/
IV. visulization on GlobalPSM sajt
==================================
-- to visualize table for each new run, use script:
-- fill table on GlobalPSM sajt: add new runs
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM
- correct file index_toAddOnlyNewRuns_EOS.sh for "dasgoclient.. dataset=..." and "k="
and then use scripts:
./index_toAddOnlyNewRuns_EOS.sh runlist_runA
./index_toAddOnlyNewRuns_EOS.sh runlist_runB
./index_toAddOnlyNewRuns_EOS.sh runlist_runC
./index_toAddOnlyNewRuns_EOS.sh runlist_runD
./index_toAddOnlyNewRuns_EOS.sh runlist_1ALCARECO
./index_toAddOnlyNewRuns_EOS.sh runlist_2ALCARECO
At that, check index_draft.html and if everything is ok, copy it:
cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/index.html
cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM/index.html
cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM/index.html
-- to visualize PSM-plots for each new run, use script:
--------------------------------------------------
cd macros/psm
./compile.csh RemoteMonitoringPSM.cc
./compile.csh RemoteMonitoringALCARECOPSM.cc
./compile.csh RemoteMonitoringIMPSM.cc
./compile.csh RemoteMonitoringIMPSMTEST.cc
cd scripts/psm
./GlobalPSM_EOS.sh runlist_all
./ALCARECOPSM_EOS.sh runlist_1ALCARECO
./ALCARECOPSM_EOS.sh runlist_2ALCARECO
./IMPSM_EOS.sh runlist_1IMPSM
(during this running, for each run, new dirs like GLOBAL_runnumber
will be created in dir.
/eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM
or
/eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM
or
/eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM
)
V. helpfull sites and dirs
==================
1) Phi-Symmetry Monitoring(PSM) Tool for IMALCARECO RUNs:
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/IMPSM/
2) Phi-Symmetry Monitoring(PSM) Tool for ALCARECO RUNs
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/ALCARECOPSM
3) Phi-Symmetry Monitoring(PSM) Tool for Global RUNs
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM/
==== example of IMPSM for Natalii (Iterative Method)
cd /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/
mkdir IMPSM
=======
ls -latr /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/
drwxr-xr-x. 2 kodolova zh 4096 Mar 13 2020 AMT
drwxr-xr-x. 2 kodolova zh 4096 Sep 28 16:21 RMT
drwxr-xr-x. 2 kodolova zh 4096 Nov 23 09:02 CMT
drwxr-xr-x. 2 51962 zh 4096 Nov 23 09:33 GlobalRMT
drwxr-xr-x. 2 zhokin zh 4096 Dec 3 13:07 GlobalPSM
drwxr-xr-x. 2 zhokin zh 4096 Dec 6 14:31 ALCARECOPSM
drwxr-xr-x. 2 zhokin zh 4096 Dec 7 10:41 IMPSM
======
sites:
-----
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/RMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/CMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalRMT
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/ALCARECOPSM
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/IMPSM
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/AMT
VI. tests:
==========
cd macros/psm
cp /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM/histos/Global_346512.root .
./compile.csh RemoteMonitoringIMPSMTEST.cc
./RemoteMonitoringIMPSMTEST.cc.exe Global_346512.root Global_346512.root Global
local:
firefox MAP.html
gnome-open RBX-HO-3plots.png
grep RBX-HB-3plotsmore *.html
ll HB_ADCampl.html
==================================
==================================
==================================
==================================
==================================
VII. use of AMT scripts
=====================
-- use macro zGlobalamtTimeDependenciesPhiSymmetryHE.cc
in dir
cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/macros/amt
- create dirs.
zamtPhiSymmetryHElin
zamtPhiSymmetryHElog
- ./compile.csh zGlobalamtTimeDependenciesPhiSymmetryHE.cc
- ./zGlobalamtTimeDependenciesPhiSymmetryHE.cc.exe
cd zamtPhiSymmetryHElog
gnome-open *.png
-- visualization:
- correct files in dir:
/eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/AMT/index.html
and mainly here:
/afs/cern.ch/cms/CAF/CMSALCA/ALCA_HCALCALIB/HCALMONITORING/AMTweb
-see site:
https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/AMT
https://cms-cpt-software.web.cern.ch/cms-cpt-software/General/Validation/SVSuite/HcalRemoteMonitoring/AMT/HE.html
|