Warning, /DPGAnalysis/HcalTools/scripts/psm/readme is written in an unsupported language. File is not indexed.
0001 !!!!!!!!!!!!!!
0002
0003
0004 !!!!!!!!!!!!!!
0005 Changes were made to add option to use also only ALCARECO collection:
0006 # if = 0 -> do not use digis at all
0007 flagToUseDigiCollectionsORNot = cms.int32(1),
0008
0009 !!!!!!!!!!!!!!
0010 Warning: before use, correct everywhere below the letters:
0011 ...qq8/CMSSW_12_3_0_pre4... for needed dirs.
0012
0013 !!!!!!!!!!!!!!
0014 setenv SCRAM_ARCH slc7_amd64_gcc10
0015 cmsrel CMSSW_12_3_0_pre4
0016 cd CMSSW_12_3_0_pre4/src
0017 cmsenv
0018 git cms-addpkg DPGAnalysis/HcalTools
0019
0020 git cms-addpkg Calibration/HcalAlCaRecoProducers
0021 git cms-addpkg Calibration/HcalCalibAlgos
0022 scram b
0023
0024
0025 !!!!!!!!!!!!!!
0026 I.SELECTION OF RUNS FOR EACH ERA
0027 ==============================
0028
0029 1. fill into files (ztmpHcalNZS_A,...E...HI) all list of runs for each
0030 dataset era (...Run2018A-v1,..E....HI..) using commands:
0031 dasgoclient --query="run dataset=/HcalNZS/Run2018A-v1/RAW" --limit=0 > ztmpHcalNZS_A
0032 or for ALCARECO collection:
0033 dasgoclient --query="run dataset=/HcalNZS/Commissioning2021-HcalCalMinBias-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalNZS_ALCARECO
0034 dasgoclient --query="run dataset=/MinimumBias/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB_ALCARECO
0035 dasgoclient --query="run dataset=/MinimumBias0/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB0_ALCARECO
0036 dasgoclient --query="run dataset=/MinimumBias1/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB1_ALCARECO
0037 dasgoclient --query="run dataset=/MinimumBias2/Commissioning2021-HcalCalIterativePhiSym-PromptReco-v1/ALCARECO" --limit=0 > ztmpHcalMB2_ALCARECO
0038
0039 2. specify in file rundate.sh the line with ...dataset=...
0040 and correct the line:
0041 mv index_selection.html index_selectionA(B...).html
0042
0043 and use obtained txt file ztmpHcalNZS_A(B...)
0044 and execute one of the commands:
0045
0046 ./rundate.sh ztmpHcalNZS_ALCARECO > ztmpNZSALCARECO
0047
0048 ./rundate.sh ztmpHcalNZS_A > ztmpA
0049 ./rundate.sh ztmpHcalNZS_B > ztmpB
0050 ./rundate.sh ztmpHcalNZS_C > ztmpC
0051 ./rundate.sh ztmpHcalNZS_D > ztmpD
0052 ./rundate.sh ztmpHcalNZS_E > ztmpE
0053 ./rundate.sh ztmpHcalNZS_HI > ztmpHI
0054
0055 (file index_selectionA(B...).html is created )
0056
0057 3. use file index_selectionA(B...).html to select runs
0058 of needed size and only 1 run per day(last one),
0059 with comparison with
0060 https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18
0061 /13TeV/PromptReco/Cert_314472-325175_13TeV_PromptReco_Collisions18_JSON.txt
0062
0063 , create txt files:
0064 runtime_runA(B...)
0065 runlist_runA(B...)
0066
0067
0068 II. running of main cc-kode (creation of root files)
0069 ===================================================
0070 1) interactively
0071 (w/o config.Data.lumiMask =
0072 'https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18
0073 /13TeV/PromptReco/Cert_314472-325175_13TeV_PromptReco_Collisions18_JSON.txt'
0074 )
0075 - necessary corrections:
0076 1. correct file "file_lists.csh" or "file_ALCARECO.csh" or "file_IMPSM.csh" for dataset used
0077 2. correct file "runlist_runA(B...)" to specify run list
0078 - running:
0079 ./mkcfg_new120.csh runlist_runA(B...)
0080 ./mkcfg_newALCARECO.csh runlist_2ALCARECO
0081
0082 - see, how many py-files in PYTHON_runlist_run dir..
0083 - run ~ 15 jobs per PC: 1-15, 15-30, ...nn1- NN using commands:
0084 ./run_interactive.csh runlist_run nn1 NN
0085
0086 2) using crab:
0087 see https://cms-service-dqm.web.cern.ch/cms-service-dqm/CAF/certification/Collisions18/13TeV/PromptReco/
0088 - correct: PSM_Global_2018_cfg.py
0089 - for each run correct and use scripts:
0090 crab_NZS2018C_1(2...).py
0091 - running:
0092 source /cvmfs/cms.cern.ch/crab3/crab.csh
0093 source /cvmfs/cms.cern.ch/crab3/crab_standalone.csh
0094 setenv SCRAM_ARCH slc7_amd64_gcc10
0095 cmsenv
0096
0097 crab submit -c crab_NZS2018C_1(2...).py
0098
0099 to check(kill) use:
0100 crab status
0101 crab status -d ./crab_20200212_101405
0102 crab tasks
0103 crab tasks --days
0104 crab kill -d ./crab_20200212_101405
0105
0106 III. copy and merge root files
0107 ==============================
0108 1) from eos, merge pices of root files into one big run-root file in your_dir on afs:
0109 (using script cp_runs_fromeos_into_longruns.sh )
0110 ./cp_runs_fromeos_into_longruns.sh runlist_runA /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
0111 ./cp_runs_fromeos_into_longruns.sh runlist_runB /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
0112 ./cp_runs_fromeos_into_longruns.sh runlist_runC /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm/longruns
0113 (input dir is /eos/cms/store/user/zhokin/PSM/HcalNZS/2018 ; see: ll /eos/cms/store/user/zhokin/PSM/HcalNZS/ )
0114
0115 [1.1 (DOESN'T WORK YET) in principle, you can use merging directly within eos dir.
0116 For that, use script glu_runs_oneos.csh directly from eos:
0117 cd /eos/cms/store/user/zhokin/PSM/HcalNZS
0118 ./glu_runs_oneos.csh /eos/cms/store/user/zhokin/PSM/HcalNZS/2018 runlist_eostest1
0119 ]
0120
0121 2) in use of interactive running use the next :
0122 cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
0123 2.1 check all root-files availability:
0124 cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
0125 ./checkoutput120.csh PYTHON_runlist_run longruns
0126
0127 2.2 merging run-root-piecies into one big run-root file:
0128 cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
0129 (crashed jobs to be re-runned interactively and copied into dir longruns )
0130 ./glu_runs_fromfile.csh longruns runlist_run
0131
0132 3) copy into /eos:
0133 cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/scripts/psm
0134 copy all files Global_"run".root from your_dir into eos:
0135 cp Global_*.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos/.
0136 cp Global_*.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM/histos/.
0137 or each run only::
0138 eos cp /your_dir/Global_345821.root /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos
0139 check:
0140 eos ls -l /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/histos/
0141
0142 IV. visulization on GlobalPSM sajt
0143 ==================================
0144 -- to visualize table for each new run, use script:
0145 -- fill table on GlobalPSM sajt: add new runs
0146 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM
0147 - correct file index_toAddOnlyNewRuns_EOS.sh for "dasgoclient.. dataset=..." and "k="
0148 and then use scripts:
0149 ./index_toAddOnlyNewRuns_EOS.sh runlist_runA
0150 ./index_toAddOnlyNewRuns_EOS.sh runlist_runB
0151 ./index_toAddOnlyNewRuns_EOS.sh runlist_runC
0152 ./index_toAddOnlyNewRuns_EOS.sh runlist_runD
0153
0154 ./index_toAddOnlyNewRuns_EOS.sh runlist_1ALCARECO
0155 ./index_toAddOnlyNewRuns_EOS.sh runlist_2ALCARECO
0156
0157 At that, check index_draft.html and if everything is ok, copy it:
0158 cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM/index.html
0159 cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM/index.html
0160 cp index_draft.html /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM/index.html
0161
0162
0163 -- to visualize PSM-plots for each new run, use script:
0164 --------------------------------------------------
0165 cd macros/psm
0166 ./compile.csh RemoteMonitoringPSM.cc
0167 ./compile.csh RemoteMonitoringALCARECOPSM.cc
0168 ./compile.csh RemoteMonitoringIMPSM.cc
0169 ./compile.csh RemoteMonitoringIMPSMTEST.cc
0170
0171 cd scripts/psm
0172 ./GlobalPSM_EOS.sh runlist_all
0173
0174 ./ALCARECOPSM_EOS.sh runlist_1ALCARECO
0175 ./ALCARECOPSM_EOS.sh runlist_2ALCARECO
0176 ./IMPSM_EOS.sh runlist_1IMPSM
0177
0178 (during this running, for each run, new dirs like GLOBAL_runnumber
0179 will be created in dir.
0180 /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/GlobalPSM
0181 or
0182 /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/ALCARECOPSM
0183 or
0184 /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM
0185
0186 )
0187
0188 V. helpfull sites and dirs
0189 ==================
0190 1) Phi-Symmetry Monitoring(PSM) Tool for IMALCARECO RUNs:
0191 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/IMPSM/
0192
0193 2) Phi-Symmetry Monitoring(PSM) Tool for ALCARECO RUNs
0194 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/ALCARECOPSM
0195
0196 3) Phi-Symmetry Monitoring(PSM) Tool for Global RUNs
0197 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM/
0198
0199 ==== example of IMPSM for Natalii (Iterative Method)
0200 cd /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/
0201 mkdir IMPSM
0202 =======
0203
0204 ls -latr /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/
0205
0206 drwxr-xr-x. 2 kodolova zh 4096 Mar 13 2020 AMT
0207 drwxr-xr-x. 2 kodolova zh 4096 Sep 28 16:21 RMT
0208 drwxr-xr-x. 2 kodolova zh 4096 Nov 23 09:02 CMT
0209 drwxr-xr-x. 2 51962 zh 4096 Nov 23 09:33 GlobalRMT
0210 drwxr-xr-x. 2 zhokin zh 4096 Dec 3 13:07 GlobalPSM
0211 drwxr-xr-x. 2 zhokin zh 4096 Dec 6 14:31 ALCARECOPSM
0212 drwxr-xr-x. 2 zhokin zh 4096 Dec 7 10:41 IMPSM
0213 ======
0214 sites:
0215 -----
0216 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/RMT
0217
0218 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/CMT
0219 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalRMT
0220
0221 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/GlobalPSM
0222 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/ALCARECOPSM
0223 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/IMPSM
0224
0225 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/AMT
0226
0227
0228 VI. tests:
0229 ==========
0230 cd macros/psm
0231 cp /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/IMPSM/histos/Global_346512.root .
0232 ./compile.csh RemoteMonitoringIMPSMTEST.cc
0233 ./RemoteMonitoringIMPSMTEST.cc.exe Global_346512.root Global_346512.root Global
0234
0235 local:
0236 firefox MAP.html
0237 gnome-open RBX-HO-3plots.png
0238 grep RBX-HB-3plotsmore *.html
0239 ll HB_ADCampl.html
0240
0241
0242 ==================================
0243 ==================================
0244 ==================================
0245 ==================================
0246 ==================================
0247 VII. use of AMT scripts
0248 =====================
0249 -- use macro zGlobalamtTimeDependenciesPhiSymmetryHE.cc
0250 in dir
0251 cd /afs/cern.ch/work/z/zhokin/hcal/qq8/CMSSW_12_3_0_pre4/src/DPGAnalysis/HcalTools/macros/amt
0252 - create dirs.
0253 zamtPhiSymmetryHElin
0254 zamtPhiSymmetryHElog
0255
0256 - ./compile.csh zGlobalamtTimeDependenciesPhiSymmetryHE.cc
0257 - ./zGlobalamtTimeDependenciesPhiSymmetryHE.cc.exe
0258
0259 cd zamtPhiSymmetryHElog
0260 gnome-open *.png
0261
0262 -- visualization:
0263 - correct files in dir:
0264 /eos/cms/store/group/dpg_hcal/comm_hcal/www/HcalRemoteMonitoring/AMT/index.html
0265 and mainly here:
0266 /afs/cern.ch/cms/CAF/CMSALCA/ALCA_HCALCALIB/HCALMONITORING/AMTweb
0267
0268 -see site:
0269
0270 https://cms-conddb.cern.ch/eosweb/hcal/HcalRemoteMonitoring/AMT
0271
0272 https://cms-cpt-software.web.cern.ch/cms-cpt-software/General/Validation/SVSuite/HcalRemoteMonitoring/AMT/HE.html
