Warning, /Alignment/OfflineValidation/README.md is written in an unsupported language. File is not indexed.
0001 # Aligment/OfflineValidation
0002
0003 We use the Boost library (Program Options, Filesystem & Property Trees) to deal with the treatment of the config file.
0004 Basic idea:
0005 - a generic config file is "projected" for each validation (*e.g.* the geometry is changed, together with the plotting style);
0006 - for each config file, a new condor config file is produced;
0007 - a DAGMAN file is also produced in order to submit the whole validation at once.
0008
0009 In principle, the `validateAlignments.py` command is enough to submit everything.
0010 However, for local testing, one may want to make a dry run: all files will be produced, but the condor jobs will not be submitted;
0011 then one can just test locally any step, or modify any parameter before simply submitting the DAGMAN.
0012
0013 ## HOWTO use
0014
0015 The main script is `validateAlignments.py`. One can check the options with:
0016 ```
0017 validateAlignments.py -h
0018 usage: validateAlignments.py [-h] [-d] [-v] [-e] [-f]
0019 [-j {espresso,microcentury,longlunch,workday,tomorrow,testmatch,nextweek}]
0020 config
0021
0022 AllInOneTool for validation of the tracker alignment
0023
0024 positional arguments:
0025 config Global AllInOneTool config (json/yaml format)
0026
0027 optional arguments:
0028 -h, --help show this help message and exit
0029 -d, --dry Set up everything, but don't run anything
0030 -v, --verbose Enable standard output stream
0031 -e, --example Print example of config in JSON format
0032 -f, --force Force creation of enviroment, possible overwritten old configuration
0033 -j {espresso,microcentury,longlunch,workday,tomorrow,testmatch,nextweek}, --job-flavour {espresso,microcentury,longlunch,workday,tomorrow,testmatch,nextweek}
0034 Job flavours for HTCondor at CERN, default is 'workday'
0035 ```
0036
0037 As input the AllInOneTool config in `yaml` or `json` file format has to be provided. One working example can be find here: `Alignment/OfflineValidation/test/test.yaml`. To create the set up and submit everything to the HTCondor batch system, one can call
0038
0039 ```
0040 validateAlignments.py $CMSSW_BASE/src/Alignment/OfflineValidation/test/test.yaml
0041
0042 ```
0043
0044 To create the set up without submitting jobs to HTCondor one can use the dry run option:
0045
0046 ```
0047 validateAlignments.py $CMSSW_BASE/src/Alignment/OfflineValidation/test/test.yaml -d
0048 ```
0049
0050 More detailed examples for a possible validation configuration can be found here: `Alignment/OfflineValidation/test/example_DMR_full.yaml`
0051
0052 ## HOWTO implement
0053
0054 To implement a new/or porting an existing validation to the new frame work, two things needs to be provided: executables and a python file providing the information for each job.
0055
0056 #### Executables
0057
0058 In the new frame work standalone executables do the job of the validations. They are designed to run indenpendently from the set up of validateAlignments.py, the executables only need a configuration file with information needed for the validation/plotting. One can implement a C++ or a python executable.
0059
0060 If a C++ executable is implemented, the source file of the executable needs to be placed in the` Alignment/OfflineValidation/bin` directory and the BuildFile.xml in this directory needs to be modified. For the readout of the configuration file, which is in JSON format, the property tree class from the boost library is used. See `bin/DMRmerge.cc as` an example of a proper C++ implementation.
0061
0062 If a python executable is implemented, the source file needs to be placed in the `Alignment/OfflineValidation/scripts` directory. In the first line of the python script a shebang like `#!/usr/bin/env python` must be written and the script itself must be changed to be executable. In the case of python the configuration file can be both in JSON/YAML, because in python both after read in are just python dictionaries. See `Example of Senne when he finished it` as an example of a proper python implementation.
0063
0064 For the special case of a cmsRun job, one needs to provide only the CMS python configuration. Because it is python again, both JSON/YAML for the configuration file are fine to use. Also for this case the execution via cmsRun is independent from the set up provided by validateAligments.py and only need the proper configuration file. See `python/TkAlAllInOneTool/DMR_cfg.py` as an example of a proper implementation.
0065
0066 #### Python file for configuration
0067
0068 For each validation several jobs can be executed, because there are several steps like nTupling, fitting, plotting or there is categorization like alignments, IOVs. The information will be encoded in a global config provided by the aligner, see `Alignment/OfflineValidation/test/test.yaml` as an example. To figure out from the global config which/how many jobs should be prepared, a python file needs to be implemented which reads the global config, extract the relevant information of the global config and yields smaller config designed to be read from the respective executable. As an example see `python/TkAlAllInOneTool/DMR.py`.
0069
0070 There is a logic which needed to be followed. Each job needs to be directionary with a structure like this:
0071
0072 ```
0073 job = {
0074 "name": Job name ##Needs to be unique!
0075 "dir": workingDirectory ##Also needs to be unique!
0076 "exe": Name of executable/or cmsRun
0077 "cms-config": path to CMS config if exe = cmsRun, else leave this out
0078 "dependencies": [name of jobs this jobs needs to wait for] ##Empty list [] if no depedencies
0079 "config": Slimmed config from global config only with information needed for this job
0080 }
0081 ```
0082
0083 The python file returns a list of jobs to the `validateAligments.py` which finally creates the directory structure/configuration files/DAG file. To let` validateAligments.py` know one validation implementation exist, import the respective python file and extend the if statements which starts at line 271. This is the only time one needs to touch `validateAligments.py`!
0084
0085
0086 ## TODO list
0087
0088 - improve exceptions handling (filesystem + own)
0089 - unification of local configuration style based on DMR/PV example
0090 - plotting style options to be implemented
0091 - change marker size for trends
0092 - accept ROOT pre-defined encoding in config (`kRed`, `kDotted`, etc.)
0093 - validations to implement:
0094 - PV (only average is missing)
0095 - Zµµ (trend)
0096 - MTS (single + merge)
0097 - overlap (single + merge + trend)
0098 - ...
0099 - documentation (this README)
0100 - tutorial for SplitV and GCP
0101 - more working examples
0102 - instructions for developers
0103 - details
0104 - results of PV validation do not end up in results directory but one above
0105 - crab submission not available for all validations
0106 (list from October 2022)
0107
0108 ## DMR validation
0109 For details read [`README_DMR.md`](https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/README_DMR.md)
0110
0111 ## PV validation
0112 For details read [`README_PV.md`](https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/README_PV.md)
0113
0114 ## JetHT validation
0115 For details read [`README_JetHT.md`](https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/README_JetHT.md)
0116
0117 ## General info about IOV/run arguments
0118 For details read [`README_IOV.md`](https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/README_IOV.md)
