Back to home page

Project CMSSW displayed by LXR

 
 

    


File indexing completed on 2021-02-14 13:28:39

0001 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
0002 <BODY bgcolor="FFFFFF">
0003 <title>
0004           CMS MessageLogger: Open Issues Concerning Features Wanted by CMS
0005 </title>
0006 
0007 <center>
0008 <h1> <img src="header-public.gif" align="center"> </h1>
0009 
0010 <font color=red>
0011 <h1>CMS MessageLogger Service
0012 <br> 
0013 Open Issues Concerning Features Wanted by CMS</h1>
0014 </font>
0015 </center>
0016 <ul>
0017 <li> <a href=#multipleID> Multiple Message Categories </a>
0018 <li> <a href=#log4cplus> Destination Reporting to log4cplus </a>
0019     <ul> <font color=blue> Implemented </font> </ul>
0020 <li> <a href=#probe> Probing for whether a message will be reported </a>
0021     <ul> <font color=blue> Interface Implemented </font> </ul>
0022 <li> <a href=#statistics> Statistics Destination </a>
0023 <li> <a href=#filtering> Other Filtering Options </a>
0024 <li> <a href=#context> Context (e.g., event number) in Messages </a>
0025 <li> <a href=#endl> Support For Use Of endl in Messages </a>
0026 <li> <a href=#messageobj> Multi-statement Building of Message Objects </a>
0027 <li> <a href=#control> Post-configuration Control of Logging Behavior </a>
0028 </ul>
0029 
0030 <hr>
0031 <a name=#multipleID></a>
0032 <h2>Multiple Message Categories </h2>
0033 
0034 <h3> The requirement </h3>
0035 A message can be issued  with multiple categoriess, as in 
0036 <pre>
0037   edm::LogWarning(tracking&amp;overflow) << "some more text";
0038 </pre>
0039 
0040 This is to be treated as both a category=tracking message, 
0041 and an category=overflow
0042 message.
0043 
0044 <h3> Questions </h3>
0045 
0046 What does it mean to be two types of message.  The easiest thing to do would
0047 be to issue two distinct messages but assumedly that is not the most desirable
0048 behavior.  The questions include:
0049 <ul>
0050 <li> How should the output message look?  (We implied that just including the 
0051 combined categories as the ID would be OK.)
0052 <li> Which destinations should react to this message?  (Probably destinations
0053 that would react to either of the combined categories.)
0054 <li> What should this do to the message counts for each category
0055  for each destination?
0056 <li> How is this treated for statistics?  (Probably one message of each type;
0057 but the first-two, last-one contexts might want to behave specially.)
0058 </ul>
0059 
0060 <h3> Proposed treatment </h3>
0061 When a message is issued
0062 <pre>
0063   edm::LogWarning(tracking&amp;overflow) << "some more text";
0064 </pre>
0065 message counts for both "tracking" and "overflow" ID's are incremented.
0066 If a given destination would react to <em>either</em> a "tracking" message 
0067 or an "overflow" message, it will react to this message, but only a single
0068 copy of the message output will appear.
0069 <p>
0070 In that message output, the category will look like tracking&amp;overflow, 
0071 and even if
0072 the combined length is longer than the normally permitted length the full
0073 combined categoriess will appear.
0074 <p> 
0075 Statistics destinations will note the appearance of one "tracking" message
0076 and one "overflow" message; thus the total of counts by message ID will no 
0077 longer match the total of counts by severity.  The context (of the first 
0078 two and last one of each type of message) kept by the statistics destination
0079 will react as it would to a single message of each of the categories.  Thus it
0080 is possible to have the same context (event number or whatever) noted twice
0081 (in two different categories) caused by this one error message.
0082    
0083 <h3> Work consequences </h3>
0084 
0085 The "issue the message twice" attitude would require no work on ErrorLogger
0086 internals, and routine work in the LogXYZ() functions. However, that is not
0087 the desired behavior.
0088 <p>
0089 The above proposal will require modifying the nature of a message ID within
0090 ErrorLogger.  At some points it is a potentially compound ID; at others (for
0091 example, in the limit maps and statistics maps) it is exactly as before.
0092 This is non-trivial, as the messageID penetrates much of the package, but 
0093 as long as the behavior to shoot for is well-defined, this appears to be a
0094 solvable task.
0095 
0096 <h3> Decision and Plan </h3>
0097 
0098 The proposed treatment described above is agreed upon. 
0099 
0100 In order that the user be able to use the multiple-category syntax as soon as 
0101 possible, we will temporarily implement the double-message treatment.
0102 
0103 <hr>
0104 <a name=#log4cplus></a>
0105 <h2>Destination Reporting to log4cplus  </h2>
0106 
0107 <h3> The requirement </h3>
0108 
0109 We need a destination which, rather than sending to a file or ostream,
0110 delivers the header and text of the message to the CMS log4cplus facility.
0111 
0112 <h3> Questions </h3>
0113 
0114 What do we do to use log4cplus, and what options does it have:
0115 <ul>
0116 <li> Should we have a default destination of log4cplus, so that the user gets it
0117 even without mention in the .cfg file?
0118 <li> Is it mature enough to just use, or do we check out the package (and
0119 if so, can we make needed tweaks)?
0120 <li> What choices (if any) should we make if there are options in how to use
0121 log4cplus?
0122 <li> Are there any options which we should leave to the user via the .cfg file?
0123 </ul>
0124 
0125 
0126 <h3> Proposed treatment </h3>
0127 
0128 <ul>
0129 <li>
0130 We create a ELlog4cplus destination class which reports to log4cplus.
0131 This is similar to ELoutput, but probably formats the header and remainder
0132 of the message separately.
0133 <li>
0134 We attach an ELlog4cplus destination always.
0135 <li>
0136 We provide a special named destination parameter "log4cplus" by which a user
0137 can control filtering of messages to that destination.
0138 <li>
0139 We are told by CMS which log4cplus options to use, and we do not provide 
0140 further flexibility from the .cfg file for the user changing those options.
0141 </ul>
0142    
0143 <h3> Work consequences </h3>
0144 
0145 There are three potentially significant areas of work:
0146 <ol>
0147 <li> Understanding how to use log4cplus may be easy (especially if the
0148 product is mature and well documented) or arbitrarily hard.  We can give no
0149 estimate on how long this will take until we try it out.
0150 <li> Creating the ELlog4cplus destination will be some non-trivial but
0151 fixed amount of semi-routine work, since it is done with some knowledge of 
0152 how ELoutput works.
0153 <li> Providing the parameters to control flexibility will take about half a day
0154 per parameter, once we know what sort of .cfg control (if any) is needed by CMS.
0155 </ol>
0156 
0157 <h3> Decision and Plan </h3>
0158 
0159 Instead of the proposed treatment, we have created a 
0160 <font color = red>MLlog4cplus</font> service which ensures that 
0161 an ELlog4cplus destination is attached to the logger.  This has the 
0162 advantage of cutting any dependencies of MessageLogger on log4cplus (which is
0163 in xdaq).
0164 
0165 The log4cplus capability is as of 12/23/05 in place and tested.  It is up to 
0166 the user to deal with log4cplus, in particular, to assign whatever appenders
0167 are needed for the job.
0168 
0169 Two temporary conditions are left:
0170 <ol>
0171 <li>
0172 We currently assign a file_appender writing to log4cplus.output whenever
0173 the MLlog4cplus service is specified in the .cfg file; it is 
0174 probable that CMS will want to change this default action.
0175 <li> 
0176 We have not yet done the work to allow the user to control, via the .cfg file,
0177 MessageLogger filtering of the ELlog4cplus destination.
0178 </ol>
0179 
0180 <hr>
0181 <a name=probe></a>
0182 <h2>Probing for whether a message will be reported </a>
0183 </ul>  </h2>
0184 
0185 <h3> The requirement </h3>
0186 
0187 User code may sometimes generate messages which under many circumstances would
0188 be ignored by all destinations.  It would be desirable to be able to quickly 
0189 probe whether any destination would respond to a given message severity
0190 and category, so that the work of preparing and formating the message items 
0191 can be skipped if appropriate.
0192 <p>
0193 
0194 There are two potential modes of usage for this capability.  The naive mode is
0195 to do the probe each time a message is to be prepared.  Another mode is to
0196 assume that the result of this probe will remain static, and to cache either
0197 the first result or the first negative result.  In that mode, a message 
0198 which would be ignored will cost only one conditional on the cached boolean.
0199 
0200 <h3> Questions </h3>
0201 
0202 Should we also try to automate this checking, to avoid the cost of the 
0203 operator &lt;&lt; when the user appends items to the line to log a message?
0204 <p>
0205 Since the cost of a probe will not be trivial, should we cache results and
0206 consider the probe of each message type to be a one-shot affair?
0207 That is,
0208 if you have learned whether this type of message is reported, 
0209 should we assume that this answer will not change. 
0210 This is not strictly true because of limits, but it may be a useful shortcut.
0211  
0212 <h3> Proposed treatment </h3>
0213 
0214 Create 4 new functions:
0215 <font color=blue>
0216 <pre>
0217   ProbeLogError   (const std::string & category);  
0218   ProbeLogWarning (const std::string & category);  
0219   ProbeLogInfo    (const std::string & category);  
0220   ProbeLogDebug   (const std::string & category);  
0221 </pre>
0222 </font>
0223 
0224 These would return true if any destination would respond to the 
0225 corresponding message.
0226 <p>
0227 The first time a message type is probed, we must issue some sort of special 
0228 two-phase command to the MessageLoggerScribe, and it must cause an actual check 
0229 useing ErrorLogger code.
0230 Once a message type has been probed in this way, the answer will be cached.
0231 We should implement this cache via a map
0232 (in the local thread, to avoid the complications of locking against new
0233 entries by other threads) of all messages already probed.
0234 When the same message type is probed again, the map is consulted.  
0235 <p>
0236 In order that we eventually get the efficiency of rapidly ignoring 
0237 which were previously reported but 
0238 have reached their limit, when a true result is found in the map,
0239 one time in about twenty we should re-check.
0240 <p>
0241 We can 
0242 automate the check of the cached map to occur 
0243 whenever messages are issued.  (The user still has the option of probing
0244 to avoid work to prepare the items to be added to the message.)
0245 The rules would be:
0246 <ul>
0247 <li>
0248 If a message has never been probed, the LogXYZ commands always send that
0249 message along to the ErrorLogger.
0250 <li>
0251 If a message has ever been probed and found not to be reportable, 
0252 then the result of a probe (automatic or explicit) will be that the 
0253 message is not reportable.  For unreportable messages, 
0254 operator &lt;&lt to the results of the LogXYZ commands become no-ops.
0255 <li>
0256 Every 11-th time the cache of results is used and delivers true,
0257 that message is re-probed in case it is now being ignored.  This interval
0258 increases via an exponential backoff. 
0259 <li>
0260 If a message has been probed and found to be reportable (either by the cache
0261 or by a new actual probe), 
0262 then LogXYZ will send that message along to the ErrorLogger.
0263 </ul>
0264 However, we can check the local map:  
0265 if a message has already been probed and found to be ignored, 
0266 then the  operator &lt;&lt can become a no-op.  
0267 <p>
0268 This presents another mode of usage, one which we can recommend:  
0269 The user probes a message category 
0270 just once, and relies on the system to obviate extra work by automatically
0271 checking when appropriate.
0272 <p>
0273 We do have to warn about the effect of this on message statistics:
0274 Messaages which are sent to the logger but ignored make it into the statistics,
0275 but messages which are never even sent do not.  (Assumedly, the user would
0276 not care about the count of messages in categories which are completely 
0277 ignored.)
0278 
0279 <h3> Quick temporary treatment </h3>
0280 
0281 We can quickly implement the propbe functions as returning true, thus 
0282 allowing user code to start using this mechanism (with no efficiency gain
0283 until we implement the actual probing) immediately.
0284 
0285 <font color=red><b>
0286 This treatment is now in place.
0287 </b></font>
0288 
0289 <h3> Work consequences </h3>
0290 
0291 The ErrorLogger currently has no means of probing for responding destinations.
0292 This will need to be created.  The difficulty should be moderate (time estimate
0293 of 1 day).
0294 <p>
0295 The MessageLoggerScribe will need a new opcode to do the probe.  The probe
0296 data structure would be a pointer to one integer (not a bool, because the 
0297 probing end has to know when the result is now valid).  The probe routine
0298 (on the client side) will need to either sleep on that value changing, or
0299 otherwise wait till the true/false value is established.  This is not as
0300 easy as the other part, but probably will take another 1 day.
0301 <p>
0302 The establishment of the cache will be easy, 
0303 but the re-writes of LogWarning etc. to take 
0304 advantage of known non-response results will be subtle.
0305 In particular, the LogDebug macro may be very subtle if we don't want 
0306 to take the cost of forming and passing __FILE__ and __LINE__ when we don't 
0307 have to.  
0308 <p> 
0309 The total time estimate for this feature is three solid days.
0310 
0311 <hr>
0312 <a name=#statistics></a>
0313 <h2>Statistics Destination </h2>
0314 
0315 <h3> The requirement </h3>
0316 
0317 The ErrorLogger has a nice ELstatistics destination for summarizing 
0318 the messages.
0319 <p>
0320 We need a way for MessageLogger service users to to specify a 
0321 statistics destination.
0322 
0323 <h3> Questions </h3>
0324 
0325 <ul>
0326 <li> Should we let the .cfg file specify where statistics are to be sent,
0327 or have some default destination?
0328 <li> What should trigger the reporting of statistics?  (For example, it could
0329 be part of the end-of-job or end-of-run activity of the MessageLogger
0330 service.)
0331 <li> How should we obtain the "short context" to report the contexts of the
0332 first two and last one of each type of message -- or should we punt that 
0333 feature?
0334 </ul>
0335 
0336 
0337 <h3> Proposed treatment </h3>
0338 
0339 <ul>
0340 <li>
0341 We treat the statistics destinations analogously to the output destinations,
0342 allowing the user to provide a list.  This allows the user to control those
0343 features of the statistics destinations (including threshold and output control)
0344 which are needed.
0345 <li>
0346 By default, we create a single statistics destination sending its output to 
0347 the first of the output destinations (or to the log4cplus destination).
0348 <li>
0349 The issue of context is solvable (we can obtain the event number, for example)
0350 but it is an open issue as to how we would do this.  For a start, we could 
0351 disable the noting of contexts.
0352 </ul>
0353    
0354 <h3> Work consequences </h3>
0355 
0356 Other than the context issue, we can probably set up statistics in a day.
0357 
0358 
0359 <hr>
0360 <a name=#filtering></a>
0361 <h2>Other Filtering Options  </h2>
0362 
0363 <h3> The requirement </h3>
0364 The ErrorLogger package supplies 
0365 other ways to specify limits and thresholds on which messages a destination will 
0366 report.  For instance, one can set a limit for all message IDs <em>except</em>
0367 a specified type.
0368 The MessageLogger service might benefit from enabling some of these further 
0369 options via the configuration file.
0370 <p>
0371 <em>This requirement may or may not finally be requested.</em>
0372 
0373 
0374 <h3> Questions </h3>
0375 
0376 <ul>
0377 <li> Do we want exclusion limits?
0378 <li> Do we want rapid-discard thresholds?
0379 <li> Other flexibility (see the 
0380 <a href=
0381 "www.fnal.gov/docs/working-groups/fpcltf/Pkg/ErrorLogger/doc/html/0ErrorLogger.html">
0382 ZOOM ErrorLogger</a> documentation for possibilities)?
0383 </ul>
0384 
0385 <h3> Proposed treatment </h3>
0386 Additional parameters in the .cfg file, either destination-specific lines
0387 within the destination PSet, or general lines within the service=MessageLogger
0388 PSet.
0389    
0390 <h3> Work consequences </h3>
0391 
0392 Requires no modifications to the ErrorLogger package.  
0393 However, each degree of flexibility will require code in MessageLoggerScribe
0394 to understand the parameter and issue appropriate calls to the ErrorLogger.
0395 Also, each feature requires CMS MessageLogger service documentation (otherwise 
0396 it is next to useless).  The total time to enable a feature is about half a day.
0397 
0398 
0399 <hr>
0400 <a name=#context></a>
0401 <h2>Context (e.g., event number) in Messages </a>
0402 </ul>  </h2>
0403 
0404 <h3> The requirement </h3>
0405 
0406 The ErrorLogger package supplies 
0407 ways to automatically append context information to messages.  The intent
0408 is to indicate event/run numbers without the message issuer having to think 
0409 about it.  Perhaps the MessageLogger service should support this. 
0410 <p>
0411 <em>This requirement may or may not finally be requested.</em>
0412 
0413 
0414 <h3> Questions </h3>
0415 
0416 What should we use as the context?
0417 <ul>
0418 <li> How do we obtain the context (error number or whatever)?
0419 <li> Is there a useful abbreviated form?
0420 <li> Does getting the context take too much time?
0421 </ul>
0422 
0423 <h3> Proposed treatment </h3>
0424 
0425 We find out how to get context from an existing EDM service, and write a 
0426 context supplier (in the ErrorLogger sense) to do so).  We propose that the
0427 format of the context not be user configurable.
0428    
0429 <h3> Work consequences </h3>
0430 
0431 Requires no modifications to the ErrorLogger package.  
0432 Writing the context supplier is likely to take less than a day, assuming the
0433 info is available from the EDM.
0434 
0435 
0436 <hr>
0437 <a name=endl></a>
0438 <h2>Support For Use Of endl in Messages </a>
0439 </ul>  </h2>
0440 
0441 <h3> The requirement </h3>
0442 
0443 Users are used to using std::endl as a line terminator.
0444 The use of \n is already supported for this, but it might be
0445 desirable to also support endl.
0446 <p>
0447 <em>This requirement may or may not finally be requested.</em>
0448 
0449 
0450 <h3> Questions </h3>
0451 
0452 Is endl any different in effect than \n?
0453 
0454 <h3> Proposed treatment </h3>
0455 
0456 The ErrorLogger package implementers originally attempted use std::endl 
0457 as a message terminator.
0458 This led to technical difficulties we could not, at that time, surmount
0459 (thus the introduction of the errmsg manipulator.
0460 <p>
0461 Jim Kowalkowski and Marc Paterno claim they know how to do this without 
0462 those difficulties.  If so, we will adapt that technique, treating endl as
0463 a "force line termination" directive (as opposed to a message terminator).
0464 <p>
0465 If this is not easy, we should not do it, as the gain is slight.   
0466 <h3> Work consequences </h3>
0467 
0468 Requires learning how to handle endl, and implementing that in ELoutput
0469 and other destinations.  The danger is that the problem issues may be subtle, in
0470 which case it is best to abandon the idea.
0471 
0472 <hr>
0473 <a name=messageobj></a>
0474 <h2>Multi-statement Building of Message Objects </a>
0475 </ul>  </h2>
0476 
0477 <h3> The requirement </h3>
0478 
0479 In the ErrorLogger package, the user can gradually build an ErrorObj containing
0480 th message, and later dispatch it ot the logger.  A similar capability
0481 might be desirable in the MessageLogger service.
0482 
0483 <p>
0484 <em>This requirement may or may not finally be requested.</em>
0485 
0486 
0487 <h3> Questions </h3>
0488 <ul>
0489 <li>
0490 What is the best way do instantiate the object?
0491 <li>
0492 How should the message be fired off to the log? 
0493 </ul>
0494 <h3> Proposed treatment </h3>
0495 
0496 The ErrorObj in the ErrorLogger package already has its severity and ID imbedded
0497 upon construction, and we propose to do the same here.  
0498 We would provide 3 classes (the Debug form has complications we choose not to
0499 deal with:  InfoMessage, WarningMessage, and ErrorMessage.  The ctor would take
0500 the message ID.
0501 <p>
0502 The user community would probably vote for some sort of send() member function
0503 or special endmsg manipulator to indicate that the message is to be dispatched,
0504 but this leaves around a dangerously already-sent message object.  The correct
0505 idiom is to dispatch the message when the object is destructed.  The user code
0506 should look like:
0507 <pre>
0508   if ( problem_is_detected ) {
0509     WarningMessage warn ("thistypeoftrouble");
0510     warn << "information is";
0511     while (there_is_more_information) {
0512       warn << get_a_piece_of_information(); 
0513     }
0514   } // here warn goes out of scope and the message is dispatched
0515 </pre>
0516 
0517 <h3> Work consequences </h3>
0518 
0519 This would be straightforward since no ErrorLogger code need be modified.  
0520 Including documentation it should take a couple of days.
0521 
0522 
0523 <hr>
0524 <a name=control></a>
0525 <h2>Post-configuration Control of Logging Behavior </a>
0526 </ul>  </h2>
0527 
0528 <h3> The requirement </h3>
0529 
0530 The current service allows control of the behavior of destinations (filtering
0531 and thresholds) and the overall logger, only via job-start configuration 
0532 parameters found in the .cfg file.
0533 We could also support modification of these choices under programmatic control.
0534 <p>
0535 <em>This requirement may or may not finally be requested.</em>
0536 
0537 
0538 <h3> Questions </h3>
0539 <ul>
0540 <li>
0541 What degree of control should we support?
0542 <li>
0543 What is the syntax of the code controlling logging behavior?
0544 </ul>
0545 
0546 <h3> Proposed treatment </h3>
0547 
0548 We immediately step onto a slippery slope (more like, step off a steep cliff)
0549 when we open up the idea of runtime control of these options.  
0550 Developing an interface for this control would take weeks at the least, and 
0551 there would inevitably be arguments over all the equally-good ways of expressing 
0552 the desired behavior.
0553 <p>
0554 The only sensible answers are to provide no post-configuration control at all,
0555 or to provide everything in the ErrorLogger package.  In the latter case, the
0556 capability is provided by having the service provide points or references to
0557 the actual ErrorLogger package objects, such as ELadministrator and the various
0558 ELdestControl handles.  The user would need to look in the ErrorLogger package
0559 documentation for how to use these.
0560 
0561 <h3> Work consequences </h3>
0562 
0563 If we can stick to our guns about just providing handles to raw ErrorLogger 
0564 objects, this should not be too hard.  There may be a bit of work in developing
0565 an interface to get the ELdestControl for a given destination, but we
0566 probably already have the structures in place for that.
0567 <p>
0568 If we need to develop a separate CMS-approved interface for post-configuration 
0569 adjustment of behavior, this will be a hopeless and endless task.
0570 
0571 
0572 <p><center>
0573 <img src="bar.gif"></center>
0574 
0575 <p><center>
0576 <a href="http://www.uscms.org/SoftwareComputing/index.html">
0577 USCMS Software and Computing Home Page </a> -
0578 <a href="MessageLogger.html">CMS MessageLogger Service Page</a>
0579 </center>
0580 
0581 <p>
0582       <hr>
0583       <address><a href="mailto:mf@fnal.gov">Mark Fischler</a></address>
0584 <!-- hhmts start -->
0585 Last modified: December 1, 2005
0586 <!-- hhmts end -->
0587 </body>