|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
|
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<BODY bgcolor="FFFFFF">
<title>
CMS MessageLogger: Open Issues Concerning Features Wanted by CMS
</title>
<center>
<h1> <img src="header-public.gif" align="center"> </h1>
<font color=red>
<h1>CMS MessageLogger Service
<br>
Open Issues Concerning Features Wanted by CMS</h1>
</font>
</center>
<ul>
<li> <a href=#multipleID> Multiple Message Categories </a>
<li> <a href=#log4cplus> Destination Reporting to log4cplus </a>
<ul> <font color=blue> Implemented </font> </ul>
<li> <a href=#probe> Probing for whether a message will be reported </a>
<ul> <font color=blue> Interface Implemented </font> </ul>
<li> <a href=#statistics> Statistics Destination </a>
<li> <a href=#filtering> Other Filtering Options </a>
<li> <a href=#context> Context (e.g., event number) in Messages </a>
<li> <a href=#endl> Support For Use Of endl in Messages </a>
<li> <a href=#messageobj> Multi-statement Building of Message Objects </a>
<li> <a href=#control> Post-configuration Control of Logging Behavior </a>
</ul>
<hr>
<a name=#multipleID></a>
<h2>Multiple Message Categories </h2>
<h3> The requirement </h3>
A message can be issued with multiple categoriess, as in
<pre>
edm::LogWarning(tracking&overflow) << "some more text";
</pre>
This is to be treated as both a category=tracking message,
and an category=overflow
message.
<h3> Questions </h3>
What does it mean to be two types of message. The easiest thing to do would
be to issue two distinct messages but assumedly that is not the most desirable
behavior. The questions include:
<ul>
<li> How should the output message look? (We implied that just including the
combined categories as the ID would be OK.)
<li> Which destinations should react to this message? (Probably destinations
that would react to either of the combined categories.)
<li> What should this do to the message counts for each category
for each destination?
<li> How is this treated for statistics? (Probably one message of each type;
but the first-two, last-one contexts might want to behave specially.)
</ul>
<h3> Proposed treatment </h3>
When a message is issued
<pre>
edm::LogWarning(tracking&overflow) << "some more text";
</pre>
message counts for both "tracking" and "overflow" ID's are incremented.
If a given destination would react to <em>either</em> a "tracking" message
or an "overflow" message, it will react to this message, but only a single
copy of the message output will appear.
<p>
In that message output, the category will look like tracking&overflow,
and even if
the combined length is longer than the normally permitted length the full
combined categoriess will appear.
<p>
Statistics destinations will note the appearance of one "tracking" message
and one "overflow" message; thus the total of counts by message ID will no
longer match the total of counts by severity. The context (of the first
two and last one of each type of message) kept by the statistics destination
will react as it would to a single message of each of the categories. Thus it
is possible to have the same context (event number or whatever) noted twice
(in two different categories) caused by this one error message.
<h3> Work consequences </h3>
The "issue the message twice" attitude would require no work on ErrorLogger
internals, and routine work in the LogXYZ() functions. However, that is not
the desired behavior.
<p>
The above proposal will require modifying the nature of a message ID within
ErrorLogger. At some points it is a potentially compound ID; at others (for
example, in the limit maps and statistics maps) it is exactly as before.
This is non-trivial, as the messageID penetrates much of the package, but
as long as the behavior to shoot for is well-defined, this appears to be a
solvable task.
<h3> Decision and Plan </h3>
The proposed treatment described above is agreed upon.
In order that the user be able to use the multiple-category syntax as soon as
possible, we will temporarily implement the double-message treatment.
<hr>
<a name=#log4cplus></a>
<h2>Destination Reporting to log4cplus </h2>
<h3> The requirement </h3>
We need a destination which, rather than sending to a file or ostream,
delivers the header and text of the message to the CMS log4cplus facility.
<h3> Questions </h3>
What do we do to use log4cplus, and what options does it have:
<ul>
<li> Should we have a default destination of log4cplus, so that the user gets it
even without mention in the .cfg file?
<li> Is it mature enough to just use, or do we check out the package (and
if so, can we make needed tweaks)?
<li> What choices (if any) should we make if there are options in how to use
log4cplus?
<li> Are there any options which we should leave to the user via the .cfg file?
</ul>
<h3> Proposed treatment </h3>
<ul>
<li>
We create a ELlog4cplus destination class which reports to log4cplus.
This is similar to ELoutput, but probably formats the header and remainder
of the message separately.
<li>
We attach an ELlog4cplus destination always.
<li>
We provide a special named destination parameter "log4cplus" by which a user
can control filtering of messages to that destination.
<li>
We are told by CMS which log4cplus options to use, and we do not provide
further flexibility from the .cfg file for the user changing those options.
</ul>
<h3> Work consequences </h3>
There are three potentially significant areas of work:
<ol>
<li> Understanding how to use log4cplus may be easy (especially if the
product is mature and well documented) or arbitrarily hard. We can give no
estimate on how long this will take until we try it out.
<li> Creating the ELlog4cplus destination will be some non-trivial but
fixed amount of semi-routine work, since it is done with some knowledge of
how ELoutput works.
<li> Providing the parameters to control flexibility will take about half a day
per parameter, once we know what sort of .cfg control (if any) is needed by CMS.
</ol>
<h3> Decision and Plan </h3>
Instead of the proposed treatment, we have created a
<font color = red>MLlog4cplus</font> service which ensures that
an ELlog4cplus destination is attached to the logger. This has the
advantage of cutting any dependencies of MessageLogger on log4cplus (which is
in xdaq).
The log4cplus capability is as of 12/23/05 in place and tested. It is up to
the user to deal with log4cplus, in particular, to assign whatever appenders
are needed for the job.
Two temporary conditions are left:
<ol>
<li>
We currently assign a file_appender writing to log4cplus.output whenever
the MLlog4cplus service is specified in the .cfg file; it is
probable that CMS will want to change this default action.
<li>
We have not yet done the work to allow the user to control, via the .cfg file,
MessageLogger filtering of the ELlog4cplus destination.
</ol>
<hr>
<a name=probe></a>
<h2>Probing for whether a message will be reported </a>
</ul> </h2>
<h3> The requirement </h3>
User code may sometimes generate messages which under many circumstances would
be ignored by all destinations. It would be desirable to be able to quickly
probe whether any destination would respond to a given message severity
and category, so that the work of preparing and formating the message items
can be skipped if appropriate.
<p>
There are two potential modes of usage for this capability. The naive mode is
to do the probe each time a message is to be prepared. Another mode is to
assume that the result of this probe will remain static, and to cache either
the first result or the first negative result. In that mode, a message
which would be ignored will cost only one conditional on the cached boolean.
<h3> Questions </h3>
Should we also try to automate this checking, to avoid the cost of the
operator << when the user appends items to the line to log a message?
<p>
Since the cost of a probe will not be trivial, should we cache results and
consider the probe of each message type to be a one-shot affair?
That is,
if you have learned whether this type of message is reported,
should we assume that this answer will not change.
This is not strictly true because of limits, but it may be a useful shortcut.
<h3> Proposed treatment </h3>
Create 4 new functions:
<font color=blue>
<pre>
ProbeLogError (const std::string & category);
ProbeLogWarning (const std::string & category);
ProbeLogInfo (const std::string & category);
ProbeLogDebug (const std::string & category);
</pre>
</font>
These would return true if any destination would respond to the
corresponding message.
<p>
The first time a message type is probed, we must issue some sort of special
two-phase command to the MessageLoggerScribe, and it must cause an actual check
useing ErrorLogger code.
Once a message type has been probed in this way, the answer will be cached.
We should implement this cache via a map
(in the local thread, to avoid the complications of locking against new
entries by other threads) of all messages already probed.
When the same message type is probed again, the map is consulted.
<p>
In order that we eventually get the efficiency of rapidly ignoring
which were previously reported but
have reached their limit, when a true result is found in the map,
one time in about twenty we should re-check.
<p>
We can
automate the check of the cached map to occur
whenever messages are issued. (The user still has the option of probing
to avoid work to prepare the items to be added to the message.)
The rules would be:
<ul>
<li>
If a message has never been probed, the LogXYZ commands always send that
message along to the ErrorLogger.
<li>
If a message has ever been probed and found not to be reportable,
then the result of a probe (automatic or explicit) will be that the
message is not reportable. For unreportable messages,
operator << to the results of the LogXYZ commands become no-ops.
<li>
Every 11-th time the cache of results is used and delivers true,
that message is re-probed in case it is now being ignored. This interval
increases via an exponential backoff.
<li>
If a message has been probed and found to be reportable (either by the cache
or by a new actual probe),
then LogXYZ will send that message along to the ErrorLogger.
</ul>
However, we can check the local map:
if a message has already been probed and found to be ignored,
then the operator << can become a no-op.
<p>
This presents another mode of usage, one which we can recommend:
The user probes a message category
just once, and relies on the system to obviate extra work by automatically
checking when appropriate.
<p>
We do have to warn about the effect of this on message statistics:
Messaages which are sent to the logger but ignored make it into the statistics,
but messages which are never even sent do not. (Assumedly, the user would
not care about the count of messages in categories which are completely
ignored.)
<h3> Quick temporary treatment </h3>
We can quickly implement the propbe functions as returning true, thus
allowing user code to start using this mechanism (with no efficiency gain
until we implement the actual probing) immediately.
<font color=red><b>
This treatment is now in place.
</b></font>
<h3> Work consequences </h3>
The ErrorLogger currently has no means of probing for responding destinations.
This will need to be created. The difficulty should be moderate (time estimate
of 1 day).
<p>
The MessageLoggerScribe will need a new opcode to do the probe. The probe
data structure would be a pointer to one integer (not a bool, because the
probing end has to know when the result is now valid). The probe routine
(on the client side) will need to either sleep on that value changing, or
otherwise wait till the true/false value is established. This is not as
easy as the other part, but probably will take another 1 day.
<p>
The establishment of the cache will be easy,
but the re-writes of LogWarning etc. to take
advantage of known non-response results will be subtle.
In particular, the LogDebug macro may be very subtle if we don't want
to take the cost of forming and passing __FILE__ and __LINE__ when we don't
have to.
<p>
The total time estimate for this feature is three solid days.
<hr>
<a name=#statistics></a>
<h2>Statistics Destination </h2>
<h3> The requirement </h3>
The ErrorLogger has a nice ELstatistics destination for summarizing
the messages.
<p>
We need a way for MessageLogger service users to to specify a
statistics destination.
<h3> Questions </h3>
<ul>
<li> Should we let the .cfg file specify where statistics are to be sent,
or have some default destination?
<li> What should trigger the reporting of statistics? (For example, it could
be part of the end-of-job or end-of-run activity of the MessageLogger
service.)
<li> How should we obtain the "short context" to report the contexts of the
first two and last one of each type of message -- or should we punt that
feature?
</ul>
<h3> Proposed treatment </h3>
<ul>
<li>
We treat the statistics destinations analogously to the output destinations,
allowing the user to provide a list. This allows the user to control those
features of the statistics destinations (including threshold and output control)
which are needed.
<li>
By default, we create a single statistics destination sending its output to
the first of the output destinations (or to the log4cplus destination).
<li>
The issue of context is solvable (we can obtain the event number, for example)
but it is an open issue as to how we would do this. For a start, we could
disable the noting of contexts.
</ul>
<h3> Work consequences </h3>
Other than the context issue, we can probably set up statistics in a day.
<hr>
<a name=#filtering></a>
<h2>Other Filtering Options </h2>
<h3> The requirement </h3>
The ErrorLogger package supplies
other ways to specify limits and thresholds on which messages a destination will
report. For instance, one can set a limit for all message IDs <em>except</em>
a specified type.
The MessageLogger service might benefit from enabling some of these further
options via the configuration file.
<p>
<em>This requirement may or may not finally be requested.</em>
<h3> Questions </h3>
<ul>
<li> Do we want exclusion limits?
<li> Do we want rapid-discard thresholds?
<li> Other flexibility (see the
<a href=
"www.fnal.gov/docs/working-groups/fpcltf/Pkg/ErrorLogger/doc/html/0ErrorLogger.html">
ZOOM ErrorLogger</a> documentation for possibilities)?
</ul>
<h3> Proposed treatment </h3>
Additional parameters in the .cfg file, either destination-specific lines
within the destination PSet, or general lines within the service=MessageLogger
PSet.
<h3> Work consequences </h3>
Requires no modifications to the ErrorLogger package.
However, each degree of flexibility will require code in MessageLoggerScribe
to understand the parameter and issue appropriate calls to the ErrorLogger.
Also, each feature requires CMS MessageLogger service documentation (otherwise
it is next to useless). The total time to enable a feature is about half a day.
<hr>
<a name=#context></a>
<h2>Context (e.g., event number) in Messages </a>
</ul> </h2>
<h3> The requirement </h3>
The ErrorLogger package supplies
ways to automatically append context information to messages. The intent
is to indicate event/run numbers without the message issuer having to think
about it. Perhaps the MessageLogger service should support this.
<p>
<em>This requirement may or may not finally be requested.</em>
<h3> Questions </h3>
What should we use as the context?
<ul>
<li> How do we obtain the context (error number or whatever)?
<li> Is there a useful abbreviated form?
<li> Does getting the context take too much time?
</ul>
<h3> Proposed treatment </h3>
We find out how to get context from an existing EDM service, and write a
context supplier (in the ErrorLogger sense) to do so). We propose that the
format of the context not be user configurable.
<h3> Work consequences </h3>
Requires no modifications to the ErrorLogger package.
Writing the context supplier is likely to take less than a day, assuming the
info is available from the EDM.
<hr>
<a name=endl></a>
<h2>Support For Use Of endl in Messages </a>
</ul> </h2>
<h3> The requirement </h3>
Users are used to using std::endl as a line terminator.
The use of \n is already supported for this, but it might be
desirable to also support endl.
<p>
<em>This requirement may or may not finally be requested.</em>
<h3> Questions </h3>
Is endl any different in effect than \n?
<h3> Proposed treatment </h3>
The ErrorLogger package implementers originally attempted use std::endl
as a message terminator.
This led to technical difficulties we could not, at that time, surmount
(thus the introduction of the errmsg manipulator.
<p>
Jim Kowalkowski and Marc Paterno claim they know how to do this without
those difficulties. If so, we will adapt that technique, treating endl as
a "force line termination" directive (as opposed to a message terminator).
<p>
If this is not easy, we should not do it, as the gain is slight.
<h3> Work consequences </h3>
Requires learning how to handle endl, and implementing that in ELoutput
and other destinations. The danger is that the problem issues may be subtle, in
which case it is best to abandon the idea.
<hr>
<a name=messageobj></a>
<h2>Multi-statement Building of Message Objects </a>
</ul> </h2>
<h3> The requirement </h3>
In the ErrorLogger package, the user can gradually build an ErrorObj containing
th message, and later dispatch it ot the logger. A similar capability
might be desirable in the MessageLogger service.
<p>
<em>This requirement may or may not finally be requested.</em>
<h3> Questions </h3>
<ul>
<li>
What is the best way do instantiate the object?
<li>
How should the message be fired off to the log?
</ul>
<h3> Proposed treatment </h3>
The ErrorObj in the ErrorLogger package already has its severity and ID imbedded
upon construction, and we propose to do the same here.
We would provide 3 classes (the Debug form has complications we choose not to
deal with: InfoMessage, WarningMessage, and ErrorMessage. The ctor would take
the message ID.
<p>
The user community would probably vote for some sort of send() member function
or special endmsg manipulator to indicate that the message is to be dispatched,
but this leaves around a dangerously already-sent message object. The correct
idiom is to dispatch the message when the object is destructed. The user code
should look like:
<pre>
if ( problem_is_detected ) {
WarningMessage warn ("thistypeoftrouble");
warn << "information is";
while (there_is_more_information) {
warn << get_a_piece_of_information();
}
} // here warn goes out of scope and the message is dispatched
</pre>
<h3> Work consequences </h3>
This would be straightforward since no ErrorLogger code need be modified.
Including documentation it should take a couple of days.
<hr>
<a name=control></a>
<h2>Post-configuration Control of Logging Behavior </a>
</ul> </h2>
<h3> The requirement </h3>
The current service allows control of the behavior of destinations (filtering
and thresholds) and the overall logger, only via job-start configuration
parameters found in the .cfg file.
We could also support modification of these choices under programmatic control.
<p>
<em>This requirement may or may not finally be requested.</em>
<h3> Questions </h3>
<ul>
<li>
What degree of control should we support?
<li>
What is the syntax of the code controlling logging behavior?
</ul>
<h3> Proposed treatment </h3>
We immediately step onto a slippery slope (more like, step off a steep cliff)
when we open up the idea of runtime control of these options.
Developing an interface for this control would take weeks at the least, and
there would inevitably be arguments over all the equally-good ways of expressing
the desired behavior.
<p>
The only sensible answers are to provide no post-configuration control at all,
or to provide everything in the ErrorLogger package. In the latter case, the
capability is provided by having the service provide points or references to
the actual ErrorLogger package objects, such as ELadministrator and the various
ELdestControl handles. The user would need to look in the ErrorLogger package
documentation for how to use these.
<h3> Work consequences </h3>
If we can stick to our guns about just providing handles to raw ErrorLogger
objects, this should not be too hard. There may be a bit of work in developing
an interface to get the ELdestControl for a given destination, but we
probably already have the structures in place for that.
<p>
If we need to develop a separate CMS-approved interface for post-configuration
adjustment of behavior, this will be a hopeless and endless task.
<p><center>
<img src="bar.gif"></center>
<p><center>
<a href="http://www.uscms.org/SoftwareComputing/index.html">
USCMS Software and Computing Home Page </a> -
<a href="MessageLogger.html">CMS MessageLogger Service Page</a>
</center>
<p>
<hr>
<address><a href="mailto:mf@fnal.gov">Mark Fischler</a></address>
<!-- hhmts start -->
Last modified: December 1, 2005
<!-- hhmts end -->
</body>
|