Subject: Re: Issue 2 (TAG)
From: Rainer Gerhards <[email-masked]>
Date: Thu, 02 Oct 2003 18:57:40 +0200

Albert,

this is how far I came with the ABNF. I didn't run it through the
validator, so there might still be some issues:

     TAG          =  full-stat-id  [full-dyn-id] (':' / SP)
     full-stat-id =  [path] progname
     full-dyn-id  = '[' proc-id [thread-part] ']'
     path         =  path-part 1*(path-sep [path])
     path-part    = 1*VISUAL
     path-sep     = '/' / '\'
     progname     = 1*VISUAL
     proc-id      = 1*ALFANUM  ; recommended: number
     thread-part  = thread-sep thread-id
     thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
     thread-id    = 1*ALFANUM  ; recommended: number
     VISUAL       = ([a-zA-Z0-9...], excusing  '['
     SP           = %d32

By updating it, some more points were raised (it is much like
implementing;)):

- - I made full-dyn-id optional - was that the intetion (mine it is;))
- - VISUAL must allow non US-ASCII characters
- - do we really need to describe the path?
- - why not fix thread-sep to ","?
- - MUST proc-id/thread-id by ALFANUM - would a SHOULD be sufficient (what
do we gain from a MUST)?
- - The ABNF is still not fully correct in the terminating SP of the
message... This can not be done only by specifying the tag. But I think
it is sufficiently to convey the idea.

I hope I got the spirit right, but you may check. All in all, I am very
happy with the propsal, except for the points raised ;)

HTH
Rainer

On Tue, 2003-09-30 at 12:03, Albert Mietus wrote:
> Based on Rainers text, the issue's/problems  of Chris, and the
> expirence in implementing it. The following text is proposed to be
> included in syslog-sign.
>
> """
>    The TAG is a string of visible (printing) characters excluding SP,
>    that MUST NOT exceed 64 characters in length.
>    The first occurrence of a SP (space) will terminate the TAG field,
>    but is not part of it.
>    It is RECOMMENDED to terminate the TAG with a colon (':'), which if
>    used, is part of the TAG.
>
>    The TAG is used to denote the sender of the message. And is
>    RECOMMENDED to have the following syntax:
>
>      TAG          =  full-stat-id  full-dyn-id ':'
>      full-stat-id =  path path-sep progname
>      full-dyn-id  = '[' proc-id thread-part ']'
>      path         = VISUAL *
>      path-sep	  = (  '/'  |  '\'  )
>      progname	  = VISUAL *
>      proc-id	  = ALFANUM *  ; recommended: number
>      thread-part  = ( EMPTY | thread-sep ALFANUM )
>      thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
>      EMPTY	  = "" ; nothing!
>      VISUAL	  = ([a-zA-Z0-9...], excusing  '['
>
> (hope I have the this syntax OK --please correct ALbert)
>
>    The PROGNAME part is special, as it is frequently used by relays to
>    determine the routing of a message.  As a note to implementors: it
>    can be found by getting the visual part before the first occurrence
>    of '[', and after the last '\' of '/' part of that segment.
>
>    An example of a TAG is: (without the quotes)
>       "/path/to/PROGNAME[123,456]:"
>
>    Systems that use both process-ID's and thead-IDs, SHOULD fill both
>    the proc-id and the thread-part. For other systems it is
>    RECOMMENDED to use the proc-id only.
>
>    Receivers SHOULD, to be consistent with the format described in
>    RFC3164, accept TAGs that terminate with a single colon, without a
>    space following it. Then the colon is both the last character of
>    that TAG, and the field separator with the next field (MSG).
> """
>
>
>

------------------------------

Date: Mon, 6 Oct 2003 09:00:51 +0200
From: Albert Mietus <[email-masked]>
Subject: Re: Issue 2 (TAG)

Rainer Gerhards writes:
 > >    The PROGNAME part is special, as it is frequently used by relays to
 > >    determine the routing of a message.  As a note to implementors: it
 > >    can be found by getting the visual part before the first occurrence
 > >    of '[', and after the last '\' of '/' part of that segment.

 > mmhhh... I can see the reasoning for this, but do you think we really
 > need to allow [ inside the path? Without it, implementing would be
 > easier. In my experience, parsing strings backwards (as would be
 > required) seems somehow more error-prone than forward parsing (but maybe
 > it's just me failing in those cases ;))

Just to inform:

* It not needed to parse backward. It can be done  in 1 for-loop
  (see my implementation)

* Currerently, several (Unix) programmes use this path approach. The
  log tags like " postfix/local[33282]:"

* I did found posting from this WG that resulted in this idea. I think
  Rainer posted some Window alinke paths as "C\to\Progrmam" ;-)

Hope it help

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Mon, 6 Oct 2003 09:19:59 +0200
From: Albert Mietus <[email-masked]>
Subject: Re: Issue 2 (TAG)

Hello again:-)

 > >    Receivers SHOULD, to be consistent with the format described in
 > >    RFC3164, accept TAGs that terminate with a single colon, without a
 > >    space following it. Then the colon is both the last character of
 > >    that TAG, and the field separator with the next field (MSG).
 >
 > I think you must somehow revert back to my wording. Above you say that
 > SP MUST terminate the tag. If you allow colon to terminate, too (I agree
 > on this need), you must also allow it - otherwise it is
 > confusing/inconsistent.

With the syntax I tried to express how a TAG SHOULD be. Senders should
use that, and receivers should expect it.

However, the real world  sometime differs. Like existing
syslog(d)'s. Which may (as rfc3164 describes) not use a SP to separat
field and use ":" to terminate.

The quoted part describes how a reciever shoul deal those "not
standard" logmessages.

Note: we can not acccept colon as a terminator. E.g. Windows used it
as in "C:\PATH\PROG[main, minor]".

By describing it, (vaguely) I leave it to the implementor how to
handle it exactly. When implementing for Unix-only this "requirement"
may get another priority the in a mainly Windows one.

This request some quirks to implement, no a stanbdard (I think)

- --ALbert

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Mon, 6 Oct 2003 10:24:24 +0200
From: Albert Mietus <[email-masked]>
Subject: Re: Issue 2 (TAG)

Rainer Gerhards writes:

 >      TAG          =  full-stat-id  [full-dyn-id] (':' / SP)
 >      full-stat-id =  [path] progname
 >      full-dyn-id  = '[' proc-id [thread-part] ']'
 >      path         =  path-part 1*(path-sep [path])
 >      path-part    = 1*VISUAL
 >      path-sep     = '/' / '\'
 >      progname     = 1*VISUAL
 >      proc-id      = 1*ALFANUM  ; recommended: number
 >      thread-part  = thread-sep thread-id
 >      thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
 >      thread-id    = 1*ALFANUM  ; recommended: number
 >      VISUAL       = ([a-zA-Z0-9...], excusing  '['
 >      SP           = %d32

Some remark (aside it look nice)

* See my other mail about the termniting colon.
* The terminating SP shouldn't be part of the tag. I think, it is a
  separator between fiels/parts (like in other parts of -sign.
* I agree on you other mail: proc/'/thread can be main/detail
* A path may start with a path-part (/like/this)

 > By updating it, some more points were raised (it is much like
 > implementing;)):

 > - I made full-dyn-id optional
Yes

 > - VISUAL must allow non US-ASCII characters
It shoud allow allo characters that are allowable in "syslog" :-)

 > - do we really need to describe the path?
Yes, Bit limmited. Only to describe how to find PROGNAME. Which is
frequently used in relays (all syslogd use it. The use PROGNAME, not
TAG!

 > - why not fix thread-sep to ","?

Allowing "many" characters, make it possible to use "native" notation
for several platforms. (I don't use thread frequently, so I made a
guess which chars to uses).

It is like allowing both '/' and '\' for directories. (Allowing ':'
for the Mac is complex, as the colon becomes used to often.

 > - MUST proc-id/thread-id by ALFANUM - would a SHOULD be sufficient (what
 > do we gain from a MUST)?

Probally, I'm not an expert on MUST and SHOULD notation, I must ( or
should:-) say.

 > - The ABNF is still not fully correct in the terminating SP of the
 > message... This can not be done only by specifying the tag. But I think
 > it is sufficiently to convey the idea.

See above,

Thansk for the help!

- -- 
ALbert Mietus
	Send prive mail to:      [email-masked]
	Send business mail to:  [email-masked]
	Don't send spam mail!

------------------------------

Date: Mon, 06 Oct 2003 18:36:27 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Re: Issue 2 (TAG)

On Mon, 2003-10-06 at 09:19, Albert Mietus wrote:
> Hello again:-)
>
>  > >    Receivers SHOULD, to be consistent with the format described in
>  > >    RFC3164, accept TAGs that terminate with a single colon, without a
>  > >    space following it. Then the colon is both the last character of
>  > >    that TAG, and the field separator with the next field (MSG).
>  >
>  > I think you must somehow revert back to my wording. Above you say that
>  > SP MUST terminate the tag. If you allow colon to terminate, too (I agree
>  > on this need), you must also allow it - otherwise it is
>  > confusing/inconsistent.
>
> With the syntax I tried to express how a TAG SHOULD be. Senders should
> use that, and receivers should expect it.

So we should probably move this into some MUST/SHOULD wording. I think
the ABNF already deals with this.
>
> However, the real world  sometime differs. Like existing
> syslog(d)'s. Which may (as rfc3164 describes) not use a SP to separat
> field and use ":" to terminate.
>
> The quoted part describes how a reciever shoul deal those "not
> standard" logmessages.
>
> Note: we can not acccept colon as a terminator. E.g. Windows used it
> as in "C:\PATH\PROG[main, minor]".

Mmmmhhh.. as of your earlier posting, I think this would be possible.
Think of this nice path name

    "C:\Program Files\GreatSyslogd[main, minor"

Do you see the space in it ;) Isn't life easy...

BTW: That was my point on backwards parsing - I think you must parse it
backwards so that the *last* occurrence of the character is acutally the
terminator. But space, as above, is really hard to cope with. It should
probably be forbidden.

>
> By describing it, (vaguely) I leave it to the implementor how to
> handle it exactly. When implementing for Unix-only this "requirement"
> may get another priority the in a mainly Windows one.

I see your point. By describing it, you place some burden on the
implementor, but the implementor at least is forced to think about it. I
fear, however, that we will see broken implementations as with the SP in
pathname above. But isn't that, too, an argument to not allow ":". After
all, when we disallow SP, we can disallow other characters, too  - the
implementor would need to parse the file name in any way. OK, he may
decide not to do this... But can we really care for this?

Rainer

------------------------------

Date: Mon, 06 Oct 2003 18:29:03 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Re: Issue 2 (TAG)

On Mon, 2003-10-06 at 10:24, Albert Mietus wrote:
> Rainer Gerhards writes:
>
>  >      TAG          =  full-stat-id  [full-dyn-id] (':' / SP)
>  >      full-stat-id =  [path] progname
>  >      full-dyn-id  = '[' proc-id [thread-part] ']'
>  >      path         =  path-part 1*(path-sep [path])
>  >      path-part    = 1*VISUAL
>  >      path-sep     = '/' / '\'
>  >      progname     = 1*VISUAL
>  >      proc-id      = 1*ALFANUM  ; recommended: number
>  >      thread-part  = thread-sep thread-id
>  >      thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
>  >      thread-id    = 1*ALFANUM  ; recommended: number
>  >      VISUAL       = ([a-zA-Z0-9...], excusing  '['
>  >      SP           = %d32
>
> Some remark (aside it look nice)
>
> * See my other mail about the termniting colon.
> * The terminating SP shouldn't be part of the tag. I think, it is a
>   separator between fiels/parts (like in other parts of -sign.

I agree. If you look at my original comments, I said that I just
included it as above because there is no way you can do this in ABNF
without specifying more than just the tag. I will try to update my
(partly not published yet) syslog ABNF to include your sample and then
post this.
> * I agree on you other mail: proc/'/thread can be main/detail
> * A path may start with a path-part (/like/this)
>
>  > By updating it, some more points were raised (it is much like
>  > implementing;)):
>
>  > - I made full-dyn-id optional
> Yes
>
>  > - VISUAL must allow non US-ASCII characters
> It shoud allow allo characters that are allowable in "syslog" :-)
In RFC3164 (as well as e.g. sysklogd real world package), they can be
non US-ASCII. So this would just save from further limitiation not
currently existiing.
>
>  > - do we really need to describe the path?
> Yes, Bit limmited. Only to describe how to find PROGNAME. Which is
> frequently used in relays (all syslogd use it. The use PROGNAME, not
> TAG!
OK, I think a language issue. My point was not if we need a path. It is
helpful. My point was if we really need to describe it in ABNF:
>  >      path         =  path-part 1*(path-sep [path])
>  >      path-part    = 1*VISUAL
>  >      path-sep     = '/' / '\'
>
wouldn't it be sufficient to say

        path            = 1*VISUAL

thus not thinking about path being separated by slashes?

>  > - why not fix thread-sep to ","?
>
> Allowing "many" characters, make it possible to use "native" notation
> for several platforms. (I don't use thread frequently, so I made a
> guess which chars to uses).
>
> It is like allowing both '/' and '\' for directories. (Allowing ':'
> for the Mac is complex, as the colon becomes used to often.

Ah, yes... I now agree.
>
>
>  > - MUST proc-id/thread-id by ALFANUM - would a SHOULD be sufficient (what
>  > do we gain from a MUST)?
>
> Probally, I'm not an expert on MUST and SHOULD notation, I must ( or
> should:-) say.

If I get it right, MUST is it really MUST - always, ever, no way around
it. SHOULD is a recommendation, but it leaves room to treat do it
differently if the implementation sees a need for it.
>
>  > - The ABNF is still not fully correct in the terminating SP of the
>  > message... This can not be done only by specifying the tag. But I think
>  > it is sufficiently to convey the idea.
That was the (somewhat hidden;)) point about SP not being correctly
represented ;)
>
> See above,
>
> Thansk for the help!
Thanks for all your help!

Rainer

------------------------------

Date: Thu, 9 Oct 2003 08:16:46 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 2:  TAG Field Definition - Update

Hi Folks,

I've updated the WG web page with a pointer to our Issues.
http://www.employees.org/~lonvick/index.shtml

Below is the latest proposal with pointers to the Archive.
(I believe that is the latest proposal from the discussions from Albert
and Rainer so please review it and make sure it's what we want.  I also
changed "of" to "or" in a sentence below so that it now reads, "after the
last '\' or '/' part of that segment.")

There is an issue of using SHOULD or MUST in some of the text.  Please
offer your suggestions on that as well.

Thanks,
Chris

- --------------------------------------------------------------------

Issue 2:  TAG Field Definition - Update

http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#HEADER

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01315.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01316.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01317.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01318.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01319.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01320.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01321.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01322.html

Albert has proposed the following text (with modifications by Rainer):
"""

   The TAG is a string of visible (printing) characters excluding SP,
   that MUST NOT exceed 64 characters in length.
   The first occurrence of a SP (space) will terminate the TAG field,
   but is not part of it.
   It is RECOMMENDED to terminate the TAG with a colon (':'), which if
   used, is part of the TAG.

   The TAG is used to denote the sender of the message. And is
   RECOMMENDED to have the following syntax:

     TAG          =  full-stat-id  [full-dyn-id] (':' / SP)
     full-stat-id =  [path] progname
     full-dyn-id  = '[' proc-id [thread-part] ']'
     path         =  path-part 1*(path-sep [path])
     path-part    = 1*VISUAL
     path-sep     = '/' / '\'
     progname     = 1*VISUAL
     proc-id      = 1*ALFANUM  ; recommended: number
     thread-part  = thread-sep thread-id
     thread-sep   = VISUAL     ; recommended: ",", or ':', or '.'
     thread-id    = 1*ALFANUM  ; recommended: number
     VISUAL       = ([a-zA-Z0-9...], excusing  '['
     SP           = %d32

   The PROGNAME part is special, as it is frequently used by relays to
   determine the routing of a message.  As a note to implementors: it
   can be found by getting the visual part before the first occurrence
   of '[', and after the last '\' or '/' part of that segment.

   An example of a TAG is: (without the quotes)
      "/path/to/PROGNAME[123,456]:"

   Systems that use both process-ID's and thead-IDs, SHOULD fill both
   the proc-id and the thread-part. For other systems it is
   RECOMMENDED to use the proc-id only.

   Receivers SHOULD, to be consistent with the format described in
   RFC3164, accept TAGs that terminate with a single colon, without a
   space following it. Then the colon is both the last character of
   that TAG, and the field separator with the next field (MSG).
"""

STATUS: This proposal has been received and is being considered.

Previously from the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01269.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01270.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01224.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01234.html
http://www.mail-archive.com/syslog-sec%40employees.org/msg01222.html

------------------------------

Date: Thu, 9 Oct 2003 08:25:01 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 10: Calculating Hashes and Signatures

Hi Everyone,

Albert has raised a new issue.  Please comment to the mailing list.

Thanks,
Chris

================================================================

Issue 10: Calculating Hashes and Signatures

Albert has raised some concerns about the clarity of the language used to
describe the way hashes and signatures are calculated.

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01314.html

Alberts' proposal:

"""
     Calculating Hashes and Signatures

     Before a Signature or Certificate Block can be send, some
     cryptographic calculations needs to be done. Elsewhere in this
     document is specified which algorithms need to be used, and
     where to place the result. This section specifies the data used
     as input for those calculations

     For each device-message (not for relayed messages), a hash SHOULD
     be calculated. It is REQUIRED to use the complete message
     including PRI, HEADER and MSG parts as input for the hashing.
     Those hashes are transited, later, in a Signature Block.

     Both, the Signature Block and the Certificate Block contain a
     digital signature. Those signatures SHOULD be calculated over the
     HASH of the partially composed message. It is REQUIRED to
     calculate the HASH of all parts and all fields of the composing
     message, but the signature-field.  Also, the separating space(s)
     direct before the signature-field NOT SHOULD be part of this
     calculation.  After calculating the HASH and the SIGNATURE, a
     space and the SIGNATURE should appemded to the message. It is
     RECOMMENDED to send this message directly, as the timestamp will
     age.
"""

------------------------------

Date: Thu, 9 Oct 2003 08:23:10 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 6:  Fragment Length - Update

Hi Folks,

We have resolved Issue 8 so I believe that we can resolve Issue 6 to be
"1-3".  Your comments will be welcome on this subject.

Thanks,
Chris

- ---------------------------------------------------------------------

Issue 6:  Fragment Length

http://www.employees.org/~lonvick/draft-ietf-syslog-sign-12.html#fraglen

From Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01229.html

Albert suggests changing the value from "1-4 characters" to "1-3"
since the payload of a syslog Certificate Block will be less than
999 characters -taking out the length of the PRI, TIMESTAMP and
HOSTNAME.  Anyone have any problems with this?

STATUS: Deferred until we resolve Issue 8

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01246.html

------------------------------

Date: Fri, 10 Oct 2003 11:26:59 +0200
From: Rainer Gerhards <[email-masked]>
Subject: Re: Issue 6:  Fragment Length - Update

On Thu, 2003-10-09 at 17:23, Chris Lonvick wrote:
> Hi Folks,
>
> We have resolved Issue 8 so I believe that we can resolve Issue 6 to be
> "1-3".  Your comments will be welcome on this subject.

Yes, at least this is my point of view. Make it "1-3".

Rainer

PS: sorry for the delay in the promised update to -sign and the ABNF.
Some high priority work got into the way and I didn't manage to push it
away...

------------------------------

Date: Mon, 13 Oct 2003 12:59:32 +0200
From: [email-masked]
Subject: XML for syslog? (off-topic)

Hello,

A small (off-topic) question,

Does anybody know wheter there is a XML description for syslog? Kind of
(defacto) standard one.

I don't mean the XML used in e.g syslog-relable (which adds fields).
But one that can be used to "store" rfc3164 messages, after spitting them in the
fields as decribed in the syslog standards. So, something like: (psuedo
notation)
   <syslog>
    <PRI><fac>8</fac><sev>5</sev></PRI>
    <HEADER>
         <timestamp> ....
         <hostname> ....
         <TAG progname="aProg">/PATH/To/AProg[1,2]</TAG>
    </HEADER>
   ....

As you see, I is easy to create one. But, it would be nice to have a standandard
one.

Thanks

ALbert

------------------------------

Date: Mon, 13 Oct 2003 15:42:50 +0200
From: Rainer Gerhards <[email-masked]>
Subject: RE: XML for syslog? (off-topic)

The closest thing to a standard XML description is in RFC 3195, in the
Cooked profile. But it looks like you don't like that. We have created
our own format for storage, as I think have others. I would try to stick
with the same names used in 3195 - don't make the mistake we made to use
others... It can become very confusing, even in your own codebase ;)

Chris, while I never thought about this, would this make up for a new
ID?

Rainer

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Monday, October 13, 2003 1:00 PM
> To: [email-masked]
> Subject: XML for syslog? (off-topic)
>
>
> Hello,
>
> A small (off-topic) question,
>
> Does anybody know wheter there is a XML description for
> syslog? Kind of
> (defacto) standard one.
>
> I don't mean the XML used in e.g syslog-relable (which adds fields).
> But one that can be used to "store" rfc3164 messages, after
> spitting them in the
> fields as decribed in the syslog standards. So, something
> like: (psuedo
> notation)
>    <syslog>
>     <PRI><fac>8</fac><sev>5</sev></PRI>
>     <HEADER>
>          <timestamp> ....
>          <hostname> ....
>          <TAG progname="aProg">/PATH/To/AProg[1,2]</TAG>
>     </HEADER>
>    ....
>
> As you see, I is easy to create one. But, it would be nice to
> have a standandard
> one.
>
> Thanks
>
>
> ALbert
>
>
>
>
>
>

------------------------------

Date: Mon, 13 Oct 2003 08:28:25 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 11: Non-US-ASCII Characters in MSG

Hi Folks,

Rainer pointed out that we overlooked an Issue that he previously brought
up.  I've updated the list of Issues to "C" here:
     http://www.employees.org/~lonvick/issues-c.txt
and included this new issue.  Please comment.

===

Issue 11: Non-US-ASCII Characters in MSG

Rainer has pointed out the RFC 3164 states that non-US-ASCII characters
have been seen in syslog messages but that is inconsistent with the
current draft of syslog-sign.

From the Archive:
http://www.mail-archive.com/syslog-sec%40employees.org/msg01258.html

Rainer has looked at a lot of existing code and found that many
implementations will accept non-US-ASCII characters so we should modify
syslog-sign to agree with that fact.  Are there any objections to this?
===

Thanks,
Chris

------------------------------

Date: Sat, 11 Oct 2003 08:08:39 -0400 (EDT)
From: [email-masked]
Subject: NYTimes.com Article: I.B.M. and Cisco Plan Venture to Develop Software Standards

This article from NYTimes.com
has been sent to you by [email-masked].

FYI - this is yet another structured log data recommendation;  see http://xml.coverpages.org/IBMCommonBaseEventV111.pdf

[email-masked]

/-------------------- advertisement -----------------------\

Explore more of Starbucks at Starbucks.com.
http://www.starbucks.com/default.asp?ci=1015
\----------------------------------------------------------/

I.B.M. and Cisco Plan Venture to Develop Software Standards

October 10, 2003
 By STEVE LOHR

I.B.M. and Cisco are expected to announce that they will
jointly develop and promote open software standards
intended to simplify the increasingly complex task of
managing corporate data networks.

http://www.nytimes.com/2003/10/10/technology/10blue.html?ex=1066874119&ei=1&en=aad78d54c3822525

- ---------------------------------

Get Home Delivery of The New York Times Newspaper. Imagine
reading The New York Times any time & anywhere you like!
Leisurely catch up on events & expand your horizons. Enjoy
now for 50% off Home Delivery! Click here:

http://www.nytimes.com/ads/nytcirc/index.html

HOW TO ADVERTISE
- ---------------------------------
For information on advertising in e-mail newsletters
or other creative advertising opportunities with The
New York Times on the Web, please contact
[email-masked] or visit our online media
kit at http://www.nytimes.com/adinfo

For general information about NYTimes.com, write to
[email-masked].

Copyright 2003 The New York Times Company

------------------------------

Date: Mon, 13 Oct 2003 09:37:35 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: RE: XML for syslog? (off-topic)

Hi Rainer,

On Mon, 13 Oct 2003, Rainer Gerhards wrote:

> The closest thing to a standard XML description is in RFC 3195, in the
> Cooked profile. But it looks like you don't like that. We have created
> our own format for storage, as I think have others. I would try to stick
> with the same names used in 3195 - don't make the mistake we made to use
> others... It can become very confusing, even in your own codebase ;)
>
> Chris, while I never thought about this, would this make up for a new
> ID?
>

I had a side-conversation with someone many months ago about this.  I
asked him to bring it up on the list as he actually had a proposal.  (It
looked good to me anyway. :-)  Let me bring it up with that person again
and we'll see where it can go.

Thanks,
Chris

> Rainer
>
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]]
> > Sent: Monday, October 13, 2003 1:00 PM
> > To: [email-masked]
> > Subject: XML for syslog? (off-topic)
> >
> >
> > Hello,
> >
> > A small (off-topic) question,
> >
> > Does anybody know wheter there is a XML description for
> > syslog? Kind of
> > (defacto) standard one.
> >
> > I don't mean the XML used in e.g syslog-relable (which adds fields).
> > But one that can be used to "store" rfc3164 messages, after
> > spitting them in the
> > fields as decribed in the syslog standards. So, something
> > like: (psuedo
> > notation)
> >    <syslog>
> >     <PRI><fac>8</fac><sev>5</sev></PRI>
> >     <HEADER>
> >          <timestamp> ....
> >          <hostname> ....
> >          <TAG progname="aProg">/PATH/To/AProg[1,2]</TAG>
> >     </HEADER>
> >    ....
> >
> > As you see, I is easy to create one. But, it would be nice to
> > have a standandard
> > one.
> >
> > Thanks
> >
> >
> > ALbert
> >
> >
> >
> >
> >
> >
>
>
>

------------------------------

Date: Tue, 21 Oct 2003 11:16:21 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 11 - Non-US-ASCII Characters in MSG - RESOLVED

Hi Folks,

Just a reminder about our open issues in draft-ietf-syslog-sign-12.txt
The issues are here:
  http://www.employees.org/~lonvick/issues-c.txt

Hearing no objections to Issue 11, I'm going to call it RESOLVED and ask
that the ID be changed to state that printable characters may be used in
syslog messages.

Thanks,
Chris

------------------------------

Date: Tue, 21 Oct 2003 12:06:08 -0700 (PDT)
From: Chris Lonvick <[email-masked]>
Subject: Issue 6 - Fragment Length - RESOLVED

Hi,

No one opposed having the Fragment Length as 1-3 characters.  Just so
everyone knows, that will mean that Certificate Block message lengths may
be a maximum of 999 bytes.  That forces the issue of Certificate Blocks
being within the 1024 byte limit.

Thanks,
Chris

------------------------------

Date: Tue, 28 Oct 2003 09:39:31 -0500
From: [email-masked]
Subject: I-D ACTION:draft-ietf-syslog-sign-13.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Security Issues in Network Event Logging Working Group of the IETF.

	Title		: Syslog-Sign Protocol
	Author(s)	: J. Kelsey, J. Callas
	Filename	: draft-ietf-syslog-sign-13.txt
	Pages		: 39
	Date		: 2003-10-27

This document describes syslog-sign, a mechanism adding origin
authentication, message integrity, replay-resistance, message
sequencing, and detection of missing messages to syslog. Syslog-sign
provides these security features in a way that has minimal
requirements and minimal impact on existing syslog implementations.
It is possible to support syslog-sign and gain some of its security
attributes by only changing the behavior of the devices generating
syslog messages. Some additional processing of the received syslog
messages and the syslog-sign messages on the relays and collectors
may realize additional security benefits.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-syslog-sign-13.txt

To remove yourself from the IETF Announcement list, send a message to
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-syslog-sign-13.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-syslog-sign-13.txt".

NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

------------------------------

Date: Thu, 30 Oct 2003 06:59:29 -0800 (PST)
From: Chris Lonvick <[email-masked]>
Subject: Web Page Updated

Hi Folks,

I've updated our web page here:
  http://www.employees.org/~lonvick/index.shtml

Syslog-sign has been updated to -13 reflecting the closed issues that
we've been discussing.  We have a modified list of open issues here:
  http://www.employees.org/~lonvick/issues-13-a.txt

Let's get these issues knocked down.  :-)  Please comment upon them to the
WG list so we can resolve the open ones.

There hasn't been much discussion of Glenn's mib document.  Would the
interested people please comment on that so we can keep this moving.
  http://www.ietf.org/internet-drafts/draft-ietf-syslog-device-mib-04.txt

Thanks,
Chris

------------------------------

Date: Thu, 30 Oct 2003 16:49:21 +0100
From: Rainer Gerhards <[email-masked]>
Subject: LF at end of syslog message

Hi list,

I have observed that at least PIX as well as the popular sysklogd package
under Linux place a LF at the end (last char) of the syslog message. I am
not sure about other syslogd implementations.

Anyhow, I wonder if we should allow LF and (enventually CR in front of it)
as the last character of the MSG part (or even after it). This question
applies to all currently discussed IDs. I propose we allow it, because that
will help a newer receivers to treat messages from those popular emitors as
well-formed. I don't see where having it optionally in would hurt.

Rainer

------------------------------

Date: Fri, 31 Oct 2003 12:26:48 +0100
From: Rainer Gerhards <[email-masked]>
Subject: -sign, Issue 13-2 (TAG)

Hi folks,

sorry, a long mail and this on a friday...

I have thought quite a while over the new specification for the TAG. In
the new wording, the "path" is explicitly defined. Let's see an excerpt:

###

		TAG          =  full-stat-id  [full-dyn-id] (':' / SP)
		full-stat-id =  [path] progname
		path         =  path-part 1*(path-sep [path])
		path-part    = 1*VISUAL
		path-sep     = '/' / '\'
		progname     = 1*VISUAL
		proc-id      = 1*ALFANUM  ; recommended: number
		VISUAL       = ([a-zA-Z0-9...], excusing  '['
		SP           = %d32
###

An example from postfix was given. I have just taken a new example from
my logs, because I could verify this better:

###
Oct 31 10:01:51 ipx10102 postfix/smtpd[19782]: disconnect from
ASte-Genev-Bois-113-1-1-241.w81-50.abo.wanadoo.fr[81.50.79.241]
###

if you look at it, the path is just a partial path. Postfix is installed
at /usr/libexec/postfix and the file run actually is
/usr/libexec/postfix/smtpd.

Also, of course, the TAG vlaue could also have been an additional
designation and not an actual path.

Besides that, there is another fundamental issue with specifying the
path - that is, we assume that the OS provides pathes as specified. In
the ABNF, we already cover *nix vs. DOS pathes ('/' vs '\'). But there
are many more path representations in other OS's. I dug I bit in my
memories and can see there are other formats:

For example on VMS (ok, a bit outdated nowadays...), this is a valid
path:

DKA0:[MYDIR.SUBDIR1.SUBDIR2]MYFILE.TXT;1

I found a description of the nameing system at
http://www.djesys.com/vms/freevms/mentor/vms_path.html (I have not
visited any other page on that site and don't know what it is all about
- - so be warned if you follow the link).

Also, on Unisys OS 1100 machines, a valid path looks like this:

sys$*data$.co$install

I have found no web reference to this system, so a quick intro: On OS
1100 (to the best of my knowledge still in widespread use), a file has
the format of qualifier*name. Then, inside a file you have an so-called
element, which is specified after a period. I guess it is nowadays
possible to include *nix like pathes inside the element names.

If we look at IBM's VMS, VSE, CMS and MVS we find more different path
notations...

Of course, under DOS/Windows it looks like "c:\bin\someprogram.exe".
Notice the colon. I have heard (but don't have experience) that on MAC,
colons also occur frequently in pathes.

In short: I don't think it is a good idea to specify how a path should
look like.

However, one essence remains: there should be a part of the tag that is
more or less STATIC and one part that is DYNAMIC. The static part
denotes the application emiting messages, the dynamic part a specific
instance of it.

I think we should settle with this. As such, I propose the following
stripped-down ABNF:

###
   ; The following line is NOT part of the TAG ABNF, but it is needed
   ; to specify the optional SP after the tag. It is taken from my
   ; full message ABNF. The other parts (TIMESTAMP...) are NOT defined
   ; below.
   HEADER          = TIMESTAMP SP HOSTNAME SP TAG [SP]

   TAG             = static-id  [full-dyn-id] [':'] ; 64 chars max
   static-id       = 1*VISUAL
   full-dyn-id     = '[' proc-id [thread-sep thread-id] ']'
   proc-id         = 1*ALFANUM  ; recommended: number
   thread-sep      = VISUAL / %d58     ; recommended: ",", or ':', or
'.'
   thread-id       = 1*ALFANUM  ; recommended: number
   VISUAL          = (%d33-57/%d59-126) ; all but SP and ":"

   LF              = %d10
   CR              = %d13
   SP              = %d32
   PRINTUSASCII    = %d33-126

   The TAG is a string of visible (printing) characters excluding SP,
   that MUST NOT exceed 64 characters in length. The first occurrence of
   a SP (space) will terminate the TAG field, but is not part of it. It
   is RECOMMENDED to terminate the TAG with a colon (':'), which if
   used, is part of the TAG.

   The TAG is used to denote the sender of the message. It MUST be in
   the syntax shown in the ABNF above.

   A typical example of a TAG is: (without the quotes)

   "/path/to/PROGNAME[123,456]:"

   Another example (from VMS) is: (without the quotes)

   "DKA0:[MYDIR.SUBDIR1.SUBDIR2]MYFILE.TXT;1[123,456]".

   Please note that in this example,
   "DKA0:[MYDIR.SUBDIR1.SUBDIR2]MYFILE.TXT;1" is the static-id while
   "[123,456]" is still the full-dyn-id. This shows that a receiver must
   be prepared for special characters like '[' to be present inside the
   static part.

   As a note to implementors: the begining of the full-dyn-id is not the
   first but the LAST occurence of '[' inside the tag and this ONLY if
   the tag ends in either "]" or "]:". If these conditions are not met,
   the '[' is part of the static-id.

   Systems that use both process-ID's and thead-IDs, SHOULD fill both
   the proc-id and the thread-part. For other systems it is RECOMMENDED
   to use the proc-id only.

   Receivers SHOULD, to be consistent with the format described in
   RFC3164, accept TAGs that terminate with a single colon, without a
   space following it. Then the colon is both the last character of that
   TAG, and the field separator with the next field (MSG).

   No specific format inside the tag is required. However, an emitor
   SHOULD use a consistent tag value.
###

This ABNF still provides the essentials and allows for pathes of all
kinds. The postfix sample would fit in neatly.

A similar issue is with the full-dyn-id part. Can we really assume that
a thread/process ID *always* fits into the above ABNF? I think it is
much more likely, but I am anyhow a bit concerned. I am of the general
position that one should not limit itself in ones options. Specifying
the dyn part as above would eventually create some issues with some
(strange) environments. On the other hands, I consider this to be very
unlikely and more or less a theoretical point - thus I left it in the
ABNF as above. I would appreciate comments especially on this. What does
the rest of the WG think?

There is one more issue with the thread-sep as specified in the ABNF
above. If we say it is VISUAL, parsing is not well defined. Let's look
at this fully-dyn-id sample:

[123]

Most obvious, the intension of the ABNF is that it should be parsed as
follows:
proc-id = 123; thread-sep = empty; thread-id = empty

However, I think I could also interpreted it as follows:
proc-id = 1; thread-sep = 2; thread-id = 3

This is, because thread-sep is VISUAL. If we collapse the ABNF a little,
that full-dyn-ip is effectively specified as

'[' 1*ALFANUM VISUAL 1*ALFANUM ']'

But ALFANUM is effectively a subset of VISUAL, so there is no way to
tell where the ALFANUM ends and the VISUAL begins in cases like above. I
think this needs to be clarified and changed in the ABNF.

I propose that we change the ABNF to replace thread-sep with this
definition:

   thread-sep      = ','|'.'

As there is no legacy to support, I think this won't hurt. Or is there
legacy to support (especially legacy that would not fit into this
definition)?

And one final comment about colons: we have several places in the ABNF
above where we allow colons (path name, thread-sep). Especially in path
names (DOS, MAC, VMS), colons seem to be used often. So if we really
intend to include path names inside the tag (which sounds like a good
idea), we probably need to drop the legacy compatibility rule that a
colon NOT followed by a SP will terminate the tag. Look at this:

"C:\bin\mwagent.exe[1234]:"

This can properly be parsed if we demand a SP after the tag. So we know
the first colon is part of the path (because it is not followed by SP)
while the last (followed by a SP not shown) is not.

With the current wording, that tag would just be parsed as "C:" and the
rest goes into the MSG part. I don't see any way to avoid this, instead
by making the SP after TAG a MUST.

So I think we need to make a tradeoff decision: we can either

a) allow colons in the path name
xor ;)
b) allow TAG NOT to be terminated by SP

Selecting
a) will break compliance for older clients (how many?)
b) will open up a can of (security?) bugs as I guess there are
   well enough implementors out there not caring about the
   restricted char

Both choices are not really good... I personally have a slight tendency
towards a), as I *feel* that the number of affected older senders is
limited (but I may be totally wrong). I also think it is "cleaner" from
an overall architectural point of view to separate ALL fields by SP and
don't make an exception for a single one. An argument for this is,
again, this can be the source for some program bugs, eventually even
security related ones (missed length check and a long message without
spaces immediately following the colon).

The above ABNF and wording is inconsistent in regard to the
"colon-issue".

Well, I think that's it for now. Looks like it gets ugly the more you
dig into detail...

Rainer

------------------------------