Subject: RE: [Syslog] Re: DISCUSS in draft-ietf-syslog-protocol - congenstioncontrol (fwd)

From: "Rainer Gerhards" <[email-masked]>
Date: Mon, 13 Aug 2007 17:25:34 +0200

Hi folks,

I am sorry for the long silence. I was extremely busy with projects and
have to admit will be more or less unavailable for the next 3 weeks.
Beginning of September things will finally clear up and I hope to be
able to publish a new, hopefully final, version of -protocol.

Even though I was not directly active in regard to -protocol, I thought
about this requirement here. I have a somewhat weak text to propose:

---
In any implementation, a situation can arise in which an originator or
relay would need to block sending messages. A common case is when an
internal queue is full. This might happen due to rate-limiting or slow
performance of the syslog application. In such cases, it is RECOMMENDED
that the originator or relay drops messages of lower severity in favor
of higher severity messages. Messages with a numerically lower SEVERITY
value have a higher severity than those with a numerically higher one.
To-be-dropped messages SHOULD simply be discarded. The syslog
application may notify a collector or relay about the fact that it drops
messages. If the underlying protocol supports congestion control,
discarding of messages SHOULD be avoided.
---

I have blogged about why I think this is the right thing to do, you may
view the details of my thoughts here (sorry, I'd like not to duplicate
text as I am already very short on time):

http://rgerhards.blogspot.com/2007/08/on-reliability-and-need-to-discard
_13.html

I would appreciate comments and, even better, improvements to the
proposed text. I may not be able to reply immediately, but I will
definitely appreciate any comment and incorporate it.

Sorry once again for the long silence.

Rainer

> -----Original Message-----
> From: Anton Okmyanskiy (aokmians) [mailto:[email-masked]]
> Sent: Thursday, July 12, 2007 7:32 PM
> To: Chris Lonvick (clonvick); [email-masked]
> Subject: RE: [Syslog] Re: DISCUSS in draft-ietf-syslog-protocol -
> congenstioncontrol (fwd)
> 
> Chris:
> 
> Can we ask Mangus to provide suggested text? He mentioned it is just a
> paragraph. This would make it a bit easier to get to the point of
> what/how he wants addressed and evaluate if we agree with it. If his
> suggested text is not too demanding on implementations, but rather a
> recommendation, it is easier to accept it.
> 
> Is he now concerned with syslog reliability (dropping of arbitrary
> messages due to congestion and queue overfill) instead of previously
> raised concern of syslog being a bad citizen and causing congestion
for
> others?
> 
> For end-to-end reliability, we have a whole section describing many
> aspects we did not intend to address in UDP draft.
> 
> Why is there an assumption of blocking application?  Maybe my syslog
> application writes messages to file first, returns to client app and
> then asynchronously transmits them while listening for ICMP errors. I
> also don't think we intended to cover any end-to-end guarantees from
> application perspective.  Even reliable network transport (TCP) does
> not
> means reliable application-to-application delivery.  We had the
> app-level-ack discussion and dismissed it.
> 
> So, yes, messages are not guaranteed to be delivered to their final
> destination and processed there.  They can be dropped, they can get
> corrupted, receiver may crash right after getting message but before
> processing it, relay may fail, etc.
> 
> Thanks,
> Anton.
> 
> > -----Original Message-----
> > From: Chris Lonvick (clonvick)
> > Sent: Wednesday, July 11, 2007 7:16 AM
> > To: [email-masked]
> > Subject: [Syslog] Re: DISCUSS in draft-ietf-syslog-protocol -
> > congenstion control (fwd)
> >
> > Hi Folks,
> >
> > Here is clarification of what Magnus wants.  We have so far
> > received Eliot's proposal but I don't think that addresses
> > the concern.
> >
> > I would like to hear from everyone.  Do we want to push back
> > on Magnus, or can someone propose some text around this?
> >
> > Thanks,
> > Chris
> >
> >
> > ---------- Forwarded message ----------
> > Date: Fri, 06 Jul 2007 18:08:12 +0200
> > From: Magnus Westerlund <[email-masked]>
> > To: David Harrington <[email-masked]>
> > Cc: Chris Lonvick <[email-masked]>, Lars Eggert
> > <[email-masked]>
> > Subject: Re: DISCUSS in draft-ietf-syslog-protocol -
> > congenstion control (fwd)
> >
> > Hi David,
> >
> > I think you are missunderstanding things here. But thanks for
> > protesting  and not accepting crap. But let me make clear
> > what I actually think is needed in regards to syslog to make
> > it a working protocol to deploy.
> >
> > First, this starts as an issue with TLS over TCP and the
> > syslog base protocol.
> > It can also arise teorethically for UDP, but as I understand
> > not in practice for todays usage. When you are using TCP, in
> > case the syslog sender generates events at an rate that is
> > higher than available rate over the path used there will be
> > build up of a queue. So I would like to have some words
> > somewhere saying what you do when you build up a queue of
> > messages that should be transmitted, but the queue simply
> > keeps growing. What do I do? To me this situation implies
> > that one starts discarding syslog messages and starts with
> > the least important ones. So I would like to have a paragraph on
> this.
> >
> > I also included UDP in this in the case that you actually
> > have reserved or determined that syslog is allowed to use a
> > particular amount of bandwidth, but not more. In this case it
> > could be possible that one implements a rate limiter and run
> > into exactly the same issue as for TCP.
> >
> > Please do understand that if syslog was designed from scratch
> > today it wouldn't get awy without a congestion control that
> > guarantees that it isn't missbehaving in any situation. But
> > being an "old" protocol we are accepting less than that.
> > However, we do require it to contain the limitations and
> > assumptions that it can be safely operated with. Using it as
> > it currently is used is not an issue because the networks it
> > is used in has many magnitudes more bandwidth that syslog
> > generates. However, what would happen if some one starts
> > using syslog in low-powered, low-bitrate sensor network for
> > carrying some information. In that situation syslog becomes a
> > potential threat to the stability of the network as it
> > doesn't react at all to congestion if run over UDP. Network
> > failures are also sitation that are problematic to ensure
> > that the inteded resources are available. Thus we do like to
> > protect the utility of what resources do exist.
> >
> > So the repeat:
> >
> > Please seriously consider adding a paragraph about how one
> > can thinn a queue of syslog messages in the base protocol.
> > This as I think it potentially applies to any transport.
> >
> > Also consider when updating the UDP draft and adds what has
> > been discussed with Lars, to add a single sentece with a
> > reference the above paragraph as a method to keep the traffic
> > within the allowed resources.
> >
> > Regards
> >
> > Magnus
> >
> >
> >
> > David Harrington skrev:
> > >
> > >
> > >  Hi Magnus,
> > >
> > >  Syslog is a fire-and-forget protocol. We have no feedback
> > mechanism
> > > that permits us to recognize congestion and rate limit
> > traffic based
> > > on that feedback.
> > >
> > >  Syslog is not a brand new protocol; it is widely used in the
> > > industry,  and other aspects of standardization has been handled
> > > through POSIX  and BSD standardization. The industry has
> > expressed no
> > > interest in  making this a two-way protocol. This protocol
> > is widely
> > > used by  operators, amd I have seen no demand from
> > operators to make
> > > this a  two-way protocol, or any demand to resolve
> > congestion control
> > > for  syslog, because congestion caused by syslog apparently
> > is not a
> > > problem in the real world.
> >
> > >
> > >  We had discussion of rate-limiting during the IETF Last Call. We
> > > actually had suggestions for ways to rate-limit syslog in
> > the earlier
> > > document, but the IETF community rejected our having that
> > text in the
> > > document because syslog is a fire-and-forget protocol, so
> > there is no
> > > reliable mechanism for detecting congestion. The IETF
> > commmunity did
> > > not ask us to make syslog two-way, so we could add rate
> > limiting to
> > > the protocol; they asked us to keep syslog working as is,
> > and remove
> > > the unreliable recommendations to rate limit.
> > >
> > >  You are placing a whole new requirement, that no implementers or
> > > operators are asking for, on a protocol that is already widely
> > > implemented and deployed. Where is the customer demand for this?
> > >
> > >  I am concerned you might drive the syslog community to not work
> > > within  the IETF, where they came to develop a standard
> > message format
> > > and a  secure transport, not to change the basic nature of
> > the protocol.
> > >
> > >  David Harrington
> > >  [email-masked]
> > >  [email-masked]
> > >  [email-masked]
> > >
> >
> > --
> >
> > Magnus Westerlund
> >
> > IETF Transport Area Director & TSVWG Chair
> >
---------------------------------------------------------------------
> -
> > Multimedia Technologies, Ericsson Research EAB/TVM/M
> >
---------------------------------------------------------------------
> -
> > Ericsson AB                | Phone +46 8 4048287
> > Torshamsgatan 23           | Fax   +46 8 7575550
> > S-164 80 Stockholm, Sweden | mailto: [email-masked]
> >
---------------------------------------------------------------------
> -
> >
> > _______________________________________________
> > Syslog mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/syslog
> >
> 
> _______________________________________________
> Syslog mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/syslog

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog