Subject: RE: [Simple] RE: Question regarding NOTIFY messages using UDP.

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 21 Nov 2001 01:58:27 -0500

I agree with Adam that defining a SIP specific fragmentation is a bad thing.
We have discussed this on the SIP list and it has been quickly squashed.

For simple, a real delta versioning solution is the best thing.

Another solution is TCP, which is generally better as far as firewall/nat
traversal is concerned, not worse, and has none of these fragmentation
issues.

-Jonathan R.

---
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, November 20, 2001 6:50 PM
> To: 'Brian Stucker'; simple
> Cc: Alex Nava; Patrick Sollee; Sanjoy Sen
> Subject: [Simple] RE: Question regarding NOTIFY messages using UDP.
> 
> 
> >From: Brian Stucker [mailto:[email-masked]]
> >
> >Ok, you've really got me confused now. I even went off to 
> RFC-0760 (IP) and
> >RFC-0768 (UDP) to make sure that I wasn't off in the weeds. 
> Even bis-05 SIP
> talks
> >about MTU problems.
> 
> It makes a recommendation. If fragmentation were really a 
> problem, SIP would
> mandate behavior. As it is, it's just trying to raise 
> awareness of some of
> the related issues (like the two I discuss in a previous mail).
> 
> >I don't think I'm off in the weeds, but I see what you're 
> referring to.
> >I now see what you're getting at with the 64k size (maximum 
> datagram size),
> but
> >since IP makes no guarantee that any particular packet reaches it's
> destination, it's
> >really IP that is the problem. I see what you're getting at 
> with the UDP
> packet
> >being fragmented to the IP next-hop MTU size, but that means 
> that portions of
> the UDP
> >packet may go missing and unknown since IP doesn't do 
> anything to ensure that
> any
> >particular datagram makes it to the destination.
> 
> That's a problem even when you don't exceed the MTU. That's why
> SIP retransmits requests.
> 
> There's a timer for reconstitution of the packet, sequence numbers,
> a checksum, and all sorts of goodies in the UDP fragmentation to make
> sure that it works. It's unreliable (but in an all-or-nothing sort
> of way, just like all UDP datagram transmissions) but it works. All
> you need is a mechanism to retransmit datagrams until they are
> acknowledged, and SIP provides such a mechanism.
> 
> >What's worse, is popular operating systems such as Microsoft Windows
> >(with the possible exception of XP) specifically drops any datagram
> >exceeding the MTU value until it receives an ARP response 
> for the first
> >datagram (thus guaranteeing it to drop all of the portions of the UDP
> >packet, except for the very last one:
> >http://support.microsoft.com/support/kb/articles/q233/4/01.asp).
> 
> I have two reactions to this:
> 
> 1. Are you *seriously* proposing that we add provisions in the
>    protocol to work around Microsoft's buggy IP implementation?
> 
> 2. Even with this special little Microsoft bug, it all works.
>    The first transmission of a UDP packet to a host might get
>    lost; however, this first transmission will load the ARP cache
>    for the destination, and the subsequent retransmissions will
>    progress as normal.
> 
> I'm not arguing against your solution. I'm pointing out that
> the "problem" you are trying to solve is not a problem. It might
> be an inconvenience in corner cases, but it's not a problem.
> 
> /a
> 
> _______________________________________________
> simple mailing list
> [email-masked]
> http://mailman.dynamicsoft.com/mailman/listinfo/simple
>

---

From: Michael Hammer <[email-masked]>
Date: Wed, 21 Nov 2001 10:10:46 -0500

Jonathan,

While not favoring any fragmentation mechanism for SIP, does this preclude 
any implementation (higher level application using SIP) from choosing to 
send the body of the Notifies as incremental parts?  It would seem that the 
SIP-level mechanisms would be unaware of the full nature of the body which 
it carries and should place no restrictions on that body.

That is, it provides no mechanism nor precludes use of such a mechanism.

Mike

---

From: "Brian Stucker" <[email-masked]>
Date: Mon, 26 Nov 2001 02:15:02 -0600

That's what I was afraid of. 

TCP is incredibly expensive to use for presence notification, in my mind,
because the connection has to be baby-sitted for extremely long periods of
time to keep any potential pinholes open in case a notify needs to be sent.
That's bad. I know there are tons of web servers out there that use TCP all
the time, with no problem, but they don't have to keep that connection up
for very long. A flash in the pan compared to what presence would do.

Hmmm....

Brian

-----Original Message-----
From: Christian Huitema [mailto:[email-masked]]
Sent: Friday, November 23, 2001 11:38 PM
To: Michael Hammer; Jonathan Rosenberg
Cc: [email-masked]; Stucker, Brian [NGB:B635:EXCH]; simple;
Nava, Alex [NGC:B634:EXCH]; Sollee, Patrick [NGC:B680:EXCH]; Sen, Sanjoy
[NGB:B692:EXCH]
Subject: RE: [Simple] RE: Question regarding NOTIFY messages using UDP.

I hope you are all aware that sending long messages over UDP is a
*really bad idea*. The failure mode is the following: 
 1) long message gets fragmented in a set of path-MTU sized IP messages;

 2) at least one message in the set is dropped in the network;
 3) application times out and resend long message;
 4) repeat step one.
The theory says that transmissions of independent messages should result
in at least one transmission of the whole set eventually, but the
independent error assumption is false in practice. There are error
situations in which every fourth or fifth message gets dropped, in which
case no amount of repetition solves the problem. This is not
theoretical: I have seen the problem occur for example in IKE, with UDP
messages containing long certificate chains. 

My rule of thumb is that trying to transmit UDP messages longer than 4K
is guaranteed to fail spectacularly in at least some circumstances, and
that only messages shorter than the IPv6 guaranteed minimal MTU are safe
-- that is, about 1K in the payload.

The practical options are, only use short presence documents, or use
TCP.

-- Christian Huitema

> From: Michael Hammer [mailto:[email-masked]]
> Sent: Wednesday, November 21, 2001 7:11 AM
> To: Jonathan Rosenberg
> Cc: [email-masked]'; 'Brian Stucker'; simple; Alex Nava;
Patrick
> Sollee; Sanjoy Sen
> Subject: RE: [Simple] RE: Question regarding NOTIFY messages using
UDP.
> send the body of the Notifies as incremental parts?  It would seem
that
> the

---

From: "Brian Stucker" <[email-masked]>
Date: Mon, 26 Nov 2001 03:04:20 -0600

Ok, here's a thought...

If we're in a position where we think we're going to need to use TCP (or
some other connection-oriented protocol) to notify the watcher (in this
example, a presence subscriber), but it's difficult to keep up a bunch of
TCP connections, we use the following general mechanism.

1. We send the watcher a NOTIFY that says in effect "you need to fetch the
current state of this resource" (in this case, probably using UDP).
2. The watcher responds to the NOTIFY with a 200 OK however it chooses.
3. The watcher sends a SUBSCRIBE in to fetch the current state of the
resource (in this case a presentity) using the transport of it's choosing.
If it knows it's behind a firewall, or a NAT, then it can pick TCP, for
instance.
4. The presence agent responds to the SUBSCRIBE with a 200 OK (because there
shouldn't be any issues about authorization or not).
5. The presence agent sends a NOTIFY to the watcher using the same
transport, IP, port, everything, that the fetch SUBSCRIBE was received with.

Simple as that. We trigger a TCP connection inbound from the client using a
UDP message. NAT or not, it saves on having to nail up a bunch of TCP
connections for a long period of time, and it's super easy to implement.

Events draft change:
All we'd need to change is to either add a new reason to the
subscription-expires header, like "stale" with a non-zero value, and add a
restriction that the state agent MUST respond to a fetch using the same
transport parameters used by the watcher from the SUBSCRIBE.

-OR-

Events draft/watcherinfo change:
We add a new event to the watcherinfo format called "stale" to the current
list (subscribe, rejected, approved, ...), and recommend that when a watcher
to eventpkg.winfo receives this event, they should perform a fetch. Then we
add the change to the events draft that when a fetch is performed, the
source parameters from the SUBSCRIBE should be used to send the NOTIFY.

Thoughts?

Brian Stucker

-----Original Message-----
From: Stucker, Brian [NGB:B635:EXCH] 
Sent: Monday, November 26, 2001 2:15 AM
To: Christian Huitema; Michael Hammer; Jonathan Rosenberg
Cc: [email-masked]; simple; Nava, Alex [NGC:B634:EXCH]; Sollee,
Patrick [NGC:B680:EXCH]; Sen, Sanjoy [NGB:B692:EXCH]
Subject: RE: [Simple] RE: Question regarding NOTIFY messages using UDP.

-----Original Message----- 
From: Christian Huitema [mailto:[email-masked]] 
Sent: Friday, November 23, 2001 11:38 PM 
To: Michael Hammer; Jonathan Rosenberg 
Cc: [email-masked]; Stucker, Brian [NGB:B635:EXCH]; simple; 

Patrick 
UDP. 
that

---

From: "Christian Huitema" <[email-masked]>
Date: Fri, 23 Nov 2001 21:38:25 -0800

-- Christian Huitema

Patrick
UDP.
that

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 26 Nov 2001 15:03:08 -0500

> Sent: Wednesday, November 21, 2001 10:11 AM
> To: Jonathan Rosenberg
> Patrick Sollee; Sanjoy Sen

The spec assumes full-state, and talks about using CSeq to determine the
most recent document, which wouldn't make sense for partial documents as you
have described.

An extension could in principle override this, and you could define a
document format that supports partial information.

However, I still assert that:

1. this is a problem in theory only
2. if the docs do become large, delta encoding is better

Honestly, the problem is more for other event packages that may convey
larger amounts of state. 

> If we're in a position where we think we're going to need to use TCP (or
some other 
> connection-oriented protocol) to notify the watcher (in this example, a
presence 
> subscriber), but it's difficult to keep up a bunch of TCP connections, we
use the 
> following general mechanism.
>
current state 
> of this resource" (in this case, probably using UDP).
>
resource (in this 
> case a presentity) using the transport of it's choosing. If it knows it's
behind a 
> firewall, or a NAT, then it can pick TCP, for instance.

Actually, we've been toying with the idea of not even using SUBSCRIBE for
this. An HTTP request is just fine for synchronously fetching a document;
thats what its designed to do.

One could think of this as another sub-package, say "alert". If you
subscribe to foo.alert, you get a NOTIFY when foo changes, but the NOTIFY
contains a body type that is always short - providing a URL for actually
retrieving the thing that has changed. In fact, the default could be as
simple as application/uri-list. So:

SUBSCRIBE sip:[email-masked] SIP/2.0
Event: presence.alert
Accept: application/uri-list

then the NOTIFY:

NOTIFY sip:[email-masked] SIP/2.0
Event: presence.alert
Content-Type: application/uri-list

http://school.edu/user/current-pres-doc.pl

One might even have multipel URI there of several schemes, supporting a
variety of pulls. 

-Jonathan R.

---

---

From: "Brian Stucker" <[email-masked]>
Date: Wed, 28 Nov 2001 00:15:13 -0600

Good idea (HTTP url)... The thought had occurred to me briefly, but I really
hadn't thought it out much.

Actually, would it be possible to simply throw a request URI into a presence
notification for any sort of a transfer?

i.e. to get around problems with message length and firewall/NAT issues, is
it important that we actually include the cpim-xpidf+xml document in the
NOTIFY, or could we simply add that you might get an
application/cpim-xpidf-url document, that the client can go grab using HTTP.
Solves all of the problems in one tidy package.

That way clients initiate the transfer of the presence document, and can
choose TCP if they wish (likely), or if they suspect they're behind a
firewall/NAT. When they make the subscription, they could include an accept
with the application/cpim-xpidf-url content type which would simply be an
HTTP or HTTPS url to where to grab the presence document.

Another added advantage would be that you could use a "cheap" transport like
UDP to send the notification, and then let the client initiate an
"expensive" transport like TCP or TLS, on demand, to go get the document
when it needs to. That would allow established secure HTTP transfers to take
place when retrieving the data that we're trying to pull.

We could stipulate that the document that the HTTP url points to must be an
application/cpim-xpidf+xml formatted document.

Regards,

Brian Stucker
Nortel Networks

-----Original Message-----
From: Jonathan Rosenberg [mailto:[email-masked]]
Sent: Monday, November 26, 2001 2:03 PM
To: 'Michael Hammer'; Jonathan Rosenberg
Cc: [email-masked]'; Stucker, Brian [NGB:B635:EXCH]; simple;

some other 
presence 
use the 
current state 
resource (in this 
behind a 

then the NOTIFY:

-Jonathan R.

---

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 4 Dec 2001 01:54:10 -0500

-----Original Message-----
From: Brian Stucker [mailto:[email-masked]]
Sent: Wednesday, November 28, 2001 1:15 AM
To: Jonathan Rosenberg; 'Michael Hammer'
Cc: [email-masked]'; simple; Alex Nava; Patrick Sollee; Sanjoy Sen
Subject: RE: [Simple] RE: Question regarding NOTIFY messages using UDP.

>Good idea (HTTP url)... 

Thanks.

>The thought had occurred to me briefly, but I really hadn't 
>thought it out much. 
>
>Actually, would it be possible to simply throw a request URI into a
presence notification 
>for any sort of a transfer? 

Huh? The request URI of the NOTIFY is determine through standard Route
header processing.

>i.e. to get around problems with message length and firewall/NAT issues, is
it important 
>that we actually include the cpim-xpidf+xml document in the NOTIFY, or
could we simply add 
>that you might get an application/cpim-xpidf-url document, that the client
can go grab 
>using HTTP. Solves all of the problems in one tidy package.

I see no need to define a URL format for each document type. You could
either:

1. subscribe to Event: presence.alert, and the Accept header has type
application/uri-list. The NOTIFY would contain URLs to get at the presence
doc, presumably with HTTP URLs. This is the proposal below.

2. subscribe to Event: presence, and the Accept header has type
application/uri-list, which becomes a valid type for NOTIFY. The NOTIFY
comes with the URL list. This is similar to what I have suggested below, but
uses the same package.

>That way clients initiate the transfer of the presence document, and can
choose TCP if 
>they wish (likely), or if they suspect they're behind a firewall/NAT. When
they make the 
>subscription, they could include an accept with the
application/cpim-xpidf-url content 
>type which would simply be an HTTP or HTTPS url to where to grab the
presence document.
>
>Another added advantage would be that you could use a "cheap" transport
like UDP to send 
>the notification, and then let the client initiate an "expensive" transport
like TCP or 
>TLS, on demand, to go get the document when it needs to. That would allow
established 
>secure HTTP transfers to take place when retrieving the data that we're
trying to pull.

Right. This is exactly what I have proposed below.

-Jonathan R.
-----Original Message----- 
From: Jonathan Rosenberg [mailto:[email-masked]] 
Sent: Monday, November 26, 2001 2:03 PM 
To: 'Michael Hammer'; Jonathan Rosenberg 
Cc: [email-masked]'; Stucker, Brian [NGB:B635:EXCH]; simple; 

have described. 

some other 
presence 
use the 

current state 
resource (in this 
behind a 

-Jonathan R. 

--- 

---

---

From: "Brian Stucker" <[email-masked]>
Date: Tue, 4 Dec 2001 09:54:25 -0600

Great.

So, we'll have the ability to specify application/url-list in the SUBSCRIBE
for presence events in the -05 draft?

Regards,

Brian

-----Original Message-----
From: Jonathan Rosenberg [mailto:[email-masked]]
Sent: Tuesday, December 04, 2001 12:54 AM
To: Stucker, Brian [NGB:B635:EXCH]; Jonathan Rosenberg; 'Michael Hammer'
Cc: [email-masked]'; simple; Nava, Alex [NGC:B634:EXCH];
Sollee, Patrick [NGC:B680:EXCH]; Sen, Sanjoy [NGB:B692:EXCH]
Subject: RE: [Simple] RE: Question regarding NOTIFY messages using UDP.

Thanks.

presence notification 

it important 
could we simply add 
can go grab 

choose TCP if 
they make the 
presence document.
like UDP to send 
like TCP or 
established 
trying to pull.

-Jonathan R.
-----Original Message----- 
From: Jonathan Rosenberg [mailto:[email-masked]] 
Sent: Monday, November 26, 2001 2:03 PM 
To: 'Michael Hammer'; Jonathan Rosenberg 
Cc: [email-masked]'; Stucker, Brian [NGB:B635:EXCH]; simple; 

have described. 

some other 
presence 
use the 

current state 
resource (in this 
behind a 

-Jonathan R. 

--- 

---

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 4 Dec 2001 11:40:57 -0500

> From: Brazier Lachlan [mailto:[email-masked]]
> Sent: Tuesday, December 04, 2001 11:32 AM
> To: 'Jonathan Rosenberg'; 'Brian Stucker'; 'Michael Hammer'
> Cc: [email-masked]'; simple; Alex Nava; Patrick 
> Sollee; Sanjoy
> Sen
> Subject: AW: [Simple] RE: Question regarding NOTIFY messages 
> using UDP.
> 
> 
> Hello,
> 
> > 
> > >Great. 
> > Well, we need to decide whether thats really a separate 
> > package, or just a
> > different data format for an existing package.
> > 
> > We also need a few more opinions on this subject, since the 
> > list has been
> > faily quiet of late.
> > 
> 
> does this mean that in the NOTIFY there is no cpim-xpidf+xml 
> document but a
> URI or a list of URI's where the client can get the 
> cpim-xpidf+xml document
> with HTTP or whatever is specified?
> 
> I this correct?

Yes.

> If yes, sounds beautiful. Haven't thought about the packages yet.

Let me state the proposal for those who haven't been following the thread.
The current proposal on the table is to allow the SUBSCRIBE to have an
Accept header that contains application/uri-list as a valid type for NOTIFY.
So:

SUBSCRIBE sip:user@domain SIP/2.0
Event: presence
Accept: application/cpim-pidf+xml;q=0.5, application/uri-list;q=1.0

This would ask for notifications preferably as URLs. A NOTIFY would have:

NOTIFY sip:subscriber@domain SIP/2.0
Event: presence
Content-Type: application/uri-list
Content-Length: 23

http://www.domain.com/user/pdoc.xml

And then you would use HTTP to fetch the document. This way, you could have
arbitrarily large documents that would not need to be transferred with SIP. 

We would probably need to define a baseline url scheme (presumably http) to
guarantee interop. Certainly, other URLs, even LDAP say, could be provided.

This is not presence specific, to be honest, and is more useful for event
types with large data formats. I don't expect presence docs to be that large
in general.

-Jonathan R.

---