Subject: [Simple] Thoughts on list template

From: "Adam Roach" <[email-masked]>
Date: Mon, 18 Nov 2002 16:04:27 -0600

After giving some thought to the issues raised in today's
meeting, it occurs to me that there may, in fact, be a
compromise. I'm throwing the rough description out on the
list in hope that, if a flaw exists, one of you will
be able to point it out.

Basically, it amounts to this: to give the capabilities
that several people want (e.g. arbitrary depth lists)
without abandoning templates, we could slightly tweak
the meaning of the ".list" template. In particular,
"foo.list" would refer to any arbitrary nesting of
foo objects; e.g., it would cover what is, under the
current draft, called any of foo.list, foo.list.list,
foo.list.list.list, ad infinitum.

I *think* this still works gracefully when you combine
it with other templates (e.g. presence.list.winfo.list).
It *does* impose more of a processing load on aggregators,
but I don't think it's crippling.

So, first: does anyone see anything that would prevent
this working, from a technical perspective?

And, second: does anyone have any philosophical objections
to this approach? (I'll admit that I have my own reservations,
but I want to find a middle ground)

/a

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 18 Nov 2002 19:13:34 -0500

> foo.list.list.list, ad infinitum.
... 
> this working, from a technical perspective?

I too hope something like this works. I think we need to dig into details a bit to figure out if it is going to work.

1) Does this imply that the server serving an instance of foo.list itself issues the subscriptions to all the leaves of the recursive list expansion? Or is it just responsible for delegating those? Or are both approaches valid? (They may have different authentication implications.) The pure template approach clearly implied delegating the responsibility for the leaf subscriptions.

2) Either way, the server for foo.list is now responsible for determining if a particular list entry is an instance of foo, or of foo.list. Do we have any better way to figure this out than trial and error? Is that acceptable? (An alternative might be to explicitly type the entries in a list.)

	Paul

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 18 Nov 2002 23:41:02 -0500

inline.

Paul Kyzivat wrote:
 >
 > Adam Roach wrote:
 >
 >> Basically, it amounts to this: to give the capabilities that
 >> several people want (e.g. arbitrary depth lists) without abandoning
 >> templates, we could slightly tweak the meaning of the ".list"
 >> template. In particular, "foo.list" would refer to any arbitrary
 >> nesting of foo objects; e.g., it would cover what is, under the
 >> So, first: does anyone see anything that would prevent this
 >> working, from a technical perspective?
 >
 >
 > I too hope something like this works. I think we need to dig into
 > details a bit to figure out if it is going to work.
 >
 > 1) Does this imply that the server serving an instance of foo.list
 > itself issues the subscriptions to all the leaves of the recursive
 > list expansion? Or is it just responsible for delegating those? Or
 > are both approaches valid? (They may have different authentication
 > implications.) The pure template approach clearly implied delegating
 > the responsibility for the leaf subscriptions.

I don't see the difference between the two things?

 >
 > 2) Either way, the server for foo.list is now responsible for
 > determining if a particular list entry is an instance of foo, or of
 > foo.list. Do we have any better way to figure this out than trial and
 > error? Is that acceptable? (An alternative might be to explicitly
 > type the entries in a list.)

This is the big issue.

It might help to step back and consider how this would work in an IDEAL 
world of a clean slate, and work backwards. Doing that is what helped us 
figure out how to do loose routing, and as it turns out, I think it 
solves this problem too.

In the ideal world, <gasp> there is no presence package. There is only 
the equivalent of "presence.list". That is, we assume at the outset that 
a presence subscription always implies a list, and a valid option is a 
list of one. Thus, I subscribe to my buddylist with package "presence", 
and each of the subscriptions it generates are for the same package, 
"presence". The owner of the URI gets to decide whether its a list of 
multiple elements, or just one.

Now, the problem of course is that all of the list features would need 
to be replicated in each package. So, the ideal solution there IMHO 
would have been to have list processing be central to rfc3265 in the 
first place, so that all packages are list enabled from the outset.

OK, so IF you agree that this is what it ought to have been, how do we 
achieve as close a goal as possible with the constraints that we have?

I would define list processing as an EXTENSION to rfc3265, not a 
template or a new package. How would that work? When a UAC subscribes, 
it includes a Supported header indicated "lists". The Allow header in 
the subscribe also lists multipart as a valid body type thats supported, 
along with the MIME type of this XML list meta-data. So, a buddy list 
subscription looks like:

SUBSCRIBE sip:[email-masked] SIP/2.0
Supported: lists
Allow: application/pidf+xml, multipart/mixed, application/elist+xml
Event: presence

The server knows that mylist is a list, but since the client supports 
lists, its OK. The server does a SUBSCRIBE to each participant in the 
list, all of them with the Event: presence header, and all of them with 
the same Supported and Allow headers. Now, should one of those happen to 
be a regular presence server that doesn't support the "lists" extension, 
it works just fine! The server will generate NOTIFYs with plain-old 
PIDF, no multipart or elist+xml. So, there is no trial-and-error; a 
subscription to an element in the list is the same in all cases, and 
doesn't require knowing what it is. If it turns out that its a list, 
multipart comes back. If not, just plain PIDF.

Should a user SUBSCRIBE to a list, and not indicate support for "lists" 
in the Supported header, the server rejects the request with a 420 and 
the appropriate Require header.

As far as I can tell, this solution buys us EVERYTHING we want:

* infinite nesting of lists, without the list server needing to know 
whether an entry is a list or not
* no trial and error
* backwards compatibility
* graceful evolution, if we want, to a revised rfc3265 that includes the 
list processing as a native capability

Comments?

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 19 Nov 2002 00:22:25 -0500

Jonathan,

I was thinking that getting out of this problem required special casing list. It looks like you have come up with a fine way to do it. At first glance this seems to work just fine as far as I can see, though I reserve the right to change my mind.

	Great idea,
	Paul

---

From: Ben Campbell <[email-masked]>
Date: Wed, 20 Nov 2002 10:36:46 -0500

This certainly seems like an elegant way to allow assymetric hierachies. 
There are a few areas that give me heartburn, but on reflection, I 
realize these issues are with the idea of recursive lists in the first 
place.

1) We talk about the idea that a watcher can infer list contents from an 
initial full state notify. But it is not entirely clear to me what a 
full state notify looks like for a list that contains other lists. Do we 
expect the watcher to be able to infer the full hiearchy?

2) Robert mentioned a scary one to me this morning: What about circular 
references in lists?

---

From: Sean Olson <[email-masked]>
Date: Wed, 20 Nov 2002 07:56:50 -0800 (PST)

> expect the watcher to be able to infer the full

I don't think this is necessary, but certainly
possible -- if a bit of a pain. The hierarchy of
the MIME plus the tuple ID plus the presentity URI
plus the From: URI in the NOTIFY gives the watcher
enough info to reconstruct this hierarchy if it
really wanted to.... but is this a requirement? 
I'm assuming the watcher has a notion of the
hiearchy of the list it is subscribing to -- even 
though that may be an incomplete view. The watcher
could render the NOTIFYs relative to his own view of
the list, relative to the hierarchy in the NOTIFYs,
or even "flat".

> 2) Robert mentioned a scary one to me this morning:

Could a common Call-ID be used to break these loops? 
/sean

__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - Let the expert host your site
http://webhosting.yahoo.com

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 21 Nov 2002 03:18:10 -0500

inline.

> This certainly seems like an elegant way to allow assymetric hierachies. 

Thanks. I definitely think this is right.

> expect the watcher to be able to infer the full hiearchy?

I thought that the presence list server (or whatever we are calling it 
these days) would subscribe to each element, and when it gets the 
notifies, will know whether each element is a single URI or itself a 
list. It then reports its own result as a list, where elements are 
either a single URI, or another list, based on its downstream 
subscribes. The result is that the hierarchy would be constructed and 
placed in the full state notify.

Now, the bigger issue is full-state notifies for big lists. They quickly 
become unwieldy. If we apply the current semantic - which is that 
NOTIFY's triggered from SUBSCRIBE contain full state, you end up with 
this huge NOTIFY after every refresh, probably not what you want.

If you assume that the list structure itself (ie., the set of 
presentities in any list), rather than the state of the presentities, is 
synchronized separately, you don't really have a strong need for full 
state updates. Instead, you can send a bunch of partial updates (state 
for a single presentity) that are then used by the subscriber to fill in 
the state of the individual leaves of the tree. Its probably OK for 
those partial notifies to be sent to the subscriber over some window of 
time after the subscribe (i.e, it takes 2 seconds for the state of all 
leaves to be transferred).

We could also define a parameter in the SUBSCRIBE somewhere that 
explicitly tells the server to send a full state in a NOTIFY. The 
deafult operation is to never send full state, only when requested.

> 2) Robert mentioned a scary one to me this morning: What about circular 

Yikes!!

We can either do a "hop counter" - every time a server triggers a 
subscription to get state for an incoming subscription it decrements a 
counter, or we can detect the loop through a persistent identifer as 
sean suggests. I think that there is no real meaning of "spiral" here, 
which would immensely complicate such an approach.

One way or another, we have to worry about this case if we accept 
recursive lists.

-Jonathan R.