From nobody Fri Apr 11 09:28:04 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD3DA1A069D for ; Fri, 11 Apr 2014 09:28:02 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.727 X-Spam-Level: ** X-Spam-Status: No, score=2.727 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, MANGLED_TOOL=2.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZuROxBoLaqEc for ; Fri, 11 Apr 2014 09:27:58 -0700 (PDT) Received: from sabertooth02.qualcomm.com (sabertooth02.qualcomm.com [65.197.215.38]) by ietfa.amsl.com (Postfix) with ESMTP id 828871A01CF for ; Fri, 11 Apr 2014 09:27:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1397233678; x=1428769678; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=pyP4hlA1wp/K15N7OUgJw9Yw2DWZNfV6uBfrY1HkUbI=; b=c1VBOcauzlT3Lhm5okaqy220+hBjFMXqLBvV60yvPRMuMRFLHFnYPUt8 8Wn0CZzw9MjH9xeFbMboKExXjbCOJ44NwDn52+GYf/misHKmPPg2NSlMT JeOoSRHq4RE+ML/q2SO5RXBWNZd3HBzwR/IgMjQnt9DDXpWUV2Qe6sdnQ k=; X-IronPort-AV: E=McAfee;i="5400,1158,7405"; a="61941679" Received: from ironmsg03-r.qualcomm.com ([172.30.46.17]) by sabertooth02.qualcomm.com with ESMTP; 11 Apr 2014 09:27:57 -0700 X-IronPort-AV: E=Sophos;i="4.97,842,1389772800"; d="scan'208";a="660649620" Received: from nasanexhc09.na.qualcomm.com ([172.30.39.8]) by Ironmsg03-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 11 Apr 2014 09:29:30 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.137]) by nasanexhc09.na.qualcomm.com ([172.30.39.8]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 09:26:37 -0700 From: "Wang, Ye-Kui" To: Jonatan Samuelsson , Jonathan Lennox Thread-Topic: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC Thread-Index: AQHPNy2E9/hww0j0CUugS6BNO6oybZrRLN1AgADMMwD//53j0IABJUUAgAASsHCAAAyLQIAAuiiA//+fAJCAATlBgIAALTaggAYyJwCAADEYgP//jXRwAGquMIAF3rF4sA== Date: Fri, 11 Apr 2014 16:26:35 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A83451FFD94@nasanexd02f.na.qualcomm.com> References: <073.d3261c7320c6cd8879101a2e0938470c@trac.tools.ietf.org> <088.5cce7c7048eaf169f8424dbdd09a8722@trac.tools.ietf.org> <8BA7D4CEACFFE04BA2D902BF11719A83387D5BD0@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C6FAB@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D6229@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C74DC@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D804D@nasanexd02f.na.qualcomm.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D8196@nasanexd02f.na.qualcomm.com> <02ACD790-4F42-4CB1-84CE-8F6490374477@vidyo.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D86D4@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C7983@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D956D@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C846A@ESESSMB109.ericsson.se> <7975B7E5-D690-4F0A-8DDD-1AE5CA13FAB2@vidyo.com> <8BA7D4CEACFFE04BA2D902BF11719A83387DBF7C@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C8DC2@ESESSMB109.ericsson.se> In-Reply-To: <634D3B5D82E3214DA9B6A36D5F50DB231C8DC2@ESESSMB109.ericsson.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/Cs9YsImfbeLGhJNHLZJLgJOQ60s Cc: "draft-ietf-payload-rtp-h265@tools.ietf.org" , "payload@ietf.org" , payload issue tracker Subject: Re: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 16:28:02 -0000 Good. It seems that we are settled down here. I will implement the agreed c= hanges in the next version, which I hope can be submitted soon (next week o= r the week after). BR, YK -----Original Message----- From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com]=20 Sent: Wednesday, March 12, 2014 5:18 AM To: Wang, Ye-Kui; Jonathan Lennox Cc: payload issue tracker; draft-ietf-payload-rtp-h265@tools.ietf.org; Step= han Wenger; payload@ietf.org Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC Hi,=20 Regarding Jonathan's question: "Bare SHOULDs (i.e., a statement that says something SHOULD be done, withou= t explaining the circumstances in which an implementation would not do it) = are generally considered undesirable in IETF standards these days. (This i= s a stylistic change since RFC 3551 was written.) What do you consider the= circumstances in which it would be appropriate for an H.265 RTP implementa= tion not to set the marker bit?" In fact I would prefer if it was explicitly stated that the marker bit MUST= be set on the last packet in an RTP stream of an access unit and that it M= UST NOT be set on a packet that is not the last packet of an RTP stream of = an access unit.=20 One concrete example of where a "lazy" MANE would be tempted not to set mar= ker bit correctly is when an RTP stream containing two temporal layers is s= plit up to two different RTP streams based on temporal_id. In order two ded= uce which packet to set the marker bit on in each stream the MANE would hav= e to buffer the entire access unit before being able to forward the last pa= cket of the stream corresponding to the higher temporal_id. Another example was the one I mentioned earlier. A "lazy" MANE that (for so= me reason) removes suffix SEI messages (perhaps it has somehow been informe= d that the receiver is not capable of making use of them) without buffering= preceding packets will not set the marker bit on the last packet if it has= already been forwarded.=20 A third example is for SHVC or MV-HEVC in which a MANE extracts a base laye= r from a single RTP stream containing multiple layers. If that stream is su= pposed to be correct according to the HEVC payload format the MANE will mos= t probably have to buffer and wait for all layers of that access unit befor= e deducing which packet was the last packet of the base layer. Anyway. I appreciate the constructive discussion on the topic (thanks) and = I think the suggestion on the table: =20 " Marker bit (M): 1 bit Set for the last packet, carried in the current RTP stream, of an access un= it, in line with the normal use of the M bit in video formats, to allow an = efficient playout buffer handling."=20 is ok and an improvement compared to the original text in the draft but I w= ould appreciate if it was made clear that this is to be interpreted as a MU= ST (not a SHOULD) and that some more text is added to, as Jonathan suggeste= d: "explicitly pointing out that in MST mode, if an access unit appears in mul= tiple packet streams, the marker bit is set on each packet stream's last pa= cket of the access unit." Best Regards Jonatan -----Original Message----- From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com]=20 Sent: den 10 mars 2014 17:30 To: Jonathan Lennox; Jonatan Samuelsson Cc: payload issue tracker; draft-ietf-payload-rtp-h265@tools.ietf.org; Step= han Wenger; payload@ietf.org Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC I have basically the same opinion here as Jonathan. Just one aspect regarding Jonatan's following comment: > Just because you have received all packets with the same timestamp it is = not necessarily the case that you can safely display the picture in that ac= cess unit, e.g. if you in the next access unit receive a picture with lower= timestamp. Thus, in my opinion, the most useful information that you can g= et from the marker bit is whether you have received all VCL NAL units of a = picture or not. [YK] I think your first sentence is correct, but I don't see the logic that= this true statement can lead to the conclusion that "the most useful infor= mation that you can get from the marker bit is whether you have received al= l VCL NAL units of a picture or not". To me, if you have reordering in enco= ding, then you are not doing low-delay, and then the marker bit is less use= ful than in low-delay where there is no reordering in encoding. BR, YK -----Original Message----- From: Jonathan Lennox [mailto:jonathan@vidyo.com]=20 Sent: Monday, March 10, 2014 9:13 AM To: Jonatan Samuelsson Cc: Wang, Ye-Kui; payload issue tracker; draft-ietf-payload-rtp-h265@tools.= ietf.org; Stephan Wenger; payload@ietf.org Subject: Re: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC On Mar 10, 2014, at 9:17 AM, Jonatan Samuelsson wrote: > Thanks for your response Ye-Kui (sorry for my late reply), >=20 > In my opinion it does make sense to distinguish between VCL data and non-= VCL data. Just because you have received all packets with the same timestam= p it is not necessarily the case that you can safely display the picture in= that access unit, e.g. if you in the next access unit receive a picture wi= th lower timestamp. Thus, in my opinion, the most useful information that y= ou can get from the marker bit is whether you have received all VCL NAL uni= ts of a picture or not. I disagree. I don't think VCL and non-VCL NAL units should be distinguishe= d for the marker bit. Presumably any encoder which is sending suffix NAL u= nits (and I would expect them to be rare) is doing so because it believes t= hey'll be useful for the decoders to fully decode the access unit. If a later packet contains an earlier timestamp, you're either doing frame = reordering (in which case it *is* safe to decode the earlier frame in decod= e order) or else using DON (in which case you need to track DON values rath= er than sequence numbers). The marker bit - by definition - can't carry more than one bit of data, so = there's no way to tell just from that bit whether it's safe to decode a fra= me given arbitrary earlier packet loss. Without packet loss, it should alw= ays be possible to decode an access unit when the marker bit is received, i= f all earlier access units in decode order (since the last refresh point) h= ave already been decoded. > Anyway, let me ask one question on the solution that you were suggesting.= =20 >=20 > " Marker bit (M): 1 bit > Set for the last packet, carried in the current RTP stream, of an access = unit, in line with the normal use of the M bit in video formats, to allow a= n efficient playout buffer handling." >=20 > Would it be equivalent to: >=20 > " Marker bit (M): 1 bit > Set when the next packet (if any), in the same RTP stream, has a differen= t timestamp than the current packet." They are equivalent when not using DON (or more specifically when sprop-dep= ack-buf-nalus > 0). When you are using DON, they are different; and I thin= k the original definition is more useful. > Another question related to >=20 > Most of these video encodings also specify that the marker bit of the > RTP header SHOULD be set to one in the last packet of a video frame > and otherwise set to zero. Thus, it is not necessary to wait for a > following packet with a different timestamp to detect that a new > frame should be displayed. >=20 > It uses "SHOULD" from RFC 2119 but in the H.265 payload format "SHOULD" i= s not used. Is the intended interpretation: >=20 > " Marker bit (M): 1 bit > SHOULD be set for the last packet..."? Bare SHOULDs (i.e., a statement that says something SHOULD be done, without= explaining the circumstances in which an implementation would not do it) a= re generally considered undesirable in IETF standards these days. (This is= a stylistic change since RFC 3551 was written.) What do you consider the = circumstances in which it would be appropriate for an H.265 RTP implementat= ion not to set the marker bit? > Best Regards Jonatan >=20 > -----Original Message----- > From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] > Sent: den 6 mars 2014 22:52 > To: Jonatan Samuelsson; Jonathan Lennox > Cc: payload issue tracker; draft-ietf-payload-rtp-h265@tools.ietf.org;=20 > Stephan Wenger; payload@ietf.org > Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >=20 > [Jonatan]: I don't think this text is really up to date with HEVC in the = sense that the actual video frame (which in my opinion, for the HEVC case, = is the VCL NAL units of an access unit) can be completely received even tho= ugh there are more packet(s) that are following with the same time stamp. >=20 > I don't understand the above sentence. I think the receiver can only conc= lude that an access unit is completely received only when no more packets w= ith the same timestamp is coming soon. >=20 > For other parts of your new message below: Any data, including non-VCL da= ta, associated with a picture in the bitstream is supposed to be used by th= e decoder side (not just the decoder) to do something that can be helpful i= n processing (decoding including error concealment and so on, display, etc.= ) of the decoded picture, even not needed for NORMATIVE decoding of the pic= ture. Thus, only when the decoder side receives all data associated with a = picture, can it be sure that no more useful information may come before sen= t the decoded picture to the display. Therefore, we should not change the w= ording from "the last NAL unit" to "the last VCL NAL unit". Also, without m= aking the setting to be stream specific, as I suggested and Jonathan agreed= , multi-layer extensions will be problematic. >=20 > BR, YK >=20 >=20 > -----Original Message----- > From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] > Sent: Thursday, March 06, 2014 2:59 AM > To: Wang, Ye-Kui; Jonathan Lennox > Cc: payload issue tracker; draft-ietf-payload-rtp-h265@tools.ietf.org;=20 > Stephan Wenger; payload@ietf.org > Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >=20 > Thanks Ye-Kui and Jonathan for your responses. >=20 > Regarding Section 5 of RFC 3551: >=20 > Most of these video encodings also specify that the marker bit of the > RTP header SHOULD be set to one in the last packet of a video frame > and otherwise set to zero. Thus, it is not necessary to wait for a > following packet with a different timestamp to detect that a new > frame should be displayed. >=20 > I don't think this text is really up to date with HEVC in the sense that = the actual video frame (which in my opinion, for the HEVC case, is the VCL = NAL units of an access unit) can be completely received even though there a= re more packet(s) that are following with the same time stamp. >=20 > Since none of the suffix SEI messages has any normative effect on the out= put of the decoder, why should it be necessary to wait for example for a de= coded_picture_hash SEI before detecting that a frame can be displayed? >=20 > Ye-Kui, you earlier wrote: > "Anyway, I admitted that the current semantics are not fully future-proof= regarding suffix SEI messages. In looking into future multi-layer extensio= ns, it appears that simply changing the wording from "last NAL unit" to "la= st VCL NAL unit" is not good enough." >=20 > But if (as I initially suggested) the change is from "last NAL unit of an= access unit" to "last VCL NAL unit of a picture" I guess it could work als= o for future extensions in which there will be several pictures in the same= access unit. Or do you see a problem here? >=20 > Best Regards Jonatan >=20 > -----Original Message----- > From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] > Sent: den 6 mars 2014 01:24 > To: Jonathan Lennox > Cc: Jonatan Samuelsson; payload issue tracker;=20 > draft-ietf-payload-rtp-h265@tools.ietf.org; Stephan Wenger;=20 > payload@ietf.org > Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >=20 > Agree to make the intention clearer, e.g. by adding a note. >=20 > On your last question: I think you are right, a non-VCL suffix NAL unit, = e.g. an SEI, end of sequence, or end of bitstream NAL unit, could have nuh_= layer_id equal to 0, and thus in MST the last NAL unit of an access unit ma= y actually be carried in the last packet from the "base" RTP stream. And ye= s this actually does not matter for using of the marker bit for the purpose= described below.=20 >=20 > BR, YK >=20 > -----Original Message----- > From: Jonathan Lennox [mailto:jonathan@vidyo.com] > Sent: Wednesday, March 05, 2014 4:05 PM > To: Wang, Ye-Kui > Cc: Jonatan Samuelsson; payload issue tracker;=20 > draft-ietf-payload-rtp-h265@tools.ietf.org; Stephan Wenger;=20 > payload@ietf.org > Subject: Re: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >=20 > This seems correct to me. I would suggest, however, explicitly pointing = out that in MST mode, if an access unit appears in multiple packet streams,= the marker bit is set on each packet stream's last packet of the access un= it. >=20 > I think this definition works even when DONs are in use (when sprop-depac= k-buf-nalus is > 0), since an AP can only contain packets from a single acc= ess unit. If packets from different access units are interleaved (in RTP o= rder) using DON, the packet stream will look odd to a naive RTP analysis, b= ut I believe the marker bit will still be useful, allowing low-delay decode= rs to flush complete access units out of the de-packetization buffer early. >=20 > Ye-Kui, a question about your comment, though: are we sure that the final= packet of the access unit will in fact always be the final packet of the h= ighest layer? In particular, in SHVC or MV-HEVC, presumably we'll be able = to split MST streams by nuh_layer_id. How will the nuh_layer_id be set for= trailing SEIs in such streams? If it's set to 0, then the final packet of= the access unit will in fact be sent in a lower layer. I don't think this= actually matters, especially since MST streams will use DON, but we should= be careful about our logic. >=20 >=20 >=20 > On Mar 5, 2014, at 7:20 PM, Wang, Ye-Kui wrote: >=20 >> I did a bit of search on the usage of the marker bit for video, and the = most relevant description I found was the following paragraph in Section 5 = of RFC 3551: >>=20 >> Most of these video encodings also specify that the marker bit of=20 >> the RTP header SHOULD be set to one in the last packet of a video=20 >> frame and otherwise set to zero. Thus, it is not necessary to wait=20 >> for a following packet with a different timestamp to detect that a=20 >> new frame should be displayed. >>=20 >> So the key usage of the marker bit is that in some low-delay application= s, output/display of a picture can be done before receiving an RTP packet w= ith a different/greater timestamp. Since non-VCL NAL unit should have the s= ame timestamp as the associated picture, thus I think we should not use the= wording of "last VCL NAL unit" in determining the setting of the marker bi= t. >>=20 >> To make the semantics extensible or future-proof for multi-layer extensi= ons, I think we should change it to be RTP stream (or equivalently packet s= tream) specific, and each receiver can just look at the marker bits of the = packets from the highest RTP stream for the usage. And this will also resol= ve the issue of requiring some entities to change the marker bit in sub-lay= er scalability usage as Jonaton raised.=20 >>=20 >> In other words, the normative semantics are to be changed as follows: >>=20 >> Marker bit (M): 1 bit >> Set for the last packet, carried in the current RTP stream, of an access= unit, in line with the normal use of the M bit in video formats, to allow = an efficient playout buffer handling. >>=20 >> Please let me know if there is any concern with such a change (and note = that we would align the terminology of RTP stream with other drafts, e.g. t= he Grouping Taxonomy draft, as discussed in responding to Magnus' reviewing= comments). >>=20 >> BR, YK >>=20 >> -----Original Message----- >> From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang,=20 >> Ye-Kui >> Sent: Wednesday, March 05, 2014 10:29 AM >> To: Jonatan Samuelsson; payload issue tracker;=20 >> draft-ietf-payload-rtp-h265@tools.ietf.org; stewe@stewe.org >> Cc: payload@ietf.org >> Subject: Re: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> Something should be done to the HEVC spec to make it clearer for the dec= oded picture hash SEI message, like that it shall not be nested and what th= e values of LayerId and LId should be, or allow add to the list of nestable= SEI message. In any case, it does not make sense to me to put a greater va= lue of TId than its associated picture. On user_data_registered_itu_t_t35 a= nd user_data_unregistered suffix SEI messages, only if it is defined by som= e application then it may make sense to put a greater value of TId than its= associated picture - currently I have not seen any such use being defined = anywhere. >>=20 >> Anyway, I admitted that the current semantics are not fully future-proof= regarding suffix SEI messages. In looking into future multi-layer extensio= ns, it appears that simply changing the wording from "last NAL unit" to "la= st VCL NAL unit" is not good enough. We might need to specify the semantics= of the marker bit to be RTP stream specific. To make sure whether such a c= hange is OK, we also need to check against the usage of the marker bit. >>=20 >> BR, YK >>=20 >> -----Original Message----- >> From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] >> Sent: Wednesday, March 05, 2014 1:07 AM >> To: Wang, Ye-Kui; payload issue tracker;=20 >> draft-ietf-payload-rtp-h265@tools.ietf.org; stewe@stewe.org >> Cc: payload@ietf.org >> Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> Thanks for your response Ye-Kui, >>=20 >> As far as I understand, out of the 6 SEI messages that have so far been = defined in the HEVC spec as possible to use as suffixes (filler_payload, us= er_data_registered_itu_t_t35, user_data_unregistered, progressive_refinemen= t_segment_end, post_filter_hint, and decoded_picture_hash) there are 3 that= are not required to have the same TId as the VCL NAL units of the access u= nit: user_data_registered_itu_t_t35, user_data_unregistered and decoded_pi= cture_hash since their payloadType numbers (4, 5 and 132) are not included = in the list: >>=20 >> "When a non-nested SEI message has payloadType equal to 2, 3, 6, 9, 15, = 16, 17, 19, 22, 23, 45, 47, 128, 131, or 134 (i.e., one of the SEI messages= that have payloadType not equal to 0, 1, or 130, and that are allowed to b= e nested SEI messages), the SEI NAL unit containing the non-nested SEI mess= age shall have TemporalId equal to the TemporalId of the access unit contai= ning the SEI NAL unit." >>=20 >> Maybe it's a mistake in the HEVC spec that 132 is not in this list, but = I think the problem would exist even if was. An encoder that (for whatever = reason) puts in user_data_registered_itu_t_t35 or user_data_unregistered su= ffix SEI messages with higher TId would put a MANE in a difficult situation= .=20 >>=20 >> I also think we should consider future extensions of HEVC (i.e. SHVC and= MV-HEVC) in which an access unit may contain many pictures with different = values of nuh_layer_id. If the marker bit is set for the last packet of an = access unit, then once again all the marker bits will end up in the highest= layer (stream) and a MANE would have to change the value of marker bits as= soon as not all layers (streams) are forwarded. >>=20 >> Best Regards Jonatan >>=20 >> -----Original Message----- >> From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] >> Sent: den 5 mars 2014 01:20 >> To: Jonatan Samuelsson; payload issue tracker;=20 >> draft-ietf-payload-rtp-h265@tools.ietf.org; stewe@stewe.org >> Cc: payload@ietf.org >> Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> Hmm, a decoded picture hash SEI message applies to (and is associated wi= th) the preceding decoding picture in decoding order and thus the TId of th= e containing suffix SEI NAL unit should be set equal to the TId of the asso= ciated picture. Thus in the example, for the suffix SEI NAL unit containing= the decoded picture hash SEI message for the first encoded picture should = be assigned TId equal to 0.=20 >>=20 >> According to the HEVC spec, currently the following SEI messages are or = can be suffix SEI messages: filler_payload, user_data_registered_itu_t_t35,= user_data_unregistered, progressive_refinement_segment_end, post_filter_hi= nt, and decoded_picture_hash. Among these, I think all of the "specified" S= EI messages (i.e. the above list excluding user_data_registered_itu_t_t35, = user_data_unregistered) contained in a suffix SEI NAL unit would apply to t= he preceding decoded picture in decoder order and should have the same TId = as the associated picture. The only SEI messages that can have a greater TI= d value than the containing access unit are the three SEI message that can = carry HRD information, i.e., buffering_period, pic_timing, and decoding_uni= t_info, but these can only be prefix SEI messages. >>=20 >> Of course, in theory, future versions of HEVC or some extensions (even d= efined by an application spec) could define suffix SEI messages that can ha= ve a greater TId value than the containing access unit. In that case, what = you described could be meaningful, and indeed if a receiver receives the "b= ase sub-layer" stream, some entity would have to set the marker bit of thos= e packets containing the last NAL unit of an access unit for that stream, a= ccording to the current semantics. >>=20 >> However, if we change the wording to the last VCL NAL unit as you sugges= ted, and in case a suffix SEI NAL unit of picture A is not carried the pack= et containing the last VCL NAL unit if the access unit, would the usage of = the marker bit be affected, e.g. in playout buffer handling? If the answer = is no, then there should be no problem to do the change. >>=20 >> Basically the situation is as follows. There is no concrete problem now.= However, there can be a potential problem in the future if a suffix SEI me= ssage is defined and can have a greater TId value than the containing acces= s unit. If the suggested change does not affect the usage of the bit, we sh= ould do the change now to make the design future-proof. So the key question= is whether the suggest change would affect the usage of the bit.=20 >>=20 >> Does anyone know how the marker bit is used in playout buffering handlin= g? An explanation or a pointer would be greatly appreciated. >>=20 >> BR, YK >>=20 >> -----Original Message----- >> From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] >> Sent: Tuesday, March 04, 2014 1:28 PM >> To: Wang, Ye-Kui; payload issue tracker;=20 >> draft-ietf-payload-rtp-h265@tools.ietf.org; stewe@stewe.org >> Cc: payload@ietf.org >> Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> Dear Ye-Kui, >>=20 >> Let me try to make the example more specific (and then also see if I hav= e understood the multi-stream transmission concept correctly). >>=20 >> Assume that the sender puts all NAL units with TId 0 in one RTP stream (= stream A) and all NAL units with TId 1 in another RTP stream (stream B). No= w for the first encoded picture the VCL NAL units are assigned TId 0 and th= en a decoded picture hash SEI message is created in the same access unit bu= t is given TId 1. Then the marker bit will be set on the packet containing = the decoded picture hash SEI message which is put in stream B. If this sche= me is applied for all access units, stream A will not contain any packets w= ith the marker bit set since these are all put in stream B. A MANE that onl= y wants to forward stream A to a receiver would be required to perform anal= ysis (and probably introduce delay) in order to correctly change the marker= bit on the "new" last packet of the access unit. >>=20 >> Is that correct, or have I misinterpreted anything? >>=20 >> Best Regards Jonatan >>=20 >> -----Original Message----- >> From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] >> Sent: den 4 mars 2014 18:21 >> To: payload issue tracker;=20 >> draft-ietf-payload-rtp-h265@tools.ietf.org; stewe@stewe.org; Jonatan=20 >> Samuelsson >> Cc: payload@ietf.org >> Subject: RE: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> Hi Jonatan, >>=20 >> I do not fully understand the following example: >>=20 >> "However, as far as I understand, it is allowed for an encoder to create= a bitstream with two temporal sub-layers (every second picture with TId 0= and every second picture with TId 1) but put post-picture SEI messages fo= r all access units in temporal layer 1 (also for access units in which the= picture has TId 0, as a hint of the relative importance of the NAL units)= . For that case I guess there will be one RTP stream (corresponding to TId = 0) completely without any marker bit set, right?" >>=20 >> Could you please check whether it can be clarified a bit? >>=20 >> BR, YK >>=20 >> -----Original Message----- >> From: payload issue tracker [mailto:trac+payload@trac.tools.ietf.org] >> Sent: Monday, March 03, 2014 2:11 PM >> To: draft-ietf-payload-rtp-h265@tools.ietf.org; Wang, Ye-Kui;=20 >> stewe@stewe.org; jonatan.samuelsson@ericsson.com >> Cc: payload@ietf.org >> Subject: Re: [payload] #10 (rtp-h265): Marker bit in H.265/HEVC >>=20 >> #10: Marker bit in H.265/HEVC >>=20 >>=20 >> Comment (by jonatan.samuelsson@ericsson.com): >>=20 >> Thanks Stefan for the comments and the example. It is exactly this kind = of cases I'm concerned about but I wouldn't characterize them as exotic. F= or conversational applications it is natural to operate in *very* low dela= y mode (i.e. not buffering packets before forwarding them) and having post= - picture SEI packetized in their own packets is the only option as soon a= s the VCL NAL units are fragmented since FUs cannot be aggregated. Having = to introduce slices just to avoid FUs and thereby being able to aggregate = SEI messages with VCL NAL units does not sound like a compelling alternati= ve. >>=20 >> The most reasonable solution for this case is probably to just not perfo= rm thinning by removing the SEI messages since they are typically not very= large in the first place. >>=20 >> However, as far as I understand, it is allowed for an encoder to create = a bitstream with two temporal sub-layers (every second picture with TId 0 = and every second picture with TId 1) but put post-picture SEI messages for= all access units in temporal layer 1 (also for access units in which the = picture has TId 0, as a hint of the relative importance of the NAL units). >> For that case I guess there will be one RTP stream (corresponding to=20 >> TId >> 0) completely without any marker bit set, right? >>=20 >> -- >> -------------------------------------+------------------------------- >> -------------------------------------+--- >> -------------------------------------+--- >> Reporter: | Owner: draft-ietf-payload- >> jonatan.samuelsson@ericsson.com | rtp-h265@tools.ietf.org >> Type: defect | Status: new >> Priority: major | Milestone: >> Component: rtp-h265 | Version: 2.0 >> Severity: - | Resolution: >> Keywords: | >> -------------------------------------+------------------------------- >> -------------------------------------+--- >> -------------------------------------+--- >>=20 >> Ticket URL:=20 >> >> payload >>=20 >> _______________________________________________ >> payload mailing list >> payload@ietf.org >> https://www.ietf.org/mailman/listinfo/payload >>=20 >> _______________________________________________ >> payload mailing list >> payload@ietf.org >> https://www.ietf.org/mailman/listinfo/payload >>=20 >=20 From nobody Fri Apr 11 09:47:51 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 31DFB1A070E for ; Fri, 11 Apr 2014 09:47:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.427 X-Spam-Level: X-Spam-Status: No, score=0.427 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T6YZ8G6rrEOD for ; Fri, 11 Apr 2014 09:47:44 -0700 (PDT) Received: from sabertooth02.qualcomm.com (sabertooth02.qualcomm.com [65.197.215.38]) by ietfa.amsl.com (Postfix) with ESMTP id 57BAB1A070F for ; Fri, 11 Apr 2014 09:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1397234863; x=1428770863; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=S+AC9/5AdKcHVtVO0z9P8dNXVkEOk3Psztbr9H85ByM=; b=YMjlRID9w/Swoa6tsTRQuxxd4CHcMHZg/5D7ZfbQaNx1TThdY1rvzFtk J7bnFEkL0DkAYM0hRftwq0tolYSewJ9RorVJ9hVUtVsEiIEoCBBybcGKL cYY8asIR5uQ/ck7zup5fVrFO70GG5uJ1fVXMAyYatwegn8e6FL4f9BZfX A=; X-IronPort-AV: E=McAfee;i="5400,1158,7405"; a="61942564" Received: from ironmsg04-r.qualcomm.com ([172.30.46.18]) by sabertooth02.qualcomm.com with ESMTP; 11 Apr 2014 09:47:43 -0700 X-IronPort-AV: E=Sophos;i="4.97,842,1389772800"; d="scan'208";a="714205597" Received: from nasanexhc11.na.qualcomm.com ([172.30.39.6]) by Ironmsg04-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 11 Apr 2014 09:49:16 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.137]) by nasanexhc11.na.qualcomm.com ([172.30.39.6]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 09:47:00 -0700 From: "Wang, Ye-Kui" To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: [payload] Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxkvhCfBfHaXEuyjN/n0X9j0ZrLRhBggBqeuYCAJvf4MA== Date: Fri, 11 Apr 2014 16:47:00 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A8345200E24@nasanexd02f.na.qualcomm.com> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D100D@nasanexd02f.na.qualcomm.com> <53270527.9010407@ericsson.com> In-Reply-To: <53270527.9010407@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/Boeu-mrRbz6BOWcVKQgkZpaz8SI Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 16:47:49 -0000 Thanks again! And sorry for not being able to reply earlier as I got stuck = in some other duties (mainly the MPEG and JCT meetings last week and half). Please see my replies inline below. Similarly, I also removed the issues wh= ere I am happy with your responses and have no further to add. Now only two= items further discussed below. >=20 > 11. Section 4.1: Sequence number (SN): 16 bits >=20 > Set and used in accordance with RFC 3550. >=20 > To my understanding when using single stream transmission the Decoding=20 > order is recovered soley from RTP sequence numbers and order of the=20 > NALUs within each RTP payload. This usage should be mentioned here as=20 > it affects the impact if anyone would inject other payloads in the=20 > same RTP packet stream. >=20 > [YK]: The current draft also supports interleaved transmission,=20 > wherein decoding order can be different from RTP sequence number=20 > order. Decoder order number (DON) values are used for=20 > de-packetization. That is really not clear. My reading of the draft, really indicated that yo= u removed all the interleaving support that was present in H.264's payload = format and that DONs really was to support scalability. If it is intended to support Interleaving, then the draft needs to be expli= cit about this support and discuss the limitations on that mechanism, inclu= ding how the use of the signalling attribute to negotiate buffer capability= . [YK]: OK, I will try to make this clearer along with your suggestion above. >=20 > 12. Section 4.1: I am missing the SSRC usage for this RTP payload. In=20 > SST that is the classical identification of a particular encoding of a=20 > media source. But for MST the SSRC usage is different as one needs to=20 > know which SSRC in which RTP sessions that are associated. I think=20 > this dependency needs to be noted. >=20 > [YK]: OK. I'd appreciate if you could provide some concrete language=20 > for this. Ok, a draft for you to word smith. synchronization source (SSRC): 32-bits Used to identify the source of the RTP packets. In SST mode a single SSR= C will be used for all the parts of a single encoding. In MST mode, each SS= RC is used for a packet stream containing a sub-set of encoding layers for = a single encoding. Thus, a receive is required to correctly associate the s= et of SSRCs that included parts of the same encoding. See Section X.Y for f= urther discussion of this association. What I considered but haven't included is if more about the dependency tree= needs to be expressed here, or if that is sufficient to be covered elsewhe= re? [YK]: Thanks! Just to confirm: are you suggesting that in MST (note that MS= T would mean multi-steam transmission, using either a single RTP session or= multiple RTP sessions) it is mandated that different packet streams must u= se different SSRC values (regardless of whether a single RTP session or mul= tiple RTP sessions are used)? As for multiple dependency trees (of layers),= I think that should be orthogonal to how they are partitioned into differe= nt packet streams, because in some scenarios it can be desirable to transmi= t two layers that are independent from each other in one packet stream. BR, YK From nobody Fri Apr 11 10:56:23 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 198FC1A0739 for ; Fri, 11 Apr 2014 10:56:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 2.727 X-Spam-Level: ** X-Spam-Status: No, score=2.727 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, MANGLED_TOOL=2.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zoPMfwEprIlF for ; Fri, 11 Apr 2014 10:56:17 -0700 (PDT) Received: from sabertooth01.qualcomm.com (sabertooth01.qualcomm.com [65.197.215.72]) by ietfa.amsl.com (Postfix) with ESMTP id 15E2B1A073C for ; Fri, 11 Apr 2014 10:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1397238977; x=1428774977; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=zgQYRnnEBC4jJUIoJhXVPsPRp31ovzwZD3hAnDB0H/I=; b=kCY18o9D4DZDUUAT57eiBS40BBi+PqM0OnpKhdRgSCjraYdA4QoByzRE 6DHi9aZtGyDnOX0O1YCfDQOD75R3L60mNIzP/Sgt0hUY6Zh+QgcOMgixU 2TujNqTegTODTJc+7SiCqtWwypukDXrgjnJHoXRMd9QoDaf7Cu9OWs669 4=; X-IronPort-AV: E=McAfee;i="5400,1158,7405"; a="61840730" Received: from ironmsg03-l.qualcomm.com ([172.30.48.18]) by sabertooth01.qualcomm.com with ESMTP; 11 Apr 2014 10:56:17 -0700 X-IronPort-AV: E=Sophos;i="4.97,843,1389772800"; d="scan'208";a="647674304" Received: from nasanexhc02.na.qualcomm.com ([172.30.48.24]) by Ironmsg03-L.qualcomm.com with ESMTP/TLS/RC4-SHA; 11 Apr 2014 10:57:57 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.137]) by NASANEXHC02.na.qualcomm.com ([172.30.48.24]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 10:56:15 -0700 From: "Wang, Ye-Kui" To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: [payload] Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxkvhCfBfHaXEuyjN/n0X9j0ZrPnXCAgBZg74CAJuPegA== Date: Fri, 11 Apr 2014 17:56:15 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> In-Reply-To: <53271A9D.4020906@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/L5EqS30lRPQf284ib7C7Ft2DDGs Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 17:56:21 -0000 Thanks more! See more discussions inline below. I also removed all issues w= here we have settled down. >=20 > 23. Section 5: Otherwise (sprop- depack-buf-nalus is equal to 0=20 > for an RTP stream), the transmission order of NAL units carried in=20 > the RTP stream MUST be the same as the NAL unit decoding order. >=20 > This fails to describe how you determines the NAL unit decoding order.=20 > I assume in RTP sequence order and for each payload in the order the=20 > NALUs are included? >=20 > [YK]: No, the NAL unit decoding order is described by the=20 > de-packetization process in Section 6, which applies in all cases,=20 > regardless of sprop-depack-buf-nalus being 0 or greater than 0. Yes,=20 > when sprop-depack-buf-nalus is equal to 0, the order would be as you=20 > assumed above. I don't see anything wrong or missing in this regard. > However, if you think more clarification is needed anywhere, please=20 > suggest and let us know where. What might not be clear in Section 5, is what is counted as transmission or= der within an aggregation packet. That is also not given for sprop-depack-buf-nalus=3D0 in Section 4.7. But, you might be correct that the actual text that I am missing is the one= that belongs to Section 6, on how to derive decoding order from received p= ackets when sprop-depack-buf-nalus=3D0. [YK] OK, I will try to make it clearer in Section 4.7 and Section 5 on what= is counted as transmission order when sprop-depack-buf-nalus=3D0 and withi= n an AP. >=20 > 24. Section 6: >=20 > When MST is in use, NAL units of all RTP streams are stored in the=20 > same de-packetization buffer. >=20 > I think it needs to be clarified that this is all RTP streams from the=20 > same encoder instance and media source. Not all RTP streams from=20 > multiple encoder instances and media sources. >=20 > [YK]: Yes. But on other hand, in my understanding, we always refer to=20 > the context of one "encoder instance and media source" in RTP payload=20 > format for a particular codec, right? Is it OK that we just make this=20 > generic clarification? "for a particular codec" can be misinterpreted as meaning some other format= , like H.264. Thus, I would prefer if you try to be more explicit so that i= t isn't misunderstood. Multi source and multi-stream (within a single media= type) hasn't been that common. OK, will try to clarify it along with your suggestion. > 26. Section 7.1: >=20 > profile-space, profile-id: Otherwise (the RTP stream is a dependent=20 > RTP stream), the following applies, with j being the value of the > sub-layer- id parameter: >=20 > o profile_space =3D sub_layer_profile_space[j] o profile_id =3D=20 > sub_layer_profile_idc[j] >=20 > The following text raises some concerns. A) These are defined as being=20 > stream specific values. It is unclear how one deals with these if one=20 > only want to configure the general capability within a RTP payload=20 > type. >=20 > [YK]: Note that these are "elementary stream" specific values, not=20 > "RTP stream" specific values. For example, if there are 3 temporal=20 > sub-layers, with TemporalId equal to 0, 1, and 2, respectively, and=20 > each is carried in one RTP stream, then the values of the RTP stream=20 > corresponding to the temporal sub-layers with TemporalId equal to 2=20 > are the property of the bitstream or elementary stream consisting of=20 > all the three temporal sub-layers. And since this RTP stream is not a=20 > dependent RTP stream (because no other RTP stream depends on it), the=20 > general capability is assigned. Well, I think you need to be explicit about what "stream" you mean in the t= ext throughout the whole payload format. [YK]: Agree. I will systematically check all instances of the "stream" word= ing and try to make the usages clear. >=20 > B) It is unclear if one may or are required to create different RTP=20 > payload types for the different scalability layers. From my=20 > perspective it is crucial that one can avoid a requirement to create=20 > layer specific payload types as this can result in total consumption=20 > of the available payload type space in an RTP session. >=20 > Thus, I see a need to clarify if and when one uses the above=20 > assignment. And how the counter part in any signalling exchange is=20 > supposed to interpret these cases. >=20 > [YK]: To me, when a spec does not disallow the use of different RTP=20 > payload types, which is the case with the current spec, then the use=20 > is allowed. Do you see any reason to impose an restriction on use of=20 > payload types for the different RTP streams in a multi-stream=20 > transmission? If not, I think we can just add a note to say that there=20 > is no restriction imposed herein for the use of payload types. No, I don't see a reason to impose a restriction. But, if something is inte= nded to be possible it is better to be explicit so that the lazy implemento= r doesn't miss that it is required to be capable of handling such a configu= ration. [YK]: Good, then let's add a note to say that this restriction is not in pl= ace. >=20 > 27. Section 7.1: profile-compatibility-indicator: >=20 > Isn't this a sprop parameter? >=20 > [YK]: It is the same as profile, tier, and level. And you can see in=20 > later text, it is part of the media format configuration. >=20 > Also it is not clear what "j" is: >=20 > If the RTP stream is not a dependent RTP stream, the following applies=20 > with j =3D 0..31: >=20 > o The 32 flags =3D general_profile_compatibility_flag[j] >=20 > Am I correct that j in this case are the integer identifying a=20 > particular HEVC profile? >=20 > [YK]: That is true. This is clear from the semantics of the flag as=20 > defined in the HEVC spec. We can try to add a bit more clarification=20 > on this. Thanks, a pointer is likely all that is necessary, but as someone who only = read a few very specific parts of the HEVC spec, I don't know these are def= ined in the HEVC specification. [YK]: OK, will make this clearer in the text. >=20 > 28. Section 7.1: profile-compatibility-indicator: >=20 > It is not clear to me that this parameter has long term applicability.=20 > If one includes this, is that a promise until signaling changes to not=20 > provide a stream that breaks the provided value? I wonder how one=20 > deals with the use cases like AD splicing. > One stream may ensure this, is it the role of the signalling to=20 > enforce that what one says applies, or will be renegotiated in=20 > signalling? >=20 > [YK]: Again, the nature and the usage of this parameter is the same as=20 > profile, tier, and level. So, yes, it would be a promise when=20 > provided, in either SDP offer/answer or declarative usage. Good, can you make this clearer somehow? [YK]: Yes, will try to make it clearer. >=20 > 29. >=20 > sub-layer-id: >=20 > This parameter MAY be used to indicate the highest allowed value of=20 > TID in the stream. When not present, the value of sub-layer-id is=20 > inferred to be equal to 6. >=20 > This parameter is written in a generic form, however the O/A Section=20 > indicates this to declare a property of the stream to be sent. But not=20 > explicit discussion of this parameter is provided. So what is the=20 > meaning of this parameter? >=20 > A) I promise that the streams I send does not use more than X=20 > sub-layers >=20 > B) I require the peer sending me a stream to not use more than X=20 > sub-layers. >=20 > [YK]: It is A. Would it be clear enough in your view if we add that=20 > this parameter shall not be used for any other purpose than indicating=20 > a property of the stream (which to me is sort of clear from its=20 > current semantics)? If it is only A, then I think it should have a sprop prefix. [YK]: Agree. Will add a sprop prefix for it. >=20 > 30. Section 7.1: sprop-vps, sprop-sps and sprop-pps: >=20 > I think one unclarity and one that existed in H.264 also, is the=20 > precedence between out-of-band and in-band parameter sets. And when=20 > they start to apply. The later is especially important in cases when=20 > you actually transmit a different set in a new signalling exchange=20 > then what has previously been used. >=20 > Can some clarification on this be included? >=20 > [YK]: OK, we can add a clarification to state that, out-of-band=20 > parameter sets can be put at the beginning of the bitstream output to=20 > the decoder, while in-band parameter sets can be sent to the decoder=20 > in the same way as the receiver outputs other types of NAL units from=20 > the de-packetization process. I am uncertain that what you try to say here. I interpret it as: As soon as you get the out-of-band parameters sets they can be added to the= front of the queue of NALUS that are given to the decoder. Is that what yo= u intended? [YK]: Exactly. Will add such a clarification (in a way that it is as clear = as possible). >=20 > 31. Section 7.1: Many of the parameters. I notices that quite a lot of=20 > the parameters are not explicitly defining the value range allowed for=20 > the parameters. This is especially true for integer style values. > See the many max- parameters. >=20 > [YK]: Right. We have been doing this all the time, including RFCs=20 > 6184/6190. Are you saying that from now that the value range of each=20 > parameter needs to be clearly specified in RTP payload formats? >From my perspective, being explicit about what values you expect reduces th= e risk for error or becoming subject to an attack as you can parse accordin= g to the spec, and if the input is malformed or malicious you can give an e= rror with a clear reason. If you are not explicit, you don't know what to e= xpect here. Thus, I think explicit ranges or at least data types, i.e. what= data and format to parse the value as. [YK]: Agree with your reasons. In the video coding standard spec, we always= clearly define a value range for each parameter unless the range is clear = (e.g. for an 8-bit unsigned integer parameter). Just that we did not follow= this practice earlier. I will try to come up with some reasonable value ra= nge for each parameter. >=20 > 32. Section 7.1: max-dpb: Formation error >=20 > [YK]: Sorry, I don't know what is error by comparing to the semantics=20 > to that of other parameters. Could you please clarify? Sorry, poorly worded. The lines on page 55 overflows the 72 column boundari= es. [YK]: OK. Will check to make sure this formatting error goes away. >=20 > 33. Section 7.1: >=20 > cap-parameter =3D tier-flag / level-id / max-lsr / max-lps / max-br >=20 > These ABNF elements are not defined. I think they need to be. >=20 > [YK]: This came from you guys. Can you please provide the missing=20 > definition? Yes, I know. Here is an proposal. tier-flag =3D "tier-flag" EQ ("0" / "1") level-id =3D "level-id" EQ 1*3DIGIT ; (0-255) max-lsr =3D "max-lsr" EQ 1*20DIGIT ; (0-18,446,744,073,709,551,615) max-lps =3D "max-lps" EQ 1*10DIGIT ; (0-4,294,967,295) max-br =3D "max-br" EQ 1*20DIGIT ; (0-18,446,744,073,709,551,615) EQ =3D "=3D" DIGIT =3D This is based on my understanding that tier-flag really only is a binary fl= ag. level-id is an unsigned 8-bit value. max-lsr and max-br clearly has the= potential for being larger than 32-bit integer, thus I used a 64-bit integ= er. I think 4G of pixel resolution is likely sufficient. Any errors in these assumptions? [YK]: Thanks a lot! I think your assumptions are correct. Will add these sy= ntax definitions. >=20 > 35. Section 7.2.2: >=20 > Informative note: The above parameters apply for any stream sent by a=20 > declaring entity with the same configuration; i.e. they are dependent=20 > on their source. Rather than being bound to the payload type, the=20 > values may have to be applied to another payload type when being sent,=20 > as they apply for the configuration. >=20 > I find this confusing. I don't know if it is a typo and should say=20 > "independent of their source", or if it is the definition of source=20 > here that is wrong. >=20 > It might be that the paragraph before the above: o The parameters=20 > sprop-depack-buf-nalus and sprop-depack-buf-bytes describe the=20 > properties of the RTP stream that the offerer or the answerer is=20 > sending for the media format configuration. This differs from the=20 > normal usage of the Offer/Answer parameters: normally such parameters=20 > declare the properties of the stream that the offerer or the answerer=20 > is able to receive. When dealing with HEVC, the offerer assumes that=20 > the answerer will be able to receive media encoded using the=20 > configuration being offered. >=20 > actually needs to separate between end point specific configurations=20 > that apply to the whole RTP session through the payload type or if is=20 > on specific media encoding basis and thus needs to be connected to the=20 > SSRC(s) that carries the particular media encoding. >=20 > [YK]: It seems to be a good catch. The same wording was inherited from=20 > RFC 6184, and appeared in RFC 3984 as well, for which you were a=20 > co-author. Could you authors of RFC 3984 (e.g. Stephan, Miska, > Magnus) try to help figure out what was the actual intent when this=20 > wording was firstly introduced into RFC 3984? What, you are holding me responsible for what I wrote 10 years ago? ;-) Considering this is SDP O/A, and looking at it again. I think it refer to t= hat an SDP agent sending a Offer or Answer with sprop-depack-buf-nalus and = sprop-depack-buf-bytes included, are responsible to ensure that any HEVC en= coding and its RTP packetization conforms to the declared parameters. This = applies to each encoding independently, even if multiple encodings are tran= smitted. Thus, I think this text would be clearer if one adds "endpoint" after sourc= e. [YK]: Thanks a lot! So holding you responsible for what you wrote 10 years = ago did help ;-) >=20 > 36. Section 7.2.2: >=20 > min_spatial_segmentation_idc equal to or larger than dec-=20 > parallel-cap.spatial-seg-idc of the capability point. A stream that=20 > is sent based on choosing a capability point with parallel tool > type 't' from dec-parallel-cap MUST have=20 > entropy_coding_sync_enabled_flag equal to 0 and=20 > min_spatial_segmentation_idc equal to or larger than dec-=20 > parallel-cap.spatial-seg-idc of the capability point. >=20 > It looks like someone managed to turn this text into straight left and=20 > right columns, this is not allowed in IETF, we use ragged rights. >=20 > [YK]: I guess you meant to align text only to the left margin, not to=20 > both left and right margins? If that is the case, we actually did this=20 > in all the places in the Word based template for preparing of the=20 > draft. Do you see any issues in other places as well? Can you please=20 > let us know some specific line for which the style should be changed? Correct, I have seen this in some few places. Lines: 33 58, for three lines 262 285 299, for 3 lines 376 etc. 3275, for 5-lines Search for ". " and you will find a large number of these issues. [YK]: Thanks! I see. I will change all alignments to ragged right alignment= . >=20 > 37. Section 7.2.2: >=20 > o An offerer has to include the size of the de-packetization buffer,=20 > sprop-depack-buf-bytes, and sprop-depack-buf-nalus, in the offer=20 > for an interleaved HEVC stream or for the MST transmission=20 > mode. To enable the offerer and answerer to inform each other about =20 > their capabilities for de-packetization buffering in receiving=20 > streams, both parties are RECOMMENDED to include depack-buf-cap. For=20 > interleaved streams or in MST, it is also RECOMMENDED to consider=20 > offering multiple payload types with different buffering requirements=20 > when the capabilities of the receiver are unknown. >=20 > This paragraph contains two mentions of "interleaved", the only two in=20 > the whole specification. I assume a copy and paste from RFC 6184 that=20 > wasn't converted properly. >=20 > [YK]: No they are intentionally left here, as interleaved=20 > packetization is allowed. Maybe we can add one sentence or two to make=20 > this more explicit. As said earlier, if this is a supported feature of the payload format, then= you need to discuss it explicitly. [YK]: Agreed. > 43. Section 7.2.2: >=20 > sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 > recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id --+ > | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | | | > profile-space C X C X P profile-id > C X C X P tier-flag C X C X P=20 > level-id C X C X P >=20 > C: configuration for sending and receiving streams >=20 > I don't get this usage or definition of C for these parameters. >=20 > So these parameters are receiver capabilities, and they needs to be=20 > symmetrically used in the signalling, however the actually used values=20 > in the transmission direction from an O/A agent only needs to be=20 > within the receivers provided parameters. Thus, they are not a=20 > "configuration" for the sending direction. >=20 > [YK]: These are configuration parameters, meaning indicating both what=20 > is to be sent and what is to be received for sendrecv and must be used=20 > symmetrically, and only the latter for recvonly. This is at a least wrong for the level-id which a peer agent in an O/A may = downgrade. The rest I agree the payload format itself requires to be kept i= n answer, or the corresponding PT dropped. [YK]: OK, level-id is downgradable but it is still a configuration, but a s= pecial configuration that can be downgraded. I'd suggest we keep using the = "C" denotation but adding a note to repeat what is said earlier that level = is downgradable, because using any other denotation is not correct. Please = let me know if you think this is not OK, and if so what kind of changes wou= ld you propose. >=20 > Also I really don't get the X's in the answer: recvonly,=20 > recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The=20 > answering agent is going to receive stream in both these cases, thus=20 > it needs to declare the payload type, and these parameters MUST be=20 > included (unless defaulted). So why is they marked with X? >=20 > [YK]: When recv-sub-layer-id is included in an SDP answer, it means=20 > that the answer chooses one of the multiple operation points=20 > corresponding to the offered or declared sub-layers in the sprop-vps,=20 > and in this case there is no need to repeat the information of=20 > profile, level etc. in the SDP, thus there are marked with X (MUST NOT=20 > be present). This appears very strange to me. The Offerer includes a VPS that contains o= ptions for what it can send. The answering party, i.e. which will receive t= his stream needs in accordance to SDP O/A rules declare which PTs it can re= ceive. This PT must be corresponding to the one that the Offerer proposed t= o send and in which the VPS was included. For me this implies the following: 1. The parameters indicating the basic configuraiton of the payload type, i= .e profile, level ,tier etc. MUST be present in the answer with the same va= lues as in the offer it responds to. Otherwise the PT matching rule fails. 2. In addition the PT to receive on MUST be declared as existing to ensure = that any middleboxes doesn't block it. Thus, explicit indication of what wi= ll go in it is really recommended, especially considering i all the other c= ases that information will be explicit here. Thus, I think the X's are plai= n wrong. Or I am still misunderstanding something here. [YK]: Let's use an example, which should help. Let's say the offer SDP incl= udes a VPS that provides two operation points, 720p@15fps and 720p@30fps, c= orresponding to recv-sub-layer-id 0 and 1, respectively. The profile, tier,= level, and so on for each operation point are signalled by the VPS. If rec= v-sub-layer-id=3D0 is included in an SDP answer, then that means the answer= er chooses to operate at 720p@15fps. In this case, the profile, tier, level= , and so on for the chosen operation point does not need to be explicitly s= ent again in the SDP, as it is clear what they are (they are signalled in t= he VPS in the SDP offer). So there is no ambiguity here. Are you suggesting= that in some use cases even there is no ambiguity the configuration inform= ation must always be explicitly signalled (i.e. values that are derivable)?= If yes, could you please explain in what use cases and why. Note that we h= ad the same mechanism specified for SVC in RFC 6190. Also, on PT use, we ha= ve allowed the use of a different PT value in an SDP answer than the matchi= ng PT value in the offer since at least RFC 3984 as long as the PT matching= is clear. >=20 > 44. Section 7.2.2: >=20 > sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 > recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id --+ > | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | | | > interop-constraints C X C X P=20 > profile-compatibility-indicator C X C X P >=20 > I don't understand how these parameters really are used for receiver=20 > capability declarations, so I don't understand why the Cs are C and=20 > not P. >=20 > [YK]: As explained earlier, these are also parts of the media type=20 > configuration, same as profile, tier, and level, and their usage are=20 > the same as those too. First of all these parameters are not included as the ones that have to be = symmetrically provided. I also think they could be SDP agent specifically g= iven. But, that can be discussed. But, according to the current text, I fai= l to see that these needs to be handled as the profile-id, level-id, etc. The only constraint I find for these two flags applies under multicast. Either they need text, or these C's need to be changed. [YK]: Maybe you were initially reading version -01 and did not check versio= n -02 on this part. In version -02, these parameters are included in the li= st of configuration parameters that are specified to be used symmetrically. BR, YK From nobody Fri Apr 11 13:17:51 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F7821A03D0 for ; Fri, 11 Apr 2014 13:17:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.272 X-Spam-Level: X-Spam-Status: No, score=-1.272 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, J_CHICKENPOX_19=0.6, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fgouhAoznxHq for ; Fri, 11 Apr 2014 13:17:42 -0700 (PDT) Received: from wolverine02.qualcomm.com (wolverine02.qualcomm.com [199.106.114.251]) by ietfa.amsl.com (Postfix) with ESMTP id BF1361A03B7 for ; Fri, 11 Apr 2014 13:17:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1397247461; x=1428783461; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=G1dTxI8FGqZN8i1IgE6uGaVzQRE/ASuejBSwX+FZrls=; b=arXzMKS8WiHNsqB6sta+KR2F3UpNp7mM3tBsZFsF71thJeeFnFESCKCd 7Y1gt5qCceAZU5Wg62kMOt4LKALqqh3Nyq0aC1UTtRGa7GxVwMAj5H9Qz ERV2/mQg0hnmTo7b1T7oUB2+GY30glh9jhD1NffweO5/Kk8ezYMw+mvo3 8=; X-IronPort-AV: E=McAfee;i="5400,1158,7405"; a="119333308" Received: from ironmsg04-l.qualcomm.com ([172.30.48.19]) by wolverine02.qualcomm.com with ESMTP; 11 Apr 2014 13:17:27 -0700 X-IronPort-AV: E=Sophos;i="4.97,843,1389772800"; d="scan'208,217";a="624066344" Received: from nasanexhc16.na.qualcomm.com ([10.45.158.213]) by Ironmsg04-L.qualcomm.com with ESMTP/TLS/RC4-SHA; 11 Apr 2014 13:19:07 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.137]) by nasanexhc16.na.qualcomm.com ([10.45.158.213]) with mapi id 14.03.0158.001; Fri, 11 Apr 2014 13:17:26 -0700 From: "Wang, Ye-Kui" To: Jonatan Samuelsson , Tom Kristensen <2mkristensen@gmail.com> Thread-Topic: [payload] #12 (rtp-h265): The max-fps parameter Thread-Index: AQHPM6U/tQCfPN0v2EGxR7CgdZmQz5rJwOwA//+msXCAB9L2AIAAAcMAgAAPZID//+cXUIAEolYAgAAFvQCAN0Ow8A== Date: Fri, 11 Apr 2014 20:17:26 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A83452011C7@nasanexd02f.na.qualcomm.com> References: <530F5A6E.3040900@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387CF4DA@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C5C94@ESESSMB109.ericsson.se> <634D3B5D82E3214DA9B6A36D5F50DB231C5CEC@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D5BA4@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C7BA8@ESESSMB109.ericsson.se> In-Reply-To: <634D3B5D82E3214DA9B6A36D5F50DB231C7BA8@ESESSMB109.ericsson.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: multipart/alternative; boundary="_000_8BA7D4CEACFFE04BA2D902BF11719A83452011C7nasanexd02fnaqu_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/nsQJuvCGlRihQhLUajuDt1BG7Fo Cc: "payload@ietf.org" , "Tom Kristensen \(tomkrist@cisco.com\)" Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 20:17:48 -0000 --_000_8BA7D4CEACFFE04BA2D902BF11719A83452011C7nasanexd02fnaqu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable We should also try to conclude herein. Tom, do you have any response to Jonatan's comment below? BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Jonatan Samuel= sson Sent: Friday, March 07, 2014 12:20 AM To: Tom Kristensen; Wang, Ye-Kui Cc: Tom Kristensen (tomkrist@cisco.com); payload@ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Thanks Tom for your feedback, For me to get a better understanding of if it is reasonable to make an exce= ption from the design principle of not re-defining level definitions from t= he video specification, it would be very valuable with answers to my earlie= r question: "If the motivation for max-fps is only for SDP O/A, why isn't the SDP attri= bute a=3Dframerate sufficient?" and to Magnus' earlier question: "The issues with the media pipeline has they been with the reception of the= high framerate of packets and forwarding them to the decoder or the playba= ck part, i.e. handling of the decoded picture?" Best Regards Jonatan From: Tom Kristensen [mailto:2mkristensen@gmail.com] Sent: den 7 mars 2014 09:00 To: Wang, Ye-Kui Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sandbakken = (geirsand); payload@ietf.org; Tom Kristensen (tomk= rist@cisco.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Well, I cannot see a reason for using SHOULD in offer/answer scenarios - as= Stephan argued for earlier here. If there is a limit and a reason for sign= alling max-fps due, there is little use waiving the flag with a SHOULD (whi= ch is often ignored, as we know). I think we should rather remove max-fps for declarative usage (or relax the= MUST to SHOULD there). -- Tom On 4 March 2014 18:16, Wang, Ye-Kui > wrote: Let me pull Tom into the discussion :-) Personally I am OK with the SHOULD = language here too as I don't have a good counter argument against Jonatan's= comment. Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this? BR, YK -----Original Message----- From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] Sent: Tuesday, March 04, 2014 2:43 AM To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1 plus= resolutions up to 4K" which can never put a level 4.1 compliant encoder in= a difficult situation (i.e. everything in level 4.1 is still allowed). If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient? Best Regards Jonatan -----Original Message----- From: Stephan Wenger [mailto:stewe@stewe.org] Sent: den 4 mars 2014 10:48 To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Let me push back once more. Overwriting the limits of the level definition= is common practice in the video conferencing industry, and has been done s= ince staid industry changed to H.264 (which first introduced the level conc= epts in video compression standards widely deployed in video conferencing).= That SHOULD will be ignored, or overwritten by system standards. Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation? (I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios. I only care about O/A scenarios.) Stephan On 4.3.14, 9:41, "Jonatan Samuelsson" > wrote: >Thanks Ye-Kui for the suggested changes. I think they improve the >situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" so that it becomes: > >"The encoder SHOULD use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters." > >I think this change would be needed in order to not redefine the level >definitions of the HEVC specification. A sender that for some reason is >not capable of producing lower picture rate (e.g. since the material is >pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not >be capable of adequately present all decoded pictures. > >Best Regards Jonatan > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, >Ye-Kui >Sent: den 27 februari 2014 19:20 >To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >I am with Magnus here. If a decoder cannot DECODE a picture rate >allowed by a level, then the decoder is not even a confirming decoder. >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason. > >I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good >semantics, but the use of the parameter is still practically the same. > >-----------------Start of the semantics, with suggested >changes-------------------------- >max-fps: >The value of max-fps is an integer indicating the maximum picture rate >in units of hundreds of pictures per second that can be efficiently >received (***CHANGE TO "effectively processed by the receiver"***). >The max-fps parameter MAY be used to signal that the receiver has a >constraint in that it is not capable of decoding (***CHANGE TO >"processing"***) video efficiently (***CHANGE TO "effectively"***) at >the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. > >The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on >maximum picture rate for all resolutions. > >Informative note: The max-fps parameter is semantically different from >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters. > >The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters. >-----------------End of the semantics, with suggested >changes-------------------------- > >Please express your opinion if this is not acceptable to you. > >BR, YK > > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Magnus >Westerlund >Sent: Thursday, February 27, 2014 7:32 AM >To: Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote: >> #12: The max-fps parameter >> >> Lessons learned from video conferencing using H.264 showed that a lot >>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when >>moving towards 120 fps in H.265. It might not be caused by >>limitations in the decoder itself, but that the surrounding media >>pipeline will not have been properly tested for 120 fps before being >>put into the wild. >> If we don't allow for a code point to be used by the "restricted" >> equipment, it will be required by the "flexible/proper" to include one. >> This is what happened with max-fps in H.264 enabling higher frame >>rates compared to lowering them. >> > >I do have concerns with defining a parameter that allow dynamically >sub-setting the profile and levels that has been defined for HEVC. This >can also create interoperability issues with any pre-encoded content to >a particular profile and level. > >The issues with the media pipeline has they been with the reception of >the high framerate of packets and forwarding them to the decoder or the >playback part, i.e. handling of the decoded picture? > >If it is predominately the later it appears very reasonable to have >this as an informative parameter. I will not make use of higher FPS >than X, if you send me higher than X I will need to reduce that to X or >lower before displaying it. That way we are not re-defining the >profiles, we are ensuring that we can handle content encoded within the >profile and still by taking care of the this in the end of your decoder >chain you can protect the rest of the media framework from not yet >supported frame-rates. > >Cheers > >Magnus Westerlund > >---------------------------------------------------------------------- >Services, Media and Network features, Ericsson Research EAB/TXM >---------------------------------------------------------------------- >Ericsson AB | Phone +46 10 7148287 >F=E4r=F6gatan 6 | Mobile +46 73 0949079 >SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com >---------------------------------------------------------------------- > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload _______________________________________________ payload mailing list payload@ietf.org https://www.ietf.org/mailman/listinfo/payload -- # Cisco | http://www.cisco.com/telepresence/ ## tomkrist@cisco.com | http://www.tandberg.co= m ### | http://folk.uio.no/tomkri/ --_000_8BA7D4CEACFFE04BA2D902BF11719A83452011C7nasanexd02fnaqu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

We should also try to con= clude herein.

 <= /p>

Tom, do you have any resp= onse to Jonatan’s comment below?

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.org] On Behalf Of Jonatan Samuelsson
Sent: Friday, March 07, 2014 12:20 AM
To: Tom Kristensen; Wang, Ye-Kui
Cc: Tom Kristensen (tomkrist@cisco.com); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

 

Thanks Tom for your feedb= ack,

 <= /p>

For me to get a better un= derstanding of if it is reasonable to make an exception from the design pri= nciple of not re-defining level definitions from the video specification, it would be very valuable with answers to my earlier questi= on:

 <= /p>

“If the moti= vation for max-fps is only for SDP O/A, why isn't the SDP attribute a=3Dfra= merate sufficient?”

 <= /p>

and to Magnus’ earl= ier question:

 <= /p>

“The issues = with the media pipeline has they been with the reception of the high framer= ate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?R= 21;

 <= /p>

Best Regards Jonatan=

 <= /p>

From: Tom Kris= tensen [mailto:2mkristensen@gmail= .com]
Sent: den 7 mars 2014 09:00
To: Wang, Ye-Kui
Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sand= bakken (geirsand); payload@ietf.org; Tom Kristensen (<= a href=3D"mailto:tomkrist@cisco.com">tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

 

Well, I cannot see a reason for using SHOULD in offe= r/answer scenarios - as Stephan argued for earlier here. If there is a limi= t and a reason for signalling max-fps due, there is little use waiving the = flag with a SHOULD (which is often ignored, as we know).

 

I think we should rather remove max-fps for declarat= ive usage (or relax the MUST to SHOULD there).

 

-- Tom

 

On 4 March 2014 18:16, Wang, Ye-Kui <yekuiw@qti.qualcomm.com> wrote:

Let me pull Tom into the discussion :-) Personally I= am OK with the SHOULD language here too as I don't have a good counter arg= ument against Jonatan's comment.

Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this?

BR, YK


-----Original Message-----
From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1= plus resolutions up to 4K" which can never put a level 4.1 compliant encoder in a difficult situation (i.e. eve= rything in level 4.1 is still allowed).

If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient?

Best Regards Jonatan

-----Original Message-----
From: Stephan Wenger [mailto:stewe@stewe= .org]
Sent: den 4 mars 2014 10:48
To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Let me push back once more.  Overwriting the limits of the level defin= ition is common practice in the video conferencing industry, and has been d= one since staid industry changed to H.264 (which first introduced the level= concepts in video compression standards widely deployed in video conferencing).  That SHOULD will be ignored,= or overwritten by system standards.
Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation?
(I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios.  I only care about O/A scenarios.) Ste= phan


On 4.3.14, 9:41, "Jonatan Samuelsson" <jonatan.samuelsson@ericsson.com>
wrote:

>Thanks Ye-Kui for the suggested changes. I think they improve the
>situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" s= o that it becomes:
>
>"The encoder SHOULD use a picture rate equal to or less than this = value.
>In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters."
>
>I think this change would be needed in order to not redefine the level<= br> >definitions of the HEVC specification. A sender that for some reason is=
>not capable of producing lower picture rate (e.g. since the material is=
>pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not=
>be capable of adequately present all decoded pictures.
>
>Best Regards Jonatan
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Wang,
>Ye-Kui
>Sent: den 27 februari 2014 19:20
>To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>I am with Magnus here. If a decoder cannot DECODE a picture rate
>allowed by a level, then the decoder is not even a confirming decoder.<= br> >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason.
>
>I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good
>semantics, but the use of the parameter is still practically the same.<= br> >
>-----------------Start of the semantics, with suggested
>changes--------------------------
>max-fps:
>The value of max-fps is an integer indicating the maximum picture rate<= br> >in units of hundreds of pictures per second that can be efficiently
>received (***CHANGE TO "effectively processed by the receiver"= ;***).
>The max-fps parameter MAY be used to signal that the receiver has a
>constraint in that it is not capable of decoding (***CHANGE TO
>"processing"***) video efficiently (***CHANGE TO  "= effectively"***) at
>the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. >
>The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on
>maximum picture rate for all resolutions.
>
>Informative note: The max-fps parameter is semantically different from<= br> >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that<= br> >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters.
>
>The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters.
>-----------------End of the semantics, with suggested
>changes--------------------------
>
>Please express your opinion if this is not acceptable to you.
>
>BR, YK
>
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Magnus
>Westerlund
>Sent: Thursday, February 27, 2014 7:32 AM
>To: Geir Sandbakken (geirsand); pay= load@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
>> #12: The max-fps parameter
>>
>> Lessons learned from video conferencing using H.264 showed that a = lot
>>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when=
>>moving towards 120 fps in H.265.  It might not be caused by >>limitations in the decoder itself,  but that the surrounding m= edia
>>pipeline will not have been properly tested for 120 fps before bein= g
>>put into the wild.
>> If we don't allow for a code point to be used by the "restric= ted"
>> equipment, it will be required by the "flexible/proper" = to include one.
>> This is what happened with max-fps in H.264 enabling higher frame<= br> >>rates compared to lowering them.
>>
>
>I do have concerns with defining a parameter that allow dynamically
>sub-setting the profile and levels that has been defined for HEVC. This=
>can also create interoperability issues with any pre-encoded content to=
>a particular profile and level.
>
>The issues with the media pipeline has they been with the reception of<= br> >the high framerate of packets and forwarding them to the decoder or the=
>playback part, i.e. handling of the decoded picture?
>
>If it is predominately the later it appears very reasonable to have
>this as an informative parameter. I will not make use of higher FPS
>than X, if you send me higher than X I will need to reduce that to X or=
>lower before displaying it. That way we are not re-defining the
>profiles, we are ensuring that we can handle content encoded within the=
>profile and still by taking care of the this in the end of your decoder=
>chain you can protect the rest of the media framework from not yet
>supported frame-rates.
>
>Cheers
>
>Magnus Westerlund
>
>----------------------------------------------------------------------<= br> >Services, Media and Network features, Ericsson Research EAB/TXM
>----------------------------------------------------------------------<= br> >Ericsson AB                 | P= hone  +46 10 7148287
>F=E4r=F6gatan 6                = | Mobile +46 73 0949079
>SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com
>----------------------------------------------------------------------<= br> >
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload

_______________________________________________
payload mailing list
payload@ietf.org
https://www.ietf.org/mailman/listinfo/payload



 

--
# Cisco                   &nbs= p;     |  http://www.cisco.com/telepresence/
## tomkrist@cisco.c= om  |  = http://www.tandberg.com
###                     &= nbsp;         |  http://folk.uio.no/tomkri/

--_000_8BA7D4CEACFFE04BA2D902BF11719A83452011C7nasanexd02fnaqu_-- From nobody Sun Apr 13 15:20:25 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DEE81A02D2 for ; Sun, 13 Apr 2014 15:20:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.099 X-Spam-Level: X-Spam-Status: No, score=0.099 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id spAbBbc0YgjY for ; Sun, 13 Apr 2014 15:20:17 -0700 (PDT) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0212.outbound.protection.outlook.com [207.46.163.212]) by ietfa.amsl.com (Postfix) with ESMTP id 4047D1A026B for ; Sun, 13 Apr 2014 15:20:16 -0700 (PDT) Received: from CO1PR07MB363.namprd07.prod.outlook.com (10.141.75.22) by CO1PR07MB362.namprd07.prod.outlook.com (10.141.75.21) with Microsoft SMTP Server (TLS) id 15.0.913.9; Sun, 13 Apr 2014 22:20:05 +0000 Received: from CO1PR07MB363.namprd07.prod.outlook.com ([169.254.3.69]) by CO1PR07MB363.namprd07.prod.outlook.com ([169.254.3.69]) with mapi id 15.00.0913.002; Sun, 13 Apr 2014 22:20:04 +0000 From: Stephan Wenger To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxmAvGxRz3pFU6I5jRks/D3MJsP7nGA Date: Sun, 13 Apr 2014 22:20:03 +0000 Message-ID: References: <5310AF7F.9050508@ericsson.com> In-Reply-To: <5310AF7F.9050508@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [50.174.124.226] x-forefront-prvs: 018093A9B5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(51854002)(199002)(189002)(51704005)(43784003)(24454002)(51444003)(54094003)(53754006)(80022001)(2656002)(66066001)(4396001)(87936001)(85852003)(83072002)(92726001)(86362001)(99396002)(99286001)(2201001)(92566001)(80976001)(20776003)(79102001)(77982001)(76482001)(83322001)(19580405001)(19580395003)(54356999)(76176999)(81542001)(50986999)(36756003)(31966008)(46102001)(74502001)(74662001)(81342001)(42262001); DIR:OUT; SFP:1101; SCL:1; SRVR:CO1PR07MB362; H:CO1PR07MB363.namprd07.prod.outlook.com; FPR:CE1CF2E5.93AA1341.A2D16D47.82F6AB02.20762; PTR:InfoNoRecords; A:0; MX:1; LANG:en; received-spf: None (: stewe.org does not designate permitted sender hosts) Content-Type: text/plain; charset="Windows-1252" Content-ID: <86950C83D7ADF843A779D3A360D6ABBA@namprd07.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: stewe.org Archived-At: http://mailarchive.ietf.org/arch/msg/payload/0yfMVJip_bezpLQd3cjSaH7nEZc Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Apr 2014 22:20:22 -0000 Hi all, especially Magnus, Magnus, thanks again for your careful review. Below inline some comments on Magnus issues #17 through #22, PACI-related. All other issues have been removed from this email. Regards, Stephan On 28.2.14, 7:47, "Magnus Westerlund" wrote: >[=8A] > >17. Section 4.9: > 0 1 2 3 > 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | RTP Header | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > |F| PACI=3D50 | LayerId | TID |A| Type | PHSsize |F0..2|X| > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | Payload Header Extension Structure (PHES) | > |=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D= +=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D+=3D| > | | > | PACI payload: NAL unit | > | . . . | > | | > | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > | :...OPTIONAL RTP padding | > +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- > >I wonder if one shouldn't mark the first 16 bits after the header as the >"payload header" and instead separate out the configuration of those for >this payload format. That would also resolve the next issue. Yes, I agree. In fact, it=B9s kind of odd that section 4.9 is the first instance where we have a figure explaining the (main) payload header itself. Of course, we have done so in textual form in some detail before, but not in a figure. My current thinking is that the right place for such a figure would be section 4.3, as these 16 bits are the same for all four packet types we have. And keep the section 4.9 figure as is, plus/minus the changes identified in issues #18 and #19 below. > >18. Section 4.9: > > PACI: 6 bits > Indicates a PACI, and must be 50. > >As this field is called Type in the other definition, why not call it >Type: 6 bits > Indicates a PACI, and must be 50. > >And to clarify the confusion with the PACI payload type, that Type field >should be called IType or something like that Both suggestions make sense (regardless of the doc structure fix of issue #17 above) and we will implement them. > >19. Section 4.9: > PHSsize: 5 bits > Indicates the total length of the PHES. The value is limited to > be less than or equal to 32 octets, to simplify encoder design > for MTU size matching. > > F0..2: 3 bits > Each of the three bits indicate, when set, the presence of an > optional field (or set of fields) in the PHES. > > X: 1 bit > The X bit, when set, indicates the presence of another octet > consisting of seven flags and another X bit, each of the seven > flags indicating the presence of more PHES fields (for future > extensions). > >The motivation of the PHSSize of limiting this to 32 octets is not >fulfilled. As the RTP payload still are variable size due to the X bit, >which adds one or more bytes. Thus if the goal is to make clear that >this will never be more than 32 bytes, than I think the bytes that X >signals must be included in the 32-bytes for the PHES. It was unwise of me to call this extension bit =B3X=B2. It is easily confu= sed with the X bit in the RTP header. We should replace the section 4.9 =B3X= =B2 with another suitable character. The reminder of my answer depends on how you interpret =B3X=B2: the PHS extension X, or the RTP header X. In case you meant PHS extension =B3X=B2: You are correct, and the fix is easy: We add a requirement that PHSsize shall include the extension flag octets (whose presence is indicated by those X bits). In case you meant RTP header =B3X=B2: The bug fix as above still needed. In addition, we could also add somewhere in section 4.1 (RTP header usage) a requirement that the total size of the RTP header, plus RTP header extension, plus the payload header and all its extensions shall not exceed a certain number. That number is not easy to calculate, as it depends on the use of PACI (and its extension mechanism), but also FU, aggregation, DON or no DON, and so on. I=B9m not in favor of doing that. However, what might be a good idea is to add an informative note into section 4.1 to the extent that a good encoder fills up the MTU whenever it can so to keep packetization overhead low, but in order to do so, it has to be able to predict or know the size of the payload header, which is highly variable due to syntax options defined in this specification, but also through the potential presence of RTP header extensions. Unrelated to the above: I can imagine cases where we would need only a single bit to signal something. In that case, it would be kind of odd to use a bit to signal the presence of another bit. I don=B9t think that needs to be specified in any detail, except by saying that the extension signalized by an F bit set to 1 can have a length of zero octets. Good idea? =20 > >20. Section 4.9: Forward compability for PHES. > >The content of the PHES field is determined using the F0..2 field and in >the future the octet(s) added using the X field, where each set bit >indicates what fields are included. However, this definition have some >severe implications regarding future compatibility. As each bit can >represent an unknown amount of fields a first gen decoder can't know how >many bits each of the bits represents. This may not matter much for the >first gen, as it only knows of the PHES data defined in this version. To >be able to decode these fields only a single thing is missing, an >assurance that the first defined data MUST be first in the PHES data >section when F0 is set. > >For a third generation implementation there is another implication. To >know where the F2 PHES payload starts it MUST know also what F0 and F1 >means in amount of fixed field data values. > >I am not that happy for this forward compatibility limiations. But, it >can work, but the rules for how to make it work MUST be explictely >stated now. > >1) The fields added for a particular FX bit MUST be fixed in length and >not depend on what other FX bits are set. >2) The FX bits must be assigned in order >3) An implementation that supports FX for any value will be required to >understand what fields are added and their size for all FX bits that are >X or less. Again, I agree with this bug report and with the proposed fix. I=B9m not sure that it is best placed in normative language, though (as your capitalization of MUST suggests). The three points made above are reminders for future spec developers, and not protocol constraints (well, mostly not). In video codec standards land, tons of such constraints are made in early versions of the spec, and never documented outside of meeting reports. Luckily, there is sufficient institutional memory in the organization that they are rarely forgotten. In the IETF in general and in the payload WG in particular, I=B9m not so sure about that institutional memory. Therefore, I propose to add your 3 constraints in informative language. See issue #21 below. If you have a to-do list for the next rev of the RTP payload how-to: I think that a section "Extension Mechanisms -- Gremlins here=B2 would be useful. Also, if there were a generic ftp payload format syntax for extensions, I think we would have used it (even if its overhead would have been larger than the one we developed here). Perhaps someone with enough time on hand could write something like that up, for future use? In video codec land, we did this once for H.263+ SEI messages, and re-used it thereafter basically forever. > >21. Section 4.9: I think the motivation for the PACI is missing. What >are the motivation for providing this information in the PACI and what >would any RTP endpoint or middlebox use the information for. I propose to add as second paragraph of 4.9: =B3 The basic payload header specified in this memo is intentionally limited to the 16 bits of the NAL unit header so to keep the packetization overhead to a minimum. However, cases have been identified where it is advisable to include control information in an easily accessible position in the packet header, despite the additional overhead. One such control information is the Temporal Scalability Control Information as specified in section 4.10/4.10.1 below. However, other types of control information may be identified in the future. For this reason, an extensible syntax is included that allows future versions of this specification, or companion specifications, to defined their own control information and include it into the payload header. When a spec developer devises a new syntax that takes advantage of the PACI extension mechanism, he/she must follow the constraints listed below; otherwise the extension mechanism may break. >22. Section 4.10: > >Can you please provide a name for the functionality set the F0 bit >represents so that we can use a name to reference it? =B3Temporal Scalability Control Information=B2. That should probably also be the section head of 4.10, or we could keep 4.10, saying that there is only a single extension defined in this memo, and then 4.10.1 =B3Temporal Scalability Control Information=B2. Preference= s, anyone? From nobody Mon Apr 14 02:49:33 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 992721A02A0 for ; Mon, 14 Apr 2014 02:49:31 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 1.46 X-Spam-Level: * X-Spam-Status: No, score=1.46 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_EQ_SE=0.35, HOST_MISMATCH_NET=0.311, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h4EIhDTb2Fpq for ; Mon, 14 Apr 2014 02:49:27 -0700 (PDT) Received: from sesbmg21.mgmt.ericsson.se (sesbmg21.ericsson.net [193.180.251.49]) by ietfa.amsl.com (Postfix) with ESMTP id 8771C1A03AF for ; Mon, 14 Apr 2014 02:49:25 -0700 (PDT) X-AuditID: c1b4fb31-b7f688e000003e64-3a-534baf220c23 Received: from ESESSHC004.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg21.mgmt.ericsson.se (Symantec Mail Security) with SMTP id FC.5F.15972.22FAB435; Mon, 14 Apr 2014 11:49:22 +0200 (CEST) Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.32) with Microsoft SMTP Server id 14.3.174.1; Mon, 14 Apr 2014 11:49:21 +0200 Message-ID: <534BAF21.3020901@ericsson.com> Date: Mon, 14 Apr 2014 11:49:21 +0200 From: Magnus Westerlund User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Wang, Ye-Kui" , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D100D@nasanexd02f.na.qualcomm.com> <53270527.9010407@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200E24@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A8345200E24@nasanexd02f.na.qualcomm.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrILMWRmVeSWpSXmKPExsUyM+Jvja7Seu9gg51NghanLu5jtbh08SyT xZM1x5gdmD2WLPnJ5LFo6jNGjy+XP7MFMEdx2aSk5mSWpRbp2yVwZfz4c4Kx4IJ4xZVtm1gb GL8JdTFyckgImEgsm/6PCcIWk7hwbz1bFyMXh5DASUaJz1efM0M4yxklNk7eygZSxSugLXH4 9V12EJtFQFXi4NlNrCA2m4CFxM0fjWA1ogLBEkvnLGaBqBeUODnzCQvIIBGBTYwSCxevASsS FnCU2NDYCrZaSOAmo0TXxVQQmxOoueXUd6AaDqCTxCV6GoNAwswCehJTrrYwQtjyEs1bZzND tGpLNDR1sE5gFJyFZN0sJC2zkLQsYGRexShZnFqclJtuZKiXm55bopdalJlcXJyfp1ecuokR GMwHt/w23ME48Zr9IUZpDhYlcV6G6Z1BQgLpiSWp2ampBalF8UWlOanFhxiZODilGhiXMb9a oRlX1ama9n3zkpkfp/E8Mtz0a5OYxIvX92u8q36unD71j8llxc3H5j56myIqdV72LtPjU3su Vd97L396VWW+2K4fGivfMsu3nNJZfl/DtHJGNMNWFVu2/2L8ej/C/3WErfGrYXpbzinz/6Ad w9HrmR2xnW6liQzTFk/kNPScuuk390MjJZbijERDLeai4kQA24kp9DQCAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/payload/g061teDbC5jhw8sAMiw-cVOnlRs Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 09:49:31 -0000 Hi, Response below to the second issue. /Magnus On 2014-04-11 18:47, Wang, Ye-Kui wrote: >> >> 12. Section 4.1: I am missing the SSRC usage for this RTP payload. >> In SST that is the classical identification of a particular >> encoding of a media source. But for MST the SSRC usage is different >> as one needs to know which SSRC in which RTP sessions that are >> associated. I think this dependency needs to be noted. >> >> [YK]: OK. I'd appreciate if you could provide some concrete >> language for this. > > Ok, a draft for you to word smith. > > synchronization source (SSRC): 32-bits > > Used to identify the source of the RTP packets. In SST mode a single > SSRC will be used for all the parts of a single encoding. In MST > mode, each SSRC is used for a packet stream containing a sub-set of > encoding layers for a single encoding. Thus, a receive is required to > correctly associate the set of SSRCs that included parts of the same > encoding. See Section X.Y for further discussion of this > association. > > What I considered but haven't included is if more about the > dependency tree needs to be expressed here, or if that is sufficient > to be covered elsewhere? > > [YK]: Thanks! Just to confirm: are you suggesting that in MST (note > that MST would mean multi-steam transmission, using either a single > RTP session or multiple RTP sessions) it is mandated that different > packet streams must use different SSRC values (regardless of whether > a single RTP session or multiple RTP sessions are used)? I think so, as people are likely to do MST in a single RTP session, the requirements and hooks for such a solution are needed. Thus, the same SSRC in different RTP sessions are not a sufficient solution. Requiring a solution that can deal with different SSRCs, thus ensure that even if the same SSRC is used different RTP sessions, that is just a unlikely event, not the basis for the solution. As for > multiple dependency trees (of layers), I think that should be > orthogonal to how they are partitioned into different packet streams, > because in some scenarios it can be desirable to transmit two layers > that are independent from each other in one packet stream. Yes, agree with that. However, I think my thoughts around multiple dependency trees arise from that these can be independent trees and thus individual sets of representation where either of the trees can be independently decoded. Thus, this gets into "simulcast" land. cheers Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 Färögatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Mon Apr 14 10:36:24 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4E4841A06C6 for ; Mon, 14 Apr 2014 10:36:21 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.873 X-Spam-Level: X-Spam-Status: No, score=-1.873 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RWBHhYiOzyWz for ; Mon, 14 Apr 2014 10:36:16 -0700 (PDT) Received: from wolverine01.qualcomm.com (wolverine01.qualcomm.com [199.106.114.254]) by ietfa.amsl.com (Postfix) with ESMTP id D09AD1A069E for ; Mon, 14 Apr 2014 10:36:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1397496973; x=1429032973; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=NNzlwEFRfsbbrSaQ8hBrafTrYWbpZeaKRojTEougVJ8=; b=aaramn8i3rXDeEh/C3p8t5JUTv9+rgOzEMybWCMb8Q+QLIgPw/CR4gJO L7oUFGwsdSH1q+jn8btnhCOto4a/kgzej6zGmLqqE0tCY2Kj6ZKE2SI7j obElSwUlmlQOr9/2jRWyQzna4xLUkn/3zOk5dNr77PZzht/BBV0Hrv00Y 4=; X-IronPort-AV: E=McAfee;i="5400,1158,7408"; a="28665399" Received: from ironmsg03-r.qualcomm.com ([172.30.46.17]) by wolverine01.qualcomm.com with ESMTP; 14 Apr 2014 10:36:04 -0700 X-IronPort-AV: E=Sophos;i="4.97,858,1389772800"; d="scan'208";a="661853640" Received: from nasanexhc11.na.qualcomm.com ([172.30.39.6]) by Ironmsg03-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 14 Apr 2014 10:39:41 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by nasanexhc11.na.qualcomm.com ([172.30.39.6]) with mapi id 14.03.0158.001; Mon, 14 Apr 2014 10:36:03 -0700 From: "Wang, Ye-Kui" To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: [payload] Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxkvhCfBfHaXEuyjN/n0X9j0ZrLRhBggBqeuYCAJvf4MIAEvP2AgAAMeQA= Date: Mon, 14 Apr 2014 17:36:02 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A8345219D98@nasanexd02f.na.qualcomm.com> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D100D@nasanexd02f.na.qualcomm.com> <53270527.9010407@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200E24@nasanexd02f.na.qualcomm.com> <534BAF21.3020901@ericsson.com> In-Reply-To: <534BAF21.3020901@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/vixne7H66DcvxSOLNGzcG95EJFI Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2014 17:36:21 -0000 Thanks again! Your suggestion on using different SSRC values for MST is goo= d to me. Cool - I think we are done with this thread - of course I still ne= ed to implement them into text. BR, YK -----Original Message----- From: Magnus Westerlund [mailto:magnus.westerlund@ericsson.com]=20 Sent: Monday, April 14, 2014 2:49 AM To: Wang, Ye-Kui; payload@ietf.org; draft-ietf-payload-rtp-h265@tools.ietf.= org Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 Hi, Response below to the second issue. /Magnus On 2014-04-11 18:47, Wang, Ye-Kui wrote: >>=20 >> 12. Section 4.1: I am missing the SSRC usage for this RTP payload. >> In SST that is the classical identification of a particular encoding=20 >> of a media source. But for MST the SSRC usage is different as one=20 >> needs to know which SSRC in which RTP sessions that are associated. I=20 >> think this dependency needs to be noted. >>=20 >> [YK]: OK. I'd appreciate if you could provide some concrete language=20 >> for this. >=20 > Ok, a draft for you to word smith. >=20 > synchronization source (SSRC): 32-bits >=20 > Used to identify the source of the RTP packets. In SST mode a single=20 > SSRC will be used for all the parts of a single encoding. In MST mode,=20 > each SSRC is used for a packet stream containing a sub-set of encoding=20 > layers for a single encoding. Thus, a receive is required to correctly=20 > associate the set of SSRCs that included parts of the same encoding.=20 > See Section X.Y for further discussion of this association. >=20 > What I considered but haven't included is if more about the dependency=20 > tree needs to be expressed here, or if that is sufficient to be=20 > covered elsewhere? >=20 > [YK]: Thanks! Just to confirm: are you suggesting that in MST (note=20 > that MST would mean multi-steam transmission, using either a single=20 > RTP session or multiple RTP sessions) it is mandated that different=20 > packet streams must use different SSRC values (regardless of whether a=20 > single RTP session or multiple RTP sessions are used)? I think so, as people are likely to do MST in a single RTP session, the req= uirements and hooks for such a solution are needed. Thus, the same SSRC in = different RTP sessions are not a sufficient solution. Requiring a solution = that can deal with different SSRCs, thus ensure that even if the same SSRC = is used different RTP sessions, that is just a unlikely event, not the basi= s for the solution. As for > multiple dependency trees (of layers), I think that should be=20 > orthogonal to how they are partitioned into different packet streams,=20 > because in some scenarios it can be desirable to transmit two layers=20 > that are independent from each other in one packet stream. Yes, agree with that. However, I think my thoughts around multiple dependen= cy trees arise from that these can be independent trees and thus individual= sets of representation where either of the trees can be independently deco= ded. Thus, this gets into "simulcast" land. cheers Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 F=E4r=F6gatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Wed Apr 23 14:01:00 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8CE371A0659 for ; Wed, 23 Apr 2014 14:00:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.972 X-Spam-Level: X-Spam-Status: No, score=-3.972 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, J_CHICKENPOX_19=0.6, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wL4hyLVmP1iG for ; Wed, 23 Apr 2014 14:00:51 -0700 (PDT) Received: from wolverine01.qualcomm.com (wolverine01.qualcomm.com [199.106.114.254]) by ietfa.amsl.com (Postfix) with ESMTP id 79C4C1A0656 for ; Wed, 23 Apr 2014 14:00:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1398286846; x=1429822846; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=iqrRg1zGwBLGARJVuxWiJ5Xle722QTh7iwak4pbe+A0=; b=pHMavF6vtljK7ZuTAQCjWkd2Ive5FKmrcrc8FzLqZFHjW3nW5CR5I5U4 ufv0uo///xlrw5jfa4szaBN0IU6HzediMJeCbQxiqD5RfDDMziptseQTd k73e6zbWaMYtCCbazj1XIZHYBFHCZPHj/gAsADEuWGCCZIOB8I+jJaHXC A=; X-IronPort-AV: E=McAfee;i="5400,1158,7417"; a="30708380" Received: from ironmsg03-r.qualcomm.com ([172.30.46.17]) by wolverine01.qualcomm.com with ESMTP; 23 Apr 2014 14:00:45 -0700 X-IronPort-AV: E=Sophos;i="4.97,913,1389772800"; d="scan'208,217";a="666865230" Received: from nasanexhc12.na.qualcomm.com ([172.30.39.187]) by Ironmsg03-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 23 Apr 2014 14:00:44 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by nasanexhc12.na.qualcomm.com ([172.30.39.187]) with mapi id 14.03.0158.001; Wed, 23 Apr 2014 14:00:44 -0700 From: "Wang, Ye-Kui" To: Jonatan Samuelsson , Tom Kristensen <2mkristensen@gmail.com> Thread-Topic: [payload] #12 (rtp-h265): The max-fps parameter Thread-Index: AQHPM6U/tQCfPN0v2EGxR7CgdZmQz5rJwOwA//+msXCAB9L2AIAAAcMAgAAPZID//+cXUIAEolYAgAAFvQCAN0Ow8IAS5CxA Date: Wed, 23 Apr 2014 21:00:42 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A834523BADE@nasanexd02f.na.qualcomm.com> References: <530F5A6E.3040900@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387CF4DA@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C5C94@ESESSMB109.ericsson.se> <634D3B5D82E3214DA9B6A36D5F50DB231C5CEC@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D5BA4@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C7BA8@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83452011C7@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A83452011C7@nasanexd02f.na.qualcomm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.48.1] Content-Type: multipart/alternative; boundary="_000_8BA7D4CEACFFE04BA2D902BF11719A834523BADEnasanexd02fnaqu_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/ATvIeJnRWkQnzJ096g1Tqp1z20Y Cc: "payload@ietf.org" , "Tom Kristensen \(tomkrist@cisco.com\)" Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 21:00:58 -0000 --_000_8BA7D4CEACFFE04BA2D902BF11719A834523BADEnasanexd02fnaqu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable We need to proceed. In case there is no more input, I suggest that we go ah= ead to change the semantics as follows: max-fps: The value of max-fps is an integer indicating the maximum picture rate in u= nits of hundreds of pictures per second that can be efficiently received ef= fectively processed by the receiver. The max-fps parameter MAY be used to = signal that the receiver has a constraint in that it is not capable of deco= ding video efficiently processing video effectively at the full picture rat= e that is implied by the highest level and, when present, one or more of th= e parameters max-lsr, max-lps, and max-br. The value of max-fps is not necessarily the picture rate at which the maxim= um picture size can be sent, it constitutes a constraint on maximum picture= rate for all resolutions. Informative note: The max-fps parameter is semantically different from max-= lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps = is used to signal a constraint, lowering the maximum picture rate from what= is implied by other parameters. The encoder MUST SHOULD use a picture rate equal to or less than this value= . An exception is when sending a pre-encoded bitstream the picture rate ma= y be greater than the value of max-fps. In cases where the max-fps paramet= er is absent the encoder is free to choose any picture rate according to th= e highest level and any signaled optional parameters. The value of max-fps MUST be smaller than or equal to the full picture rate= that is implied by the highest level and, when present, one or more of the= parameters max-lsr, max-lps, and max-br. [Note that addition of this sente= nce is to address the Magnus' comment on a value range for each parameter.] Jonatan, please suggest if you have a better exception case in mind (note t= hat nowadays generally an exception case needs to be clarified for each SHO= ULD wording). BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, Ye-Kui Sent: Friday, April 11, 2014 1:17 PM To: Jonatan Samuelsson; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter We should also try to conclude herein. Tom, do you have any response to Jonatan's comment below? BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Jonatan Samuel= sson Sent: Friday, March 07, 2014 12:20 AM To: Tom Kristensen; Wang, Ye-Kui Cc: Tom Kristensen (tomkrist@cisco.com); payload= @ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Thanks Tom for your feedback, For me to get a better understanding of if it is reasonable to make an exce= ption from the design principle of not re-defining level definitions from t= he video specification, it would be very valuable with answers to my earlie= r question: "If the motivation for max-fps is only for SDP O/A, why isn't the SDP attri= bute a=3Dframerate sufficient?" and to Magnus' earlier question: "The issues with the media pipeline has they been with the reception of the= high framerate of packets and forwarding them to the decoder or the playba= ck part, i.e. handling of the decoded picture?" Best Regards Jonatan From: Tom Kristensen [mailto:2mkristensen@gmail.com] Sent: den 7 mars 2014 09:00 To: Wang, Ye-Kui Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sandbakken = (geirsand); payload@ietf.org; Tom Kristensen (tomk= rist@cisco.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Well, I cannot see a reason for using SHOULD in offer/answer scenarios - as= Stephan argued for earlier here. If there is a limit and a reason for sign= alling max-fps due, there is little use waiving the flag with a SHOULD (whi= ch is often ignored, as we know). I think we should rather remove max-fps for declarative usage (or relax the= MUST to SHOULD there). -- Tom On 4 March 2014 18:16, Wang, Ye-Kui > wrote: Let me pull Tom into the discussion :-) Personally I am OK with the SHOULD = language here too as I don't have a good counter argument against Jonatan's= comment. Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this? BR, YK -----Original Message----- From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] Sent: Tuesday, March 04, 2014 2:43 AM To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1 plus= resolutions up to 4K" which can never put a level 4.1 compliant encoder in= a difficult situation (i.e. everything in level 4.1 is still allowed). If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient? Best Regards Jonatan -----Original Message----- From: Stephan Wenger [mailto:stewe@stewe.org] Sent: den 4 mars 2014 10:48 To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Let me push back once more. Overwriting the limits of the level definition= is common practice in the video conferencing industry, and has been done s= ince staid industry changed to H.264 (which first introduced the level conc= epts in video compression standards widely deployed in video conferencing).= That SHOULD will be ignored, or overwritten by system standards. Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation? (I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios. I only care about O/A scenarios.) Stephan On 4.3.14, 9:41, "Jonatan Samuelsson" > wrote: >Thanks Ye-Kui for the suggested changes. I think they improve the >situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" so that it becomes: > >"The encoder SHOULD use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters." > >I think this change would be needed in order to not redefine the level >definitions of the HEVC specification. A sender that for some reason is >not capable of producing lower picture rate (e.g. since the material is >pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not >be capable of adequately present all decoded pictures. > >Best Regards Jonatan > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, >Ye-Kui >Sent: den 27 februari 2014 19:20 >To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >I am with Magnus here. If a decoder cannot DECODE a picture rate >allowed by a level, then the decoder is not even a confirming decoder. >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason. > >I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good >semantics, but the use of the parameter is still practically the same. > >-----------------Start of the semantics, with suggested >changes-------------------------- >max-fps: >The value of max-fps is an integer indicating the maximum picture rate >in units of hundreds of pictures per second that can be efficiently >received (***CHANGE TO "effectively processed by the receiver"***). >The max-fps parameter MAY be used to signal that the receiver has a >constraint in that it is not capable of decoding (***CHANGE TO >"processing"***) video efficiently (***CHANGE TO "effectively"***) at >the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. > >The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on >maximum picture rate for all resolutions. > >Informative note: The max-fps parameter is semantically different from >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters. > >The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters. >-----------------End of the semantics, with suggested >changes-------------------------- > >Please express your opinion if this is not acceptable to you. > >BR, YK > > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Magnus >Westerlund >Sent: Thursday, February 27, 2014 7:32 AM >To: Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote: >> #12: The max-fps parameter >> >> Lessons learned from video conferencing using H.264 showed that a lot >>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when >>moving towards 120 fps in H.265. It might not be caused by >>limitations in the decoder itself, but that the surrounding media >>pipeline will not have been properly tested for 120 fps before being >>put into the wild. >> If we don't allow for a code point to be used by the "restricted" >> equipment, it will be required by the "flexible/proper" to include one. >> This is what happened with max-fps in H.264 enabling higher frame >>rates compared to lowering them. >> > >I do have concerns with defining a parameter that allow dynamically >sub-setting the profile and levels that has been defined for HEVC. This >can also create interoperability issues with any pre-encoded content to >a particular profile and level. > >The issues with the media pipeline has they been with the reception of >the high framerate of packets and forwarding them to the decoder or the >playback part, i.e. handling of the decoded picture? > >If it is predominately the later it appears very reasonable to have >this as an informative parameter. I will not make use of higher FPS >than X, if you send me higher than X I will need to reduce that to X or >lower before displaying it. That way we are not re-defining the >profiles, we are ensuring that we can handle content encoded within the >profile and still by taking care of the this in the end of your decoder >chain you can protect the rest of the media framework from not yet >supported frame-rates. > >Cheers > >Magnus Westerlund > >---------------------------------------------------------------------- >Services, Media and Network features, Ericsson Research EAB/TXM >---------------------------------------------------------------------- >Ericsson AB | Phone +46 10 7148287 >F=E4r=F6gatan 6 | Mobile +46 73 0949079 >SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com >---------------------------------------------------------------------- > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload _______________________________________________ payload mailing list payload@ietf.org https://www.ietf.org/mailman/listinfo/payload -- # Cisco | http://www.cisco.com/telepresence/ ## tomkrist@cisco.com | http://www.tandberg.co= m ### | http://folk.uio.no/tomkri/ --_000_8BA7D4CEACFFE04BA2D902BF11719A834523BADEnasanexd02fnaqu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

We need to proceed. In ca= se there is no more input, I suggest that we go ahead to change the semanti= cs as follows:

 <= /p>

max= -fps:

The v= alue of max-fps is an integer indicating the maximum picture rate in units = of hundreds of pictures per second that can be efficiently received effectively p= rocessed by the receiver.  The max-fps parameter MAY be used to= signal that the receiver has a constraint in that it is not capable of decoding video efficiently processing video effectively at the full picture rate that is implie= d by the highest level and, when present, one or more of the parameters max= -lsr, max-lps, and max-br.

=  

The v= alue of max-fps is not necessarily the picture rate at which the maximum pi= cture size can be sent, it constitutes a constraint on maximum picture rate= for all resolutions.

=  

Infor= mative note: The max-fps parameter is semantically different from max-lsr, = max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps is us= ed to signal a constraint, lowering the maximum picture rate from what is implied by other parameters.

=  

The e= ncoder MUST SHOULD use a picture rate equal to or less than this value.  An exception is when sending a pre-encoded bitstream the picture rate may b= e greater than the value of max-fps.  In cases where the max-fp= s parameter is absent the encoder is free to choose any picture rate accord= ing to the highest level and any signaled optional parameters.

=  

The value of max-fps MUST= be smaller than or equal to the full picture rate that is implied by the h= ighest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br. [Note that addition of this sentence is to address the Magnus’ commen= t on a value range for each parameter.]

 <= /p>

Jonatan, please suggest i= f you have a better exception case in mind (note that nowadays generally an= exception case needs to be clarified for each SHOULD wording).<= /span>

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.org] On Behalf Of Wang, Ye-Kui
Sent: Friday, April 11, 2014 1:17 PM
To: Jonatan Samuelsson; Tom Kristensen
Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

 

We should also try to con= clude herein.

 <= /p>

Tom, do you have any resp= onse to Jonatan’s comment below?

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.or= g] On Behalf Of Jonatan Samuelsson
Sent: Friday, March 07, 2014 12:20 AM
To: Tom Kristensen; Wang, Ye-Kui
Cc: Tom Kristensen (tomkrist@c= isco.com); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Thanks Tom for your feedb= ack,

 <= /p>

For me to get a better un= derstanding of if it is reasonable to make an exception from the design pri= nciple of not re-defining level definitions from the video specification, it would be very valuable with answers to my earlier questi= on:

 <= /p>

“If the moti= vation for max-fps is only for SDP O/A, why isn't the SDP attribute a=3Dfra= merate sufficient?”

 <= /p>

and to Magnus’ earl= ier question:

 <= /p>

“The issues = with the media pipeline has they been with the reception of the high framer= ate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?R= 21;

 <= /p>

Best Regards Jonatan

 <= /p>

From: Tom Kris= tensen [mailto:2mkristensen@gmail= .com]
Sent: den 7 mars 2014 09:00
To: Wang, Ye-Kui
Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sand= bakken (geirsand); payload@ietf.org; Tom Kristensen (<= a href=3D"mailto:tomkrist@cisco.com">tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Well, I cannot see a reason for using SHOULD in offe= r/answer scenarios - as Stephan argued for earlier here. If there is a limi= t and a reason for signalling max-fps due, there is little use waiving the = flag with a SHOULD (which is often ignored, as we know).

 

I think we should rather remove max-fps for declarat= ive usage (or relax the MUST to SHOULD there).

 

-- Tom

 

On 4 March 2014 18:16, Wang, Ye-Kui <yekuiw@qti.qualcomm.com> wrote:

Let me pull Tom into the discussion :-) Personally I= am OK with the SHOULD language here too as I don't have a good counter arg= ument against Jonatan's comment.

Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this?

BR, YK


-----Original Message-----
From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1= plus resolutions up to 4K" which can never put a level 4.1 compliant encoder in a difficult situation (i.e. eve= rything in level 4.1 is still allowed).

If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient?

Best Regards Jonatan

-----Original Message-----
From: Stephan Wenger [mailto:stewe@stewe= .org]
Sent: den 4 mars 2014 10:48
To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Let me push back once more.  Overwriting the limits of the level defin= ition is common practice in the video conferencing industry, and has been d= one since staid industry changed to H.264 (which first introduced the level= concepts in video compression standards widely deployed in video conferencing).  That SHOULD will be ignored,= or overwritten by system standards.
Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation?
(I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios.  I only care about O/A scenarios.) Ste= phan


On 4.3.14, 9:41, "Jonatan Samuelsson" <jonatan.samuelsson@ericsson.com>
wrote:

>Thanks Ye-Kui for the suggested changes. I think they improve the
>situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" s= o that it becomes:
>
>"The encoder SHOULD use a picture rate equal to or less than this = value.
>In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters."
>
>I think this change would be needed in order to not redefine the level<= br> >definitions of the HEVC specification. A sender that for some reason is=
>not capable of producing lower picture rate (e.g. since the material is=
>pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not=
>be capable of adequately present all decoded pictures.
>
>Best Regards Jonatan
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Wang,
>Ye-Kui
>Sent: den 27 februari 2014 19:20
>To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>I am with Magnus here. If a decoder cannot DECODE a picture rate
>allowed by a level, then the decoder is not even a confirming decoder.<= br> >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason.
>
>I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good
>semantics, but the use of the parameter is still practically the same.<= br> >
>-----------------Start of the semantics, with suggested
>changes--------------------------
>max-fps:
>The value of max-fps is an integer indicating the maximum picture rate<= br> >in units of hundreds of pictures per second that can be efficiently
>received (***CHANGE TO "effectively processed by the receiver"= ;***).
>The max-fps parameter MAY be used to signal that the receiver has a
>constraint in that it is not capable of decoding (***CHANGE TO
>"processing"***) video efficiently (***CHANGE TO  "= effectively"***) at
>the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. >
>The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on
>maximum picture rate for all resolutions.
>
>Informative note: The max-fps parameter is semantically different from<= br> >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that<= br> >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters.
>
>The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters.
>-----------------End of the semantics, with suggested
>changes--------------------------
>
>Please express your opinion if this is not acceptable to you.
>
>BR, YK
>
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Magnus
>Westerlund
>Sent: Thursday, February 27, 2014 7:32 AM
>To: Geir Sandbakken (geirsand); pay= load@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
>> #12: The max-fps parameter
>>
>> Lessons learned from video conferencing using H.264 showed that a = lot
>>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when=
>>moving towards 120 fps in H.265.  It might not be caused by >>limitations in the decoder itself,  but that the surrounding m= edia
>>pipeline will not have been properly tested for 120 fps before bein= g
>>put into the wild.
>> If we don't allow for a code point to be used by the "restric= ted"
>> equipment, it will be required by the "flexible/proper" = to include one.
>> This is what happened with max-fps in H.264 enabling higher frame<= br> >>rates compared to lowering them.
>>
>
>I do have concerns with defining a parameter that allow dynamically
>sub-setting the profile and levels that has been defined for HEVC. This=
>can also create interoperability issues with any pre-encoded content to=
>a particular profile and level.
>
>The issues with the media pipeline has they been with the reception of<= br> >the high framerate of packets and forwarding them to the decoder or the=
>playback part, i.e. handling of the decoded picture?
>
>If it is predominately the later it appears very reasonable to have
>this as an informative parameter. I will not make use of higher FPS
>than X, if you send me higher than X I will need to reduce that to X or=
>lower before displaying it. That way we are not re-defining the
>profiles, we are ensuring that we can handle content encoded within the=
>profile and still by taking care of the this in the end of your decoder=
>chain you can protect the rest of the media framework from not yet
>supported frame-rates.
>
>Cheers
>
>Magnus Westerlund
>
>----------------------------------------------------------------------<= br> >Services, Media and Network features, Ericsson Research EAB/TXM
>----------------------------------------------------------------------<= br> >Ericsson AB                 | P= hone  +46 10 7148287
>F=E4r=F6gatan 6                = | Mobile +46 73 0949079
>SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com
>----------------------------------------------------------------------<= br> >
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload

_______________________________________________
payload mailing list
payload@ietf.org
https://www.ietf.org/mailman/listinfo/payload



 

--
# Cisco                   &nbs= p;     |  http://www.cisco.com/telepresence/
## tomkrist@cisco.c= om  |  = http://www.tandberg.com
###                     &= nbsp;         |  http://folk.uio.no/tomkri/

--_000_8BA7D4CEACFFE04BA2D902BF11719A834523BADEnasanexd02fnaqu_-- From nobody Wed Apr 23 14:40:29 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9D3A71A06D2 for ; Wed, 23 Apr 2014 14:40:25 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LV_9R15vRVNF for ; Wed, 23 Apr 2014 14:40:21 -0700 (PDT) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0145.outbound.protection.outlook.com [207.46.163.145]) by ietfa.amsl.com (Postfix) with ESMTP id 7DF361A06CF for ; Wed, 23 Apr 2014 14:40:20 -0700 (PDT) Received: from CO1PR07MB363.namprd07.prod.outlook.com (10.141.75.22) by CO1PR07MB362.namprd07.prod.outlook.com (10.141.75.21) with Microsoft SMTP Server (TLS) id 15.0.921.12; Wed, 23 Apr 2014 21:40:11 +0000 Received: from CO1PR07MB363.namprd07.prod.outlook.com ([169.254.3.223]) by CO1PR07MB363.namprd07.prod.outlook.com ([169.254.3.223]) with mapi id 15.00.0929.001; Wed, 23 Apr 2014 21:40:11 +0000 From: Stephan Wenger To: "payload@ietf.org" Thread-Topic: H.265 payload format: new proposed codepoint sprop-sei Thread-Index: AQHPXzyadWEnrVkJGEuByf0BOmrzqw== Date: Wed, 23 Apr 2014 21:40:10 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [50.174.124.226] x-forefront-prvs: 01901B3451 x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(53754006)(199002)(189002)(164054003)(50986999)(54356999)(79102001)(66066001)(36756003)(99286001)(83322001)(76482001)(74662001)(74502001)(80022001)(87936001)(81542001)(31966008)(20776003)(2656002)(85852003)(81342001)(16236675002)(80976001)(46102001)(92726001)(4396001)(83072002)(92566001)(99396002)(561944002)(77982001)(86362001)(42262001); DIR:OUT; SFP:1101; SCL:1; SRVR:CO1PR07MB362; H:CO1PR07MB363.namprd07.prod.outlook.com; FPR:EC54F8EE.9FF29711.16E3737B.EE8F0F9.20485; MLV:sfv; PTR:InfoNoRecords; A:0; MX:1; LANG:en; received-spf: None (: stewe.org does not designate permitted sender hosts) Content-Type: multipart/alternative; boundary="_000_CF7D814646576stewesteweorg_" MIME-Version: 1.0 X-OriginatorOrg: stewe.org Archived-At: http://mailarchive.ietf.org/arch/msg/payload/RAbcUUxpUGunT1UsZ1KKFnEZxdQ Subject: [payload] H.265 payload format: new proposed codepoint sprop-sei X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2014 21:40:25 -0000 --_000_CF7D814646576stewesteweorg_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi all, I want to propose an additional code point sprop-sei for the H.265 payload = format, which is a WG draft. There is agreement between four of the five a= uthors (#5 is on vacation or traveling or something) that this is useful. = The ISO file format spec for H.265 has a similar code point fro similar pur= poses. The following would be added after the sprop-* parameter sets into section = 7.1 " sprop-sei: This parameter MAY be used to convey one or more SEI messages th= at describe bit stream characteristics. When present, a decoder can rely o= n the sub-stream or bitstream characteristics, as the case may be, that are= described in the SEI message for the entire duration of the session, indep= endently from the persistence rules as defined in HEVC. The value of the parameter is a comma-separated (',') list of base64 [RFC46= 48] representations of SEI message NAL units as specified in Section 7.3.2.= 4 of [HEVC]. Informative Note: Intentionally, no list of applicable or inapplicable SEI = messages is specified here. Conveying certain SEI messages in sprop-sei ma= y be sensible in some application scenarios and meaningless in others. How= ever, a few examples can be provided: In an environment where the encoded stream was created from film-based sour= ce material, and no splicing is going to occur during the lifetime of the s= ession, the film grain characteristics SEI message or the tone mapping info= rmation SEI message are likely meaningful, and sending them in sprop-sei ra= ther than in the video bitstream at each entry point may help saving bits a= nd allows to configure the renderer only once, avoiding unwanted artifacts. The structure of pictures information SEI message in sprop-sei can be used = to inform a decoder of the use of a certain structure of pictures (also kno= wn as Group of Pictures, GOP). Having such knowledge can be helpful for er= ror recovery. Examples for SEI messages that would be meaningless to be conveyed in sprop= -sei include the Decoded picture hash SEI message (it is close to impossibl= e that all decoded pictures have the same hash-tag), on a handheld device t= he Display orientation SEI message (as the display orientation may change w= hen the handheld device is turned aournd), or the filler payload SEI messag= e (as there is no point in wasting bits in SDP). " As a design choice, we could forego the comma-separated list, and just allo= w one such NAL unit, because SEI messages have their own encapsulation mech= anism (in contrast to parameter sets, multiple SEI messages can be conveyed= in a single NAL unit). OTOH, there is probably value in keeping design an= d implementation orthogonality between the various sprops, and that is why = the comma-separated list is proposed. I don't think there is much to say about this in the O/A section. With sp= rop-sei, we ONLY want to express stream characteristics as a subset of the = possible negotiated features (as indicated by profile, level, tier) and not= as an additional negotiation mechanism. Similarly, there is nothing to ad= d in the declarative use section. One wanted side effect of this proposal is the possible inclusion of user s= pecified SEI messages into the SDP. This would allow users, including down= stream standardization groups, to "label" bitstreams conforming to their sp= ecs (which they can already do today in the HEVC bitstream), and expose suc= h labeling in the SDP for use during session setup. One example would be t= o "label" a bitstream offered as conforming to an application layer specifi= cation. We plan to include the language above into the next revision of the draft, = unless there is noisy opposition here. So if you don't like what you have = read, please yell. Thanks, Stephan --_000_CF7D814646576stewesteweorg_ Content-Type: text/html; charset="iso-8859-1" Content-ID: <9CA763E21B8E634D9CF133F759C8EF14@namprd07.prod.outlook.com> Content-Transfer-Encoding: quoted-printable
Hi all,

I want to propose an addit= ional code point sprop-sei for the H.265 payload format, which is a WG draf= t.  There is agreement between four of the five authors (#5 is on vaca= tion or traveling or something) that this is useful.  The ISO file format spec for H.265 has a similar code poi= nt fro similar purposes.

The following would be add= ed after the sprop-* parameter sets into section 7.1

"
sprop-sei: This parameter MAY be used to convey= one or more SEI messages that describe bit stream characteristics.  W= hen present, a decoder can rely on the sub-stream or bitstream characterist= ics, as the case may be, that are described in the SEI message for the entire duration of the session, independently f= rom the persistence rules as defined in HEVC.
The value of the parameter is a comma-separated= (',') list of base64 [RFC4648] representations of SEI message NAL units as= specified in Section 7.3.2.4 of [HEVC].
 
Informative Note: Intentionally, no list of app= licable or inapplicable SEI messages is specified here.  Conveying cer= tain SEI messages in sprop-sei may be sensible in some application scenario= s and meaningless in others.  However, a few examples can be provided:
In an environment where the encoded stream was = created from film-based source material, and no splicing is going to occur = during the lifetime of the session, the film grain characteristics SEI mess= age or the tone mapping information SEI message are likely meaningful, and sending them in sprop-sei rather th= an in the video bitstream at each entry point may help saving bits and allo= ws to configure the renderer only once, avoiding unwanted artifacts.  =
The structure of pictures information SEI messa= ge in sprop-sei can be used to inform a decoder of the use of a certain str= ucture of pictures (also known as Group of Pictures, GOP).  Having suc= h knowledge can be helpful for error recovery.
Examples for SEI messages that would be meaning= less to be conveyed in sprop-sei include the Decoded picture hash SEI messa= ge (it is close to impossible that all decoded pictures have the same hash-= tag), on a handheld device the Display orientation SEI message (as the display orientation may change when the ha= ndheld device is turned aournd), or the filler payload SEI message (as ther= e is no point in wasting bits in SDP).
" 

As a design choice, we cou= ld forego the comma-separated list, and just allow one such NAL unit, becau= se SEI messages have their own encapsulation mechanism (in contrast to para= meter sets, multiple SEI messages can be conveyed in a single NAL unit).  OTOH, there is probably value= in keeping design and implementation orthogonality between the various spr= ops, and that is why the comma-separated list is proposed.
 
I don’t think there = is much to say about this in the O/A section.   With sprop-sei, we ONL= Y want to express stream characteristics as a subset of the possible negoti= ated features (as indicated by profile, level, tier) and not as an additional negotiation mechanism.  Similarly, the= re is nothing to add in the declarative use section.

One wanted side effect of = this proposal is the possible inclusion of user specified SEI messages into= the SDP.  This would allow users, including downstream standardizatio= n groups, to “label” bitstreams conforming to their specs (which they can already do today in the HEVC bitstream), an= d expose such labeling in the SDP for use during session setup.  One e= xample would be to “label” a bitstream offered as conforming to= an application layer specification. 

We plan to include the lan= guage above into the next revision of the draft, unless there is noisy oppo= sition here.  So if you don’t like what you have read, please ye= ll.

Thanks,
Stephan

--_000_CF7D814646576stewesteweorg_-- From nobody Wed Apr 23 23:41:37 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6A31C1A07B9 for ; Wed, 23 Apr 2014 23:41:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.6 X-Spam-Level: X-Spam-Status: No, score=-3.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, J_CHICKENPOX_19=0.6, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ayfm_sWYu8Kt for ; Wed, 23 Apr 2014 23:41:24 -0700 (PDT) Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) by ietfa.amsl.com (Postfix) with ESMTP id 1646D1A07B5 for ; Wed, 23 Apr 2014 23:41:22 -0700 (PDT) X-AuditID: c1b4fb25-f798a6d000005ede-28-5358b20ca3f0 Received: from ESESSHC005.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id 32.49.24286.C02B8535; Thu, 24 Apr 2014 08:41:16 +0200 (CEST) Received: from ESESSMB109.ericsson.se ([169.254.9.25]) by ESESSHC005.ericsson.se ([153.88.183.33]) with mapi id 14.03.0174.001; Thu, 24 Apr 2014 08:41:15 +0200 From: Jonatan Samuelsson To: "Wang, Ye-Kui" , Tom Kristensen <2mkristensen@gmail.com> Thread-Topic: [payload] #12 (rtp-h265): The max-fps parameter Thread-Index: AQHPM6U/FZ37D8KebEWrOTyjcR0I9prJKgwAgAAu2ACAB1WSwP//9wAAgAAcGYCAAGFSgIAEG2cAgAASVMCAN6yuAIAS6BEAgACndvA= Date: Thu, 24 Apr 2014 06:41:15 +0000 Message-ID: <634D3B5D82E3214DA9B6A36D5F50DB2321A64C@ESESSMB109.ericsson.se> References: <530F5A6E.3040900@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387CF4DA@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C5C94@ESESSMB109.ericsson.se> <634D3B5D82E3214DA9B6A36D5F50DB231C5CEC@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D5BA4@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C7BA8@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83452011C7@nasanexd02f.na.qualcomm.com> <8BA7D4CEACFFE04BA2D902BF11719A834523BADE@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A834523BADE@nasanexd02f.na.qualcomm.com> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [153.88.183.147] Content-Type: multipart/alternative; boundary="_000_634D3B5D82E3214DA9B6A36D5F50DB2321A64CESESSMB109ericsso_" MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJIsWRmVeSWpSXmKPExsUyM+JvjS7Ppohgg6NvxS22HH/HYnHp4lkm iytHfrFZPFlzjNmBxWPK742sHjtn3WX3WLLkJ5PHoqnPGANYorhsUlJzMstSi/TtErgyDi/4 zV4w5wFTxfOnB9kaGG9sYOpi5OSQEDCR2L/7GiuELSZx4d56ti5GLg4hgaOMErNe7WCGcBYz SnTNfgBUxcHBJmAl8f1FBEiDiEC4xOzTT9lBbGaBdIkZ09uZQWxhARuJt4+7WCFqbCUa5z9g hLDLJF58uMQOMoZFQFVi1hMfEJNXwFvieJ8VxKalrBLLO3+ClXMKBEucOfgCbCSjgKzE/e/3 WCBWiUvcejIf6n4BiSV7zjND2KISLx//A7tSQkBJYtrWNIjyfIn/Ly6BtfIKCEqcnPmEZQKj 6Cwkk2YhKZuFpAwiridxY+oUNghbW2LZwtfMELauxIx/h1iQxRcwsq9iFC1OLU7KTTcy1kst ykwuLs7P08tLLdnECIzLg1t+q+5gvPzG8RCjAAejEg8vm5pfsBBrYllxZe4hRmkOFiVx3i+3 fIKFBNITS1KzU1MLUovii0pzUosPMTJxcEo1MOptt9rxa2ZXj4LjjFlvPWQeH1gSWiud+mfz jsl73u/n8ToSldrXeu6jXvt3XtlvT6b+1p1xkflTjNe8tZEyfzdoPzrp0/4q1uxsB2Pt8wi3 VJ0dFnt2/ijaFruv6kXR3WWG232zdBJmywgLzFzVuG5G8sO2xGV1V14kT5P3v1LqV3LC+FWQ q40SS3FGoqEWc1FxIgAAG4ywrAIAAA== Archived-At: http://mailarchive.ietf.org/arch/msg/payload/XuH4gnI-sjTRx9kDJseA5SslSug Cc: "payload@ietf.org" , "Tom Kristensen \(tomkrist@cisco.com\)" Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 06:41:34 -0000 --_000_634D3B5D82E3214DA9B6A36D5F50DB2321A64CESESSMB109ericsso_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks Ye-Kui, Your proposed text looks good. Perhaps the exception sentence can be modifi= ed slightly: An exception is when sending a pre-encoded bitstream, in which= case the picture rate may be greater than the value of max-fps. When reading this section again another question comes to my mind regarding= "The value of max-fps is an integer indicating the maximum picture rate in= units of hundreds of pictures per second". It seems to me that this could = be interpreted as the value 1 means "one hundred pictures per second", the = value 2 means "two hundred pictures per second" etc. I'm sure this is not t= he intention. Wouldn't it be better to just write: "The value of max-fps is= an integer indicating the maximum picture rate in units of pictures per se= cond" so that the value 1 means "one picture per second", the value 2 means= "two pictures per second" etc. (Or maybe the intention was to say "in units of pictures per hundred second= s" in order to be able to indicate fractional frame rates such as 29.97 fps= by sending the value 2997?) Best Regards Jonatan From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] Sent: den 23 april 2014 23:01 To: Jonatan Samuelsson; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com) Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter We need to proceed. In case there is no more input, I suggest that we go ah= ead to change the semantics as follows: max-fps: The value of max-fps is an integer indicating the maximum picture rate in u= nits of hundreds of pictures per second that can be efficiently received ef= fectively processed by the receiver. The max-fps parameter MAY be used to = signal that the receiver has a constraint in that it is not capable of deco= ding video efficiently processing video effectively at the full picture rat= e that is implied by the highest level and, when present, one or more of th= e parameters max-lsr, max-lps, and max-br. The value of max-fps is not necessarily the picture rate at which the maxim= um picture size can be sent, it constitutes a constraint on maximum picture= rate for all resolutions. Informative note: The max-fps parameter is semantically different from max-= lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps = is used to signal a constraint, lowering the maximum picture rate from what= is implied by other parameters. The encoder MUST SHOULD use a picture rate equal to or less than this value= . An exception is when sending a pre-encoded bitstream the picture rate ma= y be greater than the value of max-fps. In cases where the max-fps paramet= er is absent the encoder is free to choose any picture rate according to th= e highest level and any signaled optional parameters. The value of max-fps MUST be smaller than or equal to the full picture rate= that is implied by the highest level and, when present, one or more of the= parameters max-lsr, max-lps, and max-br. [Note that addition of this sente= nce is to address the Magnus' comment on a value range for each parameter.] Jonatan, please suggest if you have a better exception case in mind (note t= hat nowadays generally an exception case needs to be clarified for each SHO= ULD wording). BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, Ye-Kui Sent: Friday, April 11, 2014 1:17 PM To: Jonatan Samuelsson; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cis= co.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter We should also try to conclude herein. Tom, do you have any response to Jonatan's comment below? BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Jonatan Samuel= sson Sent: Friday, March 07, 2014 12:20 AM To: Tom Kristensen; Wang, Ye-Kui Cc: Tom Kristensen (tomkrist@cisco.com); payload= @ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Thanks Tom for your feedback, For me to get a better understanding of if it is reasonable to make an exce= ption from the design principle of not re-defining level definitions from t= he video specification, it would be very valuable with answers to my earlie= r question: "If the motivation for max-fps is only for SDP O/A, why isn't the SDP attri= bute a=3Dframerate sufficient?" and to Magnus' earlier question: "The issues with the media pipeline has they been with the reception of the= high framerate of packets and forwarding them to the decoder or the playba= ck part, i.e. handling of the decoded picture?" Best Regards Jonatan From: Tom Kristensen [mailto:2mkristensen@gmail.com] Sent: den 7 mars 2014 09:00 To: Wang, Ye-Kui Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sandbakken = (geirsand); payload@ietf.org; Tom Kristensen (tomk= rist@cisco.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Well, I cannot see a reason for using SHOULD in offer/answer scenarios - as= Stephan argued for earlier here. If there is a limit and a reason for sign= alling max-fps due, there is little use waiving the flag with a SHOULD (whi= ch is often ignored, as we know). I think we should rather remove max-fps for declarative usage (or relax the= MUST to SHOULD there). -- Tom On 4 March 2014 18:16, Wang, Ye-Kui > wrote: Let me pull Tom into the discussion :-) Personally I am OK with the SHOULD = language here too as I don't have a good counter argument against Jonatan's= comment. Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this? BR, YK -----Original Message----- From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] Sent: Tuesday, March 04, 2014 2:43 AM To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1 plus= resolutions up to 4K" which can never put a level 4.1 compliant encoder in= a difficult situation (i.e. everything in level 4.1 is still allowed). If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient? Best Regards Jonatan -----Original Message----- From: Stephan Wenger [mailto:stewe@stewe.org] Sent: den 4 mars 2014 10:48 To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Let me push back once more. Overwriting the limits of the level definition= is common practice in the video conferencing industry, and has been done s= ince staid industry changed to H.264 (which first introduced the level conc= epts in video compression standards widely deployed in video conferencing).= That SHOULD will be ignored, or overwritten by system standards. Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation? (I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios. I only care about O/A scenarios.) Stephan On 4.3.14, 9:41, "Jonatan Samuelsson" > wrote: >Thanks Ye-Kui for the suggested changes. I think they improve the >situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" so that it becomes: > >"The encoder SHOULD use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters." > >I think this change would be needed in order to not redefine the level >definitions of the HEVC specification. A sender that for some reason is >not capable of producing lower picture rate (e.g. since the material is >pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not >be capable of adequately present all decoded pictures. > >Best Regards Jonatan > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, >Ye-Kui >Sent: den 27 februari 2014 19:20 >To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >I am with Magnus here. If a decoder cannot DECODE a picture rate >allowed by a level, then the decoder is not even a confirming decoder. >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason. > >I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good >semantics, but the use of the parameter is still practically the same. > >-----------------Start of the semantics, with suggested >changes-------------------------- >max-fps: >The value of max-fps is an integer indicating the maximum picture rate >in units of hundreds of pictures per second that can be efficiently >received (***CHANGE TO "effectively processed by the receiver"***). >The max-fps parameter MAY be used to signal that the receiver has a >constraint in that it is not capable of decoding (***CHANGE TO >"processing"***) video efficiently (***CHANGE TO "effectively"***) at >the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. > >The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on >maximum picture rate for all resolutions. > >Informative note: The max-fps parameter is semantically different from >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters. > >The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters. >-----------------End of the semantics, with suggested >changes-------------------------- > >Please express your opinion if this is not acceptable to you. > >BR, YK > > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Magnus >Westerlund >Sent: Thursday, February 27, 2014 7:32 AM >To: Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote: >> #12: The max-fps parameter >> >> Lessons learned from video conferencing using H.264 showed that a lot >>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when >>moving towards 120 fps in H.265. It might not be caused by >>limitations in the decoder itself, but that the surrounding media >>pipeline will not have been properly tested for 120 fps before being >>put into the wild. >> If we don't allow for a code point to be used by the "restricted" >> equipment, it will be required by the "flexible/proper" to include one. >> This is what happened with max-fps in H.264 enabling higher frame >>rates compared to lowering them. >> > >I do have concerns with defining a parameter that allow dynamically >sub-setting the profile and levels that has been defined for HEVC. This >can also create interoperability issues with any pre-encoded content to >a particular profile and level. > >The issues with the media pipeline has they been with the reception of >the high framerate of packets and forwarding them to the decoder or the >playback part, i.e. handling of the decoded picture? > >If it is predominately the later it appears very reasonable to have >this as an informative parameter. I will not make use of higher FPS >than X, if you send me higher than X I will need to reduce that to X or >lower before displaying it. That way we are not re-defining the >profiles, we are ensuring that we can handle content encoded within the >profile and still by taking care of the this in the end of your decoder >chain you can protect the rest of the media framework from not yet >supported frame-rates. > >Cheers > >Magnus Westerlund > >---------------------------------------------------------------------- >Services, Media and Network features, Ericsson Research EAB/TXM >---------------------------------------------------------------------- >Ericsson AB | Phone +46 10 7148287 >F=E4r=F6gatan 6 | Mobile +46 73 0949079 >SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com >---------------------------------------------------------------------- > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload _______________________________________________ payload mailing list payload@ietf.org https://www.ietf.org/mailman/listinfo/payload -- # Cisco | http://www.cisco.com/telepresence/ ## tomkrist@cisco.com | http://www.tandberg.co= m ### | http://folk.uio.no/tomkri/ --_000_634D3B5D82E3214DA9B6A36D5F50DB2321A64CESESSMB109ericsso_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Thanks Ye-Kui,=

 <= /p>

Your proposed text looks = good. Perhaps the exception sentence can be modified slightly: An exception = is when sending a pre-encoded bitstream, in which= case the picture rate may be greater than the value of max-fps.

 <= /p>

When reading this section= again another question comes to my mind regarding “The value = of max-fps is an integer indicating the maximum picture rate in units of hundreds of pictures per second”= . It seems to me that this could be interpreted as the value 1 means “= ;one hundred pictures per second”, the value 2 means “two hundr= ed pictures per second” etc. I’m sure this is not the intention. = Wouldn’t it be better to just write: “The value of max-f= ps is an integer indicating the maximum picture rate in units of pictures p= er second” so that the value 1 means “one picture per second”, the value = 2 means “two pictures per second” etc.

 <= /p>

(Or maybe the intention w= as to say “in units of pictures per hundred seconds” in order t= o be able to indicate fractional frame rates such as 29.97 fps by sending the value 2997?)

 <= /p>

Best Regards Jonatan=

 <= /p>

From: Wang, Ye= -Kui [mailto:yekuiw@qti.qualcomm.com]
Sent: den 23 april 2014 23:01
To: Jonatan Samuelsson; Tom Kristensen
Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com)
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

 

We need to proceed. In ca= se there is no more input, I suggest that we go ahead to change the semanti= cs as follows:

 <= /p>

max= -fps:

The v= alue of max-fps is an integer indicating the maximum picture rate in units = of hundreds of pictures per second that can be efficiently received effectively p= rocessed by the receiver.  The max-fps parameter MAY be used to= signal that the receiver has a constraint in that it is not capable of decoding video efficiently processing video effectively at the full picture rate that is implie= d by the highest level and, when present, one or more of the parameters max= -lsr, max-lps, and max-br.

=  

The v= alue of max-fps is not necessarily the picture rate at which the maximum pi= cture size can be sent, it constitutes a constraint on maximum picture rate= for all resolutions.

=  

Infor= mative note: The max-fps parameter is semantically different from max-lsr, = max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps is us= ed to signal a constraint, lowering the maximum picture rate from what is implied by other parameters.

=  

The e= ncoder MUST SHOULD use a picture rate equal to or less than this value.  An exception is when sending a pre-encoded bitstream the picture rate may b= e greater than the value of max-fps.  In cases where the max-fp= s parameter is absent the encoder is free to choose any picture rate accord= ing to the highest level and any signaled optional parameters.

=  

The value of max-fps MUST= be smaller than or equal to the full picture rate that is implied by the h= ighest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br. [Note that addition of this sentence is to address the Magnus’ commen= t on a value range for each parameter.]

 <= /p>

Jonatan, please suggest i= f you have a better exception case in mind (note that nowadays generally an= exception case needs to be clarified for each SHOULD wording).<= /span>

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.or= g] On Behalf Of Wang, Ye-Kui
Sent: Friday, April 11, 2014 1:17 PM
To: Jonatan Samuelsson; Tom Kristensen
Cc: payload@ietf.org; Tom Kr= istensen (tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

 

We should also try to con= clude herein.

 <= /p>

Tom, do you have any resp= onse to Jonatan’s comment below?

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.or= g] On Behalf Of Jonatan Samuelsson
Sent: Friday, March 07, 2014 12:20 AM
To: Tom Kristensen; Wang, Ye-Kui
Cc: Tom Kristensen (tomkrist@c= isco.com); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Thanks Tom for your feedb= ack,

 <= /p>

For me to get a better un= derstanding of if it is reasonable to make an exception from the design pri= nciple of not re-defining level definitions from the video specification, it would be very valuable with answers to my earlier questi= on:

 <= /p>

“If the moti= vation for max-fps is only for SDP O/A, why isn't the SDP attribute a=3Dfra= merate sufficient?”

 <= /p>

and to Magnus’ earl= ier question:

 <= /p>

“The issues = with the media pipeline has they been with the reception of the high framer= ate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?R= 21;

 <= /p>

Best Regards Jonatan

 <= /p>

From: Tom Kris= tensen [mailto:2mkristensen@gmail= .com]
Sent: den 7 mars 2014 09:00
To: Wang, Ye-Kui
Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sand= bakken (geirsand); payload@ietf.org; Tom Kristensen (<= a href=3D"mailto:tomkrist@cisco.com">tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Well, I cannot see a reason for using SHOULD in offe= r/answer scenarios - as Stephan argued for earlier here. If there is a limi= t and a reason for signalling max-fps due, there is little use waiving the = flag with a SHOULD (which is often ignored, as we know).

 

I think we should rather remove max-fps for declarat= ive usage (or relax the MUST to SHOULD there).

 

-- Tom

 

On 4 March 2014 18:16, Wang, Ye-Kui <yekuiw@qti.qualcomm.com> wrote:

Let me pull Tom into the discussion :-) Personally I= am OK with the SHOULD language here too as I don't have a good counter arg= ument against Jonatan's comment.

Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this?

BR, YK


-----Original Message-----
From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1= plus resolutions up to 4K" which can never put a level 4.1 compliant encoder in a difficult situation (i.e. eve= rything in level 4.1 is still allowed).

If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient?

Best Regards Jonatan

-----Original Message-----
From: Stephan Wenger [mailto:stewe@stewe= .org]
Sent: den 4 mars 2014 10:48
To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Let me push back once more.  Overwriting the limits of the level defin= ition is common practice in the video conferencing industry, and has been d= one since staid industry changed to H.264 (which first introduced the level= concepts in video compression standards widely deployed in video conferencing).  That SHOULD will be ignored,= or overwritten by system standards.
Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation?
(I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios.  I only care about O/A scenarios.) Ste= phan


On 4.3.14, 9:41, "Jonatan Samuelsson" <jonatan.samuelsson@ericsson.com>
wrote:

>Thanks Ye-Kui for the suggested changes. I think they improve the
>situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" s= o that it becomes:
>
>"The encoder SHOULD use a picture rate equal to or less than this = value.
>In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters."
>
>I think this change would be needed in order to not redefine the level<= br> >definitions of the HEVC specification. A sender that for some reason is=
>not capable of producing lower picture rate (e.g. since the material is=
>pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not=
>be capable of adequately present all decoded pictures.
>
>Best Regards Jonatan
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Wang,
>Ye-Kui
>Sent: den 27 februari 2014 19:20
>To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>I am with Magnus here. If a decoder cannot DECODE a picture rate
>allowed by a level, then the decoder is not even a confirming decoder.<= br> >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason.
>
>I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good
>semantics, but the use of the parameter is still practically the same.<= br> >
>-----------------Start of the semantics, with suggested
>changes--------------------------
>max-fps:
>The value of max-fps is an integer indicating the maximum picture rate<= br> >in units of hundreds of pictures per second that can be efficiently
>received (***CHANGE TO "effectively processed by the receiver"= ;***).
>The max-fps parameter MAY be used to signal that the receiver has a
>constraint in that it is not capable of decoding (***CHANGE TO
>"processing"***) video efficiently (***CHANGE TO  "= effectively"***) at
>the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. >
>The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on
>maximum picture rate for all resolutions.
>
>Informative note: The max-fps parameter is semantically different from<= br> >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that<= br> >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters.
>
>The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters.
>-----------------End of the semantics, with suggested
>changes--------------------------
>
>Please express your opinion if this is not acceptable to you.
>
>BR, YK
>
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Magnus
>Westerlund
>Sent: Thursday, February 27, 2014 7:32 AM
>To: Geir Sandbakken (geirsand); pay= load@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
>> #12: The max-fps parameter
>>
>> Lessons learned from video conferencing using H.264 showed that a = lot
>>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when=
>>moving towards 120 fps in H.265.  It might not be caused by >>limitations in the decoder itself,  but that the surrounding m= edia
>>pipeline will not have been properly tested for 120 fps before bein= g
>>put into the wild.
>> If we don't allow for a code point to be used by the "restric= ted"
>> equipment, it will be required by the "flexible/proper" = to include one.
>> This is what happened with max-fps in H.264 enabling higher frame<= br> >>rates compared to lowering them.
>>
>
>I do have concerns with defining a parameter that allow dynamically
>sub-setting the profile and levels that has been defined for HEVC. This=
>can also create interoperability issues with any pre-encoded content to=
>a particular profile and level.
>
>The issues with the media pipeline has they been with the reception of<= br> >the high framerate of packets and forwarding them to the decoder or the=
>playback part, i.e. handling of the decoded picture?
>
>If it is predominately the later it appears very reasonable to have
>this as an informative parameter. I will not make use of higher FPS
>than X, if you send me higher than X I will need to reduce that to X or=
>lower before displaying it. That way we are not re-defining the
>profiles, we are ensuring that we can handle content encoded within the=
>profile and still by taking care of the this in the end of your decoder=
>chain you can protect the rest of the media framework from not yet
>supported frame-rates.
>
>Cheers
>
>Magnus Westerlund
>
>----------------------------------------------------------------------<= br> >Services, Media and Network features, Ericsson Research EAB/TXM
>----------------------------------------------------------------------<= br> >Ericsson AB                 | P= hone  +46 10 7148287
>F=E4r=F6gatan 6                = | Mobile +46 73 0949079
>SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com
>----------------------------------------------------------------------<= br> >
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload

_______________________________________________
payload mailing list
payload@ietf.org
https://www.ietf.org/mailman/listinfo/payload



 

--
# Cisco                   &nbs= p;     |  http://www.cisco.com/telepresence/
## tomkrist@cisco.c= om  |  = http://www.tandberg.com
###                     &= nbsp;         |  http://folk.uio.no/tomkri/

--_000_634D3B5D82E3214DA9B6A36D5F50DB2321A64CESESSMB109ericsso_-- From nobody Thu Apr 24 15:13:20 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16BE91A03EF for ; Thu, 24 Apr 2014 15:13:18 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.672 X-Spam-Level: X-Spam-Status: No, score=-1.672 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, J_CHICKENPOX_19=0.6, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XSWf5op_mXAd for ; Thu, 24 Apr 2014 15:13:10 -0700 (PDT) Received: from sabertooth02.qualcomm.com (sabertooth02.qualcomm.com [65.197.215.38]) by ietfa.amsl.com (Postfix) with ESMTP id 11E331A03F8 for ; Thu, 24 Apr 2014 15:13:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1398377584; x=1429913584; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=v1WkvP83hTbUDNaNQnQwD+yRymRsNTIo582WZOY3JI4=; b=mBOJCifBOJ3hG/ACaP3LMAgUc7hXEbqSoiQex4lVVhNQ/dd6E1VmQNVF 6obhPLpiYO4dEOXkq0oQMFgPaxjj7plZ3lHkJgBJjyD51XKNUL6P56zVZ FvyubfmmJ8JQMiCg1Wb6fwbgbqlz55IOGs9yO4Y5XiZCWgHdF+AwVi0Hs I=; X-IronPort-AV: E=McAfee;i="5400,1158,7418"; a="62590558" Received: from ironmsg04-r.qualcomm.com ([172.30.46.18]) by sabertooth02.qualcomm.com with ESMTP; 24 Apr 2014 15:13:03 -0700 X-IronPort-AV: E=Sophos;i="4.97,921,1389772800"; d="scan'208,217";a="721349621" Received: from nasanexhc12.na.qualcomm.com ([172.30.39.187]) by Ironmsg04-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 24 Apr 2014 15:13:03 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by nasanexhc12.na.qualcomm.com ([172.30.39.187]) with mapi id 14.03.0158.001; Thu, 24 Apr 2014 15:13:02 -0700 From: "Wang, Ye-Kui" To: Jonatan Samuelsson , Tom Kristensen <2mkristensen@gmail.com> Thread-Topic: [payload] #12 (rtp-h265): The max-fps parameter Thread-Index: AQHPM6U/tQCfPN0v2EGxR7CgdZmQz5rJwOwA//+msXCAB9L2AIAAAcMAgAAPZID//+cXUIAEolYAgAAFvQCAN0Ow8IAS5CxAgAEbr4CAAI4BEA== Date: Thu, 24 Apr 2014 22:13:00 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A834523E4B8@nasanexd02f.na.qualcomm.com> References: <530F5A6E.3040900@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387CF4DA@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C5C94@ESESSMB109.ericsson.se> <634D3B5D82E3214DA9B6A36D5F50DB231C5CEC@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83387D5BA4@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB231C7BA8@ESESSMB109.ericsson.se> <8BA7D4CEACFFE04BA2D902BF11719A83452011C7@nasanexd02f.na.qualcomm.com> <8BA7D4CEACFFE04BA2D902BF11719A834523BADE@nasanexd02f.na.qualcomm.com> <634D3B5D82E3214DA9B6A36D5F50DB2321A64C@ESESSMB109.ericsson.se> In-Reply-To: <634D3B5D82E3214DA9B6A36D5F50DB2321A64C@ESESSMB109.ericsson.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: multipart/alternative; boundary="_000_8BA7D4CEACFFE04BA2D902BF11719A834523E4B8nasanexd02fnaqu_" MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/Da3nwBx73T1fkbEjrEskFRdabLs Cc: "payload@ietf.org" , "Tom Kristensen \(tomkrist@cisco.com\)" Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 22:13:18 -0000 --_000_8BA7D4CEACFFE04BA2D902BF11719A834523E4B8nasanexd02fnaqu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Thanks Jonatan. The change you suggested is good to me. I believe the intention of the unit is "in units of pictures per hundred se= conds", exactly to enable the representation of 29.97 fps by sending the va= lue 2997, and so on. Good catch. I will make this change too. BR, YK From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] Sent: Wednesday, April 23, 2014 11:41 PM To: Wang, Ye-Kui; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com) Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter Thanks Ye-Kui, Your proposed text looks good. Perhaps the exception sentence can be modifi= ed slightly: An exception is when sending a pre-encoded bitstream, in which= case the picture rate may be greater than the value of max-fps. When reading this section again another question comes to my mind regarding= "The value of max-fps is an integer indicating the maximum picture rate in= units of hundreds of pictures per second". It seems to me that this could = be interpreted as the value 1 means "one hundred pictures per second", the = value 2 means "two hundred pictures per second" etc. I'm sure this is not t= he intention. Wouldn't it be better to just write: "The value of max-fps is= an integer indicating the maximum picture rate in units of pictures per se= cond" so that the value 1 means "one picture per second", the value 2 means= "two pictures per second" etc. (Or maybe the intention was to say "in units of pictures per hundred second= s" in order to be able to indicate fractional frame rates such as 29.97 fps= by sending the value 2997?) Best Regards Jonatan From: Wang, Ye-Kui [mailto:yekuiw@qti.qualcomm.com] Sent: den 23 april 2014 23:01 To: Jonatan Samuelsson; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cis= co.com) Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter We need to proceed. In case there is no more input, I suggest that we go ah= ead to change the semantics as follows: max-fps: The value of max-fps is an integer indicating the maximum picture rate in u= nits of hundreds of pictures per second that can be efficiently received ef= fectively processed by the receiver. The max-fps parameter MAY be used to = signal that the receiver has a constraint in that it is not capable of deco= ding video efficiently processing video effectively at the full picture rat= e that is implied by the highest level and, when present, one or more of th= e parameters max-lsr, max-lps, and max-br. The value of max-fps is not necessarily the picture rate at which the maxim= um picture size can be sent, it constitutes a constraint on maximum picture= rate for all resolutions. Informative note: The max-fps parameter is semantically different from max-= lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps = is used to signal a constraint, lowering the maximum picture rate from what= is implied by other parameters. The encoder MUST SHOULD use a picture rate equal to or less than this value= . An exception is when sending a pre-encoded bitstream the picture rate ma= y be greater than the value of max-fps. In cases where the max-fps paramet= er is absent the encoder is free to choose any picture rate according to th= e highest level and any signaled optional parameters. The value of max-fps MUST be smaller than or equal to the full picture rate= that is implied by the highest level and, when present, one or more of the= parameters max-lsr, max-lps, and max-br. [Note that addition of this sente= nce is to address the Magnus' comment on a value range for each parameter.] Jonatan, please suggest if you have a better exception case in mind (note t= hat nowadays generally an exception case needs to be clarified for each SHO= ULD wording). BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, Ye-Kui Sent: Friday, April 11, 2014 1:17 PM To: Jonatan Samuelsson; Tom Kristensen Cc: payload@ietf.org; Tom Kristensen (tomkrist@cis= co.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter We should also try to conclude herein. Tom, do you have any response to Jonatan's comment below? BR, YK From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Jonatan Samuel= sson Sent: Friday, March 07, 2014 12:20 AM To: Tom Kristensen; Wang, Ye-Kui Cc: Tom Kristensen (tomkrist@cisco.com); payload= @ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Thanks Tom for your feedback, For me to get a better understanding of if it is reasonable to make an exce= ption from the design principle of not re-defining level definitions from t= he video specification, it would be very valuable with answers to my earlie= r question: "If the motivation for max-fps is only for SDP O/A, why isn't the SDP attri= bute a=3Dframerate sufficient?" and to Magnus' earlier question: "The issues with the media pipeline has they been with the reception of the= high framerate of packets and forwarding them to the decoder or the playba= ck part, i.e. handling of the decoded picture?" Best Regards Jonatan From: Tom Kristensen [mailto:2mkristensen@gmail.com] Sent: den 7 mars 2014 09:00 To: Wang, Ye-Kui Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sandbakken = (geirsand); payload@ietf.org; Tom Kristensen (tomk= rist@cisco.com) Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Well, I cannot see a reason for using SHOULD in offer/answer scenarios - as= Stephan argued for earlier here. If there is a limit and a reason for sign= alling max-fps due, there is little use waiving the flag with a SHOULD (whi= ch is often ignored, as we know). I think we should rather remove max-fps for declarative usage (or relax the= MUST to SHOULD there). -- Tom On 4 March 2014 18:16, Wang, Ye-Kui > wrote: Let me pull Tom into the discussion :-) Personally I am OK with the SHOULD = language here too as I don't have a good counter argument against Jonatan's= comment. Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this? BR, YK -----Original Message----- From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com] Sent: Tuesday, March 04, 2014 2:43 AM To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1 plus= resolutions up to 4K" which can never put a level 4.1 compliant encoder in= a difficult situation (i.e. everything in level 4.1 is still allowed). If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient? Best Regards Jonatan -----Original Message----- From: Stephan Wenger [mailto:stewe@stewe.org] Sent: den 4 mars 2014 10:48 To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter Let me push back once more. Overwriting the limits of the level definition= is common practice in the video conferencing industry, and has been done s= ince staid industry changed to H.264 (which first introduced the level conc= epts in video compression standards widely deployed in video conferencing).= That SHOULD will be ignored, or overwritten by system standards. Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation? (I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios. I only care about O/A scenarios.) Stephan On 4.3.14, 9:41, "Jonatan Samuelsson" > wrote: >Thanks Ye-Kui for the suggested changes. I think they improve the >situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" so that it becomes: > >"The encoder SHOULD use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters." > >I think this change would be needed in order to not redefine the level >definitions of the HEVC specification. A sender that for some reason is >not capable of producing lower picture rate (e.g. since the material is >pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not >be capable of adequately present all decoded pictures. > >Best Regards Jonatan > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Wang, >Ye-Kui >Sent: den 27 februari 2014 19:20 >To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >I am with Magnus here. If a decoder cannot DECODE a picture rate >allowed by a level, then the decoder is not even a confirming decoder. >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason. > >I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good >semantics, but the use of the parameter is still practically the same. > >-----------------Start of the semantics, with suggested >changes-------------------------- >max-fps: >The value of max-fps is an integer indicating the maximum picture rate >in units of hundreds of pictures per second that can be efficiently >received (***CHANGE TO "effectively processed by the receiver"***). >The max-fps parameter MAY be used to signal that the receiver has a >constraint in that it is not capable of decoding (***CHANGE TO >"processing"***) video efficiently (***CHANGE TO "effectively"***) at >the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. > >The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on >maximum picture rate for all resolutions. > >Informative note: The max-fps parameter is semantically different from >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters. > >The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled >optional parameters. >-----------------End of the semantics, with suggested >changes-------------------------- > >Please express your opinion if this is not acceptable to you. > >BR, YK > > >-----Original Message----- >From: payload [mailto:payload-bounces@ietf.org] On Behalf Of Magnus >Westerlund >Sent: Thursday, February 27, 2014 7:32 AM >To: Geir Sandbakken (geirsand); payload@ietf.org >Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter > >On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote: >> #12: The max-fps parameter >> >> Lessons learned from video conferencing using H.264 showed that a lot >>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when >>moving towards 120 fps in H.265. It might not be caused by >>limitations in the decoder itself, but that the surrounding media >>pipeline will not have been properly tested for 120 fps before being >>put into the wild. >> If we don't allow for a code point to be used by the "restricted" >> equipment, it will be required by the "flexible/proper" to include one. >> This is what happened with max-fps in H.264 enabling higher frame >>rates compared to lowering them. >> > >I do have concerns with defining a parameter that allow dynamically >sub-setting the profile and levels that has been defined for HEVC. This >can also create interoperability issues with any pre-encoded content to >a particular profile and level. > >The issues with the media pipeline has they been with the reception of >the high framerate of packets and forwarding them to the decoder or the >playback part, i.e. handling of the decoded picture? > >If it is predominately the later it appears very reasonable to have >this as an informative parameter. I will not make use of higher FPS >than X, if you send me higher than X I will need to reduce that to X or >lower before displaying it. That way we are not re-defining the >profiles, we are ensuring that we can handle content encoded within the >profile and still by taking care of the this in the end of your decoder >chain you can protect the rest of the media framework from not yet >supported frame-rates. > >Cheers > >Magnus Westerlund > >---------------------------------------------------------------------- >Services, Media and Network features, Ericsson Research EAB/TXM >---------------------------------------------------------------------- >Ericsson AB | Phone +46 10 7148287 >F=E4r=F6gatan 6 | Mobile +46 73 0949079 >SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com >---------------------------------------------------------------------- > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload > >_______________________________________________ >payload mailing list >payload@ietf.org >https://www.ietf.org/mailman/listinfo/payload _______________________________________________ payload mailing list payload@ietf.org https://www.ietf.org/mailman/listinfo/payload -- # Cisco | http://www.cisco.com/telepresence/ ## tomkrist@cisco.com | http://www.tandberg.co= m ### | http://folk.uio.no/tomkri/ --_000_8BA7D4CEACFFE04BA2D902BF11719A834523E4B8nasanexd02fnaqu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Thanks Jonatan. The chang= e you suggested is good to me.

 <= /p>

I believe the intention o= f the unit is “in units of pictures per hundred s= econds”, exactly to enable the representation of 29.97 fps by sending= the value 2997, and so on. Good catch. I will make this change too.=

 <= /p>

BR, YK<= /p>

 <= /p>

From: Jonatan = Samuelsson [mailto:jonatan.samuelsson@ericsson.com]
Sent: Wednesday, April 23, 2014 11:41 PM
To: Wang, Ye-Kui; Tom Kristensen
Cc: payload@ietf.org; Tom Kristensen (tomkrist@cisco.com)
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

 

Thanks Ye-Kui,=

 <= /p>

Your proposed text looks = good. Perhaps the exception sentence can be modified slightly: An exception = is when sending a pre-encoded bitstream, in which= case the picture rate may be greater than the value of max-fps.

 <= /p>

When reading this section= again another question comes to my mind regarding “The value = of max-fps is an integer indicating the maximum picture rate in units of hundreds of pictures per second”= . It seems to me that this could be interpreted as the value 1 means “= ;one hundred pictures per second”, the value 2 means “two hundr= ed pictures per second” etc. I’m sure this is not the intention. = Wouldn’t it be better to just write: “The value of max-f= ps is an integer indicating the maximum picture rate in units of pictures p= er second” so that the value 1 means “one picture per second”, the value = 2 means “two pictures per second” etc.

 <= /p>

(Or maybe the intention w= as to say “in units of pictures per hundred seconds” in order t= o be able to indicate fractional frame rates such as 29.97 fps by sending the value 2997?)

 <= /p>

Best Regards Jonatan=

 <= /p>

From: Wang, Ye= -Kui [mailto:yekuiw@qti.qualcomm= .com]
Sent: den 23 april 2014 23:01
To: Jonatan Samuelsson; Tom Kristensen
Cc: payload@ietf.org; Tom Kr= istensen (tomkrist@cisco.com)
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

 

We need to proceed. In ca= se there is no more input, I suggest that we go ahead to change the semanti= cs as follows:

 <= /p>

max= -fps:

The v= alue of max-fps is an integer indicating the maximum picture rate in units = of hundreds of pictures per second that can be efficiently received effectively p= rocessed by the receiver.  The max-fps parameter MAY be used to= signal that the receiver has a constraint in that it is not capable of decoding video efficiently processing video effectively at the full picture rate that is implie= d by the highest level and, when present, one or more of the parameters max= -lsr, max-lps, and max-br.

=  

The v= alue of max-fps is not necessarily the picture rate at which the maximum pi= cture size can be sent, it constitutes a constraint on maximum picture rate= for all resolutions.

=  

Infor= mative note: The max-fps parameter is semantically different from max-lsr, = max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps is us= ed to signal a constraint, lowering the maximum picture rate from what is implied by other parameters.

=  

The e= ncoder MUST SHOULD use a picture rate equal to or less than this value.  An exception is when sending a pre-encoded bitstream the picture rate may b= e greater than the value of max-fps.  In cases where the max-fp= s parameter is absent the encoder is free to choose any picture rate accord= ing to the highest level and any signaled optional parameters.

=  

The value of max-fps MUST= be smaller than or equal to the full picture rate that is implied by the h= ighest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br. [Note that addition of this sentence is to address the Magnus’ commen= t on a value range for each parameter.]

 <= /p>

Jonatan, please suggest i= f you have a better exception case in mind (note that nowadays generally an= exception case needs to be clarified for each SHOULD wording).<= /span>

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.or= g] On Behalf Of Wang, Ye-Kui
Sent: Friday, April 11, 2014 1:17 PM
To: Jonatan Samuelsson; Tom Kristensen
Cc: payload@ietf.org; Tom Kr= istensen (tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

 

We should also try to con= clude herein.

 <= /p>

Tom, do you have any resp= onse to Jonatan’s comment below?

 <= /p>

BR, YK<= /p>

 <= /p>

From: payload = [mailto:payload-bounces@ietf.or= g] On Behalf Of Jonatan Samuelsson
Sent: Friday, March 07, 2014 12:20 AM
To: Tom Kristensen; Wang, Ye-Kui
Cc: Tom Kristensen (tomkrist@c= isco.com); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Thanks Tom for your feedb= ack,

 <= /p>

For me to get a better un= derstanding of if it is reasonable to make an exception from the design pri= nciple of not re-defining level definitions from the video specification, it would be very valuable with answers to my earlier questi= on:

 <= /p>

“If the moti= vation for max-fps is only for SDP O/A, why isn't the SDP attribute a=3Dfra= merate sufficient?”

 <= /p>

and to Magnus’ earl= ier question:

 <= /p>

“The issues = with the media pipeline has they been with the reception of the high framer= ate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?R= 21;

 <= /p>

Best Regards Jonatan

 <= /p>

From: Tom Kris= tensen [mailto:2mkristensen@gmail= .com]
Sent: den 7 mars 2014 09:00
To: Wang, Ye-Kui
Cc: Jonatan Samuelsson; Stephan Wenger; Magnus Westerlund; Geir Sand= bakken (geirsand); payload@ietf.org; Tom Kristensen (<= a href=3D"mailto:tomkrist@cisco.com">tomkrist@cisco.com)
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter<= o:p>

 

Well, I cannot see a reason for using SHOULD in offe= r/answer scenarios - as Stephan argued for earlier here. If there is a limi= t and a reason for signalling max-fps due, there is little use waiving the = flag with a SHOULD (which is often ignored, as we know).

 

I think we should rather remove max-fps for declarat= ive usage (or relax the MUST to SHOULD there).

 

-- Tom

 

On 4 March 2014 18:16, Wang, Ye-Kui <yekuiw@qti.qualcomm.com> wrote:

Let me pull Tom into the discussion :-) Personally I= am OK with the SHOULD language here too as I don't have a good counter arg= ument against Jonatan's comment.

Tom, would the SHOULD language be fine with you as well (as the proponent o= f this parameter)? Do you see any technical problem with this?

BR, YK


-----Original Message-----
From: Jonatan Samuelsson [mailto:jonatan.samuelsson@ericsson.com]
Sent: Tuesday, March 04, 2014 2:43 AM
To: Stephan Wenger; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (geirs= and); payload@ietf.org
Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter

As far as I know, all other similar parameters in H.264 RTP PL and H.265 RT= P PL extends on the levels defined in the video specification in the sense = that you are capable of expressing for example "I can decode level 4.1= plus resolutions up to 4K" which can never put a level 4.1 compliant encoder in a difficult situation (i.e. eve= rything in level 4.1 is still allowed).

If the motivation for max-fps is only for SDP O/A, why isn't the SDP attrib= ute a=3Dframerate sufficient?

Best Regards Jonatan

-----Original Message-----
From: Stephan Wenger [mailto:stewe@stewe= .org]
Sent: den 4 mars 2014 10:48
To: Jonatan Samuelsson; Wang, Ye-Kui; Magnus Westerlund; Geir Sandbakken (g= eirsand); payload@ietf.org
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Let me push back once more.  Overwriting the limits of the level defin= ition is common practice in the video conferencing industry, and has been d= one since staid industry changed to H.264 (which first introduced the level= concepts in video compression standards widely deployed in video conferencing).  That SHOULD will be ignored,= or overwritten by system standards.
Is there any technical reason for not allowing overwriting level limitation= s in scenarios where there is two way negotiation?
(I would be fine with a SHOULD, or even the removal of max_fps entirely, in= SDP declarative use scenarios.  I only care about O/A scenarios.) Ste= phan


On 4.3.14, 9:41, "Jonatan Samuelsson" <jonatan.samuelsson@ericsson.com>
wrote:

>Thanks Ye-Kui for the suggested changes. I think they improve the
>situation to a large extent, but there is one more thing that I would >like change; replacing the "MUST" with a "SHOULD" s= o that it becomes:
>
>"The encoder SHOULD use a picture rate equal to or less than this = value.
>In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters."
>
>I think this change would be needed in order to not redefine the level<= br> >definitions of the HEVC specification. A sender that for some reason is=
>not capable of producing lower picture rate (e.g. since the material is=
>pre-encoded) should be allowed to use a picture rate higher than the >value of max-fps, but it must then be aware that the receiver might not=
>be capable of adequately present all decoded pictures.
>
>Best Regards Jonatan
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Wang,
>Ye-Kui
>Sent: den 27 februari 2014 19:20
>To: Magnus Westerlund; Geir Sandbakken (geirsand); payload@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>I am with Magnus here. If a decoder cannot DECODE a picture rate
>allowed by a level, then the decoder is not even a confirming decoder.<= br> >We usually don't standardize mechanisms for non-confirming decoders, >unless there is a very strong reason.
>
>I think making the following changes (4 places in the first paragraph >of the semantics) would resolve the issue, with conceptually good
>semantics, but the use of the parameter is still practically the same.<= br> >
>-----------------Start of the semantics, with suggested
>changes--------------------------
>max-fps:
>The value of max-fps is an integer indicating the maximum picture rate<= br> >in units of hundreds of pictures per second that can be efficiently
>received (***CHANGE TO "effectively processed by the receiver"= ;***).
>The max-fps parameter MAY be used to signal that the receiver has a
>constraint in that it is not capable of decoding (***CHANGE TO
>"processing"***) video efficiently (***CHANGE TO  "= effectively"***) at
>the full picture rate that is implied by the highest level and, when >present, one or more of the parameters max-lsr, max-lps, and max-br. >
>The value of max-fps is not necessarily the picture rate at which the >maximum picture size can be sent, it constitutes a constraint on
>maximum picture rate for all resolutions.
>
>Informative note: The max-fps parameter is semantically different from<= br> >max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that<= br> >max-fps is used to signal a constraint, lowering the maximum picture >rate from what is implied by other parameters.
>
>The encoder MUST use a picture rate equal to or less than this value. >In cases where the max-fps parameter is absent the encoder is free to >choose any picture rate according to the highest level and any signaled=
>optional parameters.
>-----------------End of the semantics, with suggested
>changes--------------------------
>
>Please express your opinion if this is not acceptable to you.
>
>BR, YK
>
>
>-----Original Message-----
>From: payload [mailto:paylo= ad-bounces@ietf.org] On Behalf Of Magnus
>Westerlund
>Sent: Thursday, February 27, 2014 7:32 AM
>To: Geir Sandbakken (geirsand); pay= load@ietf.org
>Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
>
>On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
>> #12: The max-fps parameter
>>
>> Lessons learned from video conferencing using H.264 showed that a = lot
>>of equipment designed for 30 fps could not handle 60 fps even at >>sufficiently low resolution. We do anticipate similar problems when=
>>moving towards 120 fps in H.265.  It might not be caused by >>limitations in the decoder itself,  but that the surrounding m= edia
>>pipeline will not have been properly tested for 120 fps before bein= g
>>put into the wild.
>> If we don't allow for a code point to be used by the "restric= ted"
>> equipment, it will be required by the "flexible/proper" = to include one.
>> This is what happened with max-fps in H.264 enabling higher frame<= br> >>rates compared to lowering them.
>>
>
>I do have concerns with defining a parameter that allow dynamically
>sub-setting the profile and levels that has been defined for HEVC. This=
>can also create interoperability issues with any pre-encoded content to=
>a particular profile and level.
>
>The issues with the media pipeline has they been with the reception of<= br> >the high framerate of packets and forwarding them to the decoder or the=
>playback part, i.e. handling of the decoded picture?
>
>If it is predominately the later it appears very reasonable to have
>this as an informative parameter. I will not make use of higher FPS
>than X, if you send me higher than X I will need to reduce that to X or=
>lower before displaying it. That way we are not re-defining the
>profiles, we are ensuring that we can handle content encoded within the=
>profile and still by taking care of the this in the end of your decoder=
>chain you can protect the rest of the media framework from not yet
>supported frame-rates.
>
>Cheers
>
>Magnus Westerlund
>
>----------------------------------------------------------------------<= br> >Services, Media and Network features, Ericsson Research EAB/TXM
>----------------------------------------------------------------------<= br> >Ericsson AB                 | P= hone  +46 10 7148287
>F=E4r=F6gatan 6                = | Mobile +46 73 0949079
>SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com
>----------------------------------------------------------------------<= br> >
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload
>
>_______________________________________________
>payload mailing list
>payload@ietf.org
>https://www.ietf.org/mailman/listinfo/payload

_______________________________________________
payload mailing list
payload@ietf.org
https://www.ietf.org/mailman/listinfo/payload



 

--
# Cisco                   &nbs= p;     |  http://www.cisco.com/telepresence/
## tomkrist@cisco.c= om  |  = http://www.tandberg.com
###                     &= nbsp;         |  http://folk.uio.no/tomkri/

--_000_8BA7D4CEACFFE04BA2D902BF11719A834523E4B8nasanexd02fnaqu_-- From nobody Fri Apr 25 02:36:03 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D72E41A03B5 for ; Fri, 25 Apr 2014 02:36:00 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LEjTsgwBaSys for ; Fri, 25 Apr 2014 02:35:57 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) by ietfa.amsl.com (Postfix) with ESMTP id 9C1F81A0359 for ; Fri, 25 Apr 2014 02:35:56 -0700 (PDT) X-AuditID: c1b4fb30-f791c6d000005f7c-d9-535a2c75f92f Received: from ESESSHC011.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id C7.BB.24444.57C2A535; Fri, 25 Apr 2014 11:35:49 +0200 (CEST) Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.53) with Microsoft SMTP Server id 14.3.174.1; Fri, 25 Apr 2014 11:35:49 +0200 Message-ID: <535A2C74.7020306@ericsson.com> Date: Fri, 25 Apr 2014 11:35:48 +0200 From: Magnus Westerlund User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Wang, Ye-Kui" , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrALMWRmVeSWpSXmKPExsUyM+JvjW6pTlSwwaQmE4tTF/exWly6eJbJ 4smaY8wOzB5Llvxk8lg09Rmjx5fLn9kCmKO4bFJSczLLUov07RK4MqZuXMRW8Cuuou3afuYG xu3eXYwcHBICJhKfj9R2MXICmWISF+6tZ+ti5OIQEjjKKLH29RFWCGc5o0RX5wE2kCpeAW2J qXcfsoPYLAKqEu8nn2IFsdkELCRu/mgEqxEVCJZYOmcxC0S9oMTJmU9YQAaJCGxilFi4eA1Y kbCAo8SGxlYmiA03GSV2vloJNpUTqPvJtivMEOeJS/Q0BoGEmQX0JKZcbWGEsOUlmrfOZgax hYAOamjqYJ3AKDgLyb5ZSFpmIWlZwMi8ilG0OLU4KTfdyEgvtSgzubg4P08vL7VkEyMwiA9u +W2wg/Hlc8dDjAIcjEo8vMVfIoOFWBPLiitzDzFKc7AoifN+O+seLCSQnliSmp2aWpBaFF9U mpNafIiRiYNTqoGxfDnjZXnR6KTLUz7bftJQZLrTz3S2+/zW+UrtO/gmGTQ4cIge+cupz6bB esi4fEmAwnuLZLmagLt7vzctvPk49oXAndqwD1+W7qi1+Hb1zz+jd1wiJ/jv7ZvTGzshyalD ReEnw1n9bOVm4xdZ/jw23+4kTXGzqzp2Mv6H3smLj5lj155y6tNWYinOSDTUYi4qTgQAXLB7 0kMCAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/payload/4lmnB9FJpi5Bn1DVhh9CEEJxTvw Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 09:36:01 -0000 Hi, Sorry for taking my time to respond to this. I have removed the issues where it appeared that we had a resolution. Unfortunately I think I have discovered a new issue, see the last part of this email. On 2014-04-11 19:56, Wang, Ye-Kui wrote: >> 43. Section 7.2.2: >> >> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly >> w/o recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id >> --+ | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | >> | | profile-space C X C X P profile-id C >> X C X P tier-flag C X C X P >> level-id C X C X P >> >> C: configuration for sending and receiving streams >> >> I don't get this usage or definition of C for these parameters. >> >> So these parameters are receiver capabilities, and they needs to be >> symmetrically used in the signalling, however the actually used >> values in the transmission direction from an O/A agent only needs >> to be within the receivers provided parameters. Thus, they are not >> a "configuration" for the sending direction. >> >> [YK]: These are configuration parameters, meaning indicating both >> what is to be sent and what is to be received for sendrecv and must >> be used symmetrically, and only the latter for recvonly. > > This is at a least wrong for the level-id which a peer agent in an > O/A may downgrade. The rest I agree the payload format itself > requires to be kept in answer, or the corresponding PT dropped. > > [YK]: OK, level-id is downgradable but it is still a configuration, > but a special configuration that can be downgraded. I'd suggest we > keep using the "C" denotation but adding a note to repeat what is > said earlier that level is downgradable, because using any other > denotation is not correct. Please let me know if you think this is > not OK, and if so what kind of changes would you propose. I think that is okay. But, considering that the level is here not combined, wouldn't make sense to use some other property identifier for it? > >> >> Also I really don't get the X's in the answer: recvonly, >> recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The >> answering agent is going to receive stream in both these cases, >> thus it needs to declare the payload type, and these parameters >> MUST be included (unless defaulted). So why is they marked with X? >> >> [YK]: When recv-sub-layer-id is included in an SDP answer, it means >> that the answer chooses one of the multiple operation points >> corresponding to the offered or declared sub-layers in the >> sprop-vps, and in this case there is no need to repeat the >> information of profile, level etc. in the SDP, thus there are >> marked with X (MUST NOT be present). > > This appears very strange to me. The Offerer includes a VPS that > contains options for what it can send. The answering party, i.e. > which will receive this stream needs in accordance to SDP O/A rules > declare which PTs it can receive. This PT must be corresponding to > the one that the Offerer proposed to send and in which the VPS was > included. > > For me this implies the following: > > 1. The parameters indicating the basic configuraiton of the payload > type, i.e profile, level ,tier etc. MUST be present in the answer > with the same values as in the offer it responds to. Otherwise the PT > matching rule fails. > > 2. In addition the PT to receive on MUST be declared as existing to > ensure that any middleboxes doesn't block it. Thus, explicit > indication of what will go in it is really recommended, especially > considering i all the other cases that information will be explicit > here. Thus, I think the X's are plain wrong. Or I am still > misunderstanding something here. > > [YK]: Let's use an example, which should help. Let's say the offer > SDP includes a VPS that provides two operation points, 720p@15fps and > 720p@30fps, corresponding to recv-sub-layer-id 0 and 1, respectively. > The profile, tier, level, and so on for each operation point are > signalled by the VPS. If recv-sub-layer-id=0 is included in an SDP > answer, then that means the answerer chooses to operate at > 720p@15fps. In this case, the profile, tier, level, and so on for the > chosen operation point does not need to be explicitly sent again in > the SDP, as it is clear what they are (they are signalled in the VPS > in the SDP offer). So there is no ambiguity here. Are you suggesting > that in some use cases even there is no ambiguity the configuration > information must always be explicitly signalled (i.e. values that are > derivable)? If yes, could you please explain in what use cases and > why. Note that we had the same mechanism specified for SVC in RFC > 6190. Also, on PT use, we have allowed the use of a different PT > value in an SDP answer than the matching PT value in the offer since > at least RFC 3984 as long as the PT matching is clear. Okay, I can now put the finger on what troubles me with this. I think this is violation or at least an omission of the principle of SDP Offer/Answer. The reason is that you substitute the answering explicitly declaring its receiving capabilities with a reference to a particular instantiation of a video sequence parameter set that falls within the receiver's capabilities. Thus, the offerer does not know the receiver's actually capabilities if it needs to reconfigure the encoder, or in case the RTP payload type is going to be used by multiple media streams and possibly different encoder instances. In addition it creates an issue for any middlebox that is on the signalling path that it keeps track and state for what was in the offer to know what the answer means. Thus, I think it is important that the answerer doesn't treat this situation differently, and instead declare its capabilities as normally in answer. The recv-sub-layer-id parameter is an optional parameter and it is also stream specific, while the profile, level, etc are generically applicable to the RTP payload type do applies on RTP session level. To conclude, the recv-sub-layer-id is fine as a stream specific selection mechanism, but does not replace the capability declaration the regular parameters perform. > >> >> 44. Section 7.2.2: >> >> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly >> w/o recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id >> --+ | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | >> | | interop-constraints C X C X P >> profile-compatibility-indicator C X C X P >> >> I don't understand how these parameters really are used for >> receiver capability declarations, so I don't understand why the Cs >> are C and not P. >> >> [YK]: As explained earlier, these are also parts of the media type >> configuration, same as profile, tier, and level, and their usage >> are the same as those too. > > First of all these parameters are not included as the ones that have > to be symmetrically provided. I also think they could be SDP agent > specifically given. But, that can be discussed. But, according to the > current text, I fail to see that these needs to be handled as the > profile-id, level-id, etc. > > The only constraint I find for these two flags applies under > multicast. > > Either they need text, or these C's need to be changed. > > [YK]: Maybe you were initially reading version -01 and did not check > version -02 on this part. In version -02, these parameters are > included in the list of configuration parameters that are specified > to be used symmetrically. Okay, I had missed that change in the -02 version. After having given this some thought and re-read the definitions and relevant SDP O/A parts I think I have found a new issue. This issue is related to the interpretation of "interop-constraints", and "profile-compatibility-indicator". It is not clear in which direction they should be interpreted in the various cases. Are they constraints on what the receiver/decoder accept or constraints the encoder promise to fulfil or both? I think the "profile-compatibility-indicator" is an excellent case to show that neither of the alternative appears to work or provide any benefit. This example will use a future profile that I call Baseline, which is a sub-set of Main. Receiver constraints: A sends it offers with a RTP payload type (97) which is Main but with profile-compatibility-indicator indicating Main and Baseline. This means the offerer prefers to do main but knows of baseline. Answerer is baseline only. Thus it detects a profile not supported, but compatibility indicates support. However, due to the other payload type rules, the answerer must reject PT=97 as it can't answer with 97=Baseline. Nor can it reject 97 and and 98 which is baseline, and only use 97 in answerer to offerer direction. And only receive using 98. Especially not if 97 was the only PT offered, then you have no compatibility. Instead, to my understanding to achieve receiver constraints the offerer must include one PT per profile it supports, the profile-compatibility-indicator is useless in this case. Forcing the answerer to copy it and be forced to have a the same receiver constraints is just useless. Sender constraints: If this is a sender constraint, then it actually tells something about the sender's capability of providing something that matches more profiles than what it is willing to receive. However, the answerer if it doesn't support the any of the PTs and their profile-id values then it anyway have to refuse the PT, even if it could receive the stream from the offerer. Instead, also here the offerer needs to indicate all profiles it can encoded for and offer them. Which implies also decoding capability. In addition, if this is a sender constraint, using C as classification (i.e. must copy) in answer, if the encoder in the answerer supports the payload types profile-id but, can't ensure or know about all the profile-compatibility flags, then it has to refuse the payload type despite it supporting it. Result, forcing C on a sender constraint interpretation only results in failures and no benefit. Receiver and Sender constraints. To my understanding such an interpretation only forces the case of more rejections, if the answerer doesn't have an equally capable encoder and decoder as the offerer and can match both sets of constraints. In addition the compatibility has not provided any benefit for the two O/A agents. So, the question becomes have I misunderstood something about how you intended to be able to use this parameter? If I haven't I think these parameters usage is seriously in question. If one would like to use it then the must not alter constraint for profile becomes a serious issue. If one would remove that then I see that using "profile-compatibility-indicator" could be possibly beneficial. But, I haven't done a full out design of how to resolve the PT matching and deal with asymmetric usage within the session. The same type of analysis and clarification on usage needs to happen for the "interop-constraints". Then we have the question of how one shall interpret and deal with unknown flag values. These needs to cause only failures when required for ensuring video compatibility. Not cause negotiation failures for unknown cases that doesn't affect video interoperability. Cheers Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 Färögatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Fri Apr 25 02:57:06 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 673A31A0353 for ; Fri, 25 Apr 2014 02:57:05 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.302 X-Spam-Level: X-Spam-Status: No, score=-2.302 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nxtFK1AtLE0S for ; Fri, 25 Apr 2014 02:57:02 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) by ietfa.amsl.com (Postfix) with ESMTP id B75601A0143 for ; Fri, 25 Apr 2014 02:57:01 -0700 (PDT) X-AuditID: c1b4fb30-f791c6d000005f7c-16-535a3166f7e0 Received: from ESESSHC005.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id 91.8E.24444.6613A535; Fri, 25 Apr 2014 11:56:54 +0200 (CEST) Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.35) with Microsoft SMTP Server id 14.3.174.1; Fri, 25 Apr 2014 11:56:54 +0200 Message-ID: <535A3166.3050006@ericsson.com> Date: Fri, 25 Apr 2014 11:56:54 +0200 From: Magnus Westerlund User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Stephan Wenger , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" References: <5310AF7F.9050508@ericsson.com> In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrPLMWRmVeSWpSXmKPExsUyM+JvjW6aYVSwwbRNVhanLu5jtbh08SyT xfXGTewOzB5Llvxk8li8/j2jx5fLn9kCmKO4bFJSczLLUov07RK4Mrbcv8BScDGsoufdAtYG xibHLkZODgkBE4n+/ZOZIWwxiQv31rOB2EICRxklvtyx7mLkArKXM0o8fX4OrIhXQFvi6tXd 7CA2i4CqxLZ/v1lAbDYBC4mbPxrBmkUFgiWWzlnMAlEvKHFy5hMWkEEiAqsYJV7u2QJWJCxg LjF7/woWiG0+EhOXLWQEsTkFdCW6998FWsYBdJG4RE9jEEiYWcBA4siiOawQtrxE89bZzBCt 2hINTR2sExgFZyFZNwtJyywkLQsYmVcxihanFiflphsZ6aUWZSYXF+fn6eWllmxiBIbwwS2/ DXYwvnzueIhRgINRiYe3+EtksBBrYllxZe4hRmkOFiVx3m9n3YOFBNITS1KzU1MLUovii0pz UosPMTJxcEo1MPZutf29S2PV8b17N/VcrrkntkCWp0DQWrBW8HLd3NXLZ85dZ+TyJ48/VyLn /8r4jaF7eNaXbLi6VDFPd3fl239LHA8unxxxddUhhvNV111XbHKPOrggLNJp1ueo6Ed62Qen PXnvwZp/tWOH4rVLapnJmzdvaxU7fOHfRPt8k4xYiSk//uc+uXRaiaU4I9FQi7moOBEA1v6D oUICAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/payload/h6sKds90pLHSgj4cVyqZBjCp-vU Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 09:57:05 -0000 On 2014-04-14 00:20, Stephan Wenger wrote: > Hi all, especially Magnus, > > Magnus, thanks again for your careful review. > > Below inline some comments on Magnus issues #17 through #22, PACI-related. > All other issues have been removed from this email. > > Regards, > Stephan > > > On 28.2.14, 7:47, "Magnus Westerlund" > wrote: > >> [Š] >> >> 17. Section 4.9: >> 0 1 2 3 >> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >> | RTP Header | >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >> |F| PACI=50 | LayerId | TID |A| Type | PHSsize |F0..2|X| >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >> | Payload Header Extension Structure (PHES) | >> |=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=| >> | | >> | PACI payload: NAL unit | >> | . . . | >> | | >> | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ >> | :...OPTIONAL RTP padding | >> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- >> >> I wonder if one shouldn't mark the first 16 bits after the header as the >> "payload header" and instead separate out the configuration of those for >> this payload format. That would also resolve the next issue. > > Yes, I agree. In fact, it¹s kind of odd that section 4.9 is the first > instance where we have a figure explaining the (main) payload header > itself. Of course, we have done so in textual form in some detail before, > but not in a figure. > > My current thinking is that the right place for such a figure would be > section 4.3, as these 16 bits are the same for all four packet types we > have. And keep the section 4.9 figure as is, plus/minus the changes > identified in issues #18 and #19 below. > >> >> 18. Section 4.9: >> >> PACI: 6 bits >> Indicates a PACI, and must be 50. >> >> As this field is called Type in the other definition, why not call it >> Type: 6 bits >> Indicates a PACI, and must be 50. >> >> And to clarify the confusion with the PACI payload type, that Type field >> should be called IType or something like that > > Both suggestions make sense (regardless of the doc structure fix of issue > #17 above) and we will implement them. > >> >> 19. Section 4.9: >> PHSsize: 5 bits >> Indicates the total length of the PHES. The value is limited to >> be less than or equal to 32 octets, to simplify encoder design >> for MTU size matching. >> >> F0..2: 3 bits >> Each of the three bits indicate, when set, the presence of an >> optional field (or set of fields) in the PHES. >> >> X: 1 bit >> The X bit, when set, indicates the presence of another octet >> consisting of seven flags and another X bit, each of the seven >> flags indicating the presence of more PHES fields (for future >> extensions). >> >> The motivation of the PHSSize of limiting this to 32 octets is not >> fulfilled. As the RTP payload still are variable size due to the X bit, >> which adds one or more bytes. Thus if the goal is to make clear that >> this will never be more than 32 bytes, than I think the bytes that X >> signals must be included in the 32-bytes for the PHES. > > It was unwise of me to call this extension bit ³X². It is easily confused > with the X bit in the RTP header. We should replace the section 4.9 ³X² > with another suitable character. > The reminder of my answer depends on how you interpret ³X²: the PHS > extension X, or the RTP header X. > > In case you meant PHS extension ³X²: > You are correct, and the fix is easy: We add a requirement that PHSsize > shall include the extension flag octets (whose presence is indicated by > those X bits). I meant this interpretation of X. I am fine with that solution limiting the PACI content including the Flag octets to 32 octets. > > In case you meant RTP header ³X²: > The bug fix as above still needed. In addition, we could also add > somewhere in section 4.1 (RTP header usage) a requirement that the total > size of the RTP header, plus RTP header extension, plus the payload header > and all its extensions shall not exceed a certain number. That number is > not easy to calculate, as it depends on the use of PACI (and its extension > mechanism), but also FU, aggregation, DON or no DON, and so on. I¹m not > in favor of doing that. > > However, what might be a good idea is to add an informative note into > section 4.1 to the extent that a good encoder fills up the MTU whenever it > can so to keep packetization overhead low, but in order to do so, it has > to be able to predict or know the size of the payload header, which is > highly variable due to syntax options defined in this specification, but > also through the potential presence of RTP header extensions. > > Unrelated to the above: > I can imagine cases where we would need only a single bit to signal > something. In that case, it would be kind of odd to use a bit to signal > the presence of another bit. I don¹t think that needs to be specified in > any detail, except by saying that the extension signalized by an F bit set > to 1 can have a length of zero octets. Good idea? Yes, I think that is fine. > > >> >> 20. Section 4.9: Forward compability for PHES. >> >> The content of the PHES field is determined using the F0..2 field and in >> the future the octet(s) added using the X field, where each set bit >> indicates what fields are included. However, this definition have some >> severe implications regarding future compatibility. As each bit can >> represent an unknown amount of fields a first gen decoder can't know how >> many bits each of the bits represents. This may not matter much for the >> first gen, as it only knows of the PHES data defined in this version. To >> be able to decode these fields only a single thing is missing, an >> assurance that the first defined data MUST be first in the PHES data >> section when F0 is set. >> >> For a third generation implementation there is another implication. To >> know where the F2 PHES payload starts it MUST know also what F0 and F1 >> means in amount of fixed field data values. >> >> I am not that happy for this forward compatibility limiations. But, it >> can work, but the rules for how to make it work MUST be explictely >> stated now. >> >> 1) The fields added for a particular FX bit MUST be fixed in length and >> not depend on what other FX bits are set. >> 2) The FX bits must be assigned in order >> 3) An implementation that supports FX for any value will be required to >> understand what fields are added and their size for all FX bits that are >> X or less. > > Again, I agree with this bug report and with the proposed fix. I¹m not > sure that it is best placed in normative language, though (as your > capitalization of MUST suggests). The three points made above are > reminders for future spec developers, and not protocol constraints (well, > mostly not). In video codec standards land, tons of such constraints are > made in early versions of the spec, and never documented outside of > meeting reports. Luckily, there is sufficient institutional memory in the > organization that they are rarely forgotten. In the IETF in general and > in the payload WG in particular, I¹m not so sure about that institutional > memory. > Therefore, I propose to add your 3 constraints in informative language. > See issue #21 below. > > If you have a to-do list for the next rev of the RTP payload how-to: I > think that a section "Extension Mechanisms -- Gremlins here² would be > useful. Also, if there were a generic ftp payload format syntax for > extensions, I think we would have used it (even if its overhead would have > been larger than the one we developed here). Perhaps someone with enough > time on hand could write something like that up, for future use? In video > codec land, we did this once for H.263+ SEI messages, and re-used it > thereafter basically forever. I haven't created such a list yet. I guess it might be time to start one. > >> >> 21. Section 4.9: I think the motivation for the PACI is missing. What >> are the motivation for providing this information in the PACI and what >> would any RTP endpoint or middlebox use the information for. > > I propose to add as second paragraph of 4.9: > ³ > The basic payload header specified in this memo is intentionally limited > to the 16 bits of the NAL unit header so to keep the packetization > overhead to a minimum. However, cases have been identified where it is > advisable to include control information in an easily accessible position > in the packet header, despite the additional overhead. One such control > information is the Temporal Scalability Control Information as specified > in section 4.10/4.10.1 below. However, other types of control information > may be identified in the future. For this reason, an extensible syntax is > included that allows future versions of this specification, or companion > specifications, to defined their own control information and include it > into the payload header. > > When a spec developer devises a new syntax that takes advantage of the > PACI extension mechanism, he/she must follow the constraints listed below; > otherwise the extension mechanism may break. #20². > ³ Appears reasonable. > >> >> 22. Section 4.10: >> >> Can you please provide a name for the functionality set the F0 bit >> represents so that we can use a name to reference it? > > ³Temporal Scalability Control Information². > That should probably also be the section head of 4.10, or we could keep > 4.10, saying that there is only a single extension defined in this memo, > and then 4.10.1 ³Temporal Scalability Control Information². Preferences, > anyone? > Works for me, no preference. Cheers Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 Färögatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Fri Apr 25 10:56:51 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83C191A02C8 for ; Fri, 25 Apr 2014 10:56:41 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.273 X-Spam-Level: X-Spam-Status: No, score=-2.273 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.272, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SP1pCHsl1MHa for ; Fri, 25 Apr 2014 10:56:35 -0700 (PDT) Received: from sabertooth01.qualcomm.com (sabertooth01.qualcomm.com [65.197.215.72]) by ietfa.amsl.com (Postfix) with ESMTP id A75631A03FC for ; Fri, 25 Apr 2014 10:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1398448589; x=1429984589; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=220haAtraN5OMs1wrdrchNpI880Qo47LPzEWwFm3dD4=; b=Lboj0FXWXscM6EMjSMx0SCyHpSL5VdfgAwaxZFzfWanKprVuTZXFSQ2x 7F4+xaFO3+H/lBSn4sGmokSxyzkUfTF9x1uKb1FnAb88BFxGv1tgQr7QZ JY7wCMsYoQkfmVrPBehyuCPZAkzodY83EVq7+TfkgTf/ZircTLTILTOUk w=; X-IronPort-AV: E=McAfee;i="5400,1158,7419"; a="62523344" Received: from ironmsg01-lv.qualcomm.com ([10.47.202.180]) by sabertooth01.qualcomm.com with ESMTP; 25 Apr 2014 10:56:29 -0700 X-IronPort-AV: E=Sophos;i="4.97,928,1389772800"; d="scan'208";a="30121835" Received: from nasanexhc17.na.qualcomm.com ([10.45.158.129]) by ironmsg01-lv.qualcomm.com with ESMTP/TLS/RC4-SHA; 25 Apr 2014 10:56:28 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by NASANEXHC17.na.qualcomm.com ([10.45.158.129]) with mapi id 14.03.0158.001; Fri, 25 Apr 2014 10:56:28 -0700 From: "Wang, Ye-Kui" To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: [payload] Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxkvhCfBfHaXEuyjN/n0X9j0ZrPnXCAgBZg74CAJuPegIAV/V0AgAATNgA= Date: Fri, 25 Apr 2014 17:56:27 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> <535A2C74.7020306@ericsson.com> In-Reply-To: <535A2C74.7020306@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.39.5] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/D5eXFKoD3QkBtn2L5CuENcE9Bfg Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2014 17:56:42 -0000 Thanks! Please see inline below. For the third item, which is also the last one, let me also put my response= here, to draw more attention from interested people in this list: [YK]: They are meant for Receiver and Sender constraints. What we have now = is similar to what we have in RFC 6184 and RFC 6190, with the essential dif= ference been that in AVC there is only one byte of such constraints, while = in HEVC there are more bytes of such constraints. I think what you were thi= nking of is some optimization. Considering the urgent situation that the RF= C is urgent needed by the relevant industries for HEVC deployment, I person= ally would prefer to go ahead without trying to make more optimization here= , but let it be similar as in RFC 6184 and RFC 6190. However, if the group = thinks that we should do more optimization here, then we can try it, but th= is needs careful study and investigation. >> 43. Section 7.2.2: >>=20 >> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 >> recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id >> --+ | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | >> | | profile-space C X C X P profile-id C >> X C X P tier-flag C X C X P=20 >> level-id C X C X P >>=20 >> C: configuration for sending and receiving streams >>=20 >> I don't get this usage or definition of C for these parameters. >>=20 >> So these parameters are receiver capabilities, and they needs to be =20 >> symmetrically used in the signalling, however the actually used=20 >> values in the transmission direction from an O/A agent only needs to=20 >> be within the receivers provided parameters. Thus, they are not a=20 >> "configuration" for the sending direction. >>=20 >> [YK]: These are configuration parameters, meaning indicating both=20 >> what is to be sent and what is to be received for sendrecv and must=20 >> be used symmetrically, and only the latter for recvonly. >=20 > This is at a least wrong for the level-id which a peer agent in an O/A=20 > may downgrade. The rest I agree the payload format itself requires to=20 > be kept in answer, or the corresponding PT dropped. >=20 > [YK]: OK, level-id is downgradable but it is still a configuration,=20 > but a special configuration that can be downgraded. I'd suggest we=20 > keep using the "C" denotation but adding a note to repeat what is said=20 > earlier that level is downgradable, because using any other denotation=20 > is not correct. Please let me know if you think this is not OK, and if=20 > so what kind of changes would you propose. I think that is okay. But, considering that the level is here not combined,= wouldn't make sense to use some other property identifier for it? [YK]: Could you please clarify what does it mean by "not combined", and by = "to use some other property identifier"? >=20 >>=20 >> Also I really don't get the X's in the answer: recvonly,=20 >> recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The=20 >> answering agent is going to receive stream in both these cases, thus=20 >> it needs to declare the payload type, and these parameters MUST be=20 >> included (unless defaulted). So why is they marked with X? >>=20 >> [YK]: When recv-sub-layer-id is included in an SDP answer, it means =20 >> that the answer chooses one of the multiple operation points=20 >> corresponding to the offered or declared sub-layers in the sprop-vps,=20 >> and in this case there is no need to repeat the information of=20 >> profile, level etc. in the SDP, thus there are marked with X (MUST=20 >> NOT be present). >=20 > This appears very strange to me. The Offerer includes a VPS that=20 > contains options for what it can send. The answering party, i.e. > which will receive this stream needs in accordance to SDP O/A rules=20 > declare which PTs it can receive. This PT must be corresponding to the=20 > one that the Offerer proposed to send and in which the VPS was=20 > included. >=20 > For me this implies the following: >=20 > 1. The parameters indicating the basic configuraiton of the payload=20 > type, i.e profile, level ,tier etc. MUST be present in the answer with=20 > the same values as in the offer it responds to. Otherwise the PT=20 > matching rule fails. >=20 > 2. In addition the PT to receive on MUST be declared as existing to=20 > ensure that any middleboxes doesn't block it. Thus, explicit=20 > indication of what will go in it is really recommended, especially=20 > considering i all the other cases that information will be explicit=20 > here. Thus, I think the X's are plain wrong. Or I am still=20 > misunderstanding something here. >=20 > [YK]: Let's use an example, which should help. Let's say the offer SDP=20 > includes a VPS that provides two operation points, 720p@15fps and=20 > 720p@30fps, corresponding to recv-sub-layer-id 0 and 1, respectively. > The profile, tier, level, and so on for each operation point are=20 > signalled by the VPS. If recv-sub-layer-id=3D0 is included in an SDP=20 > answer, then that means the answerer chooses to operate at 720p@15fps.=20 > In this case, the profile, tier, level, and so on for the chosen=20 > operation point does not need to be explicitly sent again in the SDP,=20 > as it is clear what they are (they are signalled in the VPS in the SDP=20 > offer). So there is no ambiguity here. Are you suggesting that in some=20 > use cases even there is no ambiguity the configuration information=20 > must always be explicitly signalled (i.e. values that are derivable)?=20 > If yes, could you please explain in what use cases and why. Note that=20 > we had the same mechanism specified for SVC in RFC 6190. Also, on PT=20 > use, we have allowed the use of a different PT value in an SDP answer=20 > than the matching PT value in the offer since at least RFC 3984 as=20 > long as the PT matching is clear. Okay, I can now put the finger on what troubles me with this. I think this = is violation or at least an omission of the principle of SDP Offer/Answer. = The reason is that you substitute the answering explicitly declaring its re= ceiving capabilities with a reference to a particular instantiation of a vi= deo sequence parameter set that falls within the receiver's capabilities. T= hus, the offerer does not know the receiver's actually capabilities if it n= eeds to reconfigure the encoder, or in case the RTP payload type is going t= o be used by multiple media streams and possibly different encoder instance= s. In addition it creates an issue for any middlebox that is on the signalling= path that it keeps track and state for what was in the offer to know what = the answer means. Thus, I think it is important that the answerer doesn't treat this situatio= n differently, and instead declare its capabilities as normally in answer. = The recv-sub-layer-id parameter is an optional parameter and it is also str= eam specific, while the profile, level, etc are generically applicable to t= he RTP payload type do applies on RTP session level. To conclude, the recv-sub-layer-id is fine as a stream specific selection m= echanism, but does not replace the capability declaration the regular param= eters perform. [YK]: OK, we will try to make this change. >=20 >>=20 >> 44. Section 7.2.2: >>=20 >> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 >> recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id >> --+ | | | sendrecv w/o recv-sub-layer-id --+ | | | | | | | >> | | interop-constraints C X C X P=20 >> profile-compatibility-indicator C X C X P >>=20 >> I don't understand how these parameters really are used for receiver=20 >> capability declarations, so I don't understand why the Cs are C and=20 >> not P. >>=20 >> [YK]: As explained earlier, these are also parts of the media type =20 >> configuration, same as profile, tier, and level, and their usage are=20 >> the same as those too. >=20 > First of all these parameters are not included as the ones that have=20 > to be symmetrically provided. I also think they could be SDP agent=20 > specifically given. But, that can be discussed. But, according to the=20 > current text, I fail to see that these needs to be handled as the=20 > profile-id, level-id, etc. >=20 > The only constraint I find for these two flags applies under=20 > multicast. >=20 > Either they need text, or these C's need to be changed. >=20 > [YK]: Maybe you were initially reading version -01 and did not check=20 > version -02 on this part. In version -02, these parameters are=20 > included in the list of configuration parameters that are specified to=20 > be used symmetrically. Okay, I had missed that change in the -02 version. After having given this some thought and re-read the definitions and releva= nt SDP O/A parts I think I have found a new issue. This issue is related to the interpretation of "interop-constraints", and = "profile-compatibility-indicator". It is not clear in which direction they = should be interpreted in the various cases. Are they constraints on what th= e receiver/decoder accept or constraints the encoder promise to fulfil or b= oth? I think the "profile-compatibility-indicator" is an excellent case to show = that neither of the alternative appears to work or provide any benefit. Thi= s example will use a future profile that I call Baseline, which is a sub-se= t of Main. Receiver constraints: A sends it offers with a RTP payload type (97) which is Main but with profi= le-compatibility-indicator indicating Main and Baseline. This means the off= erer prefers to do main but knows of baseline. Answerer is baseline only. T= hus it detects a profile not supported, but compatibility indicates support= . However, due to the other payload type rules, the answerer must reject PT= =3D97 as it can't answer with 97=3DBaseline. Nor can it reject 97 and and 9= 8 which is baseline, and only use 97 in answerer to offerer direction. And = only receive using 98. Especially not if 97 was the only PT offered, then you have no compatibilit= y. Instead, to my understanding to achieve receiver constraints the offerer mu= st include one PT per profile it supports, the profile-compatibility-indica= tor is useless in this case. Forcing the answerer to copy it and be forced to have a the same receiver c= onstraints is just useless. Sender constraints: If this is a sender constraint, then it actually tells something about the = sender's capability of providing something that matches more profiles than = what it is willing to receive. However, the answerer if it doesn't support = the any of the PTs and their profile-id values then it anyway have to refus= e the PT, even if it could receive the stream from the offerer. Instead, al= so here the offerer needs to indicate all profiles it can encoded for and o= ffer them. Which implies also decoding capability. In addition, if this is = a sender constraint, using C as classification (i.e. must copy) in answer, = if the encoder in the answerer supports the payload types profile-id but, c= an't ensure or know about all the profile-compatibility flags, then it has = to refuse the payload type despite it supporting it. Result, forcing C on a sender constraint interpretation only results in fai= lures and no benefit. Receiver and Sender constraints. To my understanding such an interpretation only forces the case of more rej= ections, if the answerer doesn't have an equally capable encoder and decode= r as the offerer and can match both sets of constraints. In addition the co= mpatibility has not provided any benefit for the two O/A agents. So, the question becomes have I misunderstood something about how you inten= ded to be able to use this parameter? If I haven't I think these parameters usage is seriously in question. If on= e would like to use it then the must not alter constraint for profile becom= es a serious issue. If one would remove that then I see that using "profile= -compatibility-indicator" could be possibly beneficial. But, I haven't done= a full out design of how to resolve the PT matching and deal with asymmetr= ic usage within the session. The same type of analysis and clarification on usage needs to happen for th= e "interop-constraints". Then we have the question of how one shall interpret and deal with unknown = flag values. These needs to cause only failures when required for ensuring = video compatibility. Not cause negotiation failures for unknown cases that = doesn't affect video interoperability. [YK]: They are meant for Receiver and Sender constraints. What we have now = is similar to what we have in RFC 6184 and RFC 6190, with the essential dif= ference been that in AVC there is only one byte of such constraints, while = in HEVC there are more bytes of such constraints. I think what you were thi= nking of is some optimization. Considering the urgent situation that the RF= C is urgent needed by the relevant industries for HEVC deployment, I person= ally would prefer to go ahead without trying to make more optimization here= , but let it be similar as in RFC 6184 and RFC 6190. However, if the group = thinks that we should do more optimization here, then we can try it, but th= is needs careful study and investigation. BR, YK From nobody Sun Apr 27 23:50:39 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 416F11A0729 for ; Sun, 27 Apr 2014 23:50:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.501 X-Spam-Level: X-Spam-Status: No, score=-1.501 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id poW71HlesYue for ; Sun, 27 Apr 2014 23:50:35 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) by ietfa.amsl.com (Postfix) with ESMTP id 91FB91A0727 for ; Sun, 27 Apr 2014 23:50:34 -0700 (PDT) X-AuditID: c1b4fb30-f791c6d000005f7c-15-535dfa38795d Received: from ESESSHC018.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id 0B.9E.24444.83AFD535; Mon, 28 Apr 2014 08:50:32 +0200 (CEST) Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.74) with Microsoft SMTP Server id 14.3.174.1; Mon, 28 Apr 2014 08:50:31 +0200 Message-ID: <535DFA38.5090901@ericsson.com> Date: Mon, 28 Apr 2014 08:50:32 +0200 From: Magnus Westerlund User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Wang, Ye-Kui" , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> <535A2C74.7020306@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrALMWRmVeSWpSXmKPExsUyM+Jvja7Fr9hgg5PdxhanLu5jtbh08SyT xZM1x5gdmD2WLPnJ5LFo6jNGjy+XP7MFMEdx2aSk5mSWpRbp2yVwZSztO8VY0F9TsXzfRvYG xt1JXYycHBICJhJnV+5hhLDFJC7cW8/WxcjFISRwlFHixIyNrBDOckaJXaebmboYOTh4BbQl et6LgjSwCKhKzLx2mR3EZhOwkLj5o5ENxBYVCJZYOmcxC4jNKyAocXLmExaQOSICmxglFi5e A1YkLOAosaGxlQliwQEmic0fGsDO4ATqnv7xGCvIMgkBcYmexiCQMLOAnsSUqy2MELa8RPPW 2cwgthDQPQ1NHawTGAVnIdk3C0nLLCQtCxiZVzGKFqcWJ+WmGxnppRZlJhcX5+fp5aWWbGIE BvHBLb8NdjC+fO54iFGAg1GJh7f4S2SwEGtiWXFl7iFGaQ4WJXHeb2fdg4UE0hNLUrNTUwtS i+KLSnNSiw8xMnFwSjUwRrp3bfp8WNpN9Xyw7gXr/bq58zSfd8d/M64/eUp8yp9eNodrnfE1 F13WHXKS35t79tuW37WWy80cj8UZsvxZmeOWqnfaneHgoe3fNQ+eLf4r3Fu2n3/uEi71z7X8 j0/eN+/yW59yeO0LS/GH4W2t/gk8v7Pys9pmL2hS+cCnWyDHPG+K+K04JZbijERDLeai4kQA QU3FSkMCAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/payload/8LIJDKw8ekW2vREpe6TgZRjsY5M Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 06:50:38 -0000 On 2014-04-25 19:56, Wang, Ye-Kui wrote: > Thanks! Please see inline below. > > For the third item, which is also the last one, let me also put my > response here, to draw more attention from interested people in this > list: > > [YK]: They are meant for Receiver and Sender constraints. What we > have now is similar to what we have in RFC 6184 and RFC 6190, with > the essential difference been that in AVC there is only one byte of > such constraints, while in HEVC there are more bytes of such > constraints. I think what you were thinking of is some optimization. > Considering the urgent situation that the RFC is urgent needed by the > relevant industries for HEVC deployment, I personally would prefer to > go ahead without trying to make more optimization here, but let it be > similar as in RFC 6184 and RFC 6190. However, if the group thinks > that we should do more optimization here, then we can try it, but > this needs careful study and investigation. No, I am not talking about optimizations here. I don't even see how they work at all. Especially the profile compatibility where they potential usage is being prevented by other payload type rules. Thus, I think we need to dig into this, or simply remove it from the parameters as it provides no benefit. Or please explain how it can provide any benefit. I think the interop flags, at least the current set will work as receiver primarily and even sender constraints for bi-directional media blocks. If you have different set of constraints that you want to express you will need to offer and answer with multiple payload types, each expressing the different sets. But, the basic issue here has to do with what type of negotiation if any and what applicability the constraint have. I assume on the basic level anyone using the PT will have to have their stream fulfil the signalled constraints. Thus, they are receiver only. By, mandating them to the "C" category, i.e. copy then they becomes sender constraints. Thus, no possibility to negotiate them. Just express them. I think I am fine with this, but the issue is that you don't know what future interop constraints that will be defined and what applicability it will have. If it is something that you actually want to negotiate, then you start exploding the PT combinations. See inline for response to the other questions. Cheers Magnus > >>> 43. Section 7.2.2: >>> >>> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly >>> w/o recv-sub-layer-id --+ | | answer: sendrecv, >>> recv-sub-layer-id --+ | | | sendrecv w/o recv-sub-layer-id --+ >>> | | | | | | | | | profile-space C X >>> C X P profile-id C X C X P tier-flag >>> C X C X P level-id C X C X P >>> >>> C: configuration for sending and receiving streams >>> >>> I don't get this usage or definition of C for these parameters. >>> >>> So these parameters are receiver capabilities, and they needs to >>> be symmetrically used in the signalling, however the actually >>> used values in the transmission direction from an O/A agent only >>> needs to be within the receivers provided parameters. Thus, they >>> are not a "configuration" for the sending direction. >>> >>> [YK]: These are configuration parameters, meaning indicating both >>> what is to be sent and what is to be received for sendrecv and >>> must be used symmetrically, and only the latter for recvonly. >> >> This is at a least wrong for the level-id which a peer agent in an >> O/A may downgrade. The rest I agree the payload format itself >> requires to be kept in answer, or the corresponding PT dropped. >> >> [YK]: OK, level-id is downgradable but it is still a configuration, >> but a special configuration that can be downgraded. I'd suggest we >> keep using the "C" denotation but adding a note to repeat what is >> said earlier that level is downgradable, because using any other >> denotation is not correct. Please let me know if you think this is >> not OK, and if so what kind of changes would you propose. > > I think that is okay. But, considering that the level is here not > combined, wouldn't make sense to use some other property identifier > for it? > > [YK]: Could you please clarify what does it mean by "not combined", > and by "to use some other property identifier"? With combined, I meant the arrangement in H.264 with profile-level-id, i.e. the profile and level was combined parameters. With other property identifier, I meant replacing the "C" with some other character expressing the property of this being allowed to downgrade. > >> >>> >>> Also I really don't get the X's in the answer: recvonly, >>> recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The >>> answering agent is going to receive stream in both these cases, >>> thus it needs to declare the payload type, and these parameters >>> MUST be included (unless defaulted). So why is they marked with >>> X? >>> >>> [YK]: When recv-sub-layer-id is included in an SDP answer, it >>> means that the answer chooses one of the multiple operation >>> points corresponding to the offered or declared sub-layers in the >>> sprop-vps, and in this case there is no need to repeat the >>> information of profile, level etc. in the SDP, thus there are >>> marked with X (MUST NOT be present). >> >> This appears very strange to me. The Offerer includes a VPS that >> contains options for what it can send. The answering party, i.e. >> which will receive this stream needs in accordance to SDP O/A rules >> declare which PTs it can receive. This PT must be corresponding to >> the one that the Offerer proposed to send and in which the VPS was >> included. >> >> For me this implies the following: >> >> 1. The parameters indicating the basic configuraiton of the payload >> type, i.e profile, level ,tier etc. MUST be present in the answer >> with the same values as in the offer it responds to. Otherwise the >> PT matching rule fails. >> >> 2. In addition the PT to receive on MUST be declared as existing to >> ensure that any middleboxes doesn't block it. Thus, explicit >> indication of what will go in it is really recommended, especially >> considering i all the other cases that information will be >> explicit here. Thus, I think the X's are plain wrong. Or I am still >> misunderstanding something here. >> >> [YK]: Let's use an example, which should help. Let's say the offer >> SDP includes a VPS that provides two operation points, 720p@15fps >> and 720p@30fps, corresponding to recv-sub-layer-id 0 and 1, >> respectively. The profile, tier, level, and so on for each >> operation point are signalled by the VPS. If recv-sub-layer-id=0 is >> included in an SDP answer, then that means the answerer chooses to >> operate at 720p@15fps. In this case, the profile, tier, level, and >> so on for the chosen operation point does not need to be explicitly >> sent again in the SDP, as it is clear what they are (they are >> signalled in the VPS in the SDP offer). So there is no ambiguity >> here. Are you suggesting that in some use cases even there is no >> ambiguity the configuration information must always be explicitly >> signalled (i.e. values that are derivable)? If yes, could you >> please explain in what use cases and why. Note that we had the same >> mechanism specified for SVC in RFC 6190. Also, on PT use, we have >> allowed the use of a different PT value in an SDP answer than the >> matching PT value in the offer since at least RFC 3984 as long as >> the PT matching is clear. > > Okay, I can now put the finger on what troubles me with this. I think > this is violation or at least an omission of the principle of SDP > Offer/Answer. The reason is that you substitute the answering > explicitly declaring its receiving capabilities with a reference to a > particular instantiation of a video sequence parameter set that falls > within the receiver's capabilities. Thus, the offerer does not know > the receiver's actually capabilities if it needs to reconfigure the > encoder, or in case the RTP payload type is going to be used by > multiple media streams and possibly different encoder instances. > > In addition it creates an issue for any middlebox that is on the > signalling path that it keeps track and state for what was in the > offer to know what the answer means. > > Thus, I think it is important that the answerer doesn't treat this > situation differently, and instead declare its capabilities as > normally in answer. The recv-sub-layer-id parameter is an optional > parameter and it is also stream specific, while the profile, level, > etc are generically applicable to the RTP payload type do applies on > RTP session level. > > To conclude, the recv-sub-layer-id is fine as a stream specific > selection mechanism, but does not replace the capability declaration > the regular parameters perform. > > [YK]: OK, we will try to make this change. Great! > >> >>> >>> 44. Section 7.2.2: >>> >>> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly >>> w/o recv-sub-layer-id --+ | | answer: sendrecv, >>> recv-sub-layer-id --+ | | | sendrecv w/o recv-sub-layer-id --+ >>> | | | | | | | | | interop-constraints C X >>> C X P profile-compatibility-indicator C X C X P >>> >>> I don't understand how these parameters really are used for >>> receiver capability declarations, so I don't understand why the >>> Cs are C and not P. >>> >>> [YK]: As explained earlier, these are also parts of the media >>> type configuration, same as profile, tier, and level, and their >>> usage are the same as those too. >> >> First of all these parameters are not included as the ones that >> have to be symmetrically provided. I also think they could be SDP >> agent specifically given. But, that can be discussed. But, >> according to the current text, I fail to see that these needs to be >> handled as the profile-id, level-id, etc. >> >> The only constraint I find for these two flags applies under >> multicast. >> >> Either they need text, or these C's need to be changed. >> >> [YK]: Maybe you were initially reading version -01 and did not >> check version -02 on this part. In version -02, these parameters >> are included in the list of configuration parameters that are >> specified to be used symmetrically. > > Okay, I had missed that change in the -02 version. > > After having given this some thought and re-read the definitions and > relevant SDP O/A parts I think I have found a new issue. > > This issue is related to the interpretation of > "interop-constraints", and "profile-compatibility-indicator". It is > not clear in which direction they should be interpreted in the > various cases. Are they constraints on what the receiver/decoder > accept or constraints the encoder promise to fulfil or both? > > I think the "profile-compatibility-indicator" is an excellent case to > show that neither of the alternative appears to work or provide any > benefit. This example will use a future profile that I call Baseline, > which is a sub-set of Main. > > Receiver constraints: > > A sends it offers with a RTP payload type (97) which is Main but with > profile-compatibility-indicator indicating Main and Baseline. This > means the offerer prefers to do main but knows of baseline. Answerer > is baseline only. Thus it detects a profile not supported, but > compatibility indicates support. However, due to the other payload > type rules, the answerer must reject PT=97 as it can't answer with > 97=Baseline. Nor can it reject 97 and and 98 which is baseline, and > only use 97 in answerer to offerer direction. And only receive using > 98. Especially not if 97 was the only PT offered, then you have no > compatibility. > > Instead, to my understanding to achieve receiver constraints the > offerer must include one PT per profile it supports, the > profile-compatibility-indicator is useless in this case. > > Forcing the answerer to copy it and be forced to have a the same > receiver constraints is just useless. > > Sender constraints: > > If this is a sender constraint, then it actually tells something > about the sender's capability of providing something that matches > more profiles than what it is willing to receive. However, the > answerer if it doesn't support the any of the PTs and their > profile-id values then it anyway have to refuse the PT, even if it > could receive the stream from the offerer. Instead, also here the > offerer needs to indicate all profiles it can encoded for and offer > them. Which implies also decoding capability. In addition, if this is > a sender constraint, using C as classification (i.e. must copy) in > answer, if the encoder in the answerer supports the payload types > profile-id but, can't ensure or know about all the > profile-compatibility flags, then it has to refuse the payload type > despite it supporting it. > > Result, forcing C on a sender constraint interpretation only results > in failures and no benefit. > > Receiver and Sender constraints. > > To my understanding such an interpretation only forces the case of > more rejections, if the answerer doesn't have an equally capable > encoder and decoder as the offerer and can match both sets of > constraints. In addition the compatibility has not provided any > benefit for the two O/A agents. > > So, the question becomes have I misunderstood something about how you > intended to be able to use this parameter? > > If I haven't I think these parameters usage is seriously in question. > If one would like to use it then the must not alter constraint for > profile becomes a serious issue. If one would remove that then I see > that using "profile-compatibility-indicator" could be possibly > beneficial. But, I haven't done a full out design of how to resolve > the PT matching and deal with asymmetric usage within the session. > > The same type of analysis and clarification on usage needs to happen > for the "interop-constraints". > > Then we have the question of how one shall interpret and deal with > unknown flag values. These needs to cause only failures when required > for ensuring video compatibility. Not cause negotiation failures for > unknown cases that doesn't affect video interoperability. > > [YK]: They are meant for Receiver and Sender constraints. What we > have now is similar to what we have in RFC 6184 and RFC 6190, with > the essential difference been that in AVC there is only one byte of > such constraints, while in HEVC there are more bytes of such > constraints. I think what you were thinking of is some optimization. > Considering the urgent situation that the RFC is urgent needed by the > relevant industries for HEVC deployment, I personally would prefer to > go ahead without trying to make more optimization here, but let it be > similar as in RFC 6184 and RFC 6190. However, if the group thinks > that we should do more optimization here, then we can try it, but > this needs careful study and investigation. > > BR, YK > > > -- Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 Färögatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Mon Apr 28 15:06:11 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D39981A883D for ; Mon, 28 Apr 2014 15:05:59 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.952 X-Spam-Level: X-Spam-Status: No, score=-4.952 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JnPLMU1Buf0X for ; Mon, 28 Apr 2014 15:05:53 -0700 (PDT) Received: from wolverine01.qualcomm.com (wolverine01.qualcomm.com [199.106.114.254]) by ietfa.amsl.com (Postfix) with ESMTP id 6D5181A8844 for ; Mon, 28 Apr 2014 15:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1398722627; x=1430258627; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=woPXS2cQzmMVnNL4RSq1IoiImbON2EjizhRbLRNra6k=; b=YQGkZ5MTngXWe1r9jWdN0NP+HU2l7MxySRO8BfoYzxla8k/U5NgXycq5 iO6lL7Id7M8Vl1PUirdMNEvD2GjgkzsUvfdDLEhHzaoIAz93n48Lb7nm7 kyyRh9On7bVGz5Fq7CI8lNRntWxTF+dH4dCvkL03BZNn2FKTLTXOYvqSZ M=; X-IronPort-AV: E=McAfee;i="5400,1158,7422"; a="31827828" Received: from ironmsg01-lv.qualcomm.com ([10.47.202.180]) by wolverine01.qualcomm.com with ESMTP; 28 Apr 2014 15:03:46 -0700 X-IronPort-AV: E=Sophos;i="4.97,946,1389772800"; d="scan'208";a="30144415" Received: from nasanexhc01.na.qualcomm.com ([172.30.48.25]) by ironmsg01-lv.qualcomm.com with ESMTP/TLS/RC4-SHA; 28 Apr 2014 15:03:45 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by NASANEXHC01.na.qualcomm.com ([172.30.48.25]) with mapi id 14.03.0158.001; Mon, 28 Apr 2014 15:03:45 -0700 From: "Wang, Ye-Kui" To: Magnus Westerlund , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" Thread-Topic: [payload] Comments on draft-ietf-payload-rtp-h265-02 Thread-Index: AQHPNJxkvhCfBfHaXEuyjN/n0X9j0ZrPnXCAgBZg74CAJuPegIAV/V0AgAATNgCABHWcAIAAffAA Date: Mon, 28 Apr 2014 22:03:44 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A8345242101@nasanexd02f.na.qualcomm.com> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> <535A2C74.7020306@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> <535DFA38.5090901@ericsson.com> In-Reply-To: <535DFA38.5090901@ericsson.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.48.1] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/jLGBeLADbOjeL0QJ65G2n-4bLlk Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Apr 2014 22:06:00 -0000 Thanks for the clarification on the first point. Now again on the third point about profile-compatibility-indicator and inte= rop-constraints. profile-compatibility-indicator tells the list of profiles the bitstream co= nforms to. The profile identified by profile-id would also be indicated by = profile-compatibility-indicator. The information that is covered by profile= -id but not by profile-compatibility-indicator is as follows (note 6 in sub= clause 7.4.4 of the HEVC spec): NOTE 6 - When the coded video sequence conf= orms to multiple profiles, general_profile_idc should indicate the profile = that provides the preferred decoded result or the preferred bitstream ident= ification, as determined by the encoder (in a manner not specified in this = Specification). Therefore, the use of profile-id and profile-compatibility-indicator togeth= er here in this payload format spec is similar as the use in RFC 6184 of pr= ofile_idc and profile-iop (the first two bytes of profile-level-id, except = the particular and weird indication of level 1b using bit 4 of profile-iop)= . What we could do to optimize the O/A herein is to allow for the change of t= he values of profile-id and profile-compatibility-indicator as long as they= represent the same set of profiles. But I am not sure whether or how many = PT values could be saved in practice.=20 interop-constraints is something new in HEVC compared to AVC. Example const= raints here include 1) whether the video is a frame-packed 3D video; and 2)= whether the coded pictures are coded frames only or there are interlaced s= tuff involved. Herein I think it is preferable to require both sides to mai= ntain the values, because if an offerer offers to communicate with clean pr= ogressive video without frame packing, it also expects the other side to ke= ep the restrictions and do the same. BR, YK -----Original Message----- From: Magnus Westerlund [mailto:magnus.westerlund@ericsson.com]=20 Sent: Sunday, April 27, 2014 11:51 PM To: Wang, Ye-Kui; payload@ietf.org; draft-ietf-payload-rtp-h265@tools.ietf.= org Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 On 2014-04-25 19:56, Wang, Ye-Kui wrote: > Thanks! Please see inline below. >=20 > For the third item, which is also the last one, let me also put my=20 > response here, to draw more attention from interested people in this > list: >=20 > [YK]: They are meant for Receiver and Sender constraints. What we have=20 > now is similar to what we have in RFC 6184 and RFC 6190, with the=20 > essential difference been that in AVC there is only one byte of such=20 > constraints, while in HEVC there are more bytes of such constraints. I=20 > think what you were thinking of is some optimization. > Considering the urgent situation that the RFC is urgent needed by the=20 > relevant industries for HEVC deployment, I personally would prefer to=20 > go ahead without trying to make more optimization here, but let it be=20 > similar as in RFC 6184 and RFC 6190. However, if the group thinks that=20 > we should do more optimization here, then we can try it, but this=20 > needs careful study and investigation. No, I am not talking about optimizations here. I don't even see how they wo= rk at all. Especially the profile compatibility where they potential usage = is being prevented by other payload type rules. Thus, I think we need to di= g into this, or simply remove it from the parameters as it provides no bene= fit. Or please explain how it can provide any benefit. I think the interop flags, at least the current set will work as receiver p= rimarily and even sender constraints for bi-directional media blocks. If yo= u have different set of constraints that you want to express you will need = to offer and answer with multiple payload types, each expressing the differ= ent sets. But, the basic issue here has to do with what type of negotiation if any an= d what applicability the constraint have. I assume on the basic level anyon= e using the PT will have to have their stream fulfil the signalled constrai= nts. Thus, they are receiver only. By, mandating them to the "C" category, i.e. copy then they becomes sender constraints. Thus, no possibil= ity to negotiate them. Just express them. I think I am fine with this, but = the issue is that you don't know what future interop constraints that will = be defined and what applicability it will have. If it is something that you= actually want to negotiate, then you start exploding the PT combinations. See inline for response to the other questions. Cheers Magnus >=20 >>> 43. Section 7.2.2: >>>=20 >>> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 >>> recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id --+=20 >>> | | | sendrecv w/o recv-sub-layer-id --+ >>> | | | | | | | | | profile-space C X >>> C X P profile-id C X C X P tier-flag >>> C X C X P level-id C X C X P >>>=20 >>> C: configuration for sending and receiving streams >>>=20 >>> I don't get this usage or definition of C for these parameters. >>>=20 >>> So these parameters are receiver capabilities, and they needs to be=20 >>> symmetrically used in the signalling, however the actually used=20 >>> values in the transmission direction from an O/A agent only needs to=20 >>> be within the receivers provided parameters. Thus, they are not a=20 >>> "configuration" for the sending direction. >>>=20 >>> [YK]: These are configuration parameters, meaning indicating both =20 >>> what is to be sent and what is to be received for sendrecv and must=20 >>> be used symmetrically, and only the latter for recvonly. >>=20 >> This is at a least wrong for the level-id which a peer agent in an=20 >> O/A may downgrade. The rest I agree the payload format itself=20 >> requires to be kept in answer, or the corresponding PT dropped. >>=20 >> [YK]: OK, level-id is downgradable but it is still a configuration, =20 >> but a special configuration that can be downgraded. I'd suggest we =20 >> keep using the "C" denotation but adding a note to repeat what is=20 >> said earlier that level is downgradable, because using any other=20 >> denotation is not correct. Please let me know if you think this is=20 >> not OK, and if so what kind of changes would you propose. >=20 > I think that is okay. But, considering that the level is here not=20 > combined, wouldn't make sense to use some other property identifier=20 > for it? >=20 > [YK]: Could you please clarify what does it mean by "not combined",=20 > and by "to use some other property identifier"? With combined, I meant the arrangement in H.264 with profile-level-id, i.e.= the profile and level was combined parameters. With other property identifier, I meant replacing the "C" with some other c= haracter expressing the property of this being allowed to downgrade. >=20 >>=20 >>>=20 >>> Also I really don't get the X's in the answer: recvonly,=20 >>> recv-sub-layer-id, and answer: sendrecv, recv-sub-layer-id. The=20 >>> answering agent is going to receive stream in both these cases, thus=20 >>> it needs to declare the payload type, and these parameters MUST be=20 >>> included (unless defaulted). So why is they marked with X? >>>=20 >>> [YK]: When recv-sub-layer-id is included in an SDP answer, it means=20 >>> that the answer chooses one of the multiple operation points=20 >>> corresponding to the offered or declared sub-layers in the=20 >>> sprop-vps, and in this case there is no need to repeat the=20 >>> information of profile, level etc. in the SDP, thus there are marked=20 >>> with X (MUST NOT be present). >>=20 >> This appears very strange to me. The Offerer includes a VPS that=20 >> contains options for what it can send. The answering party, i.e. >> which will receive this stream needs in accordance to SDP O/A rules =20 >> declare which PTs it can receive. This PT must be corresponding to=20 >> the one that the Offerer proposed to send and in which the VPS was =20 >> included. >>=20 >> For me this implies the following: >>=20 >> 1. The parameters indicating the basic configuraiton of the payload =20 >> type, i.e profile, level ,tier etc. MUST be present in the answer=20 >> with the same values as in the offer it responds to. Otherwise the PT=20 >> matching rule fails. >>=20 >> 2. In addition the PT to receive on MUST be declared as existing to =20 >> ensure that any middleboxes doesn't block it. Thus, explicit=20 >> indication of what will go in it is really recommended, especially =20 >> considering i all the other cases that information will be explicit=20 >> here. Thus, I think the X's are plain wrong. Or I am still =20 >> misunderstanding something here. >>=20 >> [YK]: Let's use an example, which should help. Let's say the offer=20 >> SDP includes a VPS that provides two operation points, 720p@15fps and=20 >> 720p@30fps, corresponding to recv-sub-layer-id 0 and 1, respectively.=20 >> The profile, tier, level, and so on for each operation point are=20 >> signalled by the VPS. If recv-sub-layer-id=3D0 is included in an SDP=20 >> answer, then that means the answerer chooses to operate at=20 >> 720p@15fps. In this case, the profile, tier, level, and so on for the=20 >> chosen operation point does not need to be explicitly sent again in=20 >> the SDP, as it is clear what they are (they are signalled in the VPS=20 >> in the SDP offer). So there is no ambiguity here. Are you suggesting=20 >> that in some use cases even there is no ambiguity the configuration=20 >> information must always be explicitly signalled (i.e. values that are=20 >> derivable)? If yes, could you please explain in what use cases and=20 >> why. Note that we had the same mechanism specified for SVC in RFC=20 >> 6190. Also, on PT use, we have allowed the use of a different PT=20 >> value in an SDP answer than the matching PT value in the offer since=20 >> at least RFC 3984 as long as the PT matching is clear. >=20 > Okay, I can now put the finger on what troubles me with this. I think=20 > this is violation or at least an omission of the principle of SDP=20 > Offer/Answer. The reason is that you substitute the answering=20 > explicitly declaring its receiving capabilities with a reference to a=20 > particular instantiation of a video sequence parameter set that falls=20 > within the receiver's capabilities. Thus, the offerer does not know=20 > the receiver's actually capabilities if it needs to reconfigure the=20 > encoder, or in case the RTP payload type is going to be used by=20 > multiple media streams and possibly different encoder instances. >=20 > In addition it creates an issue for any middlebox that is on the=20 > signalling path that it keeps track and state for what was in the=20 > offer to know what the answer means. >=20 > Thus, I think it is important that the answerer doesn't treat this=20 > situation differently, and instead declare its capabilities as=20 > normally in answer. The recv-sub-layer-id parameter is an optional=20 > parameter and it is also stream specific, while the profile, level,=20 > etc are generically applicable to the RTP payload type do applies on=20 > RTP session level. >=20 > To conclude, the recv-sub-layer-id is fine as a stream specific=20 > selection mechanism, but does not replace the capability declaration=20 > the regular parameters perform. >=20 > [YK]: OK, we will try to make this change. Great! >=20 >>=20 >>>=20 >>> 44. Section 7.2.2: >>>=20 >>> sendonly --+ answer: recvonly, recv-sub-layer-id --+ | recvonly w/o=20 >>> recv-sub-layer-id --+ | | answer: sendrecv, recv-sub-layer-id --+=20 >>> | | | sendrecv w/o recv-sub-layer-id --+ >>> | | | | | | | | | interop-constraints C X >>> C X P profile-compatibility-indicator C X C X P >>>=20 >>> I don't understand how these parameters really are used for receiver=20 >>> capability declarations, so I don't understand why the Cs are C and=20 >>> not P. >>>=20 >>> [YK]: As explained earlier, these are also parts of the media type=20 >>> configuration, same as profile, tier, and level, and their usage are=20 >>> the same as those too. >>=20 >> First of all these parameters are not included as the ones that have=20 >> to be symmetrically provided. I also think they could be SDP agent=20 >> specifically given. But, that can be discussed. But, according to the=20 >> current text, I fail to see that these needs to be handled as the=20 >> profile-id, level-id, etc. >>=20 >> The only constraint I find for these two flags applies under=20 >> multicast. >>=20 >> Either they need text, or these C's need to be changed. >>=20 >> [YK]: Maybe you were initially reading version -01 and did not check=20 >> version -02 on this part. In version -02, these parameters are=20 >> included in the list of configuration parameters that are specified=20 >> to be used symmetrically. >=20 > Okay, I had missed that change in the -02 version. >=20 > After having given this some thought and re-read the definitions and=20 > relevant SDP O/A parts I think I have found a new issue. >=20 > This issue is related to the interpretation of "interop-constraints",=20 > and "profile-compatibility-indicator". It is not clear in which=20 > direction they should be interpreted in the various cases. Are they=20 > constraints on what the receiver/decoder accept or constraints the=20 > encoder promise to fulfil or both? >=20 > I think the "profile-compatibility-indicator" is an excellent case to=20 > show that neither of the alternative appears to work or provide any=20 > benefit. This example will use a future profile that I call Baseline,=20 > which is a sub-set of Main. >=20 > Receiver constraints: >=20 > A sends it offers with a RTP payload type (97) which is Main but with=20 > profile-compatibility-indicator indicating Main and Baseline. This=20 > means the offerer prefers to do main but knows of baseline. Answerer=20 > is baseline only. Thus it detects a profile not supported, but=20 > compatibility indicates support. However, due to the other payload=20 > type rules, the answerer must reject PT=3D97 as it can't answer with=20 > 97=3DBaseline. Nor can it reject 97 and and 98 which is baseline, and=20 > only use 97 in answerer to offerer direction. And only receive using=20 > 98. Especially not if 97 was the only PT offered, then you have no=20 > compatibility. >=20 > Instead, to my understanding to achieve receiver constraints the=20 > offerer must include one PT per profile it supports, the=20 > profile-compatibility-indicator is useless in this case. >=20 > Forcing the answerer to copy it and be forced to have a the same=20 > receiver constraints is just useless. >=20 > Sender constraints: >=20 > If this is a sender constraint, then it actually tells something about=20 > the sender's capability of providing something that matches more=20 > profiles than what it is willing to receive. However, the answerer if=20 > it doesn't support the any of the PTs and their profile-id values then=20 > it anyway have to refuse the PT, even if it could receive the stream=20 > from the offerer. Instead, also here the offerer needs to indicate all=20 > profiles it can encoded for and offer them. Which implies also=20 > decoding capability. In addition, if this is a sender constraint,=20 > using C as classification (i.e. must copy) in answer, if the encoder=20 > in the answerer supports the payload types profile-id but, can't=20 > ensure or know about all the profile-compatibility flags, then it has=20 > to refuse the payload type despite it supporting it. >=20 > Result, forcing C on a sender constraint interpretation only results=20 > in failures and no benefit. >=20 > Receiver and Sender constraints. >=20 > To my understanding such an interpretation only forces the case of=20 > more rejections, if the answerer doesn't have an equally capable=20 > encoder and decoder as the offerer and can match both sets of=20 > constraints. In addition the compatibility has not provided any=20 > benefit for the two O/A agents. >=20 > So, the question becomes have I misunderstood something about how you=20 > intended to be able to use this parameter? >=20 > If I haven't I think these parameters usage is seriously in question. > If one would like to use it then the must not alter constraint for=20 > profile becomes a serious issue. If one would remove that then I see=20 > that using "profile-compatibility-indicator" could be possibly=20 > beneficial. But, I haven't done a full out design of how to resolve=20 > the PT matching and deal with asymmetric usage within the session. >=20 > The same type of analysis and clarification on usage needs to happen=20 > for the "interop-constraints". >=20 > Then we have the question of how one shall interpret and deal with=20 > unknown flag values. These needs to cause only failures when required=20 > for ensuring video compatibility. Not cause negotiation failures for=20 > unknown cases that doesn't affect video interoperability. >=20 > [YK]: They are meant for Receiver and Sender constraints. What we have=20 > now is similar to what we have in RFC 6184 and RFC 6190, with the=20 > essential difference been that in AVC there is only one byte of such=20 > constraints, while in HEVC there are more bytes of such constraints. I=20 > think what you were thinking of is some optimization. > Considering the urgent situation that the RFC is urgent needed by the=20 > relevant industries for HEVC deployment, I personally would prefer to=20 > go ahead without trying to make more optimization here, but let it be=20 > similar as in RFC 6184 and RFC 6190. However, if the group thinks that=20 > we should do more optimization here, then we can try it, but this=20 > needs careful study and investigation. >=20 > BR, YK >=20 >=20 >=20 --=20 Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 F=E4r=F6gatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Wed Apr 30 06:51:41 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 966401A0904 for ; Wed, 30 Apr 2014 06:51:39 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.201 X-Spam-Level: X-Spam-Status: No, score=-4.201 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RSKHat6tloR4 for ; Wed, 30 Apr 2014 06:51:37 -0700 (PDT) Received: from sesbmg22.ericsson.net (sesbmg22.ericsson.net [193.180.251.48]) by ietfa.amsl.com (Postfix) with ESMTP id 64D741A0900 for ; Wed, 30 Apr 2014 06:51:37 -0700 (PDT) X-AuditID: c1b4fb30-f790e6d000001067-0c-5360ffe70e30 Received: from ESESSHC005.ericsson.se (Unknown_Domain [153.88.253.124]) by sesbmg22.ericsson.net (Symantec Mail Security) with SMTP id F7.7F.04199.7EFF0635; Wed, 30 Apr 2014 15:51:35 +0200 (CEST) Received: from [127.0.0.1] (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.35) with Microsoft SMTP Server id 14.3.174.1; Wed, 30 Apr 2014 15:51:34 +0200 Message-ID: <5360FFE6.3070206@ericsson.com> Date: Wed, 30 Apr 2014 15:51:34 +0200 From: Magnus Westerlund User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: "Wang, Ye-Kui" , "payload@ietf.org" , "draft-ietf-payload-rtp-h265@tools.ietf.org" References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> <535A2C74.7020306@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> <535DFA38.5090901@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345242101@nasanexd02f.na.qualcomm.com> In-Reply-To: <8BA7D4CEACFFE04BA2D902BF11719A8345242101@nasanexd02f.na.qualcomm.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrPLMWRmVeSWpSXmKPExsUyM+Jvje7z/wnBBu9P8lucuriP1eLSxbNM Fk/WHGN2YPZYsuQnk8eiqc8YPb5c/swWwBzFZZOSmpNZllqkb5fAlXH68nnmgjaVip2X5rA1 MC6R7WLk5JAQMJHY3tnIBmGLSVy4tx7MFhI4yihxcEU1hL2cUeJ2aziIzSugLbG44xEriM0i oCrxZeYWMJtNwELi5g+IOaICwRJL5yxmgagXlDg58wmQzcUhIrCJUWLh4jVgRcICjhIbGluZ QBJAC5glZj+5xQSS4ATqPvNsJ5DNAXSRuERPYxBImFlAT2LK1RZGCFteonnrbGaI47QlGpo6 WCcwCs5Csm8WkpZZSFoWMDKvYhQtTi1Oyk03MtJLLcpMLi7Oz9PLSy3ZxAgM4YNbfhvsYHz5 3PEQowAHoxIPb/GXyGAh1sSy4srcQ4zSHCxK4rzfzroHCwmkJ5akZqemFqQWxReV5qQWH2Jk 4uCUamB0+Cb/7t6ms2JzvGMmrvljcHja2TWbDdcs/nk5ZJ+x8PLecxNZjm38vaC7Zy7XIf+G MO/yuqVHdkqIaikVenjbPryzZj7rdL8Th9oO/Wm66PBEOmuF3dbu98IRF/b22z5kf98u5W3V MmeLjC3PbadDci+r25Q8NYXEFDWf8obc2fDl/VWhppAtSizFGYmGWsxFxYkAKY3JHUICAAA= Archived-At: http://mailarchive.ietf.org/arch/msg/payload/E4Se-2i0KAaHDb65hDms30P1tzU Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 13:51:39 -0000 On 2014-04-29 00:03, Wang, Ye-Kui wrote: > Thanks for the clarification on the first point. > > Now again on the third point about profile-compatibility-indicator > and interop-constraints. > > profile-compatibility-indicator tells the list of profiles the > bitstream conforms to. The profile identified by profile-id would > also be indicated by profile-compatibility-indicator. The information > that is covered by profile-id but not by > profile-compatibility-indicator is as follows (note 6 in subclause > 7.4.4 of the HEVC spec): NOTE 6 - When the coded video sequence > conforms to multiple profiles, general_profile_idc should indicate > the profile that provides the preferred decoded result or the > preferred bitstream identification, as determined by the encoder (in > a manner not specified in this Specification). Yes, I do understand the purpose of the indicator in the relation to a particular bit-stream. However, this benefit is not possible to utilize at all in SDP O/A with the rules you specify. As my initial post in this tried to analyse there are some rule changes required to have it provide any benefit. I get the impression these values have been thrown into the payload format signalling without really considering the implications and limitations that the SDP O/A results in. > > Therefore, the use of profile-id and profile-compatibility-indicator > together here in this payload format spec is similar as the use in > RFC 6184 of profile_idc and profile-iop (the first two bytes of > profile-level-id, except the particular and weird indication of level > 1b using bit 4 of profile-iop). They may be similar, but that doesn't mean that their usage was the most appropriate or best possible before. With a new standard one has the chance to revisit this. I do understand the pressure to finish this up as quickly as possible. But, I think you should take the time to consider how this is used. > > What we could do to optimize the O/A herein is to allow for the > change of the values of profile-id and > profile-compatibility-indicator as long as they represent the same > set of profiles. But I am not sure whether or how many PT values > could be saved in practice. If there is nothing to save, then I think you should question if you gain anything from including the profile-compatibility-indicator as part of the PT signalling at all? I see that it can potentially be used to indicate which additional profiles that a encoder can produce. For that to be useful the answer needs to consider if it can add new PTs for it. Or if it is only use is to allow a richer second Offer/Answer round. The other aspect of this is if it is at all reasonable to expect the peer encoder to be able to produce the same compatibility pattern, while the main constraint is to support the actual supported profile+level+constraints. I question that aspect of using it as receiver constraint for a particular payload type. > > interop-constraints is something new in HEVC compared to AVC. Example > constraints here include 1) whether the video is a frame-packed 3D > video; and 2) whether the coded pictures are coded frames only or > there are interlaced stuff involved. Herein I think it is preferable > to require both sides to maintain the values, because if an offerer > offers to communicate with clean progressive video without frame > packing, it also expects the other side to keep the restrictions and > do the same. Yes, I understand that some of the current parameters, you really want the answering party to uphold them. One important question is if there are clear and working rules for unknown constraints. How does the answerer handle unknown constraints? Here I think a first step is to explain how unknown value are to be dealt with. The other aspect of maintaining the interop-constraints values and really not have any negotiation and instead use different PTs if one like to offer different values, is the potential explosion of combinations in the future when more constraints are added. Cheers Magnus Westerlund ---------------------------------------------------------------------- Services, Media and Network features, Ericsson Research EAB/TXM ---------------------------------------------------------------------- Ericsson AB | Phone +46 10 7148287 Färögatan 6 | Mobile +46 73 0949079 SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com ---------------------------------------------------------------------- From nobody Wed Apr 30 07:40:44 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A7D8B1A6FC2 for ; Wed, 30 Apr 2014 07:40:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.052 X-Spam-Level: X-Spam-Status: No, score=-3.052 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_EQ_DE=0.35, HELO_MISMATCH_DE=1.448, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b1IFXkrmryIw for ; Wed, 30 Apr 2014 07:40:39 -0700 (PDT) Received: from mailgw.hhi.fraunhofer.de (mail.HHI.FRAUNHOFER.DE [193.174.67.45]) by ietfa.amsl.com (Postfix) with ESMTP id 717A91A6FAB for ; Wed, 30 Apr 2014 07:40:38 -0700 (PDT) X-IMSS-DKIM-Authentication-Result: mailgw.hhi.fraunhofer.de; sigcount=0 Received: from maclaptop002.ic.tu-berlin.de (maclaptop002.ic.tu-berlin.de [130.149.228.68]) by mailgw.hhi.fraunhofer.de (Postfix) with ESMTP id 98C611C90027; Wed, 30 Apr 2014 16:40:35 +0200 (CEST) Content-Type: multipart/alternative; boundary="Apple-Mail=_FD6E134C-EB8A-45E9-9756-F2D98A5FAD36" Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) From: Yago Sanchez In-Reply-To: <5360FFE6.3070206@ericsson.com> Date: Wed, 30 Apr 2014 16:40:35 +0200 Message-Id: <8C8C86DF-6425-4D5B-8142-408146A154F0@hhi.fraunhofer.de> References: <5310AF7F.9050508@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A83387D4E31@nasanexd02f.na.qualcomm.com> <53271A9D.4020906@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345200F61@nasanexd02f.na.qualcomm.com> <535A2C74.7020306@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A834523F1EC@nasanexd02f.na.qualcomm.com> <535DFA38.5090901@ericsson.com> <8BA7D4CEACFFE04BA2D902BF11719A8345242101@nasanexd02f.na.qualcomm.com> <5360FFE6.3070206@ericsson.com> To: Magnus Westerlund X-Mailer: Apple Mail (2.1874) X-TM-AS-MML: disable X-TM-AS-Product-Ver: IMSS-7.1.0.1576-7.5.0.1017-20664.007 X-TM-AS-Result: No--23.497-11.0-31-10 X-imss-scan-details: No--23.497-11.0-31-10 X-TM-AS-User-Approved-Sender: No X-TMASE-MatchedRID: IeZYkn8zfFrYfPOPCpnfAvYiLRVJ915DqapJP7NkemG/7bplhbPCQhjE XwIlRv7QqDGoNTL3sPp0GojQBDdLHEibx2WCCMTIMIiU395I8H2FXSyuOiq3H7MMGClW/Y+oMsU /lfBsruO4A+B4C5lvoQZyC1572Xh+PxSyat8hxvW628cXbnOhT4yzstdwoG+Pqr3CBdU3C2Bux2 OjWJN7Mv3ZjdzTJnr6fDWNjWK31X0aSP1YjVcdxdUFhgTP7/bWKx5ICGp/WtFsW1v4EE7KMzWta vC5bjEWkldwGqdyC2nVhfqwZRnVGostvMh+ZJD96/xAZojbl7c026H7nOZLr1vym/gvSH4i8myX 5RK7cEUcl4Vrs6KMmOd4s9CWIqKcDHlMveoJOASVSBCoZUyqbMMdI0UcXEHzWP9qYuzytBL5oJA MEFHV19+YPMIs5f9KJuwS8u+P+pAEdG16FO0c4sLcTtMlPIaVROshHNu2/QbtrvVsQ0WdMz7LJl 82DRKFLU8P6/h6qofMf+oGq5enDhnsS71Oo/Hw5ghRQ2Zr49TAMyYDvAr9pkFB9SDJ0I2fCtRo5 dkVT3cFTme8mM+FD2wnEHSO0ybJoc4V4PjD1KfFwpJWalzawyQqzcugG1CVHWNaMBkegScR285K lILsgaXsQ7KLlQJDCuM9CC1gRGNz6wE33E1GH9sfd0p6MYVXj/xLIaDSshFJg8sTT3RzFzKglE1 ZhS7grGIyT5L+3barQ6+lLafm7GFqPXSLpNdAQr2qXCJMSV91k+gP1XamtAeLCIX046iBSrJTO1 VGhMF2+AZuxakJy9NwUCEMfRZdQiNolKxpXqqeAiCmPx4NwGmRqNBHmBveuME6WhSqqOGh5DOWG PdqWbkblkrgCLv4OwBXM346/+wmMPhsJjA25AhZXfeE/8D2f71OOhH2WV8Cq2k6TtTaiibvS15O 30gu X-IMSS-DKIM-White-List: No X-alterMIME: Yes Archived-At: http://mailarchive.ietf.org/arch/msg/payload/hLyyLKjwf6NKFeBdbDUPWmMAEmQ Cc: "draft-ietf-payload-rtp-h265@tools.ietf.org" , "payload@ietf.org" Subject: Re: [payload] Comments on draft-ietf-payload-rtp-h265-02 X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 14:40:42 -0000 --Apple-Mail=_FD6E134C-EB8A-45E9-9756-F2D98A5FAD36 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hi Magnus, I think there is a misunderstanding with the intention of using the = profile-compatibility-indicator. It is not about indicating support for = additional profiles. The profile-compatibility-indicator sets additional = constraints to the profile indicated in the profile-id. That is why = Ye-Kui was pointing that both the profile-id and the = profile-compatibility-indicator are used to express the same as the = profile_idc in RFC 6184. This means that if the offerer indicates = profile-id equal to A and includes profile B in the = profile-compatibility-indicator, the offerer would indicate capability = of encoding/decoding a stream that only contains the tools that are in = the intersection of both profiles. The offerer promise would not even be = that it is able to decode the full A profile, unless indicated in = another PT. My understanding was that this would be useful for those cases where two = endpoints support different profiles but are aware of the intersection = with other profiles, as in the case expressed above. Thus, there could = express more PTs with the profile they fully support and with subsets of = it result of intersections with other profiles. Therefore, I still think that including profile-compatibility-indicator = may be useful. Would this be reasonable or is there anything I am = missing from the discussion below? Best regards, Yago On 30 Apr 2014, at 15:51, Magnus Westerlund = wrote: > On 2014-04-29 00:03, Wang, Ye-Kui wrote: >> Thanks for the clarification on the first point. >>=20 >> Now again on the third point about profile-compatibility-indicator >> and interop-constraints. >>=20 >> profile-compatibility-indicator tells the list of profiles the >> bitstream conforms to. The profile identified by profile-id would >> also be indicated by profile-compatibility-indicator. The information >> that is covered by profile-id but not by >> profile-compatibility-indicator is as follows (note 6 in subclause >> 7.4.4 of the HEVC spec): NOTE 6 - When the coded video sequence >> conforms to multiple profiles, general_profile_idc should indicate >> the profile that provides the preferred decoded result or the >> preferred bitstream identification, as determined by the encoder (in >> a manner not specified in this Specification). >=20 > Yes, I do understand the purpose of the indicator in the relation to a > particular bit-stream. However, this benefit is not possible to = utilize > at all in SDP O/A with the rules you specify. As my initial post in = this > tried to analyse there are some rule changes required to have it = provide > any benefit. I get the impression these values have been thrown into = the > payload format signalling without really considering the implications > and limitations that the SDP O/A results in. >=20 >>=20 >> Therefore, the use of profile-id and profile-compatibility-indicator >> together here in this payload format spec is similar as the use in >> RFC 6184 of profile_idc and profile-iop (the first two bytes of >> profile-level-id, except the particular and weird indication of level >> 1b using bit 4 of profile-iop). >=20 > They may be similar, but that doesn't mean that their usage was the = most > appropriate or best possible before. With a new standard one has the > chance to revisit this. I do understand the pressure to finish this up > as quickly as possible. But, I think you should take the time to > consider how this is used. >=20 >>=20 >> What we could do to optimize the O/A herein is to allow for the >> change of the values of profile-id and >> profile-compatibility-indicator as long as they represent the same >> set of profiles. But I am not sure whether or how many PT values >> could be saved in practice. >=20 > If there is nothing to save, then I think you should question if you > gain anything from including the profile-compatibility-indicator as = part > of the PT signalling at all? >=20 > I see that it can potentially be used to indicate which additional > profiles that a encoder can produce. For that to be useful the answer > needs to consider if it can add new PTs for it. Or if it is only use = is > to allow a richer second Offer/Answer round. >=20 > The other aspect of this is if it is at all reasonable to expect the > peer encoder to be able to produce the same compatibility pattern, = while > the main constraint is to support the actual supported > profile+level+constraints. I question that aspect of using it as > receiver constraint for a particular payload type. >=20 >>=20 >> interop-constraints is something new in HEVC compared to AVC. Example >> constraints here include 1) whether the video is a frame-packed 3D >> video; and 2) whether the coded pictures are coded frames only or >> there are interlaced stuff involved. Herein I think it is preferable >> to require both sides to maintain the values, because if an offerer >> offers to communicate with clean progressive video without frame >> packing, it also expects the other side to keep the restrictions and >> do the same. >=20 > Yes, I understand that some of the current parameters, you really want > the answering party to uphold them. One important question is if there > are clear and working rules for unknown constraints. How does the > answerer handle unknown constraints? Here I think a first step is to > explain how unknown value are to be dealt with. >=20 > The other aspect of maintaining the interop-constraints values and > really not have any negotiation and instead use different PTs if one > like to offer different values, is the potential explosion of > combinations in the future when more constraints are added. >=20 >=20 > Cheers >=20 > Magnus Westerlund >=20 > ---------------------------------------------------------------------- > Services, Media and Network features, Ericsson Research EAB/TXM > ---------------------------------------------------------------------- > Ericsson AB | Phone +46 10 7148287 > F=E4r=F6gatan 6 | Mobile +46 73 0949079 > SE-164 80 Stockholm, Sweden | mailto: magnus.westerlund@ericsson.com > ---------------------------------------------------------------------- >=20 > _______________________________________________ > payload mailing list > payload@ietf.org > https://www.ietf.org/mailman/listinfo/payload --Apple-Mail=_FD6E134C-EB8A-45E9-9756-F2D98A5FAD36 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Hi = Magnus,

I think there is a misunderstanding with the = intention of using the profile-compatibility-indicator. It is not about = indicating support for additional profiles. The = profile-compatibility-indicator sets additional constraints to the = profile indicated in the profile-id. That is why Ye-Kui was pointing = that both the profile-id and the profile-compatibility-indicator are = used to express the same as  the profile_idc in RFC 6184. This = means that if the offerer indicates profile-id equal to A and includes = profile B in the profile-compatibility-indicator, the offerer would = indicate capability of encoding/decoding a stream that only contains the = tools that are in the intersection of both profiles. The offerer promise = would not even be that it is able to decode the full A profile, unless = indicated in another PT.

My understanding was = that this would be useful for those cases where two endpoints support = different profiles but are aware of the intersection with other = profiles, as in the case expressed above. Thus, there could express more = PTs with the profile they fully support and with subsets of it result of = intersections with other profiles.

Therefore, I = still think that including profile-compatibility-indicator may be = useful. Would this be reasonable or is there anything I am missing from = the discussion below?

Best = regards,
Yago

On 30 Apr 2014, at = 15:51, Magnus Westerlund <magnus.westerlund@ericsson.= com> wrote:

On 2014-04-29 00:03, Wang, Ye-Kui = wrote:
Thanks for the clarification on the = first point.

Now again on the third point about = profile-compatibility-indicator
and = interop-constraints.

profile-compatibility-indicator tells the = list of profiles the
bitstream conforms to. The profile identified by = profile-id would
also be indicated by = profile-compatibility-indicator. The information
that is covered by = profile-id but not by
profile-compatibility-indicator is as follows = (note 6 in subclause
7.4.4 of the HEVC spec): NOTE 6 - When the coded = video sequence
conforms to multiple profiles, general_profile_idc = should indicate
the profile that provides the preferred decoded = result or the
preferred bitstream identification, as determined by = the encoder (in
a manner not specified in this = Specification).

Yes, I do understand the purpose of = the indicator in the relation to a
particular bit-stream. However, = this benefit is not possible to utilize
at all in SDP O/A with the = rules you specify. As my initial post in this
tried to analyse there = are some rule changes required to have it provide
any benefit. I get = the impression these values have been thrown into the
payload format = signalling without really considering the implications
and = limitations that the SDP O/A results in.


Therefore, the use of profile-id and = profile-compatibility-indicator
together here in this payload format = spec is similar as the use in
RFC 6184 of profile_idc and profile-iop = (the first two bytes of
profile-level-id, except the particular and = weird indication of level
1b using bit 4 of = profile-iop).

They may be similar, but that doesn't = mean that their usage was the most
appropriate or best possible = before. With a new standard one has the
chance to revisit this. I do = understand the pressure to finish this up
as quickly as possible. = But, I think you should take the time to
consider how this is = used.


What we could do to optimize = the O/A herein is to allow for the
change of the values of profile-id = and
profile-compatibility-indicator as long as they represent the = same
set of profiles. But I am not sure whether or how many PT = values
could be saved in practice.

If there is = nothing to save, then I think you should question if you
gain = anything from including the profile-compatibility-indicator as = part
of the PT signalling at all?

I see that it can = potentially be used to indicate which additional
profiles that a = encoder can produce. For that to be useful the answer
needs to = consider if it can add new PTs for it. Or if it is only use is
to = allow a richer second Offer/Answer round.

The other aspect of = this is if it is at all reasonable to expect the
peer encoder to be = able to produce the same compatibility pattern, while
the main = constraint is to support the actual = supported
profile+level+constraints. I question that aspect of using = it as
receiver constraint for a particular payload = type.


interop-constraints is = something new in HEVC compared to AVC. Example
constraints here = include 1) whether the video is a frame-packed 3D
video; and 2) = whether the coded pictures are coded frames only or
there are = interlaced stuff involved. Herein I think it is preferable
to require = both sides to maintain the values, because if an offerer
offers to = communicate with clean progressive video without frame
packing, it = also expects the other side to keep the restrictions and
do the = same.

Yes, I understand that some of the current = parameters, you really want
the answering party to uphold them. One = important question is if there
are clear and working rules for = unknown constraints. How does the
answerer handle unknown = constraints? Here I think a first step is to
explain how unknown = value are to be dealt with.

The other aspect of maintaining the = interop-constraints values and
really not have any negotiation and = instead use different PTs if one
like to offer different values, is = the potential explosion of
combinations in the future when more = constraints are added.


Cheers

Magnus = Westerlund

--------------------------------------------------------= --------------
Services, Media and Network features, Ericsson = Research = EAB/TXM
---------------------------------------------------------------= -------
Ericsson AB =             &n= bsp;   | Phone  +46 10 7148287
F=E4r=F6gatan 6 =             &n= bsp;   | Mobile +46 73 0949079
SE-164 80 Stockholm, = Sweden | mailto: magnus.westerlund@ericsson.= com
---------------------------------------------------------------= -------

_______________________________________________
payload = mailing list
payload@ietf.org
https://www.ietf.or= g/mailman/listinfo/payload


=
= = --Apple-Mail=_FD6E134C-EB8A-45E9-9756-F2D98A5FAD36-- From nobody Wed Apr 30 10:33:58 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE1DE1A0942; Wed, 30 Apr 2014 10:33:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UD8r0RCJ31A7; Wed, 30 Apr 2014 10:33:54 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 848151A0910; Wed, 30 Apr 2014 10:33:54 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: internet-drafts@ietf.org To: i-d-announce@ietf.org X-Test-IDTracker: no X-IETF-IDTracker: 5.4.1 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20140430173354.18935.27971.idtracker@ietfa.amsl.com> Date: Wed, 30 Apr 2014 10:33:54 -0700 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/J801eYcSS-vTpMt8gHboxFJFrzs Cc: payload@ietf.org Subject: [payload] I-D Action: draft-ietf-payload-rtp-h265-03.txt X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 17:33:56 -0000 A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Audio/Video Transport Payloads Working Group of the IETF. Title : RTP Payload Format for High Efficiency Video Coding Authors : Ye-Kui Wang Yago Sanchez Thomas Schierl Stephan Wenger Miska M. Hannuksela Filename : draft-ietf-payload-rtp-h265-03.txt Pages : 92 Date : 2014-04-30 Abstract: This memo describes an RTP payload format for the video coding standard ITU-T Recommendation H.265 and ISO/IEC International Standard 23008-2, both also known as High Efficiency Video Coding (HEVC) [HEVC] and developed by the Joint Collaborative Team on Video Coding (JCT-VC). The RTP payload format allows for packetization of one or more Network Abstraction Layer (NAL) units in each RTP packet payload, as well as fragmentation of a NAL unit into multiple RTP packets. Furthermore, it supports transmission of an HEVC bitstream over a single as well as multiple RTP streams. The payload format has wide applicability in videoconferencing, Internet video streaming, and high bit-rate entertainment-quality video, among others. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-payload-rtp-h265/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-payload-rtp-h265-03 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=draft-ietf-payload-rtp-h265-03 Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ From nobody Wed Apr 30 11:28:42 2014 Return-Path: X-Original-To: payload@ietfa.amsl.com Delivered-To: payload@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F76C1A8867 for ; Wed, 30 Apr 2014 11:28:40 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.952 X-Spam-Level: X-Spam-Status: No, score=-4.952 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nTS3JwG-NnjE for ; Wed, 30 Apr 2014 11:28:38 -0700 (PDT) Received: from wolverine02.qualcomm.com (wolverine02.qualcomm.com [199.106.114.251]) by ietfa.amsl.com (Postfix) with ESMTP id 9330B1A096B for ; Wed, 30 Apr 2014 11:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=qti.qualcomm.com; i=@qti.qualcomm.com; q=dns/txt; s=qcdkim; t=1398882517; x=1430418517; h=from:to:subject:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version; bh=LnqNUCOX2BnwpZBBYVjecf3ofGTc341ado7IOYIixfE=; b=XRkQnuTycMHneTIeCyzZc8zaZ1+nE0c7GiFvi8LZPM7KVOCMuGDgB7ri u/KojAmKAsd1VApQPG97O88I9MLwR7mKJvhXr1UQiC2/oxnykVn3ORAuA Rq45uuKokJSajR4PakcneBHvEBx8V821s75zJ7+yXXkshVdToRkmlcQlD g=; X-IronPort-AV: E=McAfee;i="5600,1067,7424"; a="123423790" Received: from ironmsg03-r.qualcomm.com ([172.30.46.17]) by wolverine02.qualcomm.com with ESMTP; 30 Apr 2014 11:28:37 -0700 X-IronPort-AV: E=Sophos;i="4.97,959,1389772800"; d="scan'208";a="670788006" Received: from nasanexhc08.na.qualcomm.com ([172.30.39.7]) by Ironmsg03-R.qualcomm.com with ESMTP/TLS/RC4-SHA; 30 Apr 2014 11:28:37 -0700 Received: from NASANEXD02F.na.qualcomm.com ([169.254.8.134]) by nasanexhc08.na.qualcomm.com ([172.30.39.7]) with mapi id 14.03.0158.001; Wed, 30 Apr 2014 11:28:36 -0700 From: "Wang, Ye-Kui" To: "payload@ietf.org" Thread-Topic: [payload] I-D Action: draft-ietf-payload-rtp-h265-03.txt Thread-Index: AQHPZJpqbLyXIk9FYUOA5cqUKWqAxJsqbPSA Date: Wed, 30 Apr 2014 18:28:36 +0000 Message-ID: <8BA7D4CEACFFE04BA2D902BF11719A83452473D3@nasanexd02f.na.qualcomm.com> References: <20140430173354.18935.27971.idtracker@ietfa.amsl.com> In-Reply-To: <20140430173354.18935.27971.idtracker@ietfa.amsl.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.48.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: http://mailarchive.ietf.org/arch/msg/payload/qyE8Af9QqBMpUSSLWW44083000Y Subject: Re: [payload] I-D Action: draft-ietf-payload-rtp-h265-03.txt X-BeenThere: payload@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Audio/Video Transport Payloads working group discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Apr 2014 18:28:40 -0000 Dear All, Compare to -02, we have addressed most of the numerous comments and tickets= , from Magnus, Jonathan, Jonatan, Bernard, ..., as well as the following ch= anges: - Addition of sprop-sei, as suggested by Stephan on Apr. 23 to the reflecto= r, agreed by most of the authors, no objection from anyone. - Addition of an informative, brief description of the display orientation = SEI message, which is among one of those new SEI messages that have some sy= stems usage. - Addition of a definition of NAL-unit-like structure. The term was used bu= t not defined earlier. - Addition of descriptions for use of the PLI and SLI messages (specified i= n RFC 4585) and the FIR message (specified in RFC 5104) with HEVC, in addit= ion to the RFC 4585 RPSI message for which the use description was already = included, as agreed during the presentation at the previous IETF meeting Remaining open issues: - On profile-compatibility-indicator and interop-constraints (raised by Mag= nus, being discussed among Magnus, myself and Yago, no conclusion yet) - On recv-sub-layer-id and using of sprop-vps for session negotiation (rais= ed by Magnus, still being discussed among the authors and Magnus) - On source-specific sprop parameter sets (raised by Magnus, still being di= scussed among the authors and Jonathan for exact text changes) - On using a different parameter category for level-id (raised by Magnus, d= iscussed between Magnus and myself, no conclusion yet) - I just noticed that the definition of "packet stream" itself needs to be = changed back to "RTP stream" We will try to address the above open issues as soon as possible. At the sa= me time, further review and comments are welcome. BR, YK -----Original Message----- From: payload [mailto:payload-bounces@ietf.org] On Behalf Of internet-draft= s@ietf.org Sent: Wednesday, April 30, 2014 10:34 AM To: i-d-announce@ietf.org Cc: payload@ietf.org Subject: [payload] I-D Action: draft-ietf-payload-rtp-h265-03.txt A New Internet-Draft is available from the on-line Internet-Drafts director= ies. This draft is a work item of the Audio/Video Transport Payloads Working Gr= oup of the IETF. Title : RTP Payload Format for High Efficiency Video Codi= ng Authors : Ye-Kui Wang Yago Sanchez Thomas Schierl Stephan Wenger Miska M. Hannuksela Filename : draft-ietf-payload-rtp-h265-03.txt Pages : 92 Date : 2014-04-30 Abstract: This memo describes an RTP payload format for the video coding standard ITU-T Recommendation H.265 and ISO/IEC International Standard 23008-2, both also known as High Efficiency Video Coding (HEVC) [HEVC] and developed by the Joint Collaborative Team on Video Coding (JCT-VC). The RTP payload format allows for packetization of one or more Network Abstraction Layer (NAL) units in each RTP packet payload, as well as fragmentation of a NAL unit into multiple RTP packets. Furthermore, it supports transmission of an HEVC bitstream over a single as well as multiple RTP streams. The payload format has wide applicability in videoconferencing, Internet video streaming, and high bit-rate entertainment-quality video, among others. The IETF datatracker status page for this draft is: https://datatracker.ietf.org/doc/draft-ietf-payload-rtp-h265/ There's also a htmlized version available at: http://tools.ietf.org/html/draft-ietf-payload-rtp-h265-03 A diff from the previous version is available at: http://www.ietf.org/rfcdiff?url2=3Ddraft-ietf-payload-rtp-h265-03 Please note that it may take a couple of minutes from the time of submissio= n until the htmlized version and diff are available at tools.ietf.org. Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ _______________________________________________ payload mailing list payload@ietf.org https://www.ietf.org/mailman/listinfo/payload