Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

From: "Geir Sandbakken (geirsand)" <[email-masked]>
Date: Thu, 27 Feb 2014 10:18:24 +0000

#12: The max-fps parameter

Lessons learned from video conferencing using H.264 showed that a lot of equipment designed for 30 fps could not handle 60 fps even at sufficiently low resolution. We do anticipate similar problems when moving towards 120 fps in H.265.  It might not be caused by limitations in the decoder itself,  but that the surrounding media pipeline will not have been properly tested for 120 fps before being put into the wild.  If we don’t allow for a code point to be used by the “restricted" equipment, it will be required by the “flexible/proper” to include one.  This is what happened with max-fps in H.264 enabling higher frame rates compared to lowering them.

Geir A

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 27 Feb 2014 16:31:58 +0100

On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
> Lessons learned from video conferencing using H.264 showed that a lot of
> equipment designed for 30 fps could not handle 60 fps even at
> sufficiently low resolution. We do anticipate similar problems when
> moving towards 120 fps in H.265.  It might not be caused by limitations
> in the decoder itself,  but that the surrounding media pipeline will not
> have been properly tested for 120 fps before being put into the wild. 
> If we don’t allow for a code point to be used by the “restricted"
> equipment, it will be required by the “flexible/proper” to include one. 
> This is what happened with max-fps in H.264 enabling higher frame rates
> compared to lowering them.
> 

I do have concerns with defining a parameter that allow dynamically
sub-setting the profile and levels that has been defined for HEVC. This
can also create interoperability issues with any pre-encoded content to
a particular profile and level.

The issues with the media pipeline has they been with the reception of
the high framerate of packets and forwarding them to the decoder or the
playback part, i.e. handling of the decoded picture?

If it is predominately the later it appears very reasonable to have this
as an informative parameter. I will not make use of higher FPS than X,
if you send me higher than X I will need to reduce that to X or lower
before displaying it. That way we are not re-defining the profiles, we
are ensuring that we can handle content encoded within the profile and
still by taking care of the this in the end of your decoder chain you
can protect the rest of the media framework from not yet supported
frame-rates.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Roni Even" <[email-masked]>
Date: Thu, 27 Feb 2014 20:15:29 +0200

Hi Magnus,
I am not sure that the level specifies the maximum fps.
Roni

> -----Original Message-----
> From: payload [mailto:[email-masked]] On Behalf Of Magnus
> Westerlund
> Sent: 27 February, 2014 5:32 PM
> To: Geir Sandbakken (geirsand); [email-masked]
> Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter
> 
> > Lessons learned from video conferencing using H.264 showed that a lot
> > of equipment designed for 30 fps could not handle 60 fps even at
> > moving towards 120 fps in H.265.  It might not be caused by
> > limitations in the decoder itself,  but that the surrounding media
> > pipeline will not have been properly tested for 120 fps before being put
into
> the wild.
> > This is what happened with max-fps in H.264 enabling higher frame
> > rates compared to lowering them.
> >
> 
> I do have concerns with defining a parameter that allow dynamically sub-
> setting the profile and levels that has been defined for HEVC. This can
also
> create interoperability issues with any pre-encoded content to a
particular
> profile and level.
> 
> The issues with the media pipeline has they been with the reception of the
high
> framerate of packets and forwarding them to the decoder or the playback
part,
> i.e. handling of the decoded picture?
> 
as an
> informative parameter. I will not make use of higher FPS than X, if you
send me
> higher than X I will need to reduce that to X or lower before displaying
it. That
> way we are not re-defining the profiles, we are ensuring that we can
handle
> content encoded within the profile and still by taking care of the this in
the end
> of your decoder chain you can protect the rest of the media framework from
> not yet supported frame-rates.
> 
> Cheers
> 
> Magnus Westerlund
> 
> _______________________________________________
> payload mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/payload

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 27 Feb 2014 18:19:38 +0000

I am with Magnus here. If a decoder cannot DECODE a picture rate allowed by a level, then the decoder is not even a confirming decoder. We usually don't standardize mechanisms for non-confirming decoders, unless there is a very strong reason. 

I think making the following changes (4 places in the first paragraph of the semantics) would resolve the issue, with conceptually good semantics, but the use of the parameter is still practically the same.

-----------------Start of the semantics, with suggested changes--------------------------
max-fps:
The value of max-fps is an integer indicating the maximum picture rate in units of hundreds of pictures per second that can be efficiently received (***CHANGE TO "effectively processed by the receiver"***).  The max-fps parameter MAY be used to signal that the receiver has a constraint in that it is not capable of decoding (***CHANGE TO  "processing"***) video efficiently (***CHANGE TO  "effectively"***) at the full picture rate that is implied by the highest level and, when present, one or more of the parameters max-lsr, max-lps, and max-br.

The value of max-fps is not necessarily the picture rate at which the maximum picture size can be sent, it constitutes a constraint on maximum picture rate for all resolutions.

Informative note: The max-fps parameter is semantically different from max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that max-fps is used to signal a constraint, lowering the maximum picture rate from what is implied by other parameters.

The encoder MUST use a picture rate equal to or less than this value.  In cases where the max-fps parameter is absent the encoder is free to choose any picture rate according to the highest level and any signaled optional parameters.
-----------------End of the semantics, with suggested changes--------------------------

Please express your opinion if this is not acceptable to you. 

BR, YK

-----Original Message-----
From: payload [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: Thursday, February 27, 2014 7:32 AM
To: Geir Sandbakken (geirsand); [email-masked]
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

On 2014-02-27 11:18, Geir Sandbakken (geirsand) wrote:
> pipeline will not have been properly tested for 120 fps before being put into the wild.
> If we don't allow for a code point to be used by the "restricted"
> equipment, it will be required by the "flexible/proper" to include one. 

I do have concerns with defining a parameter that allow dynamically sub-setting the profile and levels that has been defined for HEVC. This can also create interoperability issues with any pre-encoded content to a particular profile and level.

The issues with the media pipeline has they been with the reception of the high framerate of packets and forwarding them to the decoder or the playback part, i.e. handling of the decoded picture?

If it is predominately the later it appears very reasonable to have this as an informative parameter. I will not make use of higher FPS than X, if you send me higher than X I will need to reduce that to X or lower before displaying it. That way we are not re-defining the profiles, we are ensuring that we can handle content encoded within the profile and still by taking care of the this in the end of your decoder chain you can protect the rest of the media framework from not yet supported frame-rates.

Cheers

Magnus Westerlund

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 27 Feb 2014 18:22:55 +0000

Hi Roni,

I sent the previous comment before seeing your below. The level specifies the max luma picture size and the max luma sample rate, which together determine the picture rate (i.e. the max fps).

BR, YK

-----Original Message-----
From: payload [mailto:[email-masked]] On Behalf Of Roni Even
Sent: Thursday, February 27, 2014 10:15 AM
To: 'Magnus Westerlund'; 'Geir Sandbakken (geirsand)'; [email-masked]
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Hi Magnus,

> > Lessons learned from video conferencing using H.264 showed that a 
> > lot of equipment designed for 30 fps could not handle 60 fps even at 
into
> sub- setting the profile and levels that has been defined for HEVC. 
> This can
also
particular
> the
high
part,
as an
send me
it. That
handle
the end
> of your decoder chain you can protect the rest of the media framework 
> from not yet supported frame-rates.
> 
> Cheers
> 
> Magnus Westerlund
>

---

From: "Wang, Ye-Kui" <[email-masked]>
Date: Thu, 27 Feb 2014 18:24:35 +0000

Typo: 4 places -> 3 places

-----------------Start of the semantics, with suggested changes--------------------------
max-fps:

-----------------End of the semantics, with suggested changes--------------------------

BR, YK

-----Original Message-----
From: payload [mailto:[email-masked]] On Behalf Of Magnus Westerlund
Sent: Thursday, February 27, 2014 7:32 AM
To: Geir Sandbakken (geirsand); [email-masked]
Subject: Re: [payload] #12 (rtp-h265): The max-fps parameter

Cheers

Magnus Westerlund

---

From: "Roni Even" <[email-masked]>
Date: Fri, 28 Feb 2014 00:30:22 +0200

Hi,
The current parameters are similar to RFC6184 allowing the receiver to
support higher resolution at lower frame rate but still with the level
definition. The max-fps was proposed in
http://tools.ietf.org/id/draft-kristensen-avt-rtp-h241param-01.txt but as
far as I can see did not progress.
Roni

> From: Wang, Ye-Kui [mailto:[email-masked]]
> Sent: 27 February, 2014 8:23 PM
> To: Roni Even; 'Magnus Westerlund'; 'Geir Sandbakken (geirsand)';
> [email-masked]
> Subject: RE: [payload] #12 (rtp-h265): The max-fps parameter
> 
> Hi Roni,
> 
> I sent the previous comment before seeing your below. The level specifies
the
> max luma picture size and the max luma sample rate, which together
> determine the picture rate (i.e. the max fps).
> 
> BR, YK
>

---

From: Tom Kristensen <[email-masked]>
Date: Fri, 28 Feb 2014 10:31:40 +0100

Thanks, YK!

These changes in semantics are fine with us and express the need and usage
of the parameter well.

On 27 February 2014 19:19, Wang, Ye-Kui <[email-masked]> wrote:

> I am with Magnus here. If a decoder cannot DECODE a picture rate allowed
> by a level, then the decoder is not even a confirming decoder. We usually
> don't standardize mechanisms for non-confirming decoders, unless there is a
> very strong reason.
>
> I think making the following changes (4 places in the first paragraph of
> the semantics) would resolve the issue, with conceptually good semantics,
> but the use of the parameter is still practically the same.
>
> The value of max-fps is an integer indicating the maximum picture rate in
> units of hundreds of pictures per second that can be efficiently received
> (***CHANGE TO "effectively processed by the receiver"***).  The max-fps
> parameter MAY be used to signal that the receiver has a constraint in that
> it is not capable of decoding (***CHANGE TO  "processing"***) video
> efficiently (***CHANGE TO  "effectively"***) at the full picture rate that
> is implied by the highest level and, when present, one or more of the
> parameters max-lsr, max-lps, and max-br.
>
> The value of max-fps is not necessarily the picture rate at which the
> maximum picture size can be sent, it constitutes a constraint on maximum
> picture rate for all resolutions.
>
> Informative note: The max-fps parameter is semantically different from
> max-lsr, max-lps, max-cpb, max-dpb, max-br, max-tr, and max-tc in that
> max-fps is used to signal a constraint, lowering the maximum picture rate
> from what is implied by other parameters.
>
> The encoder MUST use a picture rate equal to or less than this value.  In
> cases where the max-fps parameter is absent the encoder is free to choose
> any picture rate according to the highest level and any signaled optional
> parameters.
> into the wild.
> sub-setting the profile and levels that has been defined for HEVC. This can
> also create interoperability issues with any pre-encoded content to a
> particular profile and level.
>
> high framerate of packets and forwarding them to the decoder or the
> as an informative parameter. I will not make use of higher FPS than X, if
> you send me higher than X I will need to reduce that to X or lower before
> displaying it. That way we are not re-defining the profiles, we are
> ensuring that we can handle content encoded within the profile and still by
> taking care of the this in the end of your decoder chain you can protect
> the rest of the media framework from not yet supported frame-rates.
>
> Cheers
>
> Magnus Westerlund
>

-- 
# Cisco                         |  http://www.cisco.com/telepresence/
## [email-masked]  |  http://www.tandberg.com
###                               |  http://folk.uio.no/tomkri/

---

From: Tom Kristensen <[email-masked]>
Date: Fri, 28 Feb 2014 10:33:16 +0100

This draft was heavily discussed earlier on, but there was an overall
accept for the need of this parameter. However, in this case more work has
to be done with regards to backwards compatibility issues and proper
language addressing that.

-- Tom