From nobody Mon Mar 2 02:52:49 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A2F0A1A86F7 for ; Mon, 2 Mar 2015 02:52:48 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -12.612 X-Spam-Level: X-Spam-Status: No, score=-12.612 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y_TXD0WhZkWC for ; Mon, 2 Mar 2015 02:52:47 -0800 (PST) Received: from aer-iport-4.cisco.com (aer-iport-4.cisco.com [173.38.203.54]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 200EC1A86F3 for ; Mon, 2 Mar 2015 02:52:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=818; q=dns/txt; s=iport; t=1425293567; x=1426503167; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=hLs09Yjllf3WzWa3beIFCAa3MvBztXltY/adykvB1PQ=; b=ViWq3ORTLSVrr1r5QKUIGo7yeIJqNeGj4Oqxk2F6ny9h8KnIs3kJ9mWg iZlZeUXc7S6RM/1siq98dYWmvpCSODTPeLRkZvsVoamZnjSCFvP6xWVyo O02VBF9xn97D7xWTNzzH7D2Ekzlt6zWV/lcuoAoZQtwwVjBoaQa0kAic6 4=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CqBAB0QPRU/xbLJq1ag1TCDAqFcAKBbwEBAQEBAXyEEAEBBAEBATU2CgEQCw4KCRYPCQMCAQIBFTAGDQEFAgEBiCsN1RwBAQEBAQEBAQEBAQEBAQEBAQEBAQETBIsShG4HhCsBBJk/hmyMbiODbj4xgkMBAQE X-IronPort-AV: E=Sophos;i="5.09,675,1418083200"; d="scan'208";a="361766357" Received: from aer-iport-nat.cisco.com (HELO aer-core-2.cisco.com) ([173.38.203.22]) by aer-iport-4.cisco.com with ESMTP; 02 Mar 2015 10:52:46 +0000 Received: from [10.47.200.95] ([10.47.200.95]) by aer-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id t22Aqjhw022860; Mon, 2 Mar 2015 10:52:45 GMT Message-ID: <54F440FC.3050508@cisco.com> Date: Mon, 02 Mar 2015 10:52:44 +0000 From: Thomas Davies Organization: Cisco Systems User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:11.0) Gecko/20120410 Thunderbird/11.0.1 MIME-Version: 1.0 To: Thomas Daede References: <54EE0A07.3080100@mozilla.com> In-Reply-To: <54EE0A07.3080100@mozilla.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: video-codec@ietf.org Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 10:52:48 -0000 The "Class A" JCT-VC sequences were originated at 4k or 8k and cropped down for reasons of compute time. The 8k-originated ones (Nebuta and SteamLocomotive) are very noisy and were shot with an early camera and sensor. There wasn't much good 4k at the time HEVC started, and it would certainly be good to get more. regards Thomas On 25/02/15 17:44, Thomas Daede wrote: > The JCT-VC test set only includes three clips for video conferencing, > which I don't think is sufficient. It also specifies a intra frame > period of 1 second, which is not reasonable. It does not feature any > 2160p test sequences either, which is a bit strange. > _______________________________________________ video-codec mailing > list video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec From nobody Mon Mar 2 05:25:24 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E21ED1A8760 for ; Mon, 2 Mar 2015 05:25:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.723 X-Spam-Level: X-Spam-Status: No, score=0.723 tagged_above=-999 required=5 tests=[BAYES_50=0.8, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3K-fKRBmAfMS for ; Mon, 2 Mar 2015 05:25:18 -0800 (PST) Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D40F81A8764 for ; Mon, 2 Mar 2015 05:25:01 -0800 (PST) Received: by wggx12 with SMTP id x12so33372877wgg.6 for ; Mon, 02 Mar 2015 05:25:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=7PM4CWNXlmXk0xumevZ/TvhGZbmVAXm8cxEMRPlUgyM=; b=l6gLVRJbbGbpKE2VHK5xWQDttwW3U+pRXmFwMtDRN5mp5x1xJurpVzY9RN+S/6faUf 7oNUib1ignm2QwFmq2SLq/t2kToaQTUO/IQ25Z8i8oi+VV0+W/QbTmIJG5H/uk3jPF8y 9DF8z9U/edAVTjwC7wr0Dw2DQp0AonVnOmbTj/bBPqOjA372eT41DKnrA5sywOK4RBrk +BGGcknv93b4MMcfg2W8eNc9i4KktYvVQs7JMFwHD4d/5ESsKN5zsWzAwbi01vSZNUx+ U+nfwfrLB9IE0R8tyzEAPQWOZ+b/OQo6O8eyRUprxJmPFKeoJvitvSC+/EXqnfzPmqcG xXRw== X-Gm-Message-State: ALoCoQkOYEzcpevhVn/IfOtTEnsiYhCi5cktolYYlfeyetsFeuVM2B1UfppOYqUEE/o9FCvXMp9L MIME-Version: 1.0 X-Received: by 10.194.61.12 with SMTP id l12mr58404828wjr.139.1425302700588; Mon, 02 Mar 2015 05:25:00 -0800 (PST) Received: by 10.194.14.129 with HTTP; Mon, 2 Mar 2015 05:25:00 -0800 (PST) In-Reply-To: <54EE4884.7050706@mozilla.com> References: <54ED8506.30908@alvestrand.no> <54EDDDE4.1050303@xiph.org> <54EDE784.2000107@alvestrand.no> <54EE2B08.9060306@xiph.org> <54EE4884.7050706@mozilla.com> Date: Mon, 2 Mar 2015 15:25:00 +0200 Message-ID: From: Mohammed Raad To: "video-codec@ietf.org" Content-Type: multipart/alternative; boundary=047d7b86df382a95c205104e252e Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 13:25:24 -0000 --047d7b86df382a95c205104e252e Content-Type: text/plain; charset=UTF-8 Hi, Can you provide some data regarding how well these metrics correlate with subjective measurements? I expect each will be more suitable for different types of content, but it would be interesting to know how these perform. BR, On Thu, Feb 26, 2015 at 12:11 AM, Thomas Daede wrote: > To start the discussion, here is a brief overview of the four metrics we > currently use in Daala. The reference code is in the tools/dump_*.c > files in the Daala repository. Note that all of these metrics are > applied to the luma plane only. > > ## PSNR > > PSNR is a traditional signal quality metric, measured in decibels. It is > directly drived from mean square error (MSE), or its square root (RMSE). > The formula used is: > > 20 * log10 ( MAX / RMSE ) > > or, equivalently: > > 10 * log10 ( MAX^2 / MSE ) > > which is the method used in the dump_psnr.c reference implementation. > > ## PSNR-HVS-M > > The PSNR-HVS metric performs a DCT transform of 8x8 blocks of the image, > weights the coefficients, and then calculates the PSNR of those > coefficients. Several different sets of weights have been considered. > The weights used by the dump_pnsrhvs.c tool have been found to be the > best match to real MOS scores. > > ## SSIM > > SSIM (Structural Similarity Image Metric) is a still image quality > metric introduced in 2004. It computes a score for each individual > pixel, using a window of neighboring pixels. These scores can then be > averaged to produce a global score for the entire image. The original > paper produces scores ranging between 0 and 1. > > For the metric to appear more linear on BD-rate curves, the score is > converted into a nonlinear decibel scale: > > -10 * log10 (1 - SSIM) > > ## Fast Multi-Scale SSIM > > Multi-Scale SSIM is SSIM extended to multiple window sizes. This is > implemented by downscaling the image a number of times, and computing > SSIM over the same number of pixels, then averaging the SSIM scores > together. The final score is converted to decibels in the same manner as > SSIM. > > On 02/25/2015 01:39 PM, Mo Zanaty (mzanaty) wrote: > > This is perhaps getting into charter bashing, but I think we will need > > some early milestone (close to requirements) for an evaluation criteria > > document that represents the workgroup consensus on comparative testing > > methodology and selection of solution candidates or specific tools. The > > set of test sequences will only be one small part of that. Metrics will > be > > a very important part of that. While I agree designing new metrics should > > probably be beyond the scope of proposed deliverables, I think we likely > > need a thorough evaluation and discussion of various metrics and reach > > some consensus on how proposed solutions and tools will be measured and > > adopted. > > > > Mo > > > > On 2/25/15, 3:05 PM, Timothy B. Terriberry wrote: > > > > Harald Alvestrand wrote: > >> psnr values of 35 dB where x264 achieves 40 dB - it seems psnr isn't > >> particularly sensitive to the resulting blurriness). > > > > Yes, it's well-known that PSNR loves low-passing. It's not the only > > metric that's going to have these problems. FastSSIM will probably be > > similarly blind. Fixable problems, maybe, but I don't want to get in the > > business of designing my own metrics. I'm not even sure there's good > > data on human preferences for when one should downsample, but I haven't > > spent any time looking. > > > > _______________________________________________ > > video-codec mailing list > > video-codec@ietf.org > > https://www.ietf.org/mailman/listinfo/video-codec > > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --047d7b86df382a95c205104e252e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

Can you provide some data regarding= how well these metrics correlate with subjective measurements? I expect ea= ch will be more suitable for different types of content, but it would be in= teresting to know how these perform.

BR,

On Thu, Feb 26,= 2015 at 12:11 AM, Thomas Daede <tdaede@mozilla.com> wrote:=
To start the discussion, here is a brief= overview of the four metrics we
currently use in Daala. The reference code is in the tools/dump_*.c
files in the Daala repository. Note that all of these metrics are
applied to the luma plane only.

## PSNR

PSNR is a traditional signal quality metric, measured in decibels. It is directly drived from mean square error (MSE), or its square root (RMSE). The formula used is:

20 * log10 ( MAX / RMSE )

or, equivalently:

10 * log10 ( MAX^2 / MSE )

which is the method used in the dump_psnr.c reference implementation.

## PSNR-HVS-M

The PSNR-HVS metric performs a DCT transform of 8x8 blocks of the image, weights the coefficients, and then calculates the PSNR of those
coefficients. Several different sets of weights have been considered.
The weights used by the dump_pnsrhvs.c tool have been found to be the
best match to real MOS scores.

## SSIM

SSIM (Structural Similarity Image Metric) is a still image quality
metric introduced in 2004. It computes a score for each individual
pixel, using a window of neighboring pixels. These scores can then be
averaged to produce a global score for the entire image. The original
paper produces scores ranging between 0 and 1.

For the metric to appear more linear on BD-rate curves, the score is
converted into a nonlinear decibel scale:

-10 * log10 (1 - SSIM)

## Fast Multi-Scale SSIM

Multi-Scale SSIM is SSIM extended to multiple window sizes. This is
implemented by downscaling the image a number of times, and computing
SSIM over the same number of pixels, then averaging the SSIM scores
together. The final score is converted to decibels in the same manner as SSIM.

On 02/25/2015 01:39 PM, Mo Zanaty (mzanaty) wrote:
> This is perhaps getting into charter bashing, but I think we will need=
> some early milestone (close to requirements) for an evaluation criteri= a
> document that represents the workgroup consensus on comparative testin= g
> methodology and selection of solution candidates or specific tools. Th= e
> set of test sequences will only be one small part of that. Metrics wil= l be
> a very important part of that. While I agree designing new metrics sho= uld
> probably be beyond the scope of proposed deliverables, I think we like= ly
> need a thorough evaluation and discussion of various metrics and reach=
> some consensus on how proposed solutions and tools will be measured an= d
> adopted.
>
> Mo
>
> On 2/25/15, 3:05 PM, Timothy B. Terriberry <tterribe@xiph.org> wrote:
>
> Harald Alvestrand wrote:
>> psnr values of 35 dB where x264 achieves 40 dB - it seems psnr isn= 't
>> particularly sensitive to the resulting blurriness).
>
> Yes, it's well-known that PSNR loves low-passing. It's not the= only
> metric that's going to have these problems. FastSSIM will probably= be
> similarly blind. Fixable problems, maybe, but I don't want to get = in the
> business of designing my own metrics. I'm not even sure there'= s good
> data on human preferences for when one should downsample, but I haven&= #39;t
> spent any time looking.
>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
=
Mohammed Raad, PhD.
Partner
RAADTECH C= ONSULTING
P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 = 414451478
Email: mohammedsraad@raadtech.com
--047d7b86df382a95c205104e252e-- From nobody Mon Mar 2 08:53:02 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62F7D1A1B84 for ; Mon, 2 Mar 2015 08:53:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.577 X-Spam-Level: X-Spam-Status: No, score=-0.577 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r4gIPivXZZgS for ; Mon, 2 Mar 2015 08:52:58 -0800 (PST) Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EB6F51A1B88 for ; Mon, 2 Mar 2015 08:52:49 -0800 (PST) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 3F004F212C for ; Mon, 2 Mar 2015 08:52:49 -0800 (PST) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VSLIwwgEpZyL for ; Mon, 2 Mar 2015 08:52:48 -0800 (PST) Received: from [172.17.0.85] (50-78-100-113-static.hfc.comcastbusiness.net [50.78.100.113]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 81B49F210A for ; Mon, 2 Mar 2015 08:52:48 -0800 (PST) Message-ID: <54F4955F.7090805@xiph.org> Date: Mon, 02 Mar 2015 08:52:47 -0800 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "video-codec@ietf.org" References: <54ED8506.30908@alvestrand.no> <54EDDDE4.1050303@xiph.org> <54EDE784.2000107@alvestrand.no> <54EE2B08.9060306@xiph.org> <54EE4884.7050706@mozilla.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 16:53:01 -0000 Mohammed Raad wrote: > Can you provide some data regarding how well these metrics correlate > with subjective measurements? I expect each will be more suitable for > different types of content, but it would be interesting to know how > these perform. There are a number of studies available (on of the advantages of not designing our own metrics). The original PSNR-HVS-M paper [1] conducted experiments using additive Gaussian noise and spatially-correlated Gaussian noise (in regions with various masking properties), and got the following Spearman correlations (Kendall correlations showed similar results): MS-SSIM: 0.406 PSNR: 0.537 PSNR-HVS-M: 0,984 Of course, Gaussian noise is not terribly representative of compression artifacts. The original Fast SSIM paper [2] used the LIVE image database (JPEG and JPEG2000 compression artifacts, white noise, Gaussian blur, and fast-fading channel noise), and showed these Spearman rank-order correlations: PSNR: 0.8755 SSIM: 0.9244 MS-SSIM: 0.9425 Fast MS-SSIM: 0.9409 They also compared with the LIVE video database (MPEG 2 compression, H.264 compression, H.264 compressed bit streams under packet loss, and H.264 compressed bit streams with bit errors): PSNR: 0.3684 SSIM: 0.4953 MS-SSIM: 0.7593 Fast MS-SSIM: 0.6991 I expect the numbers are so much lower than the image case primarily because of the packet loss and bit error conditions, but the authors did not break out the numbers by type of distortion, so it is difficult to say. Comparing with the numbers below, it also seems clear there is a significant temporal effect that none of these still-image metrics can capture. The TID 2008 database (maintained by the authors of PSNR-HVS-M) has human rankings for a number of different distortion types, and comparisons of various metrics against them can be found [3]: JPEG JPEG 2000 JPEG JPEG 2000 transmission errors t. errors PSNR 0.899 0.8255 0.7646 0.7769 SSIM 0.8994 0.8888 0.8216 0.8395 MS-SSIM: 0.935 0.9706 0.8747 0.8585 The same authors have a new expanded TID 2013 database, with more results [4]. Sadly, they no longer break out results by the individual type of distortion, but group them into subsets. Aside from the "Full" subset, only the "Actual" subset includes both JPEG and JPEG 2000 compression artifacts ("Simple" and "Color" also include JPEG compression). Only "Exotic" includes transmission errors, along with many other distortions completely unlike what would be produced by a codec. Spearman rank-order correlations: Actual Simple Full PSNR: 0.8246 0.9134 0.6395 SSIM: 0.7877 0.8371 0.6370 MS-SSIM: 0.8871 0.9053 0.7872 PSNR-HVS-M: 0.9175 0.9379 0.6246 The results here are by no means exhaustive. The algorithms here are also by no means always the best ones in the studies cited, and new studies usually come with a new algorithm that purports to be better. There have been a number of these since 2011. But these are the metrics we've been using and have some familiarity with. [1] http://enpub.fulton.asu.edu/resp/vpqm/vpqm2007/papers/399.pdf [2] http://live.ece.utexas.edu/publications/2011/chen_rtip_2011.pdf [3] http://hdrvdp.sourceforge.net/reports/2.0/quality_tid2008/index.html [4] http://ponomarenko.info/tid2013.htm From nobody Mon Mar 2 13:00:01 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E94F1A898D for ; Mon, 2 Mar 2015 12:59:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 63JUQd5x6p9q for ; Mon, 2 Mar 2015 12:59:58 -0800 (PST) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA96F1A898B for ; Mon, 2 Mar 2015 12:59:57 -0800 (PST) Received: by lbiw7 with SMTP id w7so32881514lbi.9 for ; Mon, 02 Mar 2015 12:59:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SQFd3S/+WbobaqTHV6w5MH2mtV8Cq2Ucl+BfRVdCQmo=; b=WLEzNU7nUIfCNDc/AdFSYY2a57KiJF7Dabb94Dn2vilNVc0fE/1qoAhFMX6sbeyASi /1Y0eg+Zo24gd2e/DYgThsw56KWB2QotLWNwpyVG1iUODXxTllRDyYR2SDQ8n8ULyAn7 u29Rfvn6A7CXGyJYhz22phBJOMWdNZwFDStEyyjrpNKPRpwarocVuEY+LTQowBkwhwiX psTEjiYvlgQuA8zwz9H7dGqyW50zcyBJE3d2CnFaTQ+vOAUNUG8sec0nltG35pqj2ywq lGZk1o7MN+j87yrdAExIxHRo4Aztg5TPPIbsR/PwwsmjvzobMKrWn05a3fGR6YtCLJJ5 aLmg== MIME-Version: 1.0 X-Received: by 10.152.204.69 with SMTP id kw5mr26050058lac.3.1425329996308; Mon, 02 Mar 2015 12:59:56 -0800 (PST) Received: by 10.112.142.129 with HTTP; Mon, 2 Mar 2015 12:59:56 -0800 (PST) In-Reply-To: References: <54ED8506.30908@alvestrand.no> <54EDDDE4.1050303@xiph.org> <54EDE784.2000107@alvestrand.no> Date: Mon, 2 Mar 2015 12:59:56 -0800 Message-ID: From: Monty Montgomery To: Yushin Cho Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: Harald Alvestrand , "video-codec@ietf.org" Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 20:59:59 -0000 I agree with Yushin for practical reasons specific to testing. The various objective metrics all have somewhat tentative relationships to actual human perception, and can only be used with awareness of their limitations. The primary limitation being the fact that different classes of artifacts are not penalized consistently, and so the metrics cannot be used as a black box to compare the relative worthiness of completely different techniques or codecs. Although multiscale encoding is undeniably useful, it further complicates testing by technique into the mix that the objective metrics are known to fail particularly badly at grading. They handle blurring only slightly better than they handle color*. We may need to account for that and come up with a way to grade it anyway. But it will be work, and there's a good case to be made against designing or modifying our own metrics. It's a task empirically demonstrable to be harder than building the codec itself. Monty (*which is to say, they don't take chroma into account at all) From nobody Mon Mar 2 14:20:22 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB4D01A8A11 for ; Mon, 2 Mar 2015 14:20:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uY5GhOmJSJip for ; Mon, 2 Mar 2015 14:20:10 -0800 (PST) Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com [209.85.192.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A8A461A8A44 for ; Mon, 2 Mar 2015 14:19:51 -0800 (PST) Received: by pdjz10 with SMTP id z10so43008415pdj.0 for ; Mon, 02 Mar 2015 14:19:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=DkbnlzNCQ0n+82gupkmDt987yScj74GZBeRj9gJX2ag=; b=gqsrzSeYoZtJSeudaKXfpjVnFIu+adK0s70+vij6yc7aN/W5rG38qYK0xzSAIRnrMa HgeKNxziQYm8WkIx4dfDC1G6OUI7FXSKhVVB4O7jvo+FxSoQjiSERIYxPB1nk6u2AS08 Lroxo9nWa4bEK5+AJ7USjB2bU8cy5sei35aXIvnfI9ceuAXqlgGIJlQNye05bDwNr3iY fT4X4KMtHuo1t/YWmVkWZ+Y9IovOmf14qQbhuREdEqgw5mA/all9YqGp70puYLOzypoa AE3/vRZn9wmUTH9S72N2EZo43HzFYHiUk564TiS61i3CkKOHg9iIfSS5CkHPNQlsJM3D flVg== X-Gm-Message-State: ALoCoQm7eo6qkhMgAWGDQpCVf/Ap8ZyekS3l7k71j/W7xMt+yO7AFDQc8bikpZUorrhWYHYNOgpq X-Received: by 10.68.249.194 with SMTP id yw2mr49953817pbc.20.1425334791179; Mon, 02 Mar 2015 14:19:51 -0800 (PST) Received: from ?IPv6:2620:101:80fc:232:7e7a:91ff:fe9e:8126? ([2620:101:80fc:232:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id du13sm12936057pdb.65.2015.03.02.14.19.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Mar 2015 14:19:50 -0800 (PST) Message-ID: <54F4E204.5@mozilla.com> Date: Mon, 02 Mar 2015 14:19:48 -0800 From: Thomas Daede User-Agent: http://a.pomf.se/cszdno.opus MIME-Version: 1.0 To: Thomas Davies References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> In-Reply-To: <54F440FC.3050508@cisco.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Cc: video-codec@ietf.org Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 22:20:16 -0000 Do you know if the 1-second keyframe interval was also for reasons of compute time? (It allows each GOP to be encoded in parallel). It unfortunately seems a bit short for many streaming applications, and also prevents rate control from being tested. At the moment, I run all codecs with rate control on in their constant quality mode. Perhaps this is not the right thing to do. On 03/02/2015 02:52 AM, Thomas Davies wrote: > The "Class A" JCT-VC sequences were originated at 4k or 8k and cropped > down for reasons of compute time. The 8k-originated ones (Nebuta and > SteamLocomotive) are very noisy and were shot with an early camera and > sensor. There wasn't much good 4k at the time HEVC started, and it would > certainly be good to get more. > > regards > > Thomas > > On 25/02/15 17:44, Thomas Daede wrote: >> The JCT-VC test set only includes three clips for video conferencing, >> which I don't think is sufficient. It also specifies a intra frame >> period of 1 second, which is not reasonable. It does not feature any >> 2160p test sequences either, which is a bit strange. >> _______________________________________________ video-codec mailing >> list video-codec@ietf.org >> https://www.ietf.org/mailman/listinfo/video-codec > From nobody Mon Mar 2 18:19:47 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0A1911A90B9 for ; Mon, 2 Mar 2015 18:19:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2jZymJ02N-bz for ; Mon, 2 Mar 2015 18:19:45 -0800 (PST) Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E808C1A90AB for ; Mon, 2 Mar 2015 18:19:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=932; q=dns/txt; s=iport; t=1425349185; x=1426558785; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=ErnW+mxvFYStSFprY1AraFpNbZH8sbT/jHsvn5xMJgA=; b=PBDiU3NTWwyI9KZupRy3k+Ip1s0dMHEc5Er1PCFNhzX6pVge8IswFZ4I gFoi0hKwdFJSCBSO1XC4I1P7px0q8Qs1N/VZkcu4JGZiuF3WQN1JM5af7 nkg8MnzbrHZ8RDhAbKhT1xw0nXEK9eR6celo02gsqL7F/Vpi2s5uuCLoR 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BtBQABGfVU/5NdJa1agwKBLATHMwKBIk0BAQEBAQF8hBABAQR5EAIBCA44MiUCBAENBYgv1xMBAQEBAQEBAQEBAQEBAQEBAQEBARiLEoRuB4QrAQSKLoVKiUeTWiODbm+BRH8BAQE X-IronPort-AV: E=Sophos;i="5.09,679,1418083200"; d="scan'208";a="400348064" Received: from rcdn-core-11.cisco.com ([173.37.93.147]) by rcdn-iport-6.cisco.com with ESMTP; 03 Mar 2015 02:19:44 +0000 Received: from xhc-rcd-x04.cisco.com (xhc-rcd-x04.cisco.com [173.37.183.78]) by rcdn-core-11.cisco.com (8.14.5/8.14.5) with ESMTP id t232Jiap016861 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 3 Mar 2015 02:19:44 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.229]) by xhc-rcd-x04.cisco.com ([173.37.183.78]) with mapi id 14.03.0195.001; Mon, 2 Mar 2015 20:19:44 -0600 From: "Mo Zanaty (mzanaty)" To: Thomas Daede , "Thomas Davies (thdavies)" Thread-Topic: [video-codec] Test sequences, automated and otherwise Thread-Index: AQHQVViCGcs0Byv6jEe+mSqjvuD2xA== Date: Tue, 3 Mar 2015 02:19:43 +0000 Message-ID: References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> In-Reply-To: <54F4E204.5@mozilla.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.8.150116 x-originating-ip: [64.100.32.216] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <833D1226B5359747BDB6154CE4A5CA23@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 02:19:46 -0000 Harald will probably disagree, but the convention is to disable rate control for comparing objective metrics. Rate control is normally an application-specific configuration of the encoder which can add noise or bias if comparing different rate controls. Rate control strategies can often be abstracted to apply to any modern codec (with adaptive quantizers), so it is best to remove this source of noise/bias if you can=B9t guarantee identical (not default) strategies across the comparison. Note that some GOP structures often include some basic rate controls by default, like different QP offsets for hierarchical B/P frames. This can also be a source of noise/bias unless configured identically across the comparison. Mo On 3/2/15, 5:19 PM, Thomas Daede wrote: At the moment, I run all codecs with rate control on in their constant quality mode. Perhaps this is not the right thing to do. From nobody Tue Mar 3 00:47:29 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A715E1A1A70 for ; Tue, 3 Mar 2015 00:47:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i69ISmPiLUvB for ; Tue, 3 Mar 2015 00:47:27 -0800 (PST) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D49421A1A9E for ; Tue, 3 Mar 2015 00:47:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2222; q=dns/txt; s=iport; t=1425372446; x=1426582046; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=4v4QI40LQ10y864Y/iH3WQBXTfGPUjTeOXOCe8NRmfM=; b=G0B5Pdt5XyRFvq8iHaOzSTBWS/nK+nKrw+Lw+/kMcojC21o+72yrZTfO A/iuk0ouUOfYAsfyKn6L4o95xLmk5aDJTvT0zkg3Ev9yyBRdcfWCJjRt3 7kK6ZL4Lmuy5H7e+vi6/Xx2YbBT05CDglJYXNwspeRxtnYghPECrsEYIX w=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CyBQDxc/VU/5hdJa1agwJSWgTBKAqFcAKBIU0BAQEBAQF8hA8BAQEEAQEBNzQLDAQCAQgOAwQBAQEKFAkHJwsUCQgCBA4FCIgnDdZhAQEBAQEBAQEBAQEBAQEBAQEBAQEBEwSLEoQ9MQcGgxGBFAWPeZ0oI4Nub4FEfwEBAQ X-IronPort-AV: E=Sophos;i="5.09,680,1418083200"; d="scan'208";a="397287492" Received: from rcdn-core-1.cisco.com ([173.37.93.152]) by rcdn-iport-9.cisco.com with ESMTP; 03 Mar 2015 08:47:26 +0000 Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by rcdn-core-1.cisco.com (8.14.5/8.14.5) with ESMTP id t238lPQX005080 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 3 Mar 2015 08:47:26 GMT Received: from xmb-rcd-x07.cisco.com ([169.254.7.58]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.03.0195.001; Tue, 3 Mar 2015 02:47:25 -0600 From: "Thomas Davies (thdavies)" To: Thomas Daede Thread-Topic: [video-codec] Test sequences, automated and otherwise Thread-Index: AQHQN1XnhOmKd2cPek+FbhhIUgniXpzRuQSAgAAErYCAAdQfgIAAiDWAgC4hWICAB2iSAIAAv/cAgABIoZA= Date: Tue, 3 Mar 2015 08:47:25 +0000 Message-ID: <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> In-Reply-To: <54F4E204.5@mozilla.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.61.195.187] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 08:47:28 -0000 No, I suspect it was to be more appropriate for broadcast applications, whi= ch usually have more frequent intra for channel switching. Back in the day, 1/2 second or 12 frame GOPs were deemed essential for MPEG= -2 digital TV to mimic analogue switching times, but I think that is long g= one - now audio is switched first and then the video when possible, to give= an illusion of immediacy. I think low bit rate channels could easily have = intra periods of 2 or 3 seconds. I agree it's short for streaming applications, and could be longer. However= , including a decent proportion of intra in the metrics is probably good. Regards Thomas -----Original Message----- From: Thomas Daede [mailto:tdaede@mozilla.com]=20 Sent: Monday, March 02, 2015 10:20 PM To: Thomas Davies (thdavies) Cc: video-codec@ietf.org Subject: Re: [video-codec] Test sequences, automated and otherwise Do you know if the 1-second keyframe interval was also for reasons of compu= te time? (It allows each GOP to be encoded in parallel). It unfortunately s= eems a bit short for many streaming applications, and also prevents rate co= ntrol from being tested. At the moment, I run all codecs with rate control on in their constant qual= ity mode. Perhaps this is not the right thing to do. On 03/02/2015 02:52 AM, Thomas Davies wrote: > The "Class A" JCT-VC sequences were originated at 4k or 8k and cropped=20 > down for reasons of compute time. The 8k-originated ones (Nebuta and > SteamLocomotive) are very noisy and were shot with an early camera and=20 > sensor. There wasn't much good 4k at the time HEVC started, and it=20 > would certainly be good to get more. >=20 > regards >=20 > Thomas >=20 > On 25/02/15 17:44, Thomas Daede wrote: >> The JCT-VC test set only includes three clips for video conferencing,=20 >> which I don't think is sufficient. It also specifies a intra frame=20 >> period of 1 second, which is not reasonable. It does not feature any=20 >> 2160p test sequences either, which is a bit strange. >> _______________________________________________ video-codec mailing=20 >> list video-codec@ietf.org=20 >> https://www.ietf.org/mailman/listinfo/video-codec >=20 From nobody Tue Mar 3 01:10:57 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 667D81A1AD0 for ; Tue, 3 Mar 2015 01:10:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.977 X-Spam-Level: X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mcqN2eXZAZ_B for ; Tue, 3 Mar 2015 01:10:52 -0800 (PST) Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 254FB1A1AC6 for ; Tue, 3 Mar 2015 01:10:52 -0800 (PST) Received: by widex7 with SMTP id ex7so20976315wid.0 for ; Tue, 03 Mar 2015 01:10:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=RTJeEInD/5U6H4z7dSaI+ncfIbDrxPpor90NKrRzs80=; b=D/1U5Q8EA+fbiXleYNYkMUyp/UXQtj0GyD5y6Aa3Wf1wO0SXKrhCoyRCfvGnbMxnzy yT9K1TjRq6TMBjOBLk9ePMnX6gMCwYHIfuUgw9d4CGrGfPNHxwYZwkM5TPBZ2z3Btml8 RzuLCnN2oORiQ8IcDDfpB13I6ndfAcNVsCyt1UUCZnN/DNHxM8p9ieSpf6bxWyEQLtOk 988EMYHjdKaX0vg7iHc+dFaBGt0xojg4/cutbCRQFYPcpQ1Ao5d8nmnuXEWPdJZtDYZC jCPgKm7zfVhBoaJZPQb3gf7WXVFgH0PrN4E8GrgmqBF+7M/Tefe2xnUu86OwlBOAyfHq JOkw== X-Gm-Message-State: ALoCoQlNriCGXU/rOSGZxyzGCReSq2k9wxoR1izZhfBm4MH1TP75+u9vxQP5uTSnjw9XutfDaUcU MIME-Version: 1.0 X-Received: by 10.180.95.97 with SMTP id dj1mr703822wib.43.1425373850787; Tue, 03 Mar 2015 01:10:50 -0800 (PST) Received: by 10.194.14.129 with HTTP; Tue, 3 Mar 2015 01:10:50 -0800 (PST) In-Reply-To: <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> Date: Tue, 3 Mar 2015 11:10:50 +0200 Message-ID: From: Mohammed Raad To: "video-codec@ietf.org" Content-Type: multipart/alternative; boundary=f46d0418251e0c956405105eb614 Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 09:10:54 -0000 --f46d0418251e0c956405105eb614 Content-Type: text/plain; charset=UTF-8 At MPEG we use this to determine the codec performance for the "random access" case. I don't know why the 1 second interval was chosen and don't think that its relevant in many use cases, but MPEG codecs are aimed at satisfying a broad range of applications and so this test case has been maintained. Members of the potential IETF activity may find that this is an irrelevant test case. Regards, On Tue, Mar 3, 2015 at 10:47 AM, Thomas Davies (thdavies) < thdavies@cisco.com> wrote: > No, I suspect it was to be more appropriate for broadcast applications, > which usually have more frequent intra for channel switching. > > Back in the day, 1/2 second or 12 frame GOPs were deemed essential for > MPEG-2 digital TV to mimic analogue switching times, but I think that is > long gone - now audio is switched first and then the video when possible, > to give an illusion of immediacy. I think low bit rate channels could > easily have intra periods of 2 or 3 seconds. > > I agree it's short for streaming applications, and could be longer. > However, including a decent proportion of intra in the metrics is probably > good. > > Regards > > Thomas > > -----Original Message----- > From: Thomas Daede [mailto:tdaede@mozilla.com] > Sent: Monday, March 02, 2015 10:20 PM > To: Thomas Davies (thdavies) > Cc: video-codec@ietf.org > Subject: Re: [video-codec] Test sequences, automated and otherwise > > Do you know if the 1-second keyframe interval was also for reasons of > compute time? (It allows each GOP to be encoded in parallel). It > unfortunately seems a bit short for many streaming applications, and also > prevents rate control from being tested. > > At the moment, I run all codecs with rate control on in their constant > quality mode. Perhaps this is not the right thing to do. > > On 03/02/2015 02:52 AM, Thomas Davies wrote: > > The "Class A" JCT-VC sequences were originated at 4k or 8k and cropped > > down for reasons of compute time. The 8k-originated ones (Nebuta and > > SteamLocomotive) are very noisy and were shot with an early camera and > > sensor. There wasn't much good 4k at the time HEVC started, and it > > would certainly be good to get more. > > > > regards > > > > Thomas > > > > On 25/02/15 17:44, Thomas Daede wrote: > >> The JCT-VC test set only includes three clips for video conferencing, > >> which I don't think is sufficient. It also specifies a intra frame > >> period of 1 second, which is not reasonable. It does not feature any > >> 2160p test sequences either, which is a bit strange. > >> _______________________________________________ video-codec mailing > >> list video-codec@ietf.org > >> https://www.ietf.org/mailman/listinfo/video-codec > > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --f46d0418251e0c956405105eb614 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
At MPEG we use this to determine the codec performance for= the "random access" case. I don't know why the 1 second inte= rval was chosen and don't think that its relevant in many use cases, bu= t MPEG codecs are aimed at satisfying a broad range of applications and so = this test case has been maintained. Members of the potential IETF activity = may find that this is an irrelevant test case.

Regards,<= /div>

On Tue= , Mar 3, 2015 at 10:47 AM, Thomas Davies (thdavies) <<= a href=3D"mailto:thdavies@cisco.com" target=3D"_blank">thdavies@cisco.com> wrote:
No, I suspect it was= to be more appropriate for broadcast applications, which usually have more= frequent intra for channel switching.

Back in the day, 1/2 second or 12 frame GOPs were deemed essential for MPEG= -2 digital TV to mimic analogue switching times, but I think that is long g= one - now audio is switched first and then the video when possible, to give= an illusion of immediacy. I think low bit rate channels could easily have = intra periods of 2 or 3 seconds.

I agree it's short for streaming applications, and could be longer. How= ever, including a decent proportion of intra in the metrics is probably goo= d.

Regards

Thomas

-----Original Message-----
From: Thomas Daede [mailto:tdaede@moz= illa.com]
Sent: Monday, March 02, 2015 10:20 PM
To: Thomas Davies (thdavies)
Cc: video-codec@ietf.org
Subject: Re: [video-codec] Test sequences, automated and otherwise

Do you know if the 1-second keyframe interval was also for reasons of compu= te time? (It allows each GOP to be encoded in parallel). It unfortunately s= eems a bit short for many streaming applications, and also prevents rate co= ntrol from being tested.

At the moment, I run all codecs with rate control on in their constant qual= ity mode. Perhaps this is not the right thing to do.

On 03/02/2015 02:52 AM, Thomas Davies wrote:
> The "Class A" JCT-VC sequences were originated at 4k or 8k a= nd cropped
> down for reasons of compute time. The 8k-originated ones (Nebuta and > SteamLocomotive) are very noisy and were shot with an early camera and=
> sensor. There wasn't much good 4k at the time HEVC started, and it=
> would certainly be good to get more.
>
> regards
>
> Thomas
>
> On 25/02/15 17:44, Thomas Daede wrote:
>> The JCT-VC test set only includes three clips for video conferenci= ng,
>> which I don't think is sufficient. It also specifies a intra f= rame
>> period of 1 second, which is not reasonable. It does not feature a= ny
>> 2160p test sequences either, which is a bit strange.
>> _______________________________________________ video-codec mailin= g
>> list video-codec@ietf.org<= /a>
>> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
=
Mohammed Raad, PhD.
Partner
RAADTECH C= ONSULTING
P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 = 414451478
Email: mohammedsraad@raadtech.com
--f46d0418251e0c956405105eb614-- From nobody Tue Mar 3 04:34:33 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99CE31A3B9B for ; Tue, 3 Mar 2015 04:34:31 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.977 X-Spam-Level: X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 056ky6Viai9X for ; Tue, 3 Mar 2015 04:34:29 -0800 (PST) Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com [209.85.212.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F36421A1BDC for ; Tue, 3 Mar 2015 04:34:28 -0800 (PST) Received: by widem10 with SMTP id em10so22411963wid.1 for ; Tue, 03 Mar 2015 04:34:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=2RG9z3bEH0ZWamHgQmbxr6Z5fL4yBXBmW59u6vARbR8=; b=A82ZWff2JLATDmUw6gwQDVzesdTVzw1bbUpxNX8P6zZ2HJYs/ALuMZLX1Zj/jaqrIH 0LcXn2PR2Oc4G9nOWrV+7MvtIeZVjO1lCuVxgqS2Nzy6kIQPBCDmjEpre+8EbenkpTzI Zsmo+InCPqSWxKWwJmqkInHqy6D93li6z3pP4alrbTdhfoRltMRKELC3vzqch+i1hBBh 1nD0ndXEUt53sdYcEgs9rCDfOYrBYV5dvMgN+aqCUvvUf/z6Xfe+hNqzdNcP3J778005 S+wl0241xOO0XM+XR3H671y24yn4jvjzwlVi0++Tig7iomHyrMXmzbCcgXiVJgldkuEk N2Aw== X-Gm-Message-State: ALoCoQn8VBzRCE0TFJx719Vin9eoE6sjD/0jNRBwkEbfTSL6A0S35IGhzHaZ6lY6qUXHmvAgKRzx MIME-Version: 1.0 X-Received: by 10.194.171.136 with SMTP id au8mr70122189wjc.6.1425386067643; Tue, 03 Mar 2015 04:34:27 -0800 (PST) Received: by 10.194.14.129 with HTTP; Tue, 3 Mar 2015 04:34:27 -0800 (PST) In-Reply-To: References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> Date: Tue, 3 Mar 2015 14:34:27 +0200 Message-ID: From: Mohammed Raad To: "video-codec@ietf.org" Content-Type: multipart/alternative; boundary=089e0122f58c3ae4d10510618e55 Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2015 12:34:31 -0000 --089e0122f58c3ae4d10510618e55 Content-Type: text/plain; charset=UTF-8 Just a note on the JCT-VC sequences, not all will be available for use outside of MPEG based on the copyright terms under which these sequences were provided to MPEG. Regards, On Tue, Mar 3, 2015 at 11:10 AM, Mohammed Raad wrote: > At MPEG we use this to determine the codec performance for the "random > access" case. I don't know why the 1 second interval was chosen and don't > think that its relevant in many use cases, but MPEG codecs are aimed at > satisfying a broad range of applications and so this test case has been > maintained. Members of the potential IETF activity may find that this is an > irrelevant test case. > > Regards, > > On Tue, Mar 3, 2015 at 10:47 AM, Thomas Davies (thdavies) < > thdavies@cisco.com> wrote: > >> No, I suspect it was to be more appropriate for broadcast applications, >> which usually have more frequent intra for channel switching. >> >> Back in the day, 1/2 second or 12 frame GOPs were deemed essential for >> MPEG-2 digital TV to mimic analogue switching times, but I think that is >> long gone - now audio is switched first and then the video when possible, >> to give an illusion of immediacy. I think low bit rate channels could >> easily have intra periods of 2 or 3 seconds. >> >> I agree it's short for streaming applications, and could be longer. >> However, including a decent proportion of intra in the metrics is probably >> good. >> >> Regards >> >> Thomas >> >> -----Original Message----- >> From: Thomas Daede [mailto:tdaede@mozilla.com] >> Sent: Monday, March 02, 2015 10:20 PM >> To: Thomas Davies (thdavies) >> Cc: video-codec@ietf.org >> Subject: Re: [video-codec] Test sequences, automated and otherwise >> >> Do you know if the 1-second keyframe interval was also for reasons of >> compute time? (It allows each GOP to be encoded in parallel). It >> unfortunately seems a bit short for many streaming applications, and also >> prevents rate control from being tested. >> >> At the moment, I run all codecs with rate control on in their constant >> quality mode. Perhaps this is not the right thing to do. >> >> On 03/02/2015 02:52 AM, Thomas Davies wrote: >> > The "Class A" JCT-VC sequences were originated at 4k or 8k and cropped >> > down for reasons of compute time. The 8k-originated ones (Nebuta and >> > SteamLocomotive) are very noisy and were shot with an early camera and >> > sensor. There wasn't much good 4k at the time HEVC started, and it >> > would certainly be good to get more. >> > >> > regards >> > >> > Thomas >> > >> > On 25/02/15 17:44, Thomas Daede wrote: >> >> The JCT-VC test set only includes three clips for video conferencing, >> >> which I don't think is sufficient. It also specifies a intra frame >> >> period of 1 second, which is not reasonable. It does not feature any >> >> 2160p test sequences either, which is a bit strange. >> >> _______________________________________________ video-codec mailing >> >> list video-codec@ietf.org >> >> https://www.ietf.org/mailman/listinfo/video-codec >> > >> >> _______________________________________________ >> video-codec mailing list >> video-codec@ietf.org >> https://www.ietf.org/mailman/listinfo/video-codec >> > > > > -- > Mohammed Raad, PhD. > Partner > RAADTECH CONSULTING > P.O. Box 113 > Warrawong > NSW 2502 Australia > Phone: +61 414451478 > Email: mohammedsraad@raadtech.com > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --089e0122f58c3ae4d10510618e55 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Just a note on the JCT-VC sequences, not all will be avail= able for use outside of MPEG based on the copyright terms under which these= sequences were provided to MPEG.

Regards,=C2=A0

On Tue, Mar 3= , 2015 at 11:10 AM, Mohammed Raad <mohammedsraad@raadtech.com= > wrote:
At MP= EG we use this to determine the codec performance for the "random acce= ss" case. I don't know why the 1 second interval was chosen and do= n't think that its relevant in many use cases, but MPEG codecs are aime= d at satisfying a broad range of applications and so this test case has bee= n maintained. Members of the potential IETF activity may find that this is = an irrelevant test case.

Regards,

On T= ue, Mar 3, 2015 at 10:47 AM, Thomas Davies (thdavies) <= ;thdavies@cisco.com= > wrote:
No, I suspect it w= as to be more appropriate for broadcast applications, which usually have mo= re frequent intra for channel switching.

Back in the day, 1/2 second or 12 frame GOPs were deemed essential for MPEG= -2 digital TV to mimic analogue switching times, but I think that is long g= one - now audio is switched first and then the video when possible, to give= an illusion of immediacy. I think low bit rate channels could easily have = intra periods of 2 or 3 seconds.

I agree it's short for streaming applications, and could be longer. How= ever, including a decent proportion of intra in the metrics is probably goo= d.

Regards

Thomas

-----Original Message-----
From: Thomas Daede [mailto:tdaede@mozilla.com]
Sent: Monday, March 02, 2015 10:20 PM
To: Thomas Davies (thdavies)
Cc: video-codec@i= etf.org
Subject: Re: [video-codec] Test sequences, automated and otherwise

Do you know if the 1-second keyframe interval was also for reasons of compu= te time? (It allows each GOP to be encoded in parallel). It unfortunately s= eems a bit short for many streaming applications, and also prevents rate co= ntrol from being tested.

At the moment, I run all codecs with rate control on in their constant qual= ity mode. Perhaps this is not the right thing to do.

On 03/02/2015 02:52 AM, Thomas Davies wrote:
> The "Class A" JCT-VC sequences were originated at 4k or 8k a= nd cropped
> down for reasons of compute time. The 8k-originated ones (Nebuta and > SteamLocomotive) are very noisy and were shot with an early camera and=
> sensor. There wasn't much good 4k at the time HEVC started, and it=
> would certainly be good to get more.
>
> regards
>
> Thomas
>
> On 25/02/15 17:44, Thomas Daede wrote:
>> The JCT-VC test set only includes three clips for video conferenci= ng,
>> which I don't think is sufficient. It also specifies a intra f= rame
>> period of 1 second, which is not reasonable. It does not feature a= ny
>> 2160p test sequences either, which is a bit strange.
>> _______________________________________________ video-codec mailin= g
>> list vid= eo-codec@ietf.org
>> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.= org
https://www.ietf.org/mailman/listinfo/video-codec



<= /div>--
Mohammed Raad, PhD.
Partner
RAADTEC= H CONSULTING
P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: <= a href=3D"tel:%2B61%20414451478" value=3D"+61414451478" target=3D"_blank">+= 61 414451478
Email: mohammedsraad@raadtech.com



--
Mohammed Raad, PhD.
Partner
RAADTECH CONSULTING<= br>P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 414451478<= br>Email: m= ohammedsraad@raadtech.com
--089e0122f58c3ae4d10510618e55-- From nobody Tue Mar 10 10:15:35 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 02E0B1A6FE7 for ; Tue, 10 Mar 2015 10:15:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.277 X-Spam-Level: X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kN6fTkCcGNKF for ; Tue, 10 Mar 2015 10:15:29 -0700 (PDT) Received: from smtp.mozilla.org (mx1.corp.phx1.mozilla.com [63.245.216.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D89C61A6F3A for ; Tue, 10 Mar 2015 10:15:29 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 3751CF258F for ; Tue, 10 Mar 2015 10:15:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx1.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xhKcZ6nK0D9c for ; Tue, 10 Mar 2015 10:15:29 -0700 (PDT) Received: from [10.252.25.201] (corp.mtv2.mozilla.com [63.245.221.32]) (Authenticated sender: tterriberry@mozilla.com) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 0E26DF243C for ; Tue, 10 Mar 2015 10:15:29 -0700 (PDT) Message-ID: <54FF26B0.5080900@xiph.org> Date: Tue, 10 Mar 2015 10:15:28 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "video-codec@ietf.org" References: <20150309235110.30185.80036.idtracker@ietfa.amsl.com> In-Reply-To: <20150309235110.30185.80036.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20150309235110.30185.80036.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: [video-codec] Fwd: New Version Notification for draft-terriberry-codingtools-02.txt X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 17:15:34 -0000 This version just adds some details on our 32x32 transform, and removes some outdated text. -------- Forwarded Message -------- Subject: New Version Notification for draft-terriberry-codingtools-02.txt Date: Mon, 09 Mar 2015 16:51:10 -0700 From: internet-drafts@ietf.org To: Timothy Terriberry , Timothy B. Terriberry A new version of I-D, draft-terriberry-codingtools-02.txt has been successfully submitted by Timothy B. Terriberry and posted to the IETF repository. Name: draft-terriberry-codingtools Revision: 02 Title: Coding Tools for a Next Generation Video Codec Document date: 2015-03-09 Group: Individual Submission Pages: 18 URL: http://www.ietf.org/internet-drafts/draft-terriberry-codingtools-02.txt Status: https://datatracker.ietf.org/doc/draft-terriberry-codingtools/ Htmlized: http://tools.ietf.org/html/draft-terriberry-codingtools-02 Diff: http://www.ietf.org/rfcdiff?url2=draft-terriberry-codingtools-02 Abstract: This document proposes a number of coding tools that could be incorporated into a next-generation video codec. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Tue Mar 10 10:18:18 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 495CB1A6FCB for ; Tue, 10 Mar 2015 10:18:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.599 X-Spam-Level: X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pl6wRRR1N-E6 for ; Tue, 10 Mar 2015 10:18:13 -0700 (PDT) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E120E1A6F3A for ; Tue, 10 Mar 2015 10:18:12 -0700 (PDT) Received: by qcvp6 with SMTP id p6so3651191qcv.1 for ; Tue, 10 Mar 2015 10:18:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type; bh=L6Se7tO6D+bc4+zqeksaFVn7lqin0DW73XRlRwYsp/o=; b=gwYx0yC+a5zVWcAvw1dDjl+FX2UkHfOQjzSZ4zcb+XXWTe6H4KSTdSZ5vPn4yz3Blg rBvu2Xm5p3ntR/WegnM9TRDbW9sFr+62wpQQWNpSTlHAWcdzTxwso6am1o2us5fVKWaO iSO/f6Tsrj9Pr2vqPvc8F0yRPipDvHSAaAYaCUa8stvCcD+urpTp4tlXpJxQV4TQBZLg iM9Ttl5RHc9hbmBzp/6bkU4I8siuOfjlpQMqOKVl/bDMwHRhKqw/7xH2lbSegOE66kBl tyMMYt2CM86H1sbIjXoHh97HBK3nHmRHvy2TuVaK8X8cQ2AjJvHT8uO1+/pfnOS1UBHu qdFA== X-Gm-Message-State: ALoCoQmBYV9s2L5qZ5GrdYzEdOajTArGu8zUfTZyk/ZMEP+gLKfjPX8WbQexeUv88pfLNPQgbK1L X-Received: by 10.140.147.66 with SMTP id 63mr42675040qht.35.1426007892168; Tue, 10 Mar 2015 10:18:12 -0700 (PDT) Received: from ?IPv6:2601:a:481:46a0:120b:a9ff:fe23:8e40? ([2601:a:481:46a0:120b:a9ff:fe23:8e40]) by mx.google.com with ESMTPSA id o16sm764101qkh.25.2015.03.10.10.18.11 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Mar 2015 10:18:11 -0700 (PDT) Message-ID: <54FF2750.8090206@dgql.org> Date: Tue, 10 Mar 2015 13:18:08 -0400 From: Nathan Egge User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <20150310000027.25118.84521.idtracker@ietfa.amsl.com> In-Reply-To: <20150310000027.25118.84521.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20150310000027.25118.84521.idtracker@ietfa.amsl.com> Content-Type: multipart/alternative; boundary="------------030807010803000602090500" Archived-At: Subject: [video-codec] Fwd: New Version Notification for draft-egge-videocodec-tdlt-01.txt X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 17:18:17 -0000 This is a multi-part message in MIME format. --------------030807010803000602090500 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit This version adds details about applying lapped transforms with multiple block sizes and includes references to work done on frequency domain intra prediction (for use with lapped transforms). -------- Forwarded Message -------- Subject: New Version Notification for draft-egge-videocodec-tdlt-01.txt Date: Mon, 09 Mar 2015 17:00:27 -0700 From: internet-drafts@ietf.org To: Nathan E. Egge , Nathan E. Egge , Timothy Terriberry , Timothy B. Terriberry A new version of I-D, draft-egge-videocodec-tdlt-01.txt has been successfully submitted by Nathan E. Egge and posted to the IETF repository. Name: draft-egge-videocodec-tdlt Revision: 01 Title: Time Domain Lapped Transforms for Video Coding Document date: 2015-03-09 Group: Individual Submission Pages: 18 URL: http://www.ietf.org/internet-drafts/draft-egge-videocodec-tdlt-01.txt Status: https://datatracker.ietf.org/doc/draft-egge-videocodec-tdlt/ Htmlized: http://tools.ietf.org/html/draft-egge-videocodec-tdlt-01 Diff: http://www.ietf.org/rfcdiff?url2=draft-egge-videocodec-tdlt-01 Abstract: This proposes the use of Time Domain Lapped Transforms (TDLT) as the transform step for video coding. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat --------------030807010803000602090500 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit This version adds details about applying lapped transforms with multiple block sizes and includes references to work done on frequency domain intra prediction (for use with lapped transforms).


-------- Forwarded Message --------
Subject: New Version Notification for draft-egge-videocodec-tdlt-01.txt
Date: Mon, 09 Mar 2015 17:00:27 -0700
From: internet-drafts@ietf.org
To: Nathan E. Egge <negge@dgql.org>, Nathan E. Egge <negge@dgql.org>, Timothy Terriberry <tterribe@xiph.org>, Timothy B. Terriberry <tterribe@xiph.org> 


A new version of I-D, draft-egge-videocodec-tdlt-01.txt
has been successfully submitted by Nathan E. Egge and posted to the
IETF repository.

Name:		draft-egge-videocodec-tdlt
Revision:	01
Title:		Time Domain Lapped Transforms for Video Coding
Document date:	2015-03-09
Group:		Individual Submission
Pages:		18
URL:            http://www.ietf.org/internet-drafts/draft-egge-videocodec-tdlt-01.txt
Status:         https://datatracker.ietf.org/doc/draft-egge-videocodec-tdlt/
Htmlized:       http://tools.ietf.org/html/draft-egge-videocodec-tdlt-01
Diff:           http://www.ietf.org/rfcdiff?url2=draft-egge-videocodec-tdlt-01

Abstract:
   This proposes the use of Time Domain Lapped Transforms (TDLT) as the
   transform step for video coding.

                                                                                  


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat


--------------030807010803000602090500-- From nobody Tue Mar 10 10:28:52 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BD1F21A1B21 for ; Tue, 10 Mar 2015 10:28:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VvgTcedraFJU for ; Tue, 10 Mar 2015 10:28:49 -0700 (PDT) Received: from mail-ig0-f182.google.com (mail-ig0-f182.google.com [209.85.213.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 219D71A012D for ; Tue, 10 Mar 2015 10:28:49 -0700 (PDT) Received: by igal13 with SMTP id l13so5507256iga.0 for ; Tue, 10 Mar 2015 10:28:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=/PmLzztOakDtjWETKbGx+hM9oCfWih2QeB7NaktGUk4=; b=bzGDzoV/yPT8+VV/qBxjNgaCV9Vf5jVgX1UOgJYzMutQ8wqDZj6/LoyrUrw0nvqEk7 HqQPmRsln+sOAZvQsw3ribTK6doRL9D950TQiuL/2XTAYqibtDrNaOR5eaLgbHoAW326 zTHZs0SbcospSgAc1NCykFjnzVdM38Z4CEltHd+8eNopHbb5fqqM1nDp4jezdY6UbiK7 oZWSAB6Lm2Us61JnaI5ZYT37kOCNHQldQiQnOUoJXV3FsD9/VyyMg6bAAbUekfeoPWSe b8mMyL/7I+6BH/0Vp/kAOme60WD8j1xXz+BRuHCMf0ee5fNrpmQNS95GAaiRHojL+ZLy cIEw== X-Gm-Message-State: ALoCoQkobWcn6OeLJC753hZbZF61amQuub20iXtPh6+Jy8VMmbEVXHnnymuP4mN8T3nRN3ZdQmnp X-Received: by 10.42.79.15 with SMTP id p15mr36544853ick.54.1426008528519; Tue, 10 Mar 2015 10:28:48 -0700 (PDT) Received: from panoramix.jmvalin.ca (modemcable074.170-201-24.mc.videotron.ca. [24.201.170.74]) by mx.google.com with ESMTPSA id qr1sm1917884igb.18.2015.03.10.10.28.47 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Mar 2015 10:28:48 -0700 (PDT) Message-ID: <54FF29CF.3060201@jmvalin.ca> Date: Tue, 10 Mar 2015 13:28:47 -0400 From: Jean-Marc Valin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <20150309215735.25966.33609.idtracker@ietfa.amsl.com> In-Reply-To: <20150309215735.25966.33609.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20150309215735.25966.33609.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Subject: [video-codec] Fwd: New Version Notification for draft-valin-videocodec-pvq-02.txt X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 17:28:50 -0000 This is just a minor update on the PVQ draft, also adding links to a demo and to a paper that was written on the topic. -------- Forwarded Message -------- Subject: New Version Notification for draft-valin-videocodec-pvq-02.txt Date: Mon, 09 Mar 2015 14:57:35 -0700 From: internet-drafts@ietf.org To: Jean-Marc Valin , Jean-Marc Valin A new version of I-D, draft-valin-videocodec-pvq-02.txt has been successfully submitted by Jean-Marc Valin and posted to the IETF repository. Name: draft-valin-videocodec-pvq Revision: 02 Title: Pyramid Vector Quantization for Video Coding Document date: 2015-03-09 Group: Individual Submission Pages: 7 URL: http://www.ietf.org/internet-drafts/draft-valin-videocodec-pvq-02.txt Status: https://datatracker.ietf.org/doc/draft-valin-videocodec-pvq/ Htmlized: http://tools.ietf.org/html/draft-valin-videocodec-pvq-02 Diff: http://www.ietf.org/rfcdiff?url2=draft-valin-videocodec-pvq-02 Abstract: This proposes applying pyramid vector quantization (PVQ) to video coding. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Tue Mar 10 11:20:34 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 81FB31A8746 for ; Tue, 10 Mar 2015 11:20:33 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.278 X-Spam-Level: X-Spam-Status: No, score=-1.278 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vvXGmB6ACW-J for ; Tue, 10 Mar 2015 11:20:32 -0700 (PDT) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E84DB1A872C for ; Tue, 10 Mar 2015 11:20:31 -0700 (PDT) Received: by lbiw7 with SMTP id w7so3685411lbi.7 for ; Tue, 10 Mar 2015 11:20:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=IOP2/dOLslXQcrxyPxIOy9+oC/0ZzPNu/DX7ylb73EM=; b=EcCYBH+fBEdjsaudkFZmdwDq9pQYs+n+CjuwDx83FWq+GrAOp8CITfxsmIQunx4P9s weUT93f/91Jpjv7Ysl/y0Sc0AOc1S3JT+e8s3BDgB9ejGHyD55QRse8eYspOl/BffgpF ewfXkGjuyZ7EzrxTNbSSzym3EhdYU/is5bZz0wdZYIQoOw2w7lIkOk7NMk14dWUK3Mqg wrPUa+PNke6sOYbt3mS3NCmuia2/PIsOCSDSLslwikVaCP52k/KVLZkX2s3CNCIugCF7 rDMiRw1EoGPFxMZgvYB1YQsQ7G26ekvowaOjzcNsyvMjEhKesKLNzH4DxuLB6XbS+0gn BBlQ== MIME-Version: 1.0 X-Received: by 10.152.27.232 with SMTP id w8mr26921326lag.51.1426011630286; Tue, 10 Mar 2015 11:20:30 -0700 (PDT) Sender: metajack@gmail.com Received: by 10.25.14.206 with HTTP; Tue, 10 Mar 2015 11:20:30 -0700 (PDT) In-Reply-To: <20150310000316.3179.32896.idtracker@ietfa.amsl.com> References: <20150310000316.3179.32896.idtracker@ietfa.amsl.com> Date: Tue, 10 Mar 2015 12:20:30 -0600 X-Google-Sender-Auth: bcH8tgfSxvEquHcrokt3RFZzJuU Message-ID: From: Jack Moffitt To: "video-codec@ietf.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: [video-codec] Fwd: New Version Notification for draft-moffitt-netvc-requirements-00.txt X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 18:20:33 -0000 This draft attempts to lay out requirements and considerations for the video codec effort. This first version doesn't have a lot of details yet, but I've summarized the main points. Input and co-authors very welcome! jack. ---------- Forwarded message ---------- From: Date: Mon, Mar 9, 2015 at 6:03 PM Subject: New Version Notification for draft-moffitt-netvc-requirements-00.txt To: Mo Zanaty , Jack Moffitt A new version of I-D, draft-moffitt-netvc-requirements-00.txt has been successfully submitted by Jack Moffitt and posted to the IETF repository. Name: draft-moffitt-netvc-requirements Revision: 00 Title: Video Codec Requirements Document date: 2015-03-09 Group: Individual Submission Pages: 6 URL: http://www.ietf.org/internet-drafts/draft-moffitt-netvc-requirements-00.txt Status: https://datatracker.ietf.org/doc/draft-moffitt-netvc-requirements/ Htmlized: http://tools.ietf.org/html/draft-moffitt-netvc-requirements-00 Abstract: This document provides specific requirements for an Internet video codec. These requirements address quality, bit-rate, and packet-loss robustness, as well as other desirable properties. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Tue Mar 10 16:15:49 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4A9D71A8880 for ; Tue, 10 Mar 2015 16:15:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h_HJ0fPN1OOi for ; Tue, 10 Mar 2015 16:15:45 -0700 (PDT) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8F4171A87C6 for ; Tue, 10 Mar 2015 16:15:45 -0700 (PDT) Received: by pdjp10 with SMTP id p10so5933635pdj.10 for ; Tue, 10 Mar 2015 16:15:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=hgVR+dvlQ79/CcecQP2GyQEjfAF/Pia/dQ63kmpYOA8=; b=VT4pPlSsSgD/ryARWuURnZad4K0XQyJDsqMP/fbAP8UiAJD6FAAJc6IQI9HDmhXElu 6ARcoWuABdeOkF1GPZwZfb6+j0pZ9QXQsk52s6ztQ7/PdX/pZwde75PkW6DPJe3l89gA CRXfZgdk2uToi8op3LHjQ6S2/FGIgsRf26ze+eJe+Z3FD/zU/OYZd6OK4HYPig6ahWfa gl2ZxSIM/pu3tGkxHd1mTty4wowllN+idMAwKcP4fg6ogGgPb2ef8mk2nXQgoB7oOWXg 64n1XSkpnwgFOkSJCwBSYc4mVymCqSTgb01IUlQ09vlFRwIoCMegjr0EFBTMbM7/OSFX 5e5Q== X-Gm-Message-State: ALoCoQnyakzZokIwAdHKfNSS1yplTpD4LngCnnPpwxzvL2Ir5SS1s1O7txrWIfSpsQ1nzQJJs3iF X-Received: by 10.66.189.105 with SMTP id gh9mr70316212pac.41.1426029345128; Tue, 10 Mar 2015 16:15:45 -0700 (PDT) Received: from ?IPv6:2620:101:80fc:224:7e7a:91ff:fe9e:8126? ([2620:101:80fc:224:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id y2sm2769066pdm.31.2015.03.10.16.15.44 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Mar 2015 16:15:44 -0700 (PDT) Message-ID: <54FF7B1F.3030709@mozilla.com> Date: Tue, 10 Mar 2015 16:15:43 -0700 From: Thomas Daede User-Agent: http://a.pomf.se/cszdno.opus MIME-Version: 1.0 To: video-codec@ietf.org References: <20150309231747.6652.10862.idtracker@ietfa.amsl.com> In-Reply-To: <20150309231747.6652.10862.idtracker@ietfa.amsl.com> X-Forwarded-Message-Id: <20150309231747.6652.10862.idtracker@ietfa.amsl.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: [video-codec] Fwd: New Version Notification for draft-daede-netvc-testing-00.txt X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Mar 2015 23:15:48 -0000 This is the first version of the testing draft, which is based on my experience with AreWeCompressedYet, as well as feedback on the list. Please take a look to see if it makes sense. There is still work to be done on filling out the test categories, as well as operation modes. More authors would also be welcome. -------- Forwarded Message -------- Subject: New Version Notification for draft-daede-netvc-testing-00.txt Date: Mon, 09 Mar 2015 16:17:47 -0700 From: internet-drafts@ietf.org To: Thomas Daede , Jack Moffitt , Thomas Daede , Jack Moffitt A new version of I-D, draft-daede-netvc-testing-00.txt has been successfully submitted by Thomas Daede and posted to the IETF repository. Name: draft-daede-netvc-testing Revision: 00 Title: Video Codec Testing and Quality Measurement Document date: 2015-03-09 Group: Individual Submission Pages: 7 URL: http://www.ietf.org/internet-drafts/draft-daede-netvc-testing-00.txt Status: https://datatracker.ietf.org/doc/draft-daede-netvc-testing/ Htmlized: http://tools.ietf.org/html/draft-daede-netvc-testing-00 Abstract: This document describes guidelines and procedures for evaluating an internet video codec specified at the IETF. This covers subjective and objective tests, test conditions, and materials used for the test. Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org. The IETF Secretariat From nobody Sun Mar 15 03:29:53 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5F2631A00B8 for ; Sun, 15 Mar 2015 03:29:51 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.51 X-Spam-Level: X-Spam-Status: No, score=-1.51 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6N-PqEGWy308 for ; Sun, 15 Mar 2015 03:29:49 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [158.38.152.117]) by ietfa.amsl.com (Postfix) with ESMTP id 54AF31A0092 for ; Sun, 15 Mar 2015 03:29:49 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 410DE7C4A63 for ; Sun, 15 Mar 2015 11:29:48 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eCrRlLNIFpQm for ; Sun, 15 Mar 2015 11:29:47 +0100 (CET) Received: from [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f] (unknown [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f]) by mork.alvestrand.no (Postfix) with ESMTPSA id 4133A7C4A47 for ; Sun, 15 Mar 2015 11:29:47 +0100 (CET) Message-ID: <55055F1A.7090009@alvestrand.no> Date: Sun, 15 Mar 2015 11:29:46 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> In-Reply-To: <54F4E204.5@mozilla.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 10:29:51 -0000 Den 02. mars 2015 23:19, skrev Thomas Daede: > Do you know if the 1-second keyframe interval was also for reasons of > compute time? (It allows each GOP to be encoded in parallel). It > unfortunately seems a bit short for many streaming applications, and > also prevents rate control from being tested. > > At the moment, I run all codecs with rate control on in their constant > quality mode. Perhaps this is not the right thing to do. I don't think that's even possible; when I looked at the VP8 code, I saw that if you turn on constant quality mode, rate control will disable itself - the code path for rate control is simply never reached. Other codecs may have other ways to deal with such a combination of parameters. One easy test to see how this works is to encode a clip four times: 1 - Rate A, quality X 2 - Rate B, quality X 3 - Rate A, quality Y 4 - Rate B, quality Y Most likely, the results will be pairwise equal, and you will then know which setting overrides the other. From nobody Sun Mar 15 03:32:06 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 33DE41A0110 for ; Sun, 15 Mar 2015 03:32:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.21 X-Spam-Level: X-Spam-Status: No, score=-4.21 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OxQaY1qOec8g for ; Sun, 15 Mar 2015 03:32:05 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [158.38.152.117]) by ietfa.amsl.com (Postfix) with ESMTP id C415F1A00B8 for ; Sun, 15 Mar 2015 03:32:04 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 120497C4A63 for ; Sun, 15 Mar 2015 11:32:04 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6ER3RW34f10h for ; Sun, 15 Mar 2015 11:32:02 +0100 (CET) Received: from [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f] (unknown [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f]) by mork.alvestrand.no (Postfix) with ESMTPSA id D13237C4A47 for ; Sun, 15 Mar 2015 11:32:02 +0100 (CET) Message-ID: <55055FA2.8060703@alvestrand.no> Date: Sun, 15 Mar 2015 11:32:02 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [video-codec] Test sequences, automated and otherwise X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 10:32:06 -0000 Den 03. mars 2015 03:19, skrev Mo Zanaty (mzanaty): > Harald will probably disagree, but the convention is to disable rate > control for comparing objective metrics. Oh no, I completely agree that this is the convention. I just don't agree that it's useful :-) Rate control is normally an > application-specific configuration of the encoder which can add noise or > bias if comparing different rate controls. Rate control strategies can > often be abstracted to apply to any modern codec (with adaptive > quantizers), so it is best to remove this source of noise/bias if you > can¹t guarantee identical (not default) strategies across the comparison. > Note that some GOP structures often include some basic rate controls by > default, like different QP offsets for hierarchical B/P frames. This can > also be a source of noise/bias unless configured identically across the > comparison. > > Mo > > On 3/2/15, 5:19 PM, Thomas Daede wrote: > At the moment, I run all codecs with rate control on in their constant > quality mode. Perhaps this is not the right thing to do. > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > From nobody Sun Mar 15 03:36:46 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F36FA1A012D for ; Sun, 15 Mar 2015 03:36:44 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.21 X-Spam-Level: X-Spam-Status: No, score=-4.21 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Oau_HGR2Nxc6 for ; Sun, 15 Mar 2015 03:36:43 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [158.38.152.117]) by ietfa.amsl.com (Postfix) with ESMTP id B20C21A007B for ; Sun, 15 Mar 2015 03:36:43 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 024897C4AB8 for ; Sun, 15 Mar 2015 11:36:43 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 771CCLTEjwcC for ; Sun, 15 Mar 2015 11:36:40 +0100 (CET) Received: from [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f] (unknown [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f]) by mork.alvestrand.no (Postfix) with ESMTPSA id 58BE17C4AA8 for ; Sun, 15 Mar 2015 11:36:40 +0100 (CET) Message-ID: <550560B7.6090501@alvestrand.no> Date: Sun, 15 Mar 2015 11:36:39 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Archived-At: Subject: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 10:36:45 -0000 Forking the subject.... Den 03. mars 2015 10:10, skrev Mohammed Raad: > At MPEG we use this to determine the codec performance for the "random > access" case. I don't know why the 1 second interval was chosen and > don't think that its relevant in many use cases, but MPEG codecs are > aimed at satisfying a broad range of applications and so this test case > has been maintained. Members of the potential IETF activity may find > that this is an irrelevant test case. The reasons for having non-initial I-frames that I know of: - To allow seeking in stored media. To jump to a specific point, you locate the last I-frame before the point you want to go to and run the decoder from that point to the point where you really want to go. With DASH being a prevalent mode of delivering stored media, and each DASH chunk starting with an I-frame, 5 seconds would be realistic for this. - To allow recovery from a known good state in streaming media. For interactive two-way media, you can explicitly ask for an I-frame and have the headend produce it; for other media, such as broadcast (the original use case for half-second intervals, as mentioned above), more complex schemes are needed (I've heard of technologies for storing a reference frame in some other form and transferring it out of band so that the decoder can initialize its state and start decoding without an I-frame, but I don't know if these are used in practice.) From nobody Sun Mar 15 06:26:28 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4FAEA1A1A65 for ; Sun, 15 Mar 2015 06:26:26 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7M3wGvTOIOei for ; Sun, 15 Mar 2015 06:26:24 -0700 (PDT) Received: from rcdn-iport-2.cisco.com (rcdn-iport-2.cisco.com [173.37.86.73]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 793751A0AFE for ; Sun, 15 Mar 2015 06:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1989; q=dns/txt; s=iport; t=1426425984; x=1427635584; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=fULL0lHB1y1ECMASBecIvALQ2trc6bvl3OBG0SYyJAY=; b=meee4tli0Xx6HOfsvAh+b8Rv+bDMnJEkFiK/EZC39/AWQ/7OLx+rAgT3 6MjnhB4BducUUnW+GwVi2tEnZBbYXVB+WEm+tU3Hgz1YHkcexojUS3WuR bvmVJP1RioT2viM2VfLD2v36oyyRMPcKC7aLOB9t6dsSw/ejRqwjVaXwj I=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0A4BQA5iAVV/4YNJK1bgwZSWsQRCoV0AoEVTAEBAQEBAX2EEAEBBAEBATc0CxACAQg2ECcLJQIECgQFiC8NyEQBAQEBAQEBAQEBAQEBAQEBAQEBAQETBIsXhD4zB4MXgRYFkDqJZpQUI4IygTxvgkMBAQE X-IronPort-AV: E=Sophos;i="5.11,404,1422921600"; d="scan'208";a="404133387" Received: from alln-core-12.cisco.com ([173.36.13.134]) by rcdn-iport-2.cisco.com with ESMTP; 15 Mar 2015 13:26:03 +0000 Received: from xhc-rcd-x10.cisco.com (xhc-rcd-x10.cisco.com [173.37.183.84]) by alln-core-12.cisco.com (8.14.5/8.14.5) with ESMTP id t2FDQ34o000465 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 15 Mar 2015 13:26:03 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-rcd-x10.cisco.com ([173.37.183.84]) with mapi id 14.03.0195.001; Sun, 15 Mar 2015 08:26:03 -0500 From: "Mo Zanaty (mzanaty)" To: Harald Alvestrand Thread-Topic: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) Thread-Index: AQHQXwvyydCo1wS3MEuqEifMwZXd/50diRIY Date: Sun, 15 Mar 2015 13:26:02 +0000 Message-ID: <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> In-Reply-To: <550560B7.6090501@alvestrand.no> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 13:26:26 -0000 Another reason for testing with periodic intra is the recent rise in RTP mi= xers which switch rather than transcode media. Setting the intra interval t= o the average active speaker switching time gives a good estimate of real-w= orld codec efficiency in these topologies.=20 Mo On Mar 15, 2015, at 6:36 AM, Harald Alvestrand wrote= : Forking the subject.... Den 03. mars 2015 10:10, skrev Mohammed Raad: > At MPEG we use this to determine the codec performance for the "random > access" case. I don't know why the 1 second interval was chosen and > don't think that its relevant in many use cases, but MPEG codecs are > aimed at satisfying a broad range of applications and so this test case > has been maintained. Members of the potential IETF activity may find > that this is an irrelevant test case. The reasons for having non-initial I-frames that I know of: - To allow seeking in stored media. To jump to a specific point, you locate the last I-frame before the point you want to go to and run the decoder from that point to the point where you really want to go. With DASH being a prevalent mode of delivering stored media, and each DASH chunk starting with an I-frame, 5 seconds would be realistic for this. - To allow recovery from a known good state in streaming media. For interactive two-way media, you can explicitly ask for an I-frame and have the headend produce it; for other media, such as broadcast (the original use case for half-second intervals, as mentioned above), more complex schemes are needed (I've heard of technologies for storing a reference frame in some other form and transferring it out of band so that the decoder can initialize its state and start decoding without an I-frame, but I don't know if these are used in practice.) _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec From nobody Sun Mar 15 07:06:49 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 44A0C1A1A6F for ; Sun, 15 Mar 2015 07:06:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.277 X-Spam-Level: X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eTREo0ll7UZo for ; Sun, 15 Mar 2015 07:06:46 -0700 (PDT) Received: from smtp.mozilla.org (mx1.corp.phx1.mozilla.com [63.245.216.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 714951A1A65 for ; Sun, 15 Mar 2015 07:06:46 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 05EEDF218C for ; Sun, 15 Mar 2015 07:06:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx1.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4h0-nRjf0zEx for ; Sun, 15 Mar 2015 07:06:44 -0700 (PDT) Received: from [172.17.0.107] (50-78-100-113-static.hfc.comcastbusiness.net [50.78.100.113]) (Authenticated sender: tterriberry@mozilla.com) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 916CBF2015 for ; Sun, 15 Mar 2015 07:06:44 -0700 (PDT) Message-ID: <550591F2.7020809@xiph.org> Date: Sun, 15 Mar 2015 07:06:42 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "video-codec@ietf.org" References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> In-Reply-To: <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 14:06:48 -0000 Mo Zanaty (mzanaty) wrote: > Another reason for testing with periodic intra is the recent rise in RTP mixers which switch rather than transcode media. Setting the intra interval to the average active speaker switching time gives a good estimate of real-world codec efficiency in these topologies. Do either you or Harald have some statistics with which to estimate that time? From nobody Sun Mar 15 09:54:54 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 30B721A0358 for ; Sun, 15 Mar 2015 09:54:53 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Np2Ue7Vs4qYT for ; Sun, 15 Mar 2015 09:54:49 -0700 (PDT) Received: from rcdn-iport-4.cisco.com (rcdn-iport-4.cisco.com [173.37.86.75]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3156E1A01EC for ; Sun, 15 Mar 2015 09:54:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1049; q=dns/txt; s=iport; t=1426438490; x=1427648090; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=OrAIdQLt+1dlzt4jqn+Ohp6+GVikx5Cd6mVpIKH++Iw=; b=YHG0Fxn+GRHmtU91Vl6cnmdvkgm1K3acDJce6dlrgfaFSf0WkSrUO3oT cRTc8el/PL2MU++lQQmmvR5dqa7CYvy8qkyHk9Vxbq8rXVYMt5/GivY/F ehy0COnctfd+y+xgyLcQDuTn1fhbcIiF8p19tXMo/slPVotqHlveUba7k o=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0A4BQAluAVV/4YNJK1bgwZSWsQQCoV0AoEVTAEBAQEBAX2EEAEBBAEBATc0CxACAQgYHhAnCyUCBA4FiC8NyAABAQEBAQEBAQEBAQEBAQEBAQEBAQETBIsXhD4zB4MXgRYFkDqFT4QXlBQjgjKBPG+CQwEBAQ X-IronPort-AV: E=Sophos;i="5.11,404,1422921600"; d="scan'208";a="403870505" Received: from alln-core-12.cisco.com ([173.36.13.134]) by rcdn-iport-4.cisco.com with ESMTP; 15 Mar 2015 16:54:49 +0000 Received: from xhc-aln-x11.cisco.com (xhc-aln-x11.cisco.com [173.36.12.85]) by alln-core-12.cisco.com (8.14.5/8.14.5) with ESMTP id t2FGsmxv019829 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 15 Mar 2015 16:54:48 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-aln-x11.cisco.com ([173.36.12.85]) with mapi id 14.03.0195.001; Sun, 15 Mar 2015 11:54:48 -0500 From: "Mo Zanaty (mzanaty)" To: "Timothy B. Terriberry" Thread-Topic: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) Thread-Index: AQHQXwvyydCo1wS3MEuqEifMwZXd/50diRIYgABfLgD//9slHg== Date: Sun, 15 Mar 2015 16:54:47 +0000 Message-ID: References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com>,<550591F2.7020809@xiph.org> In-Reply-To: <550591F2.7020809@xiph.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 16:54:53 -0000 I don't have good stats, but I'm sure it's much longer than all current tes= t sequences, which tend to be only a few seconds due to uncompressed video = size constraints.=20 It is important to understand intra efficiency during a stream switch, reco= very, join, random access, etc. Perhaps we can just test All-Intra configs = as a separate data point for this rather than mixing periodic intra.=20 Mo On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry wro= te: Mo Zanaty (mzanaty) wrote: > Another reason for testing with periodic intra is the recent rise in RTP = mixers which switch rather than transcode media. Setting the intra interval= to the average active speaker switching time gives a good estimate of real= -world codec efficiency in these topologies. Do either you or Harald have some statistics with which to estimate that ti= me? _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec From nobody Sun Mar 15 10:47:32 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 15FED1A00EA for ; Sun, 15 Mar 2015 10:47:32 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rCRo6OO6wTuC for ; Sun, 15 Mar 2015 10:47:29 -0700 (PDT) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 96F8B1A038E for ; Sun, 15 Mar 2015 10:47:29 -0700 (PDT) Received: by wgdm6 with SMTP id m6so23377137wgd.2 for ; Sun, 15 Mar 2015 10:47:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=79+VmLhVJ/ONylI72UjhFpMVgP5lqYLR54F1G4wxgCs=; b=YnhrvahL/jirFoZBk1Ii+HoJwE2rwwZlrsuzj5Xtvo4YCIEnUXHkqeiKD/uJPpqMSF ycZfucqejfL7wLLy28OiG6dzzZS6NHHmHvYE6bTasYy0QQqF78Ok5Mf6haASpQ8J4GNk Jd8PmGXZuKVYq2An1m45LHJlrEDD5p0Px5MFBpSTwPkr4fYLHDGA7fJTdLp/Sz++1B2U FHMlq/U+ekHyji3voCcgPMxWTe0vm2IT8+xeS26WH9wAjoBhuBU0pnax584h8qoTadI+ Y87v2bayJcQ1UnvvXiNMbW4hHLrp8qybpN3U6EGBg8UbnyeCXgG6h2x7jCcLjgWqZCrj NOUw== X-Received: by 10.194.236.200 with SMTP id uw8mr112332683wjc.10.1426441647988; Sun, 15 Mar 2015 10:47:27 -0700 (PDT) Received: from [192.168.0.194] ([95.61.111.78]) by mx.google.com with ESMTPSA id lj13sm11812989wic.9.2015.03.15.10.47.22 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Mar 2015 10:47:27 -0700 (PDT) Message-ID: <5505C5AA.1030703@gmail.com> Date: Sun, 15 Mar 2015 18:47:22 +0100 From: Sergio Garcia Murillo User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> In-Reply-To: <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 17:47:32 -0000 Hi, Based on my hands-on-experience, RTP mixer should not depend on periodic Intras for working, instead they should send an RTCP FIR/PLI request to the new active speaker and wait until the I-Frame is received before switching the RTP stream. In fact, in my mcu I configure the encoders to not send periodic i frames at all and only send them by request (on received RTCP FIR/PLI). I agree with Harald on the other use cases for non-initial I-frames, especially the seeking on storing media one. But I think there could be other alternatives that provide viable solutions while not having a big impact on quality/bitrate as Iframes have. BTW, not sure if it is the right place/thread, but as an implementator, I find it quite important to be able to tell if a frame is iframe/recovery/seekable without to deep diving into the encoded frame bitstream. Best regards Sergio On 15/03/2015 14:26, Mo Zanaty (mzanaty) wrote: > Another reason for testing with periodic intra is the recent rise in RTP mixers which switch rather than transcode media. Setting the intra interval to the average active speaker switching time gives a good estimate of real-world codec efficiency in these topologies. > > Mo > > > > On Mar 15, 2015, at 6:36 AM, Harald Alvestrand wrote: > > Forking the subject.... > > Den 03. mars 2015 10:10, skrev Mohammed Raad: >> At MPEG we use this to determine the codec performance for the "random >> access" case. I don't know why the 1 second interval was chosen and >> don't think that its relevant in many use cases, but MPEG codecs are >> aimed at satisfying a broad range of applications and so this test case >> has been maintained. Members of the potential IETF activity may find >> that this is an irrelevant test case. > The reasons for having non-initial I-frames that I know of: > > - To allow seeking in stored media. To jump to a specific point, you > locate the last I-frame before the point you want to go to and run the > decoder from that point to the point where you really want to go. > > With DASH being a prevalent mode of delivering stored media, and each > DASH chunk starting with an I-frame, 5 seconds would be realistic for this. > > - To allow recovery from a known good state in streaming media. > For interactive two-way media, you can explicitly ask for an I-frame and > have the headend produce it; for other media, such as broadcast (the > original use case for half-second intervals, as mentioned above), more > complex schemes are needed (I've heard of technologies for storing a > reference frame in some other form and transferring it out of band so > that the decoder can initialize its state and start decoding without an > I-frame, but I don't know if these are used in practice.) > > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec From nobody Sun Mar 15 11:01:52 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B868F1A1B61 for ; Sun, 15 Mar 2015 11:01:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ICuw8s14yYJh for ; Sun, 15 Mar 2015 11:01:49 -0700 (PDT) Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0A20B1A1B5E for ; Sun, 15 Mar 2015 11:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3681; q=dns/txt; s=iport; t=1426442509; x=1427652109; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=J4rLaZcmzJ1WsVUvRPEFGsBgvk1/faHyhkaGuc229Kw=; b=MgzD4gxC7cBo6GSZ36vqAs5ghkzV/nmlzPAx0EXJ8hKbJUpbAtFNi4nw ou6bJhCbz2CuBn6HnkfJ6zjBPGNFcDOgKJgr4uXiG31NIAhz7VtPmEup7 fRbx3lRkxrBf1XdzR5wp28Ad/PdG424clKlLZlk+G33evjnMhYo6WRIxh 0=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BUBQCqaeJU/4ENJK1bgwZSWsI2CoVxAoEYQwEBAQEBAXyEDQEBBAEBATc0CxACAQgYHhAhBgslAgQKBAUbh34DEQ3IYw2FOAEBAQEBAQEBAQEBAQEBAQEBAQEBARMEiwyCQ4F3MweDFoEUBY84h3GBRo0JhgQigjKBPG8BgkIBAQE X-IronPort-AV: E=Sophos;i="5.11,405,1422921600"; d="scan'208";a="403582613" Received: from alln-core-9.cisco.com ([173.36.13.129]) by rcdn-iport-8.cisco.com with ESMTP; 15 Mar 2015 18:01:48 +0000 Received: from xhc-aln-x03.cisco.com (xhc-aln-x03.cisco.com [173.36.12.77]) by alln-core-9.cisco.com (8.14.5/8.14.5) with ESMTP id t2FI1mRo015388 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 15 Mar 2015 18:01:48 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-aln-x03.cisco.com ([173.36.12.77]) with mapi id 14.03.0195.001; Sun, 15 Mar 2015 13:01:47 -0500 From: "Mo Zanaty (mzanaty)" To: Sergio Garcia Murillo Thread-Topic: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) Thread-Index: AQHQXwvyydCo1wS3MEuqEifMwZXd/50diRIYgACc1QD//7A2Xg== Date: Sun, 15 Mar 2015 18:01:47 +0000 Message-ID: References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com>, <5505C5AA.1030703@gmail.com> In-Reply-To: <5505C5AA.1030703@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 18:01:50 -0000 Regarding indication of intra frames, draft-berger-avtext-framemarking prop= oses a header extension for intra and other critical info for codec agnosti= c switching. It will be discussed in avtext.=20 Mo On Mar 15, 2015, at 1:47 PM, Sergio Garcia Murillo wrote: Hi, Based on my hands-on-experience, RTP mixer should not depend on periodic In= tras for working, instead they should send an RTCP FIR/PLI request to the n= ew active speaker and wait until the I-Frame is received before switching t= he RTP stream. In fact, in my mcu I configure the encoders to not send peri= odic i frames at all and only send them by request (on received RTCP FIR/PL= I). I agree with Harald on the other use cases for non-initial I-frames, especi= ally the seeking on storing media one. But I think there could be other alt= ernatives that provide viable solutions while not having a big impact on qu= ality/bitrate as Iframes have. BTW, not sure if it is the right place/thread, but as an implementator, I f= ind it quite important to be able to tell if a frame is iframe/recovery/see= kable without to deep diving into the encoded frame bitstream. Best regards Sergio > On 15/03/2015 14:26, Mo Zanaty (mzanaty) wrote: > Another reason for testing with periodic intra is the recent rise in RTP = mixers which switch rather than transcode media. Setting the intra interval= to the average active speaker switching time gives a good estimate of real= -world codec efficiency in these topologies. >=20 > Mo >=20 >=20 >=20 > On Mar 15, 2015, at 6:36 AM, Harald Alvestrand wro= te: >=20 > Forking the subject.... >=20 > Den 03. mars 2015 10:10, skrev Mohammed Raad: >> At MPEG we use this to determine the codec performance for the "random >> access" case. I don't know why the 1 second interval was chosen and >> don't think that its relevant in many use cases, but MPEG codecs are >> aimed at satisfying a broad range of applications and so this test case >> has been maintained. Members of the potential IETF activity may find >> that this is an irrelevant test case. > The reasons for having non-initial I-frames that I know of: >=20 > - To allow seeking in stored media. To jump to a specific point, you > locate the last I-frame before the point you want to go to and run the > decoder from that point to the point where you really want to go. >=20 > With DASH being a prevalent mode of delivering stored media, and each > DASH chunk starting with an I-frame, 5 seconds would be realistic for thi= s. >=20 > - To allow recovery from a known good state in streaming media. > For interactive two-way media, you can explicitly ask for an I-frame and > have the headend produce it; for other media, such as broadcast (the > original use case for half-second intervals, as mentioned above), more > complex schemes are needed (I've heard of technologies for storing a > reference frame in some other form and transferring it out of band so > that the decoder can initialize its state and start decoding without an > I-frame, but I don't know if these are used in practice.) >=20 >=20 >=20 > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec >=20 > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec From nobody Sun Mar 15 11:14:23 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AFE441A1B62 for ; Sun, 15 Mar 2015 11:14:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.277 X-Spam-Level: X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2OxEBlNFSzm9 for ; Sun, 15 Mar 2015 11:14:21 -0700 (PDT) Received: from smtp.mozilla.org (mx1.corp.phx1.mozilla.com [63.245.216.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 679E41A1B5F for ; Sun, 15 Mar 2015 11:14:21 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 9E459F20F4; Sun, 15 Mar 2015 11:14:20 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx1.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GdP8zoNmHwh0; Sun, 15 Mar 2015 11:14:20 -0700 (PDT) Received: from [172.17.0.108] (50-78-100-113-static.hfc.comcastbusiness.net [50.78.100.113]) (Authenticated sender: tterriberry@mozilla.com) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 1A4FAF208F; Sun, 15 Mar 2015 11:14:14 -0700 (PDT) Message-ID: <5505CBF5.9040906@xiph.org> Date: Sun, 15 Mar 2015 11:14:13 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "Mo Zanaty (mzanaty)" , Sergio Garcia Murillo References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com>, <5505C5AA.1030703@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 18:14:22 -0000 Mo Zanaty (mzanaty) wrote: > Regarding indication of intra frames, draft-berger-avtext-framemarking proposes a header extension for intra and other critical info for codec agnostic switching. It will be discussed in avtext. Right, naturally most codecs include this up-front (in Daala it is the very first bit of the packet). But the other feedback I've received for MCUs is that it is often useful to know these things without decrypting the stream, which makes the header extension an attractive solution. Of course, if we know the header extension will be used, we can possibly avoid redundantly transmitting some of that data in the codec. From nobody Sun Mar 15 12:53:59 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1F8061A0302 for ; Sun, 15 Mar 2015 12:53:58 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s9Fp46skHN0P for ; Sun, 15 Mar 2015 12:53:54 -0700 (PDT) Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CAC9E1A026C for ; Sun, 15 Mar 2015 12:53:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1453; q=dns/txt; s=iport; t=1426449234; x=1427658834; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=kDtGzoRysMp+ynjJqzXhVKJnyhB/0Y6foYs0bCi/HSg=; b=Xl5ZbZBo0Dm88neYCwcuDzfU8XWGvKQsCZ5ODGfDTEkD1CKWsTfpc4gk GJLo+BdMo24+rNyoWYxlKHLfBdJCMy6Rxy49XOJq90P1QWmKOW/geukab G47SlUUFJA2LWmLMey9ix1h5UIni780owPT9KEQxCiQi3Zg1730a2ME0i U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0A4BQDN4gVV/4UNJK1bgwZSWsQQCoV0AoEWTAEBAQEBAX2EEAEBBAEBATc0CxACAQgYHhAnCyUCBA4FG4gUDcYIAQEBAQEBAQEBAQEBAQEBAQEBAQEBEwSLF4Q+MweDF4EWBZA6iWaUFCOCMoE8b4JDAQEB X-IronPort-AV: E=Sophos;i="5.11,405,1422921600"; d="scan'208";a="132164249" Received: from alln-core-11.cisco.com ([173.36.13.133]) by alln-iport-1.cisco.com with ESMTP; 15 Mar 2015 19:53:54 +0000 Received: from xhc-aln-x01.cisco.com (xhc-aln-x01.cisco.com [173.36.12.75]) by alln-core-11.cisco.com (8.14.5/8.14.5) with ESMTP id t2FJrr7x003361 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 15 Mar 2015 19:53:53 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-aln-x01.cisco.com ([173.36.12.75]) with mapi id 14.03.0195.001; Sun, 15 Mar 2015 14:53:52 -0500 From: "Mo Zanaty (mzanaty)" To: "Timothy B. Terriberry" Thread-Topic: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) Thread-Index: AQHQXwvyydCo1wS3MEuqEifMwZXd/50diRIYgACc1QD//7A2XoAAV0uA///IBi8= Date: Sun, 15 Mar 2015 19:53:52 +0000 Message-ID: References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com>, <5505C5AA.1030703@gmail.com> ,<5505CBF5.9040906@xiph.org> In-Reply-To: <5505CBF5.9040906@xiph.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Cc: "video-codec@ietf.org" , Sergio Garcia Murillo Subject: Re: [video-codec] I-frame spacing (or not) (Re: Test sequences, automated and otherwise) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 19:53:58 -0000 Encrypted streams were the primary driver, especially new proposed architec= tures where media is encrypted end-to-end with no decryption keys at switch= ing mixers.=20 The codec must still have its own intra indication for internal use, since = it can't rely on any system layer info like RTP headers. But it doesn't nec= essarily have to make this easy to parse for non-decoders if there are more= efficient ways to signal this internally, while the system layer handles t= he job of providing easily parsed critical info.=20 Mo On Mar 15, 2015, at 2:14 PM, Timothy B. Terriberry wrot= e: Mo Zanaty (mzanaty) wrote: > Regarding indication of intra frames, draft-berger-avtext-framemarking pr= oposes a header extension for intra and other critical info for codec agnos= tic switching. It will be discussed in avtext. Right, naturally most codecs include this up-front (in Daala it is the very= first bit of the packet). But the other feedback I've received for MCUs is= that it is often useful to know these things without decrypting the stream= , which makes the header extension an attractive solution. Of course, if we= know the header extension will be used, we can possibly avoid redundantly = transmitting some of that data in the codec. _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec From nobody Sun Mar 15 14:06:58 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F04161A1B8D for ; Sun, 15 Mar 2015 14:06:56 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pi22TFfI2BqK for ; Sun, 15 Mar 2015 14:06:52 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [IPv6:2001:700:1:2::117]) by ietfa.amsl.com (Postfix) with ESMTP id 11D8D1A1B8C for ; Sun, 15 Mar 2015 14:06:52 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id AACF67C3A6A for ; Sun, 15 Mar 2015 22:06:50 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KxYCTLQfuj1o for ; Sun, 15 Mar 2015 22:06:46 +0100 (CET) Received: from [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f] (unknown [IPv6:2001:470:de0a:27:60d1:75ab:ab20:815f]) by mork.alvestrand.no (Postfix) with ESMTPSA id 5C4E47C3A21 for ; Sun, 15 Mar 2015 22:06:46 +0100 (CET) Message-ID: <5505F464.8080107@alvestrand.no> Date: Sun, 15 Mar 2015 22:06:44 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> , <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com>, <550591F2.7020809@xiph.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise)) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2015 21:06:57 -0000 (side note: I *hate* the current setting of Thunderbird where it doesn't wrap lines before replying - it interferes destructively with Mo's Outlook settings that send plain text with extremely long lines... apologies for the broken result.) Den 15. mars 2015 17:54, skrev Mo Zanaty (mzanaty): > I don't have good stats, but I'm sure it's much longer than all current test sequences, which tend to be only a few seconds due to uncompressed video size constraints. > > It is important to understand intra efficiency during a stream switch, recovery, join, random access, etc. Perhaps we can just test All-Intra configs as a separate data point for this rather than mixing periodic intra. > By "All-Intra", do you mean streams with only I-frames? I think they are probably too unrealistic to tell us much except the still image performance. Even adding 3 p-frames after the i-frame will *significantly* increase compression of the video stream. But it would indeed be nice to have specific data on how the encoding efficiency varies with recovery point interval. There should be some point where the average efficiency is not significantly affected by regular I-frames, because the I-frame size is small compared to the accumulated p-frames (but the bumpy bitrate caused by large I-frames could still cause interesting effects in bandwidth-constrained situations.) > Mo > > > > On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry wrote: > > Mo Zanaty (mzanaty) wrote: >> Another reason for testing with periodic intra is the recent rise in RTP mixers which switch rather than transcode media. Setting the intra interval to the average active speaker switching time gives a good estimate of real-world codec efficiency in these topologies. > > Do either you or Harald have some statistics with which to estimate that time? > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > From nobody Mon Mar 16 00:14:54 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EECFE1A1EB7 for ; Mon, 16 Mar 2015 00:14:52 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.977 X-Spam-Level: X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gvPISESbtnvH for ; Mon, 16 Mar 2015 00:14:48 -0700 (PDT) Received: from mail-we0-f169.google.com (mail-we0-f169.google.com [74.125.82.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 586EF1A1C00 for ; Mon, 16 Mar 2015 00:14:48 -0700 (PDT) Received: by weop45 with SMTP id p45so6002796weo.0 for ; Mon, 16 Mar 2015 00:14:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Z94tiHXofug7jyA2HH98ZgojFS1zc/cZ1DqV0e12PKY=; b=lkWMOiUzFouu3VvHlxqWhk4vWwmMLk51FJVDRO07rQzs3ZSEnmWnBYKfpijDuOh8ro NHqdAsRHVZVrNrCe4fY6HCLs1NK3QizeGZhhDBl5mzt55woeih0eReuxVMAgs6quheI0 9011ql3OVs2L+WtBp/XJkURa034vzQoIDxxf5ttZtum7rmYjsRyBr9oVc7XK4d7Nwx5/ iEp4+AGYQsspqvnJ3HBtqP2M5puLbDA7RV6122IIaLKazoC0HK2PHO6AXBtK1ImeUfCK bp8OSmUitqsBc/5Ctg9F4IqFtU4GM+rKmE3D07UbUPt6/xTffw9Jb/nxf2PS5QfUI5qg 9oBQ== X-Gm-Message-State: ALoCoQmeAaURS+IH2K/tlrK6e4nvwoxXxP1DvCaXvQHJOfBmaWm9HbLGe9KgQOveOOyVruSxnFZz MIME-Version: 1.0 X-Received: by 10.180.77.166 with SMTP id t6mr58133550wiw.52.1426490086991; Mon, 16 Mar 2015 00:14:46 -0700 (PDT) Received: by 10.194.14.129 with HTTP; Mon, 16 Mar 2015 00:14:46 -0700 (PDT) In-Reply-To: <5505F464.8080107@alvestrand.no> References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> <550591F2.7020809@xiph.org> <5505F464.8080107@alvestrand.no> Date: Mon, 16 Mar 2015 09:14:46 +0200 Message-ID: From: Mohammed Raad To: Harald Alvestrand Content-Type: multipart/alternative; boundary=f46d043d645fe93d9a0511629a2e Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise)) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2015 07:14:53 -0000 --f46d043d645fe93d9a0511629a2e Content-Type: text/plain; charset=UTF-8 Are there any use-cases besides web streaming and communication that is being considered for this effort? If people are thinking of using a new codec for something like digital cinema then an all intra case makes sense. Mohammed On Sun, Mar 15, 2015 at 11:06 PM, Harald Alvestrand wrote: > (side note: I *hate* the current setting of Thunderbird where it doesn't > wrap lines before replying - it interferes destructively with Mo's > Outlook settings that send plain text with extremely long lines... > apologies for the broken result.) > > > Den 15. mars 2015 17:54, skrev Mo Zanaty (mzanaty): > > I don't have good stats, but I'm sure it's much longer than all current > test sequences, which tend to be only a few seconds due to uncompressed > video size constraints. > > > > It is important to understand intra efficiency during a stream switch, > recovery, join, random access, etc. Perhaps we can just test All-Intra > configs as a separate data point for this rather than mixing periodic intra. > > > > By "All-Intra", do you mean streams with only I-frames? I think they are > probably too unrealistic to tell us much except the still image > performance. Even adding 3 p-frames after the i-frame will > *significantly* increase compression of the video stream. > > But it would indeed be nice to have specific data on how the encoding > efficiency varies with recovery point interval. There should be some > point where the average efficiency is not significantly affected by > regular I-frames, because the I-frame size is small compared to the > accumulated p-frames (but the bumpy bitrate caused by large I-frames > could still cause interesting effects in bandwidth-constrained situations.) > > > Mo > > > > > > > > On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry > wrote: > > > > Mo Zanaty (mzanaty) wrote: > >> Another reason for testing with periodic intra is the recent rise in > RTP mixers which switch rather than transcode media. Setting the intra > interval to the average active speaker switching time gives a good estimate > of real-world codec efficiency in these topologies. > > > > Do either you or Harald have some statistics with which to estimate that > time? > > > > _______________________________________________ > > video-codec mailing list > > video-codec@ietf.org > > https://www.ietf.org/mailman/listinfo/video-codec > > > > _______________________________________________ > > video-codec mailing list > > video-codec@ietf.org > > https://www.ietf.org/mailman/listinfo/video-codec > > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --f46d043d645fe93d9a0511629a2e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Are there any use-cases besides web streaming and communic= ation that is being considered for this effort? If people are thinking of u= sing a new codec for something like digital cinema then an all intra case m= akes sense.

Mohammed

On Sun, Mar 15, 2015 at 11:06 PM, Harald = Alvestrand <harald@alvestrand.no> wrote:
(side note: I *hate* the current setting of Thunderbird w= here it doesn't
wrap lines before replying - it interferes destructively with Mo's
Outlook settings that send plain text with extremely long lines...
apologies for the broken result.)


Den 15. mars 2015 17:54, skrev Mo Zanaty (mzanaty):
> I don't have good stats, but I'm sure it's much longer tha= n all current test sequences, which tend to be only a few seconds due to un= compressed video size constraints.
>
> It is important to understand intra efficiency during a stream switch,= recovery, join, random access, etc. Perhaps we can just test All-Intra con= figs as a separate data point for this rather than mixing periodic intra. >

By "All-Intra", do you mean streams with only I-frames? I think t= hey are
probably too unrealistic to tell us much except the still image
performance. Even adding 3 p-frames after the i-frame will
*significantly* increase compression of the video stream.

But it would indeed be nice to have specific data on how the encoding
efficiency varies with recovery point interval. There should be some
point where the average efficiency is not significantly affected by
regular I-frames, because the I-frame size is small compared to the
accumulated p-frames (but the bumpy bitrate caused by large I-frames
could still cause interesting effects in bandwidth-constrained situations.)=

> Mo
>
>
>
> On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry <tterribe@xiph.org> wrote:
>
> Mo Zanaty (mzanaty) wrote:
>> Another reason for testing with periodic intra is the recent rise = in RTP mixers which switch rather than transcode media. Setting the intra i= nterval to the average active speaker switching time gives a good estimate = of real-world codec efficiency in these topologies.
>
> Do either you or Harald have some statistics with which to estimate th= at time?
>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
Mohammed Raad, PhD.
Partner
RAADTECH CONSULTING<= br>P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 414451478<= br>Email: m= ohammedsraad@raadtech.com
--f46d043d645fe93d9a0511629a2e-- From nobody Mon Mar 16 05:46:24 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECBAE1A871F for ; Mon, 16 Mar 2015 05:46:19 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.51 X-Spam-Level: X-Spam-Status: No, score=-14.51 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0Aqk7YIpFrn7 for ; Mon, 16 Mar 2015 05:46:13 -0700 (PDT) Received: from rcdn-iport-5.cisco.com (rcdn-iport-5.cisco.com [173.37.86.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 710CF1A8718 for ; Mon, 16 Mar 2015 05:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=9547; q=dns/txt; s=iport; t=1426509973; x=1427719573; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=LjInjBZyFeBmJZi4h/zbLEP7gJMjY4s/vk6c1TM2TMI=; b=hxrqz3pK+YkIU37bd+cL2ieXErkJCL40kOEkh6DhVU2tRaTDwWSKATOS F8PqlS72R3Hi0CKhNjdcyhDIIcwwjALiJuX/NiWhrqBXBayTDK0aKgGDt Ccs8C7s0ZKeMp6VDE8KiDqz+4tV1kkQ/w61VWn2tQasASU25C+KCR8+Y4 Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0AkBQCtzwZV/4wNJK1bgwZSWsRXAQmFdAKBLUwBAQEBAQF9hA8BAQEDAQEBAWsDCAUJAgIBCBgjBAcbDAsUEQIEDgWIJwgNxHsBAQEBAQEBAQEBAQEBAQEBAQEBAQETBASLE4QPEQEkKAQHgxeBFgWQOoVPhBeUFCOCAhwUgTxvgQuBOAEBAQ X-IronPort-AV: E=Sophos;i="5.11,409,1422921600"; d="scan'208,217";a="403969797" Received: from alln-core-7.cisco.com ([173.36.13.140]) by rcdn-iport-5.cisco.com with ESMTP; 16 Mar 2015 12:46:12 +0000 Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by alln-core-7.cisco.com (8.14.5/8.14.5) with ESMTP id t2GCkCBB027237 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 16 Mar 2015 12:46:12 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.03.0195.001; Mon, 16 Mar 2015 07:46:12 -0500 From: "Mo Zanaty (mzanaty)" To: Mohammed Raad Thread-Topic: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise)) Thread-Index: AQHQX2P8nwrcwXoIjEKq/apvp7QTDJ0fBs0AgAAIyFY= Date: Mon, 16 Mar 2015 12:46:11 +0000 Message-ID: <73E73F17-6194-426C-B1CE-AE3094EDDF2A@cisco.com> References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> <550591F2.7020809@xiph.org> <5505F464.8080107@alvestrand.no>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_73E73F176194426CB1CEAE3094EDDF2Aciscocom_" MIME-Version: 1.0 Archived-At: Cc: Harald Alvestrand , "video-codec@ietf.org" Subject: Re: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise)) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2015 12:46:20 -0000 --_000_73E73F176194426CB1CEAE3094EDDF2Aciscocom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I don't think we should restrict any use case, but the focus should be to o= ptimize performance in common cases. To clarify my all-intra comment, I did indeed mean a test config of all int= ra frames (like jct-vc), but the primary purpose of those results would be = a separate data point for general intra efficiency in all use cases, not sp= ecifically for an all-intra use case, which I agree is rather rare outside = some professional applications. Mo On Mar 16, 2015, at 3:14 AM, Mohammed Raad > wrote: Are there any use-cases besides web streaming and communication that is bei= ng considered for this effort? If people are thinking of using a new codec = for something like digital cinema then an all intra case makes sense. Mohammed On Sun, Mar 15, 2015 at 11:06 PM, Harald Alvestrand > wrote: (side note: I *hate* the current setting of Thunderbird where it doesn't wrap lines before replying - it interferes destructively with Mo's Outlook settings that send plain text with extremely long lines... apologies for the broken result.) Den 15. mars 2015 17:54, skrev Mo Zanaty (mzanaty): > I don't have good stats, but I'm sure it's much longer than all current t= est sequences, which tend to be only a few seconds due to uncompressed vide= o size constraints. > > It is important to understand intra efficiency during a stream switch, re= covery, join, random access, etc. Perhaps we can just test All-Intra config= s as a separate data point for this rather than mixing periodic intra. > By "All-Intra", do you mean streams with only I-frames? I think they are probably too unrealistic to tell us much except the still image performance. Even adding 3 p-frames after the i-frame will *significantly* increase compression of the video stream. But it would indeed be nice to have specific data on how the encoding efficiency varies with recovery point interval. There should be some point where the average efficiency is not significantly affected by regular I-frames, because the I-frame size is small compared to the accumulated p-frames (but the bumpy bitrate caused by large I-frames could still cause interesting effects in bandwidth-constrained situations.) > Mo > > > > On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry > wrote: > > Mo Zanaty (mzanaty) wrote: >> Another reason for testing with periodic intra is the recent rise in RTP= mixers which switch rather than transcode media. Setting the intra interva= l to the average active speaker switching time gives a good estimate of rea= l-world codec efficiency in these topologies. > > Do either you or Harald have some statistics with which to estimate that = time? > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com _______________________________________________ video-codec mailing list video-codec@ietf.org https://www.ietf.org/mailman/listinfo/video-codec --_000_73E73F176194426CB1CEAE3094EDDF2Aciscocom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
I don't think we should restrict any use case, but the focus should be= to optimize performance in common cases. 

To clarify my all-intra comment, I did indeed mean a test config of al= l intra frames (like jct-vc), but the primary purpose of those results woul= d be a separate data point for general intra efficiency in all use cases, n= ot specifically for an all-intra use case, which I agree is rather rare outside some professional applicati= ons. 

Mo



On Mar 16, 2015, at 3:14 AM, Mohammed Raad <mohammedsraad@raadtech.com> wrote:

Are there any use-cases besides web streaming and communic= ation that is being considered for this effort? If people are thinking of u= sing a new codec for something like digital cinema then an all intra case m= akes sense.

Mohammed

On Sun, Mar 15, 2015 at 11:06 PM, Harald Alvestr= and <harald@alvest= rand.no> wrote:
(side note: I *hate* the current setting of Thunderbird where it doesn't wrap lines before replying - it interferes destructively with Mo's
Outlook settings that send plain text with extremely long lines...
apologies for the broken result.)


Den 15. mars 2015 17:54, skrev Mo Zanaty (mzanaty):
> I don't have good stats, but I'm sure it's much longer than all curren= t test sequences, which tend to be only a few seconds due to uncompressed v= ideo size constraints.
>
> It is important to understand intra efficiency during a stream switch,= recovery, join, random access, etc. Perhaps we can just test All-Intra con= figs as a separate data point for this rather than mixing periodic intra. >

By "All-Intra", do you mean streams with only I-frames? I think t= hey are
probably too unrealistic to tell us much except the still image
performance. Even adding 3 p-frames after the i-frame will
*significantly* increase compression of the video stream.

But it would indeed be nice to have specific data on how the encoding
efficiency varies with recovery point interval. There should be some
point where the average efficiency is not significantly affected by
regular I-frames, because the I-frame size is small compared to the
accumulated p-frames (but the bumpy bitrate caused by large I-frames
could still cause interesting effects in bandwidth-constrained situations.)=

> Mo
>
>
>
> On Mar 15, 2015, at 10:07 AM, Timothy B. Terriberry <tterribe@xiph.org> wrote:
>
> Mo Zanaty (mzanaty) wrote:
>> Another reason for testing with periodic intra is the recent rise = in RTP mixers which switch rather than transcode media. Setting the intra i= nterval to the average active speaker switching time gives a good estimate = of real-world codec efficiency in these topologies.
>
> Do either you or Harald have some statistics with which to estimate th= at time?
>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
Mohammed Raad, PhD.
Partner
RAADTECH CONSULTING
P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 414451478
Email: moha= mmedsraad@raadtech.com
_______________________________________________
video-codec mailing list
video-codec@ietf.org
https:/= /www.ietf.org/mailman/listinfo/video-codec
--_000_73E73F176194426CB1CEAE3094EDDF2Aciscocom_-- From nobody Mon Mar 16 11:38:51 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D98311A8A72 for ; Mon, 16 Mar 2015 11:38:49 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.277 X-Spam-Level: X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qt6xe3uiN5dX for ; Mon, 16 Mar 2015 11:38:47 -0700 (PDT) Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BA4531A8A6B for ; Mon, 16 Mar 2015 11:38:45 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 10BDCF217F for ; Mon, 16 Mar 2015 11:38:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RBdf1qEAxE2u for ; Mon, 16 Mar 2015 11:38:44 -0700 (PDT) Received: from [10.252.25.12] (corp.mtv2.mozilla.com [63.245.221.32]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id E6211F214E for ; Mon, 16 Mar 2015 11:38:44 -0700 (PDT) Message-ID: <55072334.1060308@xiph.org> Date: Mon, 16 Mar 2015 11:38:44 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "video-codec@ietf.org" References: <54EE0A07.3080100@mozilla.com> <54F440FC.3050508@cisco.com> <54F4E204.5@mozilla.com> <9C2FAEDF6B678042ADE3B6686D7C6E151E48D573@xmb-rcd-x07.cisco.com> <550560B7.6090501@alvestrand.no> <6AAC496C-F6F4-4AC7-86AB-FF6A14DB50DE@cisco.com> <550591F2.7020809@xiph.org> <5505F464.8080107@alvestrand.no>, <73E73F17-6194-426C-B1CE-AE3094EDDF2A@cisco.com> In-Reply-To: <73E73F17-6194-426C-B1CE-AE3094EDDF2A@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] Test cases with i-frames (Re: I-frame spacing (or not) (Re: Test sequences, automated and otherwise)) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2015 18:38:50 -0000 Mo Zanaty (mzanaty) wrote: > To clarify my all-intra comment, I did indeed mean a test config of all > intra frames (like jct-vc), but the primary purpose of those results > would be a separate data point for general intra efficiency in all use I agree, particularly because there are a number of coding tools which one would expect to help primarily on intra frames, and it makes it much easier to measure their performance when not cluttered by the noise of two orders of magnitude more inter frames (though it's still important to test with inter frames to make sure they don't have some deleterious effect on prediction). Personally, I think it's better to have a wide variety of still images on which to measure intra performance, rather than coding an existing video test sequence with all-intra. The main reason being that individual frames in a video clip are highly correlated, but the space of images is very large. It's a better use of testing resources to use uncorrelated images to explore that space as widely as possible. That's what informed the construction of the four still-image subsets in draft-daede-netvc-testing Section 5.2. From nobody Thu Mar 19 08:33:44 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1CC21ACDA2 for ; Thu, 19 Mar 2015 08:33:42 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=5 tests=[BAYES_50=0.8, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A6gw--L91lgU for ; Thu, 19 Mar 2015 08:33:41 -0700 (PDT) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 674961A19E5 for ; Thu, 19 Mar 2015 08:33:38 -0700 (PDT) Received: by lbcgn8 with SMTP id gn8so55490795lbc.2 for ; Thu, 19 Mar 2015 08:33:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=OtHUG40oBTgvFgjVdBNy7K1MZOxtoRErOUAAaPvD9Rk=; b=xWW/JliSAmM8uta8AmCHLgweK17hhuEcPgPZjGo/CPhQaRvmXIacQqS+1CtUn2amDJ ChAt2Vn4HREjX3ybgCKH8q3afS3ghq5SVrEPAnNxvGG+m+59kD+4di6xJayBgt96V8Fe BYrVVj1befo7JoApS/JvWfUc/cA00pXWWvHyOluZT4TN8m0o25wrWAZxZcSP2IWaa3rX CIm9mTCGsKJ3PNR0dD1kna7327Wb44juPa2xr8RULLBhWDI+g9xDjaGR2ciZPV+Mz/Z0 9ZPz9uQDSlcKTYgZIKYqoSnwwzZQJGjyu1EgUAAZkLsJGImauXBDo5QJBqPt2GZzsHB5 y0Ug== MIME-Version: 1.0 X-Received: by 10.112.29.231 with SMTP id n7mr66462803lbh.99.1426779216860; Thu, 19 Mar 2015 08:33:36 -0700 (PDT) Received: by 10.112.188.134 with HTTP; Thu, 19 Mar 2015 08:33:36 -0700 (PDT) Date: Thu, 19 Mar 2015 11:33:36 -0400 Message-ID: From: Monty Montgomery To: "video-codec@ietf.org" Content-Type: text/plain; charset=UTF-8 Archived-At: Subject: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2015 15:33:42 -0000 Hello to everyone interested in Daala, NetVC, and the run up to the netvc BoF on Tuesday at IETF92 in Dallas! While I'm mentioning the BoF I'd also like to remind folks that the Hackathon on Saturday and Sunday includes NetVC and a decent portion of our own Daala team will be there. Physical attendance at the hackathon required registration that's already closed due to the limits of the room, but the Daala team will be in the #daala IRC channel on Freenode the whole time, pretty much like we always are... online participation is always welcome and encouraged! And now to the pep talk... In 2013, Cisco announced the OpenH264 licensing hack as a step toward ending a long-standing and frustratingly non-technical roadblock to WebRTC: h.264, though dominant, is unlicenseable by Free and Open Source entities. I wrote at the time that although I supported the practical aspects of the OpenH264 deal, "Licensing caused this problem, and more licensing is not a solution. [...] We've merely kicked the can down the road and set a dangerous precedent for next time around."* We're now standing where the can takes its first bounce. NetVC is our collective opportunity to reverse that precedent and to give the Internet and Open Web the only fundamental Free technology it lacks: a fully modern video codec with no 'permission required' strings attached. The elevator pitch here is very simple: We must make for video what Opus is for audio. Video will never be a first-class citizen on the Net so long as it is only available to the 'haves' who are able to license the required commercial technology. Judging by the relative calm of the netvc mailing list, I don't think we (the proponets of netvc) face the same kind of controversy this time around as we did at the time of 'codec' BoFs. That's a bit ironic, actually, as a video codec is a substantially more challenging technical undertaking. I do think there likely is some quiet skepticism-- there would have to be-- and so I'd like to invite the skeptics and Devil's Advocates to speak up here in the days before the BoF. I doubt it will shorten the lines at the microphones, but perhaps the arguments will be more focused and better honed at that time. I'm not saying 'speak now or forever hold your peace'. I'm just trying to shift the discussions that would normally start in a cramped uncomfortable room to the mailing list a little early. Cheers, Monty of Xiph Moz Daala * http://xiphmont.livejournal.com/61927.html From nobody Thu Mar 19 17:17:13 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5DBB31A0194 for ; Thu, 19 Mar 2015 17:17:11 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.277 X-Spam-Level: X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vdmpt4_sVb7a for ; Thu, 19 Mar 2015 17:17:09 -0700 (PDT) Received: from mail-yh0-x22d.google.com (mail-yh0-x22d.google.com [IPv6:2607:f8b0:4002:c01::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C7BDA1A01A8 for ; Thu, 19 Mar 2015 17:17:08 -0700 (PDT) Received: by yhpt93 with SMTP id t93so32864577yhp.0 for ; Thu, 19 Mar 2015 17:17:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=OSv1I6PDoAQH3qSMS5l3K0677E2gn6xgQOPtmF9kiYI=; b=Nl067K+9LGKRUyeuleVEzIr7LHrlDhnt7rklPQZHHJjd7/qvaVL42Ad80NxptPwknn rPsqfl3JmAg0/XxEJN5waBUuHteBNF4EOZy0LYJni/Lrage3P1Urvtc20X6ijbw/4B3W 9JNRrAEUPBrTdHk1V/t4D/u9wFC17fEgu17AjwIu9BlA+OcG8dvR+4l60tH5x9auUtLG TE402Vi+0g68/qbaXcFMDxjigPY01gMshdG/b0y9ivj7sVANbZ1ChxQTaZVaT7V9E2xk 33Nl+kp9uMSdyB2LD0B19A4qWGTmEegjJATxlO9OIVCSJEEQCzNanCtRe92FJvvN/33E M9Tg== MIME-Version: 1.0 X-Received: by 10.236.20.138 with SMTP id p10mr80174857yhp.161.1426810628165; Thu, 19 Mar 2015 17:17:08 -0700 (PDT) Sender: winstein@gmail.com Received: by 10.170.96.195 with HTTP; Thu, 19 Mar 2015 17:17:08 -0700 (PDT) Received: by 10.170.96.195 with HTTP; Thu, 19 Mar 2015 17:17:08 -0700 (PDT) In-Reply-To: References: Date: Thu, 19 Mar 2015 17:17:08 -0700 X-Google-Sender-Auth: a3sLZBI78ozT6Ro42MIAqUFZtIc Message-ID: From: Keith Winstein To: Monty Montgomery Content-Type: multipart/alternative; boundary=089e015385f4a773520511ad3c63 Archived-At: Cc: video-codec@ietf.org Subject: Re: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: keithw@mit.edu List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2015 00:17:11 -0000 --089e015385f4a773520511ad3c63 Content-Type: text/plain; charset=UTF-8 Trying to reply in the spirit intended, here's what I would say with a devil's advocate hat on just for fun: Daala seems like an extremely important research project to develop a next-generation coded video representation that will also be free of practical licensing obstacles. But what are the merits of an IETF working group performing this kind of high-risk, high-reward research, versus doing something much more boring like "writing a specification for VP9 to enable interoperable implementations, and then iterating on that technology"? Google said in May 2013 that "a draft bitstream specification is well underway." For whatever reason, they still haven't published it yet. (There is also no independent implementation of a decoder written by a non-Google-employee, afaik, much less of an encoder.) But the VP9 format is certainly specifiable given effort, and the licensing situation is probably about as well-understood as Daala's will ever be, given the MPEG LA license. So the devil's advocate question would be, if the problem is that "video will never be a first-class citizen on the Net so long as it is only available to the 'haves' who are able to license the required commercial technology," is rallying around VP8/VP9 (perhaps with an actual high-quality spec) perhaps a better/safer approach? -Keith On Mar 19, 2015 8:33 AM, "Monty Montgomery" wrote: > Hello to everyone interested in Daala, NetVC, and the run up to > the netvc BoF on Tuesday at IETF92 in Dallas! > > While I'm mentioning the BoF I'd also like to remind folks that > the Hackathon on Saturday and Sunday includes NetVC and a decent > portion of our own Daala team will be there. Physical attendance > at the hackathon required registration that's already closed due > to the limits of the room, but the Daala team will be in the > #daala IRC channel on Freenode the whole time, pretty much like > we always are... online participation is always welcome and > encouraged! > > And now to the pep talk... > > In 2013, Cisco announced the OpenH264 licensing hack as a step > toward ending a long-standing and frustratingly non-technical > roadblock to WebRTC: h.264, though dominant, is unlicenseable > by Free and Open Source entities. I wrote at the time that although I > supported the practical aspects of the OpenH264 deal, "Licensing > caused this problem, and more licensing is not a solution. [...] > We've merely kicked the can down the road and set a dangerous > precedent for next time around."* > > We're now standing where the can takes its first bounce. > > NetVC is our collective opportunity to reverse that precedent and > to give the Internet and Open Web the only fundamental Free > technology it lacks: a fully modern video codec with no > 'permission required' strings attached. The elevator pitch here > is very simple: We must make for video what Opus is for audio. > Video will never be a first-class citizen on the Net so long as > it is only available to the 'haves' who are able to license the > required commercial technology. > > Judging by the relative calm of the netvc mailing list, I don't > think we (the proponets of netvc) face the same kind of > controversy this time around as we did at the time of 'codec' > BoFs. That's a bit ironic, actually, as a video codec is a > substantially more challenging technical undertaking. > > I do think there likely is some quiet skepticism-- there would > have to be-- and so I'd like to invite the skeptics and Devil's > Advocates to speak up here in the days before the BoF. I doubt > it will shorten the lines at the microphones, but perhaps the > arguments will be more focused and better honed at that time. > > I'm not saying 'speak now or forever hold your peace'. I'm just > trying to shift the discussions that would normally start in > a cramped uncomfortable room to the mailing list a little early. > > Cheers, > Monty > of Xiph Moz Daala > > * http://xiphmont.livejournal.com/61927.html > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > --089e015385f4a773520511ad3c63 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Trying to reply in the spirit intended, here's what I wo= uld say with a devil's advocate hat on just for fun:

Daala seems like an extremely important research project to = develop a next-generation coded video representation that will also be free= of practical licensing obstacles.

But what are the merits of an IETF working group performing = this kind of high-risk, high-reward research, versus doing something much m= ore boring like "writing a specification for VP9 to enable interoperab= le implementations, and then iterating on that technology"?

Google said in May 2013 that "a draft bitstream specifi= cation is well underway." For whatever reason, they still haven't = published it yet. (There is also no independent implementation of a decoder= written by a non-Google-employee, afaik, much less of an encoder.)

But the VP9 format is certainly specifiable given effort, an= d the licensing situation is probably about as well-understood as Daala'= ;s will ever be, given the MPEG LA license.

So the devil's advocate question would be, if the proble= m is that "video will never be a first-class citizen on the Net so lon= g as=C2=A0it is only available to the 'haves' who are able to licen= se the=C2=A0required commercial technology," is rallying around VP8/VP= 9 (perhaps with an actual high-quality spec) perhaps a better/safer approac= h?

-Keith

On Mar 19, 2015 8:33 AM, "Monty Montgomery&= quot; <xiphmont@gmail.com> = wrote:
Hello to ever= yone interested in Daala, NetVC, and the run up to
the netvc BoF on Tuesday at IETF92 in Dallas!

While I'm mentioning the BoF I'd also like to remind folks that
the Hackathon on Saturday and Sunday includes NetVC and a decent
portion of our own Daala team will be there.=C2=A0 Physical attendance
at the hackathon required registration that's already closed due
to the limits of the room, but the Daala team will be in the
#daala IRC channel on Freenode the whole time, pretty much like
we always are... online participation is always welcome and
encouraged!

And now to the pep talk...

In 2013, Cisco announced the OpenH264 licensing hack as a step
toward ending a long-standing and frustratingly non-technical
roadblock to WebRTC: h.264, though dominant, is unlicenseable
by Free and Open Source entities.=C2=A0 =C2=A0I wrote at the time that alth= ough I
supported the practical aspects of the OpenH264 deal, "Licensing
caused this problem, and more licensing is not a solution. [...]
We've merely kicked the can down the road and set a dangerous
precedent for next time around."*

We're now standing where the can takes its first bounce.

NetVC is our collective opportunity to reverse that precedent and
to give the Internet and Open Web the only fundamental Free
technology it lacks: a fully modern video codec with no
'permission required' strings attached.=C2=A0 The elevator pitch he= re
is very simple: We must make for video what Opus is for audio.
Video will never be a first-class citizen on the Net so long as
it is only available to the 'haves' who are able to license the
required commercial technology.

Judging by the relative calm of the netvc mailing list, I don't
think we (the proponets of netvc) face the same kind of
controversy this time around as we did at the time of 'codec'
BoFs.=C2=A0 That's a bit ironic, actually, as a video codec is a
substantially more challenging technical undertaking.

I do think there likely is some quiet skepticism-- there would
have to be-- and so I'd like to invite the skeptics and Devil's
Advocates to speak up here in the days before the BoF.=C2=A0 I doubt
it will shorten the lines at the microphones, but perhaps the
arguments will be more focused and better honed at that time.

I'm not saying 'speak now or forever hold your peace'.=C2=A0 I&= #39;m just
trying to shift the discussions that would normally start in
a cramped uncomfortable room to the mailing list a little early.

Cheers,
Monty
of Xiph Moz Daala

* = http://xiphmont.livejournal.com/61927.html

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec
--089e015385f4a773520511ad3c63-- From nobody Thu Mar 19 17:41:56 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A18781A1F20 for ; Thu, 19 Mar 2015 17:41:55 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cH1pdJ1m37Ut for ; Thu, 19 Mar 2015 17:41:52 -0700 (PDT) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 120051A1F16 for ; Thu, 19 Mar 2015 17:41:50 -0700 (PDT) Received: by pacwe9 with SMTP id we9so91296538pac.1 for ; Thu, 19 Mar 2015 17:41:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=9VP5WMxVvi0w3CnKBD9dCoiPcHSgIG5+u/S5UF1639E=; b=FQri1QFAx0DyGKuOCyAP4/AcD/iLsDMBJDcKb1hTbuM/94KWWbuH9Z1RV3bwpiFuC6 XSXlqJMYLz27ZIGTMbX1Yn13g0dwmxyYSqlZXdiyyur1o4iqr25dX6boWvKBc8NrLJ47 ctw5XjaFrm36sxe+nPWKRnY3/LL+t25+RT8MTcj2QJlEmwAqVNN4s2wNJrCxZo9ekpDD FgZ+Z2esAPXWkoB6rYMzek+Pc3XNf/ZHQECpJmkgfIivg1Og+xpglkVIYQCCclh0ZGM+ HwcMdmb5hteeb3NHlpZVui/bzriLNvpL0hlHBrpiS92w6Q8pH953Dh+YUSVEPjB4ctKo 2bag== X-Gm-Message-State: ALoCoQm6VSw0Z58oDsUNTX53DGKFTnb3NS5ifFAn6Ftqwpa1GamhulOvi1fCuJWQ6CTRIW24E3f9 X-Received: by 10.66.55.74 with SMTP id q10mr181291501pap.94.1426812110518; Thu, 19 Mar 2015 17:41:50 -0700 (PDT) Received: from ?IPv6:2620:101:80fc:224:7e7a:91ff:fe9e:8126? ([2620:101:80fc:224:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id tr8sm5015928pab.4.2015.03.19.17.41.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Mar 2015 17:41:49 -0700 (PDT) Message-ID: <550B6CCC.70706@mozilla.com> Date: Thu, 19 Mar 2015 17:41:48 -0700 From: Thomas Daede User-Agent: http://a.pomf.se/cszdno.opus MIME-Version: 1.0 To: video-codec@ietf.org References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2015 00:41:55 -0000 On 03/19/2015 05:17 PM, Keith Winstein wrote: > But what are the merits of an IETF working group performing this kind of > high-risk, high-reward research, versus doing something much more boring > like "writing a specification for VP9 to enable interoperable > implementations, and then iterating on that technology"? If VP9 could be shown to meet our requirements for performance and licensing, I would be supportive of that path. However, I don't believe that it currently does. In the testing draft, while I specified methods of testing codecs, I did not specify an absolute performance goal. We will need to set one. Opus achieved state-of-the-art performance, which was highly beneficial to its adoption. VP9 does not achieve this yet. In regards to licensing, it is clear that many companies are quite happy shipping VP9 right now. However, others are not nearly as comfortable with it, as seen on RTCWEB. These concerns arise from other, non MPEG-LA rights holders. We need to make sure that we address those concerns in a working group one way or another. One way to do that is just to stay well clear of the patents in question. > Google said in May 2013 that "a draft bitstream specification is well > underway." For whatever reason, they still haven't published it yet. (There > is also no independent implementation of a decoder written by a > non-Google-employee, afaik, much less of an encoder.) ffmpeg's VP9 decoder was written by a couple of non-Google employees: https://blogs.gnome.org/rbultje/ From nobody Thu Mar 19 18:31:28 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 49EBB1A9062 for ; Thu, 19 Mar 2015 18:31:27 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.277 X-Spam-Level: X-Spam-Status: No, score=-1.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FM_FORGED_GMAIL=0.622, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EJcPDvuG2IZa for ; Thu, 19 Mar 2015 18:31:25 -0700 (PDT) Received: from mail-yk0-x22a.google.com (mail-yk0-x22a.google.com [IPv6:2607:f8b0:4002:c07::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A93051A886C for ; Thu, 19 Mar 2015 18:31:25 -0700 (PDT) Received: by ykfc206 with SMTP id c206so35763578ykf.1 for ; Thu, 19 Mar 2015 18:31:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=xYaBeKIcYH5CLMF8v8QdYB4L9g1G14BtZXEkT4mFipQ=; b=0x9j5Co31Vp1x/mAVIwElaB3KPs0pENFgeqD1tV8lnYY02NwfF6rrIAOWzVnCRYGdS 9wIixj6bjA4Ug6/34KrvCakB1gEwlaeAs1mXpFru8LQrhnvcJm5osqpVOElz6M2C2shO 1Ww99fEGTFvb8qyQx3DdJVJ0tVy0Iut6YHapgg7neeqVMMdvKtpli+b/R4VKPe8hWcWh PN5b2Dnr3vxk7YD7kZ9MRxmgi+6xTiRgLPot0oq1RjtyURh0sCLuYS0RXj/p6k9i3BBJ heYCJ5ZVu0ob/MwO0cT8VapinxXkiCtXXzoFGSouq9i/p/4pQdYEV0PpjDwpO8PAFZJO h+sQ== X-Received: by 10.236.63.228 with SMTP id a64mr80335249yhd.41.1426815085100; Thu, 19 Mar 2015 18:31:25 -0700 (PDT) MIME-Version: 1.0 Sender: winstein@gmail.com Received: by 10.170.96.195 with HTTP; Thu, 19 Mar 2015 18:30:43 -0700 (PDT) In-Reply-To: <550B6CCC.70706@mozilla.com> References: <550B6CCC.70706@mozilla.com> From: Keith Winstein Date: Thu, 19 Mar 2015 18:30:43 -0700 X-Google-Sender-Auth: WBUIfAP26A8n33OWZ2r9Yiv0WpM Message-ID: To: Thomas Daede Content-Type: multipart/alternative; boundary=089e0129440c4ec4ef0511ae4694 Archived-At: Cc: video-codec@ietf.org Subject: Re: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: keithw@mit.edu List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Mar 2015 01:31:27 -0000 --089e0129440c4ec4ef0511ae4694 Content-Type: text/plain; charset=UTF-8 On Thu, Mar 19, 2015 at 5:41 PM, Thomas Daede wrote: > In the testing draft, while I specified methods of testing codecs, I did > not specify an absolute performance goal. We will need to set one. Opus > achieved state-of-the-art performance, which was highly beneficial to > its adoption. VP9 does not achieve this yet. Makes sense. > > Google said in May 2013 that "a draft bitstream specification is well > > underway." For whatever reason, they still haven't published it yet. > (There > > is also no independent implementation of a decoder written by a > > non-Google-employee, afaik, much less of an encoder.) > > ffmpeg's VP9 decoder was written by a couple of non-Google employees: > https://blogs.gnome.org/rbultje/ No disrespect intended to Ronald Bultje and his prolific work for free software, but he worked for Google on libvpx for more than two years as they developed VP9 (rbultje@google.com has >580 commits between April 2011 and July 2013) and "was one of the people involved in creating" VP9 in the first place, as he wrote on that blog post. I don't think ffvp9 really counts for what you want to see in a well-specified format. -Keith --089e0129440c4ec4ef0511ae4694 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
= On Thu, Mar 19, 2015 at 5:41 PM, Thomas Daede=C2=A0<tdaede@mozilla.com>=C2=A0wrote:
In the testing draft, while I = specified methods of testing codecs, I did
not specify an absolute perfo= rmance goal. We will need to set one. Opus
achieved state-of-the-art per= formance, which was highly beneficial to
its adoption. VP9 does not achi= eve this yet.

Makes sense.
=C2=A0=C2=A0
> Google said = in May 2013 that "a draft bitstream specification is well
> unde= rway." For whatever reason, they still haven't published it yet. (= There
> is also no independent implementation of a decoder written by= a
> non-Google-employee, afaik, much less of an encoder.)

ffm= peg's VP9 decoder was written by a couple of non-Google employees:
<= a href=3D"https://blogs.gnome.org/rbultje/" target=3D"_blank">https://blogs= .gnome.org/rbultje/

No disrespect intended to Ronald=C2=A0Bultje and= his prolific work for free software, but he worked for Google on libvpx fo= r more than two years as they developed VP9 (rbultje@google.com=C2=A0has >580 commits b= etween April 2011 and July 2013) and "was one of the people involved i= n creating" VP9 in the first place, as he wrote on that blog post. I d= on't think ffvp9 really counts for what you want to see in a well-speci= fied format.

-Keith
--089e0129440c4ec4ef0511ae4694-- From nobody Sun Mar 22 08:22:52 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B7EA51AC3CB for ; Sun, 22 Mar 2015 08:22:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -0.101 X-Spam-Level: X-Spam-Status: No, score=-0.101 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tHP9M8PKZv-v for ; Sun, 22 Mar 2015 08:22:49 -0700 (PDT) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 521E01AC3B3 for ; Sun, 22 Mar 2015 08:22:36 -0700 (PDT) Received: by lbbsy1 with SMTP id sy1so103564361lbb.1 for ; Sun, 22 Mar 2015 08:22:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ua+AMAK4IM51b4eU/rhJ62tV9dmIlsbJRAY2uqU6Hf4=; b=UrSRTluII8eVEIAr6uMepYu8YC76osR6QoegCk9YrQXoLFu5NYlOJXUfcsjo7q/JHT q3PTIckIGtkCiTSTRwGHPNtER3Q/Grudmnfgj/ugZCe67ESYGUegrwZoq35csPdW6272 zW4pjnDQepZJui/5dzgHY/Alvth9TdlxWluNTTc6DnupCJ0SxOAgVfZLgGSWZWmv94jD wjsqFK3chQ2usk87FBmFgqn+RQY12tQlpJBEGuJwOlcul54Isl+Sp+/64LzU5jsgXDpV aCdDMHlzm7yTPHl6rEKE2lP9AFXWVZ3eTPhE5SiThXWoZD6o7QVirWtbyCaWgyNUDxGq Ha3Q== MIME-Version: 1.0 X-Received: by 10.112.144.41 with SMTP id sj9mr80156683lbb.3.1427037754818; Sun, 22 Mar 2015 08:22:34 -0700 (PDT) Received: by 10.112.188.134 with HTTP; Sun, 22 Mar 2015 08:22:34 -0700 (PDT) In-Reply-To: References: Date: Sun, 22 Mar 2015 11:22:34 -0400 Message-ID: From: Monty Montgomery To: Keith Winstein Content-Type: text/plain; charset=UTF-8 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Mar 2015 15:22:50 -0000 I know Thomas already responded, but... On Thu, Mar 19, 2015 at 7:35 PM, Keith Winstein wrote: > But what are the merits of an IETF working group performing this kind of > high-risk, high-reward research, versus doing something much more boring > like "writing a specification for VP9 to enable interoperable > implementations, and then iterating on that technology"? Personally, I'd like to see Google submit their VPx work as an input to this process and even more interested in seeing them participate in development. Just like with Opus, I don't think the IETF is interested (and I'm certainly not interested) in rubber-stamping an existing codec. ...it would also go a long way toward repairing VP9's image in the marketplace. Like it or not, the conventional wisdom is 'VP9 is a proprietary codec with uncertain IPR status'. IP is obviously a major concern of ours, and we don't believe a strategy of 'we'll just win in court' is likely to win an adoption war. Monty From nobody Sun Mar 22 18:19:49 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 064EF1A8734 for ; Sun, 22 Mar 2015 18:19:48 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -3.277 X-Spam-Level: X-Spam-Status: No, score=-3.277 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Foz4cE5POdSm for ; Sun, 22 Mar 2015 18:19:46 -0700 (PDT) Received: from smtp.mozilla.org (mx1.corp.phx1.mozilla.com [63.245.216.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 569501A872F for ; Sun, 22 Mar 2015 18:19:46 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 7C026F217D for ; Sun, 22 Mar 2015 18:19:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx1.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0ZDrkecYkxRm for ; Sun, 22 Mar 2015 18:19:45 -0700 (PDT) Received: from [31.133.161.125] (dhcp-a17d.meeting.ietf.org [31.133.161.125]) (Authenticated sender: tterriberry@mozilla.com) by mx1.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id 0B4E9F2152 for ; Sun, 22 Mar 2015 18:19:44 -0700 (PDT) Message-ID: <550F6A30.5080403@xiph.org> Date: Sun, 22 Mar 2015 18:19:44 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: "video-codec@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] The Can Has Landed X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 01:19:48 -0000 Monty Montgomery wrote: > I know Thomas already responded, but... > > On Thu, Mar 19, 2015 at 7:35 PM, Keith Winstein wrote: >> But what are the merits of an IETF working group performing this kind of >> high-risk, high-reward research, versus doing something much more boring >> like "writing a specification for VP9 to enable interoperable >> implementations, and then iterating on that technology"? > > Personally, I'd like to see Google submit their VPx work as an input > to this process and even more interested in seeing them participate in > development. Just like with Opus, I don't think the IETF is Well, I think that's what Keith meant by "iterating on that technology". I think if Google wanted to contribute their VPx work we should be very happy to see it. VP9 has real-time encoding that shows significant performance gains over prior generations and a decoder faster than H.264's. Google got usable open source encoder and decoder implementations to market before HEVC, and significant hardware support agreements with some major chip vendors. If Google codec engineers are going to actively participate and bring VPx to the party, that would be a real asset. From nobody Mon Mar 23 05:42:58 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 57FD81A8901 for ; Mon, 23 Mar 2015 05:42:57 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.21 X-Spam-Level: X-Spam-Status: No, score=-4.21 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FbKFQv1Ghp8h for ; Mon, 23 Mar 2015 05:42:54 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [158.38.152.117]) by ietfa.amsl.com (Postfix) with ESMTP id 4C5521A8913 for ; Mon, 23 Mar 2015 05:42:54 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 06B4F7C432E for ; Mon, 23 Mar 2015 13:42:53 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TE2fSMSlndvH for ; Mon, 23 Mar 2015 13:42:52 +0100 (CET) Received: from [IPv6:2001:67c:370:136:a856:ac35:952d:b940] (unknown [IPv6:2001:67c:370:136:a856:ac35:952d:b940]) by mork.alvestrand.no (Postfix) with ESMTPSA id 5E7747C4304 for ; Mon, 23 Mar 2015 13:41:10 +0100 (CET) Message-ID: <551009A9.1010108@alvestrand.no> Date: Mon, 23 Mar 2015 13:40:09 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <550B6CCC.70706@mozilla.com> In-Reply-To: <550B6CCC.70706@mozilla.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 12:42:57 -0000 On 03/20/2015 01:41 AM, Thomas Daede wrote: > On 03/19/2015 05:17 PM, Keith Winstein wrote: >> But what are the merits of an IETF working group performing this kind = of >> high-risk, high-reward research, versus doing something much more bori= ng >> like "writing a specification for VP9 to enable interoperable >> implementations, and then iterating on that technology"? > If VP9 could be shown to meet our requirements for performance and > licensing, I would be supportive of that path. However, I don't believe= > that it currently does. > > In the testing draft, while I specified methods of testing codecs, I di= d > not specify an absolute performance goal. We will need to set one. Opus= > achieved state-of-the-art performance, which was highly beneficial to > its adoption. VP9 does not achieve this yet. Just as a matter of curiosity, which state of the art performance is it that you think VP9 hasn't achieved yet? I also wonder a bit about how we should treat evaluation of performance - a traditional video codec development technique has been to develop a codec that takes hours-per-second to encode a video, and then seek to optimize it by a factor of a hundred before shipping it. But sometimes, there's just limits to how much one can achieve. How should we treat that aspect in evaluation? From nobody Mon Mar 23 05:49:30 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 519841A89AC for ; Mon, 23 Mar 2015 05:49:29 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eij8-TGNs4Vr for ; Mon, 23 Mar 2015 05:49:27 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [IPv6:2001:700:1:2::117]) by ietfa.amsl.com (Postfix) with ESMTP id 599801A8928 for ; Mon, 23 Mar 2015 05:49:27 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 409757C432E for ; Mon, 23 Mar 2015 13:49:26 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i-8Nb4XhwZi0 for ; Mon, 23 Mar 2015 13:49:25 +0100 (CET) Received: from [IPv6:2001:67c:370:136:a856:ac35:952d:b940] (unknown [IPv6:2001:67c:370:136:a856:ac35:952d:b940]) by mork.alvestrand.no (Postfix) with ESMTPSA id D07A37C4304 for ; Mon, 23 Mar 2015 13:49:22 +0100 (CET) Message-ID: <55100BD0.8030301@alvestrand.no> Date: Mon, 23 Mar 2015 13:49:20 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: video-codec@ietf.org References: <550B6CCC.70706@mozilla.com> In-Reply-To: <550B6CCC.70706@mozilla.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Archived-At: Subject: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 12:49:29 -0000 On 03/20/2015 01:41 AM, Thomas Daede wrote: > On 03/19/2015 05:17 PM, Keith Winstein wrote: >> But what are the merits of an IETF working group performing this kind = of >> high-risk, high-reward research, versus doing something much more bori= ng >> like "writing a specification for VP9 to enable interoperable >> implementations, and then iterating on that technology"? > If VP9 could be shown to meet our requirements for performance and > licensing, I would be supportive of that path. However, I don't believe= > that it currently does. > > In the testing draft, while I specified methods of testing codecs, I di= d > not specify an absolute performance goal. We will need to set one. Opus= > achieved state-of-the-art performance, which was highly beneficial to > its adoption. VP9 does not achieve this yet. Just as a matter of curiosity, which state of the art performance is it that you think VP9 hasn't achieved yet? I also wonder a bit about how we should treat evaluation of performance - a traditional video codec development technique has been to develop a codec that takes hours-per-second to encode a video, and then seek to optimize it by a factor of a hundred before shipping it. But sometimes, there's just limits to how much one can achieve. How should we treat that aspect in evaluation? From nobody Mon Mar 23 05:52:34 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 19A901A892A for ; Mon, 23 Mar 2015 05:52:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.977 X-Spam-Level: X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GazxGZxcHrtD for ; Mon, 23 Mar 2015 05:52:32 -0700 (PDT) Received: from mail-wg0-f47.google.com (mail-wg0-f47.google.com [74.125.82.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DC01F1A891A for ; Mon, 23 Mar 2015 05:52:31 -0700 (PDT) Received: by wgra20 with SMTP id a20so145098440wgr.3 for ; Mon, 23 Mar 2015 05:52:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=O2MjCLLgv/sGkJ9QCVrPM3J/vnX6ERXdI38o3zkutUk=; b=hzOYg53V7v90GgbJayPIMH0jHNKFvz/Wrr0QTAFs/qvgMaEOpZJ6K9WSnsDmvnbv0g WPpnLZBwCgc7+3mnwoT/0PIMfpNIQQLxnUUs8dzRdyo3+8uQwRdU02KZjBVofM7gJ1AN Bt2KLux6DpiDY9mPYpuHDPC6rpX1ECB21Mgqu5GmkG5UphG0ePuBKQ93llZOq2UCad1o AzRTbIWW+Os7P8CmflvmHKc10SEYKGrF5ZwDn9V725K6Y4wNpOfgF737ZInMF+JthY2W aGSlyc/xKNRZUgAXWqihbfDhcigwInvg++AZ3SK0z5d5d1cqv74ToFUqgxUAeljj+Ou3 A1SQ== X-Gm-Message-State: ALoCoQlvEWh+9VCYLdb6daIXKyZru/KKVVx2Vi7vqpjU37VlM3aqIflvkeXuG7dOtzf2oZ0d77Vd MIME-Version: 1.0 X-Received: by 10.194.61.12 with SMTP id l12mr184135017wjr.139.1427115150320; Mon, 23 Mar 2015 05:52:30 -0700 (PDT) Received: by 10.194.14.129 with HTTP; Mon, 23 Mar 2015 05:52:30 -0700 (PDT) In-Reply-To: <551009A9.1010108@alvestrand.no> References: <550B6CCC.70706@mozilla.com> <551009A9.1010108@alvestrand.no> Date: Mon, 23 Mar 2015 14:52:30 +0200 Message-ID: From: Mohammed Raad To: Harald Alvestrand Content-Type: multipart/alternative; boundary=047d7b86df3896bddd0511f42308 Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 12:52:34 -0000 --047d7b86df3896bddd0511f42308 Content-Type: text/plain; charset=UTF-8 Well, the second part (how one should judge the potential optimization) typically has been avoided in video standardization and so far there seems to be little willingness to work on the development of a metric or a set of metrics that can be used to determine how "optimizable" a codec or tool is. It would actually be good if people at the IETF can develop such a metric. Also, keep in mind that the incremental development of a codec means that prior decisions limit what can be done with tools proposed later in the process. Mohammed On Mon, Mar 23, 2015 at 2:40 PM, Harald Alvestrand wrote: > On 03/20/2015 01:41 AM, Thomas Daede wrote: > > On 03/19/2015 05:17 PM, Keith Winstein wrote: > >> But what are the merits of an IETF working group performing this kind of > >> high-risk, high-reward research, versus doing something much more boring > >> like "writing a specification for VP9 to enable interoperable > >> implementations, and then iterating on that technology"? > > If VP9 could be shown to meet our requirements for performance and > > licensing, I would be supportive of that path. However, I don't believe > > that it currently does. > > > > In the testing draft, while I specified methods of testing codecs, I did > > not specify an absolute performance goal. We will need to set one. Opus > > achieved state-of-the-art performance, which was highly beneficial to > > its adoption. VP9 does not achieve this yet. > > Just as a matter of curiosity, which state of the art performance is it > that you think VP9 hasn't achieved yet? > > I also wonder a bit about how we should treat evaluation of performance > - a traditional video codec development technique has been to develop a > codec that takes hours-per-second to encode a video, and then seek to > optimize it by a factor of a hundred before shipping it. But sometimes, > there's just limits to how much one can achieve. > > How should we treat that aspect in evaluation? > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --047d7b86df3896bddd0511f42308 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Well, the second part (how one should judge the potential = optimization) typically has been avoided in video standardization and so fa= r there seems to be little willingness to work on the development of a metr= ic or a set of metrics that can be used to determine how "optimizable&= quot; a codec or tool is. It would actually be good if people at the IETF c= an develop such a metric.

Also, keep in mind that the in= cremental development of a codec means that prior decisions limit what can = be done with tools proposed later in the process.

= Mohammed

On Mon, Mar 23, 2015 at 2:40 PM, Harald Alvestrand <= ;harald@alvestran= d.no> wrote:
On 03/20/2015 = 01:41 AM, Thomas Daede wrote:
> On 03/19/2015 05:17 PM, Keith Winstein wrote:
>> But what are the merits of an IETF working group performing this k= ind of
>> high-risk, high-reward research, versus doing something much more = boring
>> like "writing a specification for VP9 to enable interoperable=
>> implementations, and then iterating on that technology"?
> If VP9 could be shown to meet our requirements for performance and
> licensing, I would be supportive of that path. However, I don't be= lieve
> that it currently does.
>
> In the testing draft, while I specified methods of testing codecs, I d= id
> not specify an absolute performance goal. We will need to set one. Opu= s
> achieved state-of-the-art performance, which was highly beneficial to<= br> > its adoption. VP9 does not achieve this yet.

Just as a matter of curiosity, which state of the art performance is it
that you think VP9 hasn't achieved yet?

I also wonder a bit about how we should treat evaluation of performance
- a traditional video codec development technique has been to develop a
codec that takes hours-per-second to encode a video, and then seek to
optimize it by a factor of a hundred before shipping it. But sometimes,
there's just limits to how much one can achieve.

How should we treat that aspect in evaluation?


_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
Mohammed Raad, PhD.
Partner
RAADTECH CONSULTING<= br>P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 414451478<= br>Email: m= ohammedsraad@raadtech.com
--047d7b86df3896bddd0511f42308-- From nobody Mon Mar 23 07:10:06 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4BA611A8ACF for ; Mon, 23 Mar 2015 07:10:04 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.91 X-Spam-Level: X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KteOpDdsUMZc for ; Mon, 23 Mar 2015 07:10:02 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [IPv6:2001:700:1:2::117]) by ietfa.amsl.com (Postfix) with ESMTP id CB5A91A8ABC for ; Mon, 23 Mar 2015 07:10:01 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 1A2837C4337; Mon, 23 Mar 2015 15:10:01 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eqwA_deCtmTV; Mon, 23 Mar 2015 15:09:59 +0100 (CET) Received: from [IPv6:2001:67c:370:176:a856:ac35:952d:b940] (unknown [IPv6:2001:67c:370:176:a856:ac35:952d:b940]) by mork.alvestrand.no (Postfix) with ESMTPSA id 085397C4330; Mon, 23 Mar 2015 15:09:58 +0100 (CET) Message-ID: <55101EB4.8020809@alvestrand.no> Date: Mon, 23 Mar 2015 15:09:56 +0100 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Mohammed Raad References: <550B6CCC.70706@mozilla.com> <551009A9.1010108@alvestrand.no> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Archived-At: Cc: "video-codec@ietf.org" Subject: Re: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 14:10:04 -0000 On 03/23/2015 01:52 PM, Mohammed Raad wrote: > Well, the second part (how one should judge the potential > optimization) typically has been avoided in video standardization and > so far there seems to be little willingness to work on the development > of a metric or a set of metrics that can be used to determine how > "optimizable" a codec or tool is. It would actually be good if people > at the IETF can develop such a metric. I was thinking more of whether experience with the encoding speed of the test implementations should be allowed as an argument or not. Sometimes one can use dataflow analysis and similar tools to show that certain techniques prevent optimization beyond a certain level (or not) - but most of the time, I think "this runs at speed x / quality y" is a more compelling argument than "I think this could be optimized to speed x / quality y". > > Also, keep in mind that the incremental development of a codec means > that prior decisions limit what can be done with tools proposed later > in the process. Yes. Especially when the decision is "we can't use this tool for nontechnical reasons", it means that we can't accept anything that is a refinement of this tool either. From nobody Mon Mar 23 07:13:19 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 316411A8A8F for ; Mon, 23 Mar 2015 07:13:17 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.378 X-Spam-Level: X-Spam-Status: No, score=-1.378 tagged_above=-999 required=5 tests=[BAYES_20=-0.001, HELO_MISMATCH_ORG=0.611, HOST_MISMATCH_COM=0.311, RCVD_IN_DNSWL_MED=-2.3, SPF_FAIL=0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H6Jmr02I9op3 for ; Mon, 23 Mar 2015 07:13:14 -0700 (PDT) Received: from smtp.mozilla.org (mx2.corp.phx1.mozilla.com [63.245.216.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F11731A8ABE for ; Mon, 23 Mar 2015 07:13:03 -0700 (PDT) Received: from localhost (localhost6.localdomain [127.0.0.1]) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTP id 268D4F225E for ; Mon, 23 Mar 2015 07:13:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at mozilla.org Received: from smtp.mozilla.org ([127.0.0.1]) by localhost (mx2.mail.corp.phx1.mozilla.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Evt-L1LuEtfX for ; Mon, 23 Mar 2015 07:13:02 -0700 (PDT) Received: from [31.133.165.93] (dhcp-a55d.meeting.ietf.org [31.133.165.93]) (Authenticated sender: tterriberry@mozilla.com) by mx2.mail.corp.phx1.mozilla.com (Postfix) with ESMTPSA id AED6AF2122 for ; Mon, 23 Mar 2015 07:13:02 -0700 (PDT) Message-ID: <55101F6D.8020905@xiph.org> Date: Mon, 23 Mar 2015 07:13:01 -0700 From: "Timothy B. Terriberry" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 SeaMonkey/2.26 MIME-Version: 1.0 To: video-codec@ietf.org References: <550B6CCC.70706@mozilla.com> <55100BD0.8030301@alvestrand.no> In-Reply-To: <55100BD0.8030301@alvestrand.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Archived-At: Subject: Re: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 14:13:17 -0000 Harald Alvestrand wrote: > Just as a matter of curiosity, which state of the art performance is it > that you think VP9 hasn't achieved yet? I think if we were trying to develop a competitor to HEVC, we could say "Ship VP9" and be done. In fact, that is exactly what Firefox did. But I am (personally) worried about what comes after HEVC, and I think we have an opportunity to get ahead of the curve here like we did with Opus. > How should we treat that aspect in evaluation? I have been thinking about this a lot since Mo asked the question a few weeks ago, and I am having a hard time coming up with an answer that is not "use the normal IETF consensus process". We've certainly tried to focus on practical algorithms in Daala. However, sometimes it is nice to know what the theoretical best we could do actually is, so we know how well some of our cheap hacks are performing. To use a recent example, until the last few weeks, the way we had been implementing our lapped transforms, one needed to know the block sizes of your neighbors to know what lapping size to apply. That meant we couldn't do an exhaustive search of block sizes without making some kind of gross simplifications. We restructured things to remove this dependency, even though it meant the filtering was theoretically worse. That allowed us to write an exhaustive search, which turned out to be between 3% and 11% better than the heuristics we had been using to pick block sizes. Until we did this, we didn't have a good idea what kind of gains were possible, nor how to change our heuristics to produce better results. Now we have something we can look at to tell us both. So having both the heuristic and the exhaustive search is the ideal situation, and sometimes asking for it is reasonable. Sometimes it may be more work than people are willing to do up-front to make a decision about what is a good idea. But I will also give a second example, that is even more recent. One issue Thomas Davies from Cisco pointed out to us was some kind of noise buildup along block boundaries in one of their test clips at low rates, something we had not seen in our own testing. We were able to track down the problem to certain rounding operations in the pre/post filters and in converting from the 12-bit data we use in our transforms down to 8-bit data we store in our reference frames. Storing 12-bit reference frames makes the problem go away, but increases the memory bandwidth required by the codec by 50%. But we've observed gains between 0.2 dB and (on specially constructed examples) 2 dB by doing so. Is that a cost people are willing to pay? I don't know the answer to that question, but I do know that the easiest way to measure that, x86 cycle counts, is also probably the least informative. It also matters what other solutions there are (we have at least one more, with a different set of unappealing costs), or how hard we want to look for them. I have a real hard time seeing how to design an up-front process that answers questions like this. Maybe others here have better ideas than I do. From nobody Mon Mar 23 07:55:49 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B90101A8BC0 for ; Mon, 23 Mar 2015 07:55:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -14.511 X-Spam-Level: X-Spam-Status: No, score=-14.511 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O2Ydk0y3ihAK for ; Mon, 23 Mar 2015 07:55:46 -0700 (PDT) Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 542FE1A8BAF for ; Mon, 23 Mar 2015 07:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1376; q=dns/txt; s=iport; t=1427122546; x=1428332146; h=from:to:subject:date:message-id:references:in-reply-to: content-id:content-transfer-encoding:mime-version; bh=vZXsL/7930ED6m+2AeMvsWHJ2XVBBQJPk5IroAC/Bq8=; b=Xp1DYfMOGkgvb4C9+3nTk/3mKab190y2yrBKcwOb1SwTVuiSxGflitom 1WdNiJIeU76mgFcxPSIxEQNpsAN57Mn4CKAHQxMy2Jwn+Ky5OW2gNfGcU xjThMYlSPaE+S243zMGyV+pg46TDabGs2A+TIY1VJdZ0I9myOQOQi16Gj o=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0AJBQA6KRBV/4QNJK1cgwaBLATMQAKBKUwBAQEBAQF9hBUBAQQ6OhUCAQg2EDIlAgQBEogvyAwBAQEBAQEBAwEBAQEBAQEbiyGEfIQtAQSQT4lulCoig25vgUR/AQEB X-IronPort-AV: E=Sophos;i="5.11,452,1422921600"; d="scan'208";a="134587742" Received: from alln-core-10.cisco.com ([173.36.13.132]) by alln-iport-2.cisco.com with ESMTP; 23 Mar 2015 14:55:45 +0000 Received: from xhc-aln-x12.cisco.com (xhc-aln-x12.cisco.com [173.36.12.86]) by alln-core-10.cisco.com (8.14.5/8.14.5) with ESMTP id t2NEtjcT016784 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Mon, 23 Mar 2015 14:55:45 GMT Received: from xmb-rcd-x14.cisco.com ([169.254.4.61]) by xhc-aln-x12.cisco.com ([173.36.12.86]) with mapi id 14.03.0195.001; Mon, 23 Mar 2015 09:55:45 -0500 From: "Mo Zanaty (mzanaty)" To: Harald Alvestrand , "video-codec@ietf.org" Thread-Topic: [video-codec] Performances, measurement of same (Re: The Can Has Landed) Thread-Index: AQHQZXlwNdqDcNEL0E6Z9vBFpmqNQw== Date: Mon, 23 Mar 2015 14:55:44 +0000 Message-ID: References: <550B6CCC.70706@mozilla.com> <551009A9.1010108@alvestrand.no> In-Reply-To: <551009A9.1010108@alvestrand.no> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.8.150116 x-originating-ip: [64.100.32.216] Content-Type: text/plain; charset="us-ascii" Content-ID: <355F548D72B7E34A8155F44DDB02E167@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Archived-At: Subject: Re: [video-codec] Performances, measurement of same (Re: The Can Has Landed) X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Mar 2015 14:55:47 -0000 Complexity (or more broadly, resource requirements, which encompass aspects beyond compute cycles like memory and memory bandwidth) should be a fundamental part of the evaluation methodology. But there are many pitfalls to avoid here. Intrinsic algorithmic complexity is difficult to discern with any precision, since any metric necessarily depends on the level of implementation optimization (from simple loop unrolling to vector intrinsics or full assembly). Optimization can hamper experimentation, so it is usually not a good idea in the research stage. Configuration points are also important to consider. Having configuration points that are impractical for real-world use is still valuable for steering the research. This should not be the primary evaluation criteria, which should attempt to focus on real-world use. But such configuration points should be allowed and analyzed. Mo On 3/23/15, 8:40 AM, Harald Alvestrand wrote: I also wonder a bit about how we should treat evaluation of performance - a traditional video codec development technique has been to develop a codec that takes hours-per-second to encode a video, and then seek to optimize it by a factor of a hundred before shipping it. But sometimes, there's just limits to how much one can achieve. How should we treat that aspect in evaluation? From nobody Tue Mar 24 09:18:40 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A419E1A9042 for ; Tue, 24 Mar 2015 09:18:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -113.959 X-Spam-Level: X-Spam-Status: No, score=-113.959 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FUZZY_AMBIEN=0.552, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bBR2zLnvBYGY for ; Tue, 24 Mar 2015 09:18:32 -0700 (PDT) Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B7C141A903B for ; Tue, 24 Mar 2015 09:18:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=11105; q=dns/txt; s=iport; t=1427213902; x=1428423502; h=mime-version:subject:from:date:cc: content-transfer-encoding:message-id:to; bh=Ll6i5OwKSqZb8IZIkMcvopkdVpl+Ae+1Fnh3b0OXYA0=; b=XuGY/xfb5eqkSphHM1XWwjHLyGWHgJhr0glHcBmx6t2tbBSDSJfEq9ln DgjMYcJsalgW6Q2a0CMtwh9WF8sxCtl7TPEDtdLco1XAQ6KMVmqpV4gCV uTm6NafoE2Q3MCtKFmcZ4crjrxsqe5410A84pRW9m2s8r6LcEKkC7SthJ Q=; X-IronPort-AV: E=Sophos;i="5.11,459,1422921600"; d="scan'208";a="134977486" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by alln-iport-5.cisco.com with ESMTP; 24 Mar 2015 16:18:22 +0000 Received: from [127.0.0.1] (ssh-sjc-2.cisco.com [171.68.46.188]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id t2OGILvR005104; Tue, 24 Mar 2015 16:18:21 GMT Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) From: Cullen Jennings Date: Tue, 24 Mar 2015 11:18:20 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <1BFB68D0-0A6C-459E-B7E9-24E8ACE0E2A4@cisco.com> To: Cullen Jennings , video-codec@ietf.org X-Mailer: Apple Mail (2.2070.6) Archived-At: Cc: Richard Barnes , Adam Roach , Jon Peterson Subject: [video-codec] Notes from NETVC BOF X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Mar 2015 16:18:38 -0000 Chairs - Jon Peterson, Adam Roach Note takers - Varum Singh, Cullen Jennings=20 AD Set Context=20 At IETF75, we tried to decide if we should do an audio codec. That = resulted in OPUS an widely successful audio codec. Later we decided to = start looking at video codec. Today we are trying to decide what we are = going to a video codec WG.=20 Chairs -------- - see slide for clear set of goals of what we want to accomplish=20 AD & Chairs clarified purpose of today was to decide if we should do = something in this space and what it should be=20 Key Consideration presentation by Mo -------------------------------------------------- Mo discussed several of the issues that are bit different for an codec = meant for the Internet.=20 Discussed how our need for fast fine rate control is a different set of = requirements than what most video codec are optimized for.=20 Algorithm Agility=20 - brilliant ideas solicited for way to avoid IPR risk and to do that = technically to allow algorithm agility to minimize IPR risk=20 Question Q what resolutions and bit depth? A: Very rough requirements draft mentions this but the WG needs to Q. Keith - new codecs make new problems (transcoding). Should be in = requirements that don't make transcoding more difficult. Concern that we = can address the delay problem in transcoding=20 Q. Tim T - IPR around spatial scaling is nasty, do you think it is = possible to more than?=20 A.=20 Q. Bernard - in many cases spatial scaling is less efficient than = simulcast Opus had an ease of configuration that made it successful. Do we want in = band FEC, are there are other things like this. When doing switching, = you send FIR and get huge hit in bandwidth, and spacial support in = bitstream might be able to help with that.=20 Q Jean-Marc - opus was designed for RTP and we need to do the same thing = here=20 Tim T presentation on the Daala project=20 Strategy for getting RF free - replace major chunks of the codec with very different technology to = avoid IPR=20 - this is high risk but high reward - helps avoid large swaths of patents including ones we are not aware of=20= - moves us area that is much less patent s Explained several areas that are huge parts of the key way most codecs = work today and then went on to explain the very different tackiness that = daala is exploring to to avoid theses techniques.=20 Example given in sides that show an example where image looks better but = PSNR is worse.=20 Contributions to daala code from 37 people - many different companies - = many of them IETF participants Q - Frank Bosson - Questions about the type of encoding and which 265. A - Using just one I frame on sequence 1 second long and trying to set = up for an interactive type codec.=20 Q - what about other codec A - yes and don't look as good as this but like some other metrics too=20= A - Time suggested taking results with full shaker of salt Q- Mo Z. - External view of this work means that we need to really focus = on evaluation. We need metrics to focus on true real time performance. = Traditional codecs have fallen way below reference implementation when = used in real time practical implementations.=20 Q. Eric Rescorla - ask about graph=20 A. For 1080p you tube around 6 and low quality video conferencing around = 1mbps or 0.5 Q about continuous improvement A. have web site (https://arewecompressedyet.com/) that shows how = compression is going on data sets provided by anyone on many codecs Q. how fast does the test framework need to be=20 A. run of a bunch of video sequences takes about 20 minutes=20 Q Frank Bosson - Probably want to be at around 2-3 mbps for 1080p not = the 6 mbps listed before.=20 Q. Ross Finlayson - has everyone signed the equivalent of Note Well=20 A. no=20 Charter Discussion ------------------------- Chairs presented objective and the charter to the room=20 Q. Keith D asked about used for interactive or non interactive A. yes this is for interactive=20 A. pointed at point 2 and 3 and processes 1/2 slide Keith D felt what was in chart was pretty good but might send a few = words to slightly more mention real time Bernard - We should say where it is better. He thinks it has to be = better. Opus is a quantum leap better than stuff before it. Would like = it to be better than existing stuff in way other that RF. He wants = something better like auto configuration to do in band FEC. Some area = can say just want to be competitive with, but in some area we want to = say we are way better - and not just "freer" - We should be very generic in how we address this because we are not = the=20 HTA - disagrees with Bernard. If we start saying things like MUST be = better than popular stuff and goal is 2 years out, . Opus is just a = little bit better than AMR-WB. Like charter text as is.=20 Richard Barnes - the must be better is not really relevant to charter. = Its the consensus of WG to say it is done that is key thing. The IESG is = not evaluating if it is better Bernard A. - In band FEC is something that has not been done. Just need = something to=20 Keith Drage - If the only thing we can say about it is a RF codec, then = we have failed. RF is just a part of it. Charter should say we want to = develop a better codec Steve Bosco - we might want to add the words "low latency" somewhere to = indicate where we are focusing it to be better. Good to a Gaelle Martin-Cocher -=20 Q - compare to codecs when it is done is vague. It should say it needs = to compare to something specific, VP9, 265 etc Q. have you considered allow a core profile that is RF and non core = profile that is not RF=20 Q. ITU has been trying to do RF codec for awhile now=20 Eric Rescorla=20 - need to remove a } on some bullet=20 - would be nice to have some word around diversity of software like if = we want to be able to do software implementation on mobile phones we = should say that. Is the assumptions software on=20 Thomas Edwards with box=20 - might be worth doing gap analysis on existing codecs=20 - people are working on jpeg2000 profile that can provide a low latency = video codec=20 - even with mepg2 we are seeing improvements over time as people learn = to use the tools better. - with H265 we are seeing it is not appropriate for software for all = resolutions and frame rates. It is only now that we are getting H265=20 Tim Terriberry - responding to Gaelle questions - we have tried the approach of RF but not as good - theora for example = - not as successful as we want=20 - with 264 we had plan to make baseline RF and then other profiles non = RF but baseline did not end up RF Gaelle - We should reach out to other places Chairs - we try and look and see if we expertise. Same thing with opus, = many people doubted this is right place but we took that risk. Downside = is we could have RFC that does not light world on fire.=20 Gaelle - If you want to mitigate risk of IPR, you will want to have the = ITU and may have better change to get a broader set of companies to = signal IPR on it.=20 Gaelle - how do you know what IRP risk are acceptable=20 Chairs - IETF process leaves that the participants of the WG=20 Ted Hardie - some of these issues are dealt with on further slides Mo Z - Might be good to have a list of useful IPR=20 Randall Jesup - Better is a rat hole. RF and competitive for interactive = / real time is what we need.=20 Thomas - There is always the risk that there is IPR you will not know = about.=20 Chairs - this is true about all the stuff we work on=20 Bernard - it will be better to get very low bit rates because of lower = price HTMl phones in Africa.=20 Mo - propose strike the 256 kbps text.=20 Tim support **** Chair ask if anyone want to strike the text on 256 kbps lower bow. = No one objected.=20 Joe - we have general liaison to ITU but do we need specific liaison to = to SQ16 **** Chair will follow up offline to see what need Stephan Wenger - currently liaison to mpeg. Most currently work is = joint work between ITU and MPEG. Stephen=20 Bernard - no payload format.=20 Answer - we will do that in payload at same time Ted Hardie - first document are huge rat whole and mostly unless Mo - be good to have a survey of relevant IPR that is expired or soon = expiring=20 Chairs - might what as living doc not a deliverable that IETF consensus evaluation metrics should be evaluation methodology perhaps we should define encoded bitstream and decoder operation instead = of defining encoder and decoder Frank - right now read as we will normatively define encoder. is that = really what we want=20 Cullen - explain desire for the point 1 text Keith - point one on IRP could be read as changing the way this WG will = deal IPR=20 Ted - will have to have argument about what licensing terms are before = they know what they will contribute=20 We could have the WG try to say if some of the well known licenses are = acceptable to WG.=20 Tim - What ted was proposing is in line with what intended here and = could ted send text=20 Stephen Wenger - done patent work for decade. When you say you have a = goal of RF, people can dream up something. Recommendation is to stop = this effort. Go ahead and do this for open source.=20 Harald A. - BCP 79 does not have licensing term.=20 Eric Rescorla - don't feel strongly about it. . One proposal would move = it down to item 4. Another possible possibility to rephrase as example = terms.=20 ***** No one felt strongly the IPR number one bullet point should stay = in the charter.=20 On to milestones slides - could strike the IPR milestone as well=20 Frank - Question about dates - how long are we talking Tim T - hope to freeze bitstream by end of year. A year is optimistic = but that type of range. Opus took 3 years from point is chartering. This = is a larger chunk works. More people working on it. Gaelle - ? Answer - invite anyone who show up and contribute encoder / decoder=20 Keith - take a pass though and check milestones match=20 Moving into Questions The question - Is there a problem worth solving ? Thomas - ask about what the problem is=20 Strong and unanimous for the minutes in favor of YES=20 Is the scope of problem well defined and understood? Is there agreement = on what a WG would need to deliver? Strongly consensus with a little bit of decidedly in favor of this Are there people will do do the work=20 7 people willing to writ the documents=20 lots of people willing to review drafts=20 big chunk of people willing to implement=20 How many people feel that a WG should not be formed at the IETF ? Heard maybe 1 person=20 Majority of room strongly in favor of forming a WG.=20 =20 From nobody Wed Mar 25 16:19:55 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A3FE61B2A07 for ; Wed, 25 Mar 2015 16:19:54 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3XYG3eF-zIfd for ; Wed, 25 Mar 2015 16:19:51 -0700 (PDT) Received: from mail-pd0-f179.google.com (mail-pd0-f179.google.com [209.85.192.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F13A1A90E8 for ; Wed, 25 Mar 2015 16:19:51 -0700 (PDT) Received: by pdbop1 with SMTP id op1so43257604pdb.2 for ; Wed, 25 Mar 2015 16:19:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=0VM52ZvtOmOXin6hz1nbrf4B7a76JhnVaKAx9HXh1xc=; b=aMlx+n2gvp2sMsToy1U2sNCBMYK7Xeqh8KosQnlvvLRR5w8h7yITJq6anxoTKTT6J1 /nP9nq+05I7Wdop4SBqsA9QlyPMQtrAfec0P/Wo+/P/T2BEwne2FPm+SZhXOhZA9kzQq uD8JzFFueWLDWikHiNXQFs1bgrOFaMcZck6RKG/inDUdNrETuwIjXVPUS8gqAyH+FpmF bp76NRqHWdphZOXBLrwfGMfDsrBbVgor9mA2cBnrKv+zDR1nWMKq1FIxdIbOraCuFMBH JU2S/cTHMgnMDER785DBIO1kcEcsd9Z03mmR6n/9xmLE4S9weblXLjecLDd0wm7Ajm0h wZvw== X-Gm-Message-State: ALoCoQla9XX9lygm1RVMTESADA7igh0y2c64uWs1twPbNwAMlSktd/L5iXEBSvEszVS/DEGPyleB X-Received: by 10.70.0.176 with SMTP id 16mr21751991pdf.78.1427325591220; Wed, 25 Mar 2015 16:19:51 -0700 (PDT) Received: from ?IPv6:2620:101:80fc:224:7e7a:91ff:fe9e:8126? ([2620:101:80fc:224:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id ox4sm3541570pdb.36.2015.03.25.16.19.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Mar 2015 16:19:50 -0700 (PDT) Message-ID: <55134294.2030803@mozilla.com> Date: Wed, 25 Mar 2015 16:19:48 -0700 From: Thomas Daede User-Agent: http://a.pomf.se/cszdno.opus MIME-Version: 1.0 To: video-codec@ietf.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Archived-At: Subject: [video-codec] Command-line options for encoder comparisons X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Mar 2015 23:19:54 -0000 I am updating the scripts that AreWeCompressedYet uses. I would like to select what rate control options are used in the comparison. This is for purposes of absolute comparison between different codecs, and to indicate when a particular performance point is reached. This is not for adding tools to an existing codec. There are a couple of different rate control modes that we can use to compare: Constant Quantizer: This is not as simple as it might sound. Should referenced frames still be allowed a boost in this mode? Should adaptive quantization be allowed? Can quantization matrices be switched? I tried to come up with a sane way to compare different codecs with a constant quantizer and was not successful. Constant Quality: Here the codecs are set to use all features available within temporal constraints. For stored video, this means full lookahead (or two pass, if that's what the codec uses). For realtime, this means no lookahead, but otherwise all quality-oriented rate control is allowed. Example rate control parameters: VP9 parameters: vpxenc --codec=vp9 --cq-level=$x --end-usage=q x264 parameters: x264 --crf=$x x265 parameters: x265 --crf=$x Constant Bitrate: This is the hardest category to compare, as it means that all of the CBR implementations need to be set to have the same buffer constraints. It is an important use case, so I think we will need to figure out a way to test it. I'd be okay with adding this one later, though. From nobody Sun Mar 29 20:16:46 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 538E41A1A9E for ; Sun, 29 Mar 2015 20:16:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.51 X-Spam-Level: X-Spam-Status: No, score=-1.51 tagged_above=-999 required=5 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_MED=-2.3, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h4Wd2pynDpuf for ; Sun, 29 Mar 2015 20:16:42 -0700 (PDT) Received: from mork.alvestrand.no (mork.alvestrand.no [158.38.152.117]) by ietfa.amsl.com (Postfix) with ESMTP id 40BE31A1A8B for ; Sun, 29 Mar 2015 20:16:42 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mork.alvestrand.no (Postfix) with ESMTP id 7A43E7C55A0; Mon, 30 Mar 2015 05:16:41 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at alvestrand.no Received: from mork.alvestrand.no ([127.0.0.1]) by localhost (mork.alvestrand.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DX8etVJg3l-a; Mon, 30 Mar 2015 05:16:39 +0200 (CEST) Received: from [IPv6:2001:470:de0a:27:8d54:322b:45f1:74a0] (unknown [IPv6:2001:470:de0a:27:8d54:322b:45f1:74a0]) by mork.alvestrand.no (Postfix) with ESMTPSA id 316AC7C559F; Mon, 30 Mar 2015 05:16:39 +0200 (CEST) Message-ID: <5518C016.7030309@alvestrand.no> Date: Mon, 30 Mar 2015 05:16:38 +0200 From: Harald Alvestrand User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Thomas Daede , video-codec@ietf.org References: <55134294.2030803@mozilla.com> In-Reply-To: <55134294.2030803@mozilla.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [video-codec] Command-line options for encoder comparisons X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 03:16:45 -0000 I know I'm being contrarian.... what's the model you are trying to construct, that we perform comparisions on? Is it one where you generate a PSNR/rate graph for each codec, you need one parameter to vary to make a plot, and you want to have a parameter that has roughly the same influence on all codecs? Den 26. mars 2015 00:19, skrev Thomas Daede: > I am updating the scripts that AreWeCompressedYet uses. I would like to > select what rate control options are used in the comparison. This is for > purposes of absolute comparison between different codecs, and to > indicate when a particular performance point is reached. This is not for > adding tools to an existing codec. There are a couple of different rate > control modes that we can use to compare: > > Constant Quantizer: > This is not as simple as it might sound. Should referenced frames still > be allowed a boost in this mode? Should adaptive quantization be > allowed? Can quantization matrices be switched? I tried to come up with > a sane way to compare different codecs with a constant quantizer and was > not successful. While I believe that constant quantizer is useless, I don't see a problem here. If you need a single set of points, ask that there be a rule for deriving all the q values from other q values, and model each rule as a different codec; keep the one that performs best. In the compare-codecs tool, see the codecs "vp8-mpeg" and "vp8-mpeg-1d" for examples - one uses three constant quantizers, the other one wraps the three into one parameter and a formula. > Constant Quality: > Here the codecs are set to use all features available within temporal > constraints. For stored video, this means full lookahead (or two pass, > if that's what the codec uses). For realtime, this means no lookahead, > but otherwise all quality-oriented rate control is allowed. Example rate > control parameters: > > VP9 parameters: > vpxenc --codec=vp9 --cq-level=$x --end-usage=q > > x264 parameters: > x264 --crf=$x > > x265 parameters: > x265 --crf=$x What's the value of this comparision, given that there's no uniformity among the codecs on what the "cq-level" / "crf" means? It might work well if we only use it to generate a spread of encoding results that can be plotted on a bitrate/quality graph, but will not work at all if we start saying "for x from 15 to 35, do this" - since there's no uniformity in the range or scaling form for the "quality" parameter. > > Constant Bitrate: > This is the hardest category to compare, as it means that all of the CBR > implementations need to be set to have the same buffer constraints. It > is an important use case, so I think we will need to figure out a way to > test it. I'd be okay with adding this one later, though. This is the default mode (ie the one I've been spending the most time with) on compare-codecs. I'd argue that we should be setting some kind of constraint on what we're doing (such as "encoding process uses no more than X MB of RAM), but otherwise leave each codec to optimize within those constraints - for my tests, I've not put any controls on buffering strategies so far. The argument I'd make here is that setting a rate should produce at most (roughly) that rate - that is, we need to have some means of removing configurations where the bitrate grows beyond reasonable. Most codecs (especially in 1-pass) have problems hitting their bitrate targets on the short test clips; production usage usually assumes that bitrate can be averaged over longer periods. (Note: I'm not sure this is actually a reasonable assumption for a DASH environment.) > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > From nobody Mon Mar 30 02:24:51 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BEEA31ACD55 for ; Mon, 30 Mar 2015 02:24:50 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.977 X-Spam-Level: X-Spam-Status: No, score=-1.977 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mi6O5zd4LPsa for ; Mon, 30 Mar 2015 02:24:48 -0700 (PDT) Received: from mail-wg0-f41.google.com (mail-wg0-f41.google.com [74.125.82.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 527A81ACD4F for ; Mon, 30 Mar 2015 02:24:48 -0700 (PDT) Received: by wgbgs4 with SMTP id gs4so75404613wgb.0 for ; Mon, 30 Mar 2015 02:24:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=DGJACeLTGQJZoiJrMnubT9MEH06oaZwHWUgNdK2GXFQ=; b=h8JmbRLolIErzcrkE53TpjyZ50MmXlg52g3hcatPPRmiUWmsTDtR2sBtlonFp6oyLW D3JZmprmc9zgOjbU7B1ARgcyfsiihWLjT+mLNL4pz+UzmW2Re6UPTkI4pDl/5x7EvEOB q7hMgudeWafRS0eG++8ScuTum1cl4b5wkSF6fQHe7KqtBVrVnSV/eFMwD5g3jObAbDpZ e2jkPLqxXmwOuETjle4IajuLp1ahSDKHuVsp+M4L3BdB18mpn/AK7ny9Tm8u/uZHHTlp tGSrGYoqhcnqnhickTnbCNbUsso1PbDCxo3M/Fyl6GL8DOFspqAUL2A8GaKFNc4vfVLb aP3Q== X-Gm-Message-State: ALoCoQmmyAmr/lh260CbtYQ4NpNC7cfEPptNLhOA8UTBf5nD09br9As0+W/0aDu4huZqeQufiBGz MIME-Version: 1.0 X-Received: by 10.180.86.201 with SMTP id r9mr14707340wiz.56.1427707486495; Mon, 30 Mar 2015 02:24:46 -0700 (PDT) Received: by 10.194.14.129 with HTTP; Mon, 30 Mar 2015 02:24:46 -0700 (PDT) In-Reply-To: <5518C016.7030309@alvestrand.no> References: <55134294.2030803@mozilla.com> <5518C016.7030309@alvestrand.no> Date: Mon, 30 Mar 2015 12:24:46 +0300 Message-ID: From: Mohammed Raad To: "video-codec@ietf.org" Content-Type: multipart/alternative; boundary=f46d0442861c93799205127e0da2 Archived-At: Subject: Re: [video-codec] Command-line options for encoder comparisons X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 09:24:50 -0000 --f46d0442861c93799205127e0da2 Content-Type: text/plain; charset=UTF-8 It's interesting that you mention a DASH environment at the end - shouldn't this use case form the basis of a test scenario? On Mon, Mar 30, 2015 at 6:16 AM, Harald Alvestrand wrote: > I know I'm being contrarian.... > > what's the model you are trying to construct, that we perform > comparisions on? > > Is it one where you generate a PSNR/rate graph for each codec, you need > one parameter to vary to make a plot, and you want to have a parameter > that has roughly the same influence on all codecs? > > Den 26. mars 2015 00:19, skrev Thomas Daede: > > I am updating the scripts that AreWeCompressedYet uses. I would like to > > select what rate control options are used in the comparison. This is for > > purposes of absolute comparison between different codecs, and to > > indicate when a particular performance point is reached. This is not for > > adding tools to an existing codec. There are a couple of different rate > > control modes that we can use to compare: > > > > Constant Quantizer: > > This is not as simple as it might sound. Should referenced frames still > > be allowed a boost in this mode? Should adaptive quantization be > > allowed? Can quantization matrices be switched? I tried to come up with > > a sane way to compare different codecs with a constant quantizer and was > > not successful. > > While I believe that constant quantizer is useless, I don't see a > problem here. If you need a single set of points, ask that there be a > rule for deriving all the q values from other q values, and model each > rule as a different codec; keep the one that performs best. > > In the compare-codecs tool, see the codecs "vp8-mpeg" and "vp8-mpeg-1d" > for examples - one uses three constant quantizers, the other one wraps > the three into one parameter and a formula. > > > Constant Quality: > > Here the codecs are set to use all features available within temporal > > constraints. For stored video, this means full lookahead (or two pass, > > if that's what the codec uses). For realtime, this means no lookahead, > > but otherwise all quality-oriented rate control is allowed. Example rate > > control parameters: > > > > VP9 parameters: > > vpxenc --codec=vp9 --cq-level=$x --end-usage=q > > > > x264 parameters: > > x264 --crf=$x > > > > x265 parameters: > > x265 --crf=$x > > What's the value of this comparision, given that there's no uniformity > among the codecs on what the "cq-level" / "crf" means? > > It might work well if we only use it to generate a spread of encoding > results that can be plotted on a bitrate/quality graph, but will not > work at all if we start saying "for x from 15 to 35, do this" - since > there's no uniformity in the range or scaling form for the "quality" > parameter. > > > > > Constant Bitrate: > > This is the hardest category to compare, as it means that all of the CBR > > implementations need to be set to have the same buffer constraints. It > > is an important use case, so I think we will need to figure out a way to > > test it. I'd be okay with adding this one later, though. > > This is the default mode (ie the one I've been spending the most time > with) on compare-codecs. I'd argue that we should be setting some kind > of constraint on what we're doing (such as "encoding process uses no > more than X MB of RAM), but otherwise leave each codec to optimize > within those constraints - for my tests, I've not put any controls on > buffering strategies so far. > > The argument I'd make here is that setting a rate should produce at most > (roughly) that rate - that is, we need to have some means of removing > configurations where the bitrate grows beyond reasonable. > > Most codecs (especially in 1-pass) have problems hitting their bitrate > targets on the short test clips; production usage usually assumes that > bitrate can be averaged over longer periods. (Note: I'm not sure this is > actually a reasonable assumption for a DASH environment.) > > > > > _______________________________________________ > > video-codec mailing list > > video-codec@ietf.org > > https://www.ietf.org/mailman/listinfo/video-codec > > > > _______________________________________________ > video-codec mailing list > video-codec@ietf.org > https://www.ietf.org/mailman/listinfo/video-codec > -- Mohammed Raad, PhD. Partner RAADTECH CONSULTING P.O. Box 113 Warrawong NSW 2502 Australia Phone: +61 414451478 Email: mohammedsraad@raadtech.com --f46d0442861c93799205127e0da2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
It's interesting that you mention a DASH environment a= t the end - shouldn't this use case form the basis of a test scenario?<= /div>

On Mon, Mar = 30, 2015 at 6:16 AM, Harald Alvestrand <harald@alvestrand.no> wrote:
I know I'm being contrari= an....

what's the model you are trying to construct, that we perform
comparisions on?

Is it one where you generate a PSNR/rate graph for each codec, you need
one parameter to vary to make a plot, and you want to have a parameter
that has roughly the same influence on all codecs?

Den 26. mars 2015 00:19, skrev Thomas Daede:
> I am updating the scripts that AreWeCompressedYet uses. I would like t= o
> select what rate control options are used in the comparison. This is f= or
> purposes of absolute comparison between different codecs, and to
> indicate when a particular performance point is reached. This is not f= or
> adding tools to an existing codec. There are a couple of different rat= e
> control modes that we can use to compare:
>
> Constant Quantizer:
> This is not as simple as it might sound. Should referenced frames stil= l
> be allowed a boost in this mode? Should adaptive quantization be
> allowed? Can quantization matrices be switched? I tried to come up wit= h
> a sane way to compare different codecs with a constant quantizer and w= as
> not successful.

While I believe that constant quantizer is useless, I don't see = a
problem here. If you need a single set of points, ask that there be a
rule for deriving all the q values from other q values, and model each
rule as a different codec; keep the one that performs best.

In the compare-codecs tool, see the codecs "vp8-mpeg" and "v= p8-mpeg-1d"
for examples - one uses three constant quantizers, the other one wraps
the three into one parameter and a formula.

> Constant Quality:
> Here the codecs are set to use all features available within temporal<= br> > constraints. For stored video, this means full lookahead (or two pass,=
> if that's what the codec uses). For realtime, this means no lookah= ead,
> but otherwise all quality-oriented rate control is allowed. Example ra= te
> control parameters:
>
> VP9 parameters:
> vpxenc --codec=3Dvp9 --cq-level=3D$x --end-usage=3Dq
>
> x264 parameters:
> x264 --crf=3D$x
>
> x265 parameters:
> x265 --crf=3D$x

What's the value of this comparision, given that there's no = uniformity
among the codecs on what the "cq-level" / "crf" means?<= br>
It might work well if we only use it to generate a spread of encoding
results that can be plotted on a bitrate/quality graph, but will not
work at all if we start saying "for x from 15 to 35, do this" - s= ince
there's no uniformity in the range or scaling form for the "qualit= y"
parameter.

>
> Constant Bitrate:
> This is the hardest category to compare, as it means that all of the C= BR
> implementations need to be set to have the same buffer constraints. It=
> is an important use case, so I think we will need to figure out a way = to
> test it. I'd be okay with adding this one later, though.

This is the default mode (ie the one I've been spending the most= time
with) on compare-codecs. I'd argue that we should be setting some kind<= br> of constraint on what we're doing (such as "encoding process uses = no
more than X MB of RAM), but otherwise leave each codec to optimize
within those constraints - for my tests, I've not put any controls on buffering strategies so far.

The argument I'd make here is that setting a rate should produce at mos= t
(roughly) that rate - that is, we need to have some means of removing
configurations where the bitrate grows beyond reasonable.

Most codecs (especially in 1-pass) have problems hitting their bitrate
targets on the short test clips; production usage usually assumes that
bitrate can be averaged over longer periods. (Note: I'm not sure this i= s
actually a reasonable assumption for a DASH environment.)

>
> _______________________________________________
> video-codec mailing list
> video-codec@ietf.org
> https://www.ietf.org/mailman/listinfo/video-codec
>

_______________________________________________
video-codec mailing list
video-codec@ietf.org
https://www.ietf.org/mailman/listinfo/video-codec



--
=
Mohammed Raad, PhD.
Partner
RAADTECH C= ONSULTING
P.O. Box 113
Warrawong
NSW 2502 Australia
Phone: +61 = 414451478
Email: mohammedsraad@raadtech.com
--f46d0442861c93799205127e0da2-- From nobody Mon Mar 30 10:19:17 2015 Return-Path: X-Original-To: video-codec@ietfa.amsl.com Delivered-To: video-codec@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2F1401A8AA1 for ; Mon, 30 Mar 2015 10:19:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.601 X-Spam-Level: X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GMwoTIOz6oXx for ; Mon, 30 Mar 2015 10:19:12 -0700 (PDT) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D54521A8AF2 for ; Mon, 30 Mar 2015 10:19:12 -0700 (PDT) Received: by pdbni2 with SMTP id ni2so181832381pdb.1 for ; Mon, 30 Mar 2015 10:19:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=WyPDJNuvdwO4i831BPvriuK/n97XGR0iUU1tOVejUPs=; b=MSLeNNPTScBypPIRSWMXq0CzaTJBouZyqvc4jVY+KohSAxwgBxSnwU+rHq46MX3YnA WH78tqYmFHDY5fh/GbX1wSEe5S5ExiB6DBXwhktbpKk6WOuTX+M0ljaKqmqplDnBpIev nIdS2o9ryNmtgCPJr3r/ZZPWoQJk2hSTu724slsRYyPxgHVrje7h3N7p4uZy2duO96Su /JhujNBXthPAtZr3khwC9vQtfkepvGtJZfnORoKVD4im/xprtIgvBqxd6QrfA+7JCcLP HvUrdbKEdyGnEeir+aoO8x4+wg/3czqEXHloaKvfrBMYbzv2YJNmO8HQtl8B9A0mzOgA G8DA== X-Gm-Message-State: ALoCoQmyeKH+LZ0rPPE6z8hL/yst1waQHodv6GhDBOSPkHfWzpYBwoYUBNBMDRao/HwJ6oD6jZMg X-Received: by 10.66.100.165 with SMTP id ez5mr18547581pab.153.1427735952328; Mon, 30 Mar 2015 10:19:12 -0700 (PDT) Received: from ?IPv6:2620:101:80fc:224:7e7a:91ff:fe9e:8126? ([2620:101:80fc:224:7e7a:91ff:fe9e:8126]) by mx.google.com with ESMTPSA id dh3sm11268650pdb.69.2015.03.30.10.19.10 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 10:19:11 -0700 (PDT) Message-ID: <5519858D.30200@mozilla.com> Date: Mon, 30 Mar 2015 10:19:09 -0700 From: Thomas Daede User-Agent: http://a.pomf.se/cszdno.opus MIME-Version: 1.0 To: Harald Alvestrand , video-codec@ietf.org References: <55134294.2030803@mozilla.com> <5518C016.7030309@alvestrand.no> In-Reply-To: <5518C016.7030309@alvestrand.no> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Archived-At: Subject: Re: [video-codec] Command-line options for encoder comparisons X-BeenThere: video-codec@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Video codec BoF discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 17:19:16 -0000 On 03/29/2015 08:16 PM, Harald Alvestrand wrote: > I know I'm being contrarian.... > > what's the model you are trying to construct, that we perform > comparisions on? > > Is it one where you generate a PSNR/rate graph for each codec, you need > one parameter to vary to make a plot, and you want to have a parameter > that has roughly the same influence on all codecs? Yes, that's the model I was trying to construct. Don't worry about being contrarian, I don't have any strong opinions about this. > In the compare-codecs tool, see the codecs "vp8-mpeg" and "vp8-mpeg-1d" > for examples - one uses three constant quantizers, the other one wraps > the three into one parameter and a formula. Thanks - I looked in compare-codecs before posting, but didn't realize what those were for until now. > What's the value of this comparision, given that there's no uniformity > among the codecs on what the "cq-level" / "crf" means? > > It might work well if we only use it to generate a spread of encoding > results that can be plotted on a bitrate/quality graph, but will not > work at all if we start saying "for x from 15 to 35, do this" - since > there's no uniformity in the range or scaling form for the "quality" > parameter. Right, I was planning to crop the graphs to a certain bitrate range afterwards. This would be less important if the rate control was always precise in hitting the specified file size, which I imagine it is for 2 pass. Am I correct in assuming that the vp9 cq-level command will give identical quality to target-bitrate in 2 pass mode? > This is the default mode (ie the one I've been spending the most time > with) on compare-codecs. I'd argue that we should be setting some kind > of constraint on what we're doing (such as "encoding process uses no > more than X MB of RAM), but otherwise leave each codec to optimize > within those constraints - for my tests, I've not put any controls on > buffering strategies so far. Right, I looked at compare-codecs, but you also allow 2 pass. This is essentially having an unlimited buffer, and as far as I am aware, is equivalent to "constant quality". I guess what I was going for in this section was for realtime streaming where you have to hit a bitrate target in a very short window with little or no lookahead. > The argument I'd make here is that setting a rate should produce at most > (roughly) that rate - that is, we need to have some means of removing > configurations where the bitrate grows beyond reasonable. Youtube seems to use libvpx's cq mode, which is constant quality, but enforces a maximum bitrate over some window. Is that what you're thinking of? It seems the most reasonable for a DASH-like scenario.