commit 930118c5c551025934d5e775c325b91f95f92da9
author Guillaume Martres <smarter@ubuntu.com> Tue Dec 06 20:26:22 2016 -1000
committer Guillaume Martres <smarter3@gmail.com> Thu Dec 15 20:37:04 2016 +0000
tree d0845181f6c4e97c2de38a5e7381c01b8df92cfb
parent 6dbacf285939dd3dce313376b5e418b2c03be4d6

PVQ: Fix incorrect calculation of rd_stats

When PVQ is on, we reencode at the end of choose_tx_size_type_from_rd to
get the entropy contexts right, previously this was done using
txfm_rd_in_plane but this is different from the encodes done in the loop
which use txfm_yrd, the result is that rd_stats is set incorrectly at
the end of choose_tx_size_type_from_rd when PVQ is on.

Results on objective-1-fast with --limit=5:

   PSNR | PSNR Cb | PSNR Cr | PSNR HVS |    SSIM | MS SSIM | CIEDE 2000
-0.5803 | -1.0598 | -1.4565 |  -0.3377 | -0.8153 | -0.5934 |    -0.9943

See https://goo.gl/Hvv0E2

Change-Id: Iccc7b0afaff849f959a0084eb48dbb838bc3cb1a