tree 41f92dc47fe287b59f04a1e8a58d056d68380daa
parent 36042696002856ab98f477991d5df424b81ffdf4
author Rachel Barker <rachelbarker@google.com> 1673293478 +0000
committer Rachel Barker <rachelbarker@google.com> 1674078748 +0000

Fix bug in global motion model refinement

When refining a global motion model, we change one model parameter at a
time and re-calculate the prediction. For TRANSLATION and AFFINE type
models, we can just change one matrix element at a time, but for ROTZOOM
models we need to change two at a time because the constraints that
wmmat[4] = -wmmat[3] and wmmat[5] = wmmat[2].

This constraint was being applied in warp_plane()/highbd_warp_plane(),
but unfortunately the shear parameters (alpha, beta, gamma, delta),
which are used internally by the warp filter, were being calculated
before the constraint was applied.

This meant that, for ROTZOOM type models, the predictions used during
global motion refinement search were just slightly wrong, leading to
suboptimal decisions.

This patch fixes the issue by applying the ROTZOOM constraints in
av1_warp_error(), just before the shear parameters are calculated.

Causes noise-level changes to both BDRATE and encode time

STATS_CHANGED

Change-Id: Ib1b2b9cea50b0df83d4ee111786c5cb6e4cc6ebb