Work around a spec bug in gm_get_motion_vector()
For TRANSLATION type global warp models, gm_get_motion_vector()
returns an incorrect motion vector. As this is a bug in the
specification, we cannot change this logic, but we can work around
it as follows:
* Add a comment explaining the bug
* Change the encoder logic so that we never select a TRANSLATION
type global motion model. If the global motion search process
produces a TRANSLATION type model, we reset to an IDENTITY
type model.
The impact of this change is ~neutral overall, with no video
changing by more than +/- 0.2% BDRATE.
Overall impact on 150 frames, speed 1, const q mode:
lowres2: -0.002% BDRATE, -0.003% instr count
midres2: +0.005% BDRATE, +0.005% instr count
hdres2: -0.001% BDRATE, -0.035% instr count
STATS_CHANGED
Bug: aomedia:3328
Change-Id: Ibbbefa5107739de0c8cdb867901994ef554daee7
diff --git a/av1/encoder/bitstream.c b/av1/encoder/bitstream.c
index 8b23e64..a9e9fdc 100644
--- a/av1/encoder/bitstream.c
+++ b/av1/encoder/bitstream.c
@@ -2681,6 +2681,12 @@
struct aom_write_bit_buffer *wb, int allow_hp) {
const TransformationType type = params->wmtype;
+ // As a workaround for an AV1 spec bug, we avoid choosing TRANSLATION
+ // type models. Check here that we don't accidentally pick one somehow.
+ // See comments in gm_get_motion_vector() for details on the bug we're
+ // working around here
+ assert(type != TRANSLATION);
+
aom_wb_write_bit(wb, type != IDENTITY);
if (type != IDENTITY) {
aom_wb_write_bit(wb, type == ROTZOOM);
