tree bc4bdedc78a4757293f2a12ce13d9a6cb275a284
parent 18d44f560fa791d8840dedb5d93684d18b44cbfd
author Rachel Barker <rachelbarker@google.com> 1680628436 +0000
committer Rachel Barker <rachelbarker@google.com> 1680629408 +0000

Fix size of warp ref list

The warp ref list (WRL) and other related arrays were sized for
SINGLE_REF_FRAMES many entries. This value seems to be intended for
use with the compacted ref frame set (see COMPACT_INDEX0_NRS), which
includes intra and the TIP frame as possible references.

However, the WRL is only intended to be used with "true" single inter
refs at the moment, so these arrays only need to be sized for
INTER_REFS_PER_FRAME many entries.

As well as saving space, this fixes a bug caused by a mixup of the
compacted single ref set constructed by COMPACT_INDEX0_NRS,
and the combined ref index constructed by av1_ref_frame_type().

Specifically, the line
    derive_wrl &= (ref_frame < SINGLE_REF_FRAMES);
ends up allowing the WRL to be derived for the TIP frame and for
compound blocks with the ref pair (0, 1). This behaviour is not
intended, and ends up leading to an out-of-bounds read of
xd->global_motion, since that only has INTER_REFS_PER_FRAME
many entries.

Note: As the modes which use the WRL are only enabled for single-ref
inter blocks, this does not affect the encoder or decoder output.