tree 28d865742679c95201729f18c9fafbbd91fb5cd6
parent 2754f83f65eb57fb9aff7029f355ad57421b57f3
author Wan-Teh Chang <wtc@google.com> 1535668487 -0700
committer Wan-Teh Chang <wtc@google.com> 1536078184 +0000

Record whether mc_buf[i] stores highbd pointers.

Add a mc_buf_use_highbd boolean field to the ThreadData struct to record
whether the byte pointers stored in the mc_buf array were converted from
highbd pointers. This allows us to free the old pointers in mc_buf array
correctly when the use_highbitdepth field of the SequenceHeader struct
changes. As a result, we don't need to pass the use_highbd argument to
av1_free_mc_tmp_buf().

Add a test case to test/invalid_file_test.cc. This test case consists of
the first two valid IVF frames in the reproducer test case for
oss-fuzz:10117. Without this fix, the test crashes with a segmentation
fault in av1_free_mc_tmp_buf() because it passes an invalid pointer to
aom_free().

Tested:
cmake ../aom -DCONFIG_LOWBITDEPTH=1
make
make testdata
./test_libaom --gtest_filter=*InvalidFileTest*

BUG=oss-fuzz:10117

Change-Id: Ie1c1010771ccd4fcb5ffba3dac71ba9c296ba85e