tree 4bc3a9c68049648de5a1d2a756791d2fa4e175f4
parent 8295883e3df5e94ba80442645337fe9de0cb8dbe
author Ruiling Song <ruiling.song@intel.com> 1561512799 +0800
committer Fyodor Kyslov <kyslov@google.com> 1562093967 +0000

AV1 RT: Use correct block-size in estimate_block_intra()

This patch fix a segment fault introduced by accessing an unaligned
memory address in aom_subtract_block_sse2().

Consider a block-size of 16x8. We will choose a TX size of 8x8 in
av1_foreach_transformed_block_in_plane(). Then iterate over the two
transform blocks of size 8x8. But when visiting the second 8x8
transform block, we obviouly only get a 8-byte-aligned block starting address.
But the plane_bsize is still 16x8, which means we are telling the aom_subtract_block()
to do subtraction on a whole 16x8 block, which is obviously wrong.
When the program execution goes inside aom_subtract_block_sse2,
it would fall into '.case_16', which assume a 16-byte-aligned block starting
address. Then crash happens when accessing the unaligned address.

BUG=aomedia:2417

Change-Id: I914bbb2754307dead719970b9901adf7dd521274