Temporary fix for 4X8 block intra prediction. Currently the RD loop traverses 4X8 blocks in inverted N order while the bitstream stores blocks smaller than 8x8 in Z order. This causes a discrepancy where the RD loop reads uninitialized data while performing intra prediction. As a temporary fix simply disable the use of the extended right edge for 4X8 blocks, until the bitstream can be changed to match the logical structure of the blocks. Change-Id: I44a9e4fc1a15cd551a7b38c3c1227bc5dac77e9a
diff --git a/av1/common/reconintra.c b/av1/common/reconintra.c index f573334..668892a 100644 --- a/av1/common/reconintra.c +++ b/av1/common/reconintra.c
@@ -83,6 +83,14 @@ int ss_x) { if (!right_available) return 0; + // TODO(bshacklett, huisu): Currently the RD loop traverses 4X8 blocks in + // inverted N order while in the bitstream the subblocks are stored in Z + // order. This discrepancy makes this function incorrect when considering 4X8 + // blocks in the RD loop, so we disable the extended right edge for these + // blocks. The correct solution is to change the bitstream to store these + // blocks in inverted N order, and then update this function appropriately. + if (bsize == BLOCK_4X8 && y == 1) return 0; + if (y == 0) { int wl = mi_width_log2_lookup[bsize]; int hl = mi_height_log2_lookup[bsize];