[NORMATIVE-DECODING, intra-edge] Fix bug in is_smooth() Because the mbmi pointer passed into is_smooth comes from the above/left block, it might be an inter block. If this happens, we correctly deduce that the above/left block does not use a smooth intra mode. However, inter blocks do not set mbmi->uv_mode, so in the UV case we end up reading stale data. This may result in is_smooth() returning the wrong value, if (whatever was previously written into) mbmi->uv_mode happens to be a smooth intra mode. Fix this by including an explicit check for inter blocks. BUG=aomedia:1362 Change-Id: I3ec9faef9b6297e22915176067b5704003bc4664
diff --git a/av1/common/reconintra.c b/av1/common/reconintra.c index 5e1b5de..b9d266f 100644 --- a/av1/common/reconintra.c +++ b/av1/common/reconintra.c
@@ -1398,6 +1398,10 @@ return (mode == SMOOTH_PRED || mode == SMOOTH_V_PRED || mode == SMOOTH_H_PRED); } else { + // uv_mode is not set for inter blocks, so need to explicitly + // detect that case. + if (is_inter_block(mbmi)) return 0; + const UV_PREDICTION_MODE uv_mode = mbmi->uv_mode; return (uv_mode == UV_SMOOTH_PRED || uv_mode == UV_SMOOTH_V_PRED || uv_mode == UV_SMOOTH_H_PRED);