CDEF: Do not filter chroma if subsampling_x != subsampling_y
Since CDEF looks uses the luma direction for chroma, CDEF would have
to change significantly to support formats like 4:2:2. The limited
use of such formats does not justify the complexity to support this,
so the simple solution is to mandate that the chroma planes aren't
filtered if subsampling_x != subsampling_y. Most of the visual gain
is in luma, anyway.
This also means that the chroma strengths and chroma skip condition
shall not be sent if subsampling_x != subsampling_y.
BUG=aomedia:720
Change-Id: I35c184a6fe0908ae0fee1e74494b6904fa9a3c82
diff --git a/av1/encoder/bitstream.c b/av1/encoder/bitstream.c
index 21e3c19..cfa6d07 100644
--- a/av1/encoder/bitstream.c
+++ b/av1/encoder/bitstream.c
@@ -3407,7 +3407,8 @@
aom_wb_write_literal(wb, cm->cdef_bits, 2);
for (i = 0; i < cm->nb_cdef_strengths; i++) {
aom_wb_write_literal(wb, cm->cdef_strengths[i], CDEF_STRENGTH_BITS);
- aom_wb_write_literal(wb, cm->cdef_uv_strengths[i], CDEF_STRENGTH_BITS);
+ if (cm->subsampling_x == cm->subsampling_y)
+ aom_wb_write_literal(wb, cm->cdef_uv_strengths[i], CDEF_STRENGTH_BITS);
}
}
#endif
