Support rectangular tx size in cb4x4 mode
This commit makes ext-tx and rect-tx experiments supported in the
cb4x4 mode. It resolves an enc/dec mismatch issue when all the
transform experiments are enabled.
The coding gains are
ext-tx + rect-tx cb4x4 vartx total
lowres 4.0% 2.3% 0.5% 6.9%
The encoding speed is about the same when cb4x4 and vartx are
further enabled.
BUG=aomedia:139
Change-Id: I3fdabc6d5de23ceb78ac0751a9bf7332ebc0a3ac
diff --git a/av1/common/pred_common.h b/av1/common/pred_common.h
index 75948b6..7825f3c 100644
--- a/av1/common/pred_common.h
+++ b/av1/common/pred_common.h
@@ -184,7 +184,7 @@
int left_ctx = (has_left && !left_mbmi->skip)
? (int)txsize_sqr_map[left_mbmi->tx_size]
: max_tx_size;
- assert(xd->mi[0]->mbmi.sb_type >= BLOCK_8X8);
+
if (!has_left) left_ctx = above_ctx;
if (!has_above) above_ctx = left_ctx;