Reset tx_type in recursive transform block partitioning Temporarily reset the transform type in the inter modes when recursive transform block partitioning is used. This resolves an enc/dec mismatch issue in nextgenv2 codebase when both var-tx and ext-tx experiments are turned on. Change-Id: I2543f0a567243da95b237752d46964b07b669ad9
diff --git a/vp10/encoder/rdopt.c b/vp10/encoder/rdopt.c index dfec9b4..43eeb06 100644 --- a/vp10/encoder/rdopt.c +++ b/vp10/encoder/rdopt.c
@@ -1442,6 +1442,10 @@ *sse = 0; *skippable = 1; +#if CONFIG_EXT_TX + xd->mi[0]->mbmi.tx_type = DCT_DCT; +#endif + if (is_cost_valid) { const struct macroblockd_plane *const pd = &xd->plane[0]; const BLOCK_SIZE plane_bsize = get_plane_block_size(bsize, pd);