[CFL] Store luma as prediction for chroma
Stores the reconstructed luma pixels for each transform block inside a
prediction block. Rectangular transform blocks are supported.
As for RDO, after all the modes have been tested for luma, an extra
encoding is perform in order to store the reconstructed pixel values of
the best mode. These values are then used for RDO on the chromatic
planes.
Change-Id: I354d9827e32fd41065f1b2ce02832d943a6fa156
diff --git a/av1/encoder/rdopt.c b/av1/encoder/rdopt.c
index 458f2b4..f13c585 100644
--- a/av1/encoder/rdopt.c
+++ b/av1/encoder/rdopt.c
@@ -3714,6 +3714,16 @@
od_encode_rollback(&x->daala_enc, &post_buf);
#endif // CONFIG_PVQ
+#if CONFIG_CFL
+ // Perform one extra txfm_rd_in_plane() call, this time with the best value so
+ // we can store reconstructed luma values
+ RD_STATS this_rd_stats;
+ x->cfl_store_y = 1;
+ txfm_rd_in_plane(x, cpi, &this_rd_stats, INT64_MAX, 0, bsize,
+ mic->mbmi.tx_size, cpi->sf.use_fast_coef_costing);
+ x->cfl_store_y = 0;
+#endif
+
#if CONFIG_PALETTE
if (try_palette) {
rd_pick_palette_intra_sby(cpi, x, bsize, palette_y_mode_ctx,
