tree 3f0929b3752171c9ef242c39b6ca32619bbb5bd9
parent 7ee731b0f3cc923fa95406eba2be3dcd85ef5c16
author Debargha Mukherjee <debargha@google.com> 1747846648 +0000
committer Urvang Joshi <urvang@google.com> 1747846648 +0000

F054 fix to setup av1_setup_pred_block() correctly

Fixes #697.

With the integration of F054, a discrepancy arises when fetching reference samples outside the picture boundary. Specifically, the reference samples used during encoder RD (via `av1_setup_pred_block()`) differ from those used during final encoding (via `av1_setup_pre_planes()`). This mismatch is due to a missing update from F054 in `av1_setup_pred_block()`.

As a result, in certain cases—such as when lossless coding is enabled—the encoder sets skip_txfm=1 based on the RD path. However, during final encoding, the prediction differs while the residual is treated as zero, leading to a reconstructed block that is no longer lossless.

This MR aligns the prediction behavior between the RD path and final encoding, ensuring consistency and avoiding incorrect skip_txfm settings.