Use adative number of frames in temporal filtering
Temporal filtering uses a sequence of frames to filter a target frame.
Before this CL, the number of filtering frames is fixed as
`arnr_max_frames` (commonly set as 7).
This CL adaptively changes the number of frames used for filtering. In
particular, if the target frame is with low noise level, we will
increase the number of filtering frames. The reason is that we would
like the filtered frame to fuse context information from more frames
such that it can provide better predictions for more frames in the
following encoding process. Also, using more frames is beneficial for
de-noising. However, we will not increase the number of frames if the
target frame is with high noise level. The reason is that temporal
filtering is based on motion search. Large noise will hinder the search
engine from finding the proper block, and further harm the filtering
performance.
NOTE 1: This adaptive change of number of frames is applicable to both
Key Frames and ARFs. However, we treat them a little bit differently.
More concretely, when the `q` is small enough (especially for lossless
compression), we will NOT increase the number of frames for Key Frame
filtering. The reason is that AV1 does not support adding an overlay to
the Key Frame. Over-filtering may cause the visual quality drop of the
Key Frame. On the contrary, AV1 supports adding an overlay to ARFs.
Hence, it will be much safer to use more frames for ARF filtering.
NOTE 2: When user sets `arnr_max_frames` as 1 to disable temporal
filtering, the number of frames will NOT be changed (i.e., will be kept
as 1). This CL only takes effect when temporal filtering is active.
NOTE 3: The performance drop on lowres2 and midres2 under speed-1 might
be brought back by tuning the hyper-parameters in CL:
https://aomedia-review.googlesource.com/c/aom/+/111381
Experimental results:
Under Speed-4 (two-pass mode):
avg PSNR ovr PSNR SSIM
ugc360p -0.498 -0.681 -0.622
lowres -0.266 -0.358 -0.314
lowres2 0.088 0.047 0.025
midres -0.043 -0.139 -0.082
midres2 0.019 -0.024 -0.006
hdres -0.356 -0.446 -0.301
hdres2 -0.029 -0.064 -0.032
Under Speed-1 (two-pass mode):
avg PSNR ovr PSNR SSIM
ugc360p -0.428 -0.570 -0.531
lowres -0.212 -0.296 -0.290
lowres2 0.199 0.162 0.124
midres -0.121 -0.122 -0.031
midres2 0.118 0.091 0.109
hdres -0.390 -0.485 -0.375
hdres2 0.002 -0.037 -0.040
STATS_CHANGED
Change-Id: I6d36de6b5dc8e085a262d6e1487f5ed4098ade75
diff --git a/av1/encoder/encode_strategy.c b/av1/encoder/encode_strategy.c
index 3f01b4e..ba896e6 100644
--- a/av1/encoder/encode_strategy.c
+++ b/av1/encoder/encode_strategy.c
@@ -884,7 +884,7 @@
frame_params->frame_type == KEY_FRAME &&
oxcf->enable_keyframe_filtering && !is_stat_generation_stage(cpi) &&
!frame_params->show_existing_frame &&
- cpi->rc.frames_to_key > TF_NUM_FILTERING_FRAMES_FOR_KEY_FRAME &&
+ cpi->rc.frames_to_key > cpi->oxcf.arnr_max_frames &&
!is_lossless_requested(oxcf) && oxcf->arnr_max_frames > 0;
if (apply_filtering) {
const double y_noise_level = av1_estimate_noise_from_single_plane(
