Correlate od_compute_dist_8x8 with sum of squared error
Piecewise linear fit without activity masking on subset3 intra
and objective-1-fast inter, by simple linear regression.
In combination with 79c0f32c "Remove DCT from od_compute_dist_8x8",
this gives a even trade between perceptual and non-perceptual metrics.
av1_daala_dist_30f@2017-01-23T21:32:27.902Z
-> daala_dist_scale10_30f@2017-01-31T21:52:07.635Z
PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | CIEDE 2000
2.1080 | 2.9645 | 3.4697 | -2.2086 | 0.2541 | -2.5232 | 2.1645
Piecewise linear-quadratic fit with activity masking, by same method.
The total effect of activity masking and daala-dist changes, with PVQ:
av1_pvq_5f@2017-01-31T01:05:24.219Z
-> av1_float_pvq_dist_scale_AM_5f_Jan31_crfix@2017-02-02T15:14:40.477Z
PSNR | PSNR Cb | PSNR Cr | PSNR HVS | SSIM | MS SSIM | CIEDE 2000
22.5041 | 42.6349 | 40.8516 | -10.3510 | -6.8030 | -16.9057 | 21.2613
Change-Id: I9b513509a03aa058dc5c1479c01d62c8fc363a34
diff --git a/av1/encoder/rdopt.c b/av1/encoder/rdopt.c
index 34780d0..e08baa3 100644
--- a/av1/encoder/rdopt.c
+++ b/av1/encoder/rdopt.c
@@ -544,8 +544,6 @@
double sum;
sum = 0;
- (void)qindex;
-
assert(bsize_w >= 8 && bsize_h >= 8);
if (qm == OD_FLAT_QM) {
@@ -594,16 +592,17 @@
bsize_w);
}
}
- /* Compensate for the fact that the quantization matrix lowers the
- distortion value. We tried a half-dozen values and picked the one where
- we liked the ntt-short1 curves best. The tuning is approximate since
- the different metrics go in different directions. */
- /*Start interpolation at coded_quantizer 1.7=f(36) and end it at 1.2=f(47)*/
- // TODO(yushin): Check whether qindex of AV1 work here, replacing daala's
- // coded_quantizer.
- /*sum *= qindex >= 47 ? 1.2 :
- qindex <= 36 ? 1.7 :
- 1.7 + (1.2 - 1.7)*(qindex - 36)/(47 - 36);*/
+ /* Scale according to linear regression against SSE, for 8x8 blocks. */
+ if (activity_masking) {
+ sum *= 2.2 + (1.7 - 2.2) * (qindex - 99) / (210 - 99) +
+ (qindex < 99 ? 2.5 * (qindex - 99) / 99 * (qindex - 99) / 99 : 0);
+ } else {
+ sum *= qindex >= 128
+ ? 1.4 + (0.9 - 1.4) * (qindex - 128) / (209 - 128)
+ : qindex <= 43
+ ? 1.5 + (2.0 - 1.5) * (qindex - 43) / (16 - 43)
+ : 1.5 + (1.4 - 1.5) * (qindex - 43) / (128 - 43);
+ }
}
return sum;
}
