optimize_b_greedy: Avoid left shift of negative value. Before: ------- accu_error was always <= 0: - It was initialized to zero, and - Became more and more negative through the iterations. This meant that RDOPT() macro usage on accu_error was causing left shift of negative value. Now: ---- - accu_error is initialized to a large positive value. - It becomes smaller on every iteration but remains >= always. This is functionally equivalent: verified that there's no change in output due to this patch. BUG=aomedia:600 Change-Id: I654572a72234c588c8f332d5b66a5587fc6610b4
diff --git a/av1/encoder/encodemb.c b/av1/encoder/encodemb.c index c2a99fb..0f1b799 100644 --- a/av1/encoder/encodemb.c +++ b/av1/encoder/encodemb.c
@@ -218,7 +218,9 @@ const int ctx0 = ctx; /* Record the r-d cost */ int64_t accu_rate = 0; - int64_t accu_error = 0; + // Initialized to the worst possible error for the largest transform size. + // This ensures that it never goes negative. + int64_t accu_error = ((int64_t)1) << 50; rate0 = get_token_bit_costs(*(token_costs_ptr + band_translate[0]), 0, ctx0, EOB_TOKEN); @@ -413,6 +415,7 @@ token_cache[rc] = av1_pt_energy_class[t0]; } + assert(accu_error >= 0); x_prev = qcoeff[rc];