tree 0f501e6538a675a5684785dbc18b6a1b415b091b
parent 7aabe9d093746c7b5e23aa8a034695f7817af98c
author Yunqing Wang <yunqingwang@google.com> 1547866122 -0800
committer Yunqing Wang <yunqingwang@google.com> 1548182118 +0000

Cost calculation for out of range probability

In some situations, the cdf probability can be out of range
[1, CDF_PROB_TOP - 1]. Then, corresponding cost calculation
would be wrong. This CL clamps the probabilty to resolve the
issue.

Ran Borg tests on lowres, midres, and hdres, and average PSNR
change was 0. Saw slight PSNR changes for following 4 files.
bridge_far_cif.y4m          -0.003
into_tree_480p.y4m          -0.006
station2_480p25.y4m          0.004
tears_of_steel2_480p.y4m     0.001

STATS_CHANGED

BUG=aomedia:2297

Change-Id: Ia725af7edda830c46f7cced4a8e3a1d49559cbbf