Make the first cdf prob < CDF_PROB_TOP to avoid infinite loop The probability training tool generates all CDF_PROB_TOP table if the counts are all zero. Although it means such cases are never used in actual encoding, in rd cost pre-calculation, we still sometimes run into such cases and the token cost calcualtion function cannot handle all-CDF_PROB_TOP cdfs (Will trigger assertion when debug mode is on). Change-Id: I259620e210fea0f6e1f7848724fa006fa06da226
diff --git a/tools/aom_entropy_optimizer.c b/tools/aom_entropy_optimizer.c index 8acdc0d..3108a0d 100644 --- a/tools/aom_entropy_optimizer.c +++ b/tools/aom_entropy_optimizer.c
@@ -212,6 +212,7 @@ else cdf[i] = (csum[i] * CDF_PROB_TOP + round_shift) / sum; } + if (sum <= 0) cdf[0] = CDF_PROB_TOP - 1; return 0; }