| commit | 8620dea882a7bfce428d995782514f21ec4d90a6 | [log] [tgz] |
|---|---|---|
| author | Alex Converse <aconverse@google.com> | Wed May 04 11:30:00 2016 -0700 |
| committer | Alex Converse <aconverse@google.com> | Wed May 04 11:40:55 2016 -0700 |
| tree | 5867a7ae1429ecbd3e235af8ad32ced33f3b955e | |
| parent | 3c2231807ff1c540d1c0ce5ea13642bc57422055 [diff] |
variance aq: Fix a variance calculation overflow bug. This is an actual overflow where the result of the calculation is materially changed, not just a negative value that is stored in an unsigned. Caught with fsanitize=integer on the VP9/AqSegmentTest.TestNoMisMatchAQ2/1 test. Change-Id: I514b0ef4ae7ad50e3e08c0079aa204d59fa679aa