tree 5e89fc19aba6fecc526f26b7b5d506cf63a9a71e
parent f127b3f3134ae9a7637ed6d289ed466d0ad63556
author Wan-Teh Chang <wtc@google.com> 1534941566 -0700
committer Wan-Teh Chang <wtc@google.com> 1535054038 +0000

Bound the total allocated memory of frame buffers.

By setting a debugger breakpoint in aom_memalign() (called by both
aom_malloc() and aom_calloc()) for 'size' greater than 10 MB, I found
that the frame buffer pool is one of the two biggest consumers of heap
memory in the decoder. This CL tries to bound the memory consumption of
the frame buffer pool. We could define a new macro for the maximum frame
buffer pool size, but for now I am borrowing the existing
AOM_MAX_ALLOCABLE_MEMORY macro for this purpose.

Tested:
cmake ../aom -DCONFIG_PIC=1 -DCONFIG_LOWBITDEPTH=1 -DCONFIG_SIZE_LIMIT=1 -DDECODE_HEIGHT_LIMIT=12288 -DDECODE_WIDTH_LIMIT=12288 -DAOM_EXTRA_C_FLAGS="-DAOM_MAX_ALLOCABLE_MEMORY=536870912 -DDO_RANGE_CHECK_CLAMP=1"
make
./test_libaom --gtest_filter=*TestVectorTest*

BUG=oss-fuzz:9678
BUG=oss-fuzz:9697
BUG=oss-fuzz:9956
BUG=oss-fuzz:9992
BUG=oss-fuzz:9993

Change-Id: Iab894920bb5e85aa13d92598fd18e88c84ae64fd