Rule. Always verify that return pointers from dynamic memory allocation functions (aom_malloc, aom_calloc, aom_memalign, aom_realloc, malloc, calloc, realloc) and opaque resource handles are non-NULL using runtime checks (CHECK_MEM_ERROR, AOM_CHECK_MEM_ERROR, explicit NULL checks, or abort()) rather than debug-only assert() or aom_assert(). In codec routines where an error context (AV1_COMMON *cm or struct aom_internal_error_info *error_info) is available, wrap allocations in CHECK_MEM_ERROR(cm, ptr, expr) or AOM_CHECK_MEM_ERROR(error_info, ptr, expr). On allocation failure, validation error, or early-exit path, explicitly release all locally allocated buffers (aom_free / free) and destroy acquired handles, or route execution through a shared cleanup label (goto fail; or goto cleanup;). Additionally, helper functions that perform dynamic memory allocation or fallible operations must return a status code (e.g., non-zero on success, zero on error, or an aom_codec_err_t) rather than returning void, and all call sites must check and handle the returned status. Never rely on assert() or aom_assert() for allocation checks since assertions are stripped in release builds when NDEBUG is defined.
Why. Dynamic memory allocation and handle creation functions can return NULL upon memory exhaustion or invalid sizing. Dereferencing, indexing, or performing pointer arithmetic on unchecked pointers causes segmentation faults and undefined behavior. Relying on assert() or aom_assert() leaves release builds unprotected because assertions are compiled out when NDEBUG is defined. Using CHECK_MEM_ERROR or AOM_CHECK_MEM_ERROR raises aom_internal_error with AOM_CODEC_MEM_ERROR safely, while explicit runtime NULL checks combined with proper cleanup prevent progressive memory and handle leaks along error paths. Furthermore, returning void from fallible functions causes callers to proceed with uninitialized or invalid buffers, leading to silent data corruption or crashes.
Bad:
// Variant 1: Unchecked allocation or missing cleanup size_t size = width * height * 3; uint8_t *src_rgb = (uint8_t *)aom_malloc(size); uint8_t *dst_rgb = (uint8_t *)aom_malloc(size); I420ToRGB24Matrix(source->y_buffer, ..., src_rgb, dst_rgb); // Variant 2: assert() stripped in release builds with NDEBUG syms = realloc(syms, sizeof(Symbol) * count); assert(syms != NULL); // Variant 3: Leaking handle or void function ignoring allocation error JxlButteraugliApi *api = JxlButteraugliApiCreate(NULL); JxlButteraugliResult *res = JxlButteraugliCompute(api, ...); if (!res) return; // Leaks api handle and caller cannot detect error
Good:
// Variant 1: Verify allocations, free prior buffers on failure size_t size = width * height * 3; uint8_t *src_rgb = (uint8_t *)aom_malloc(size); uint8_t *dst_rgb = (uint8_t *)aom_malloc(size); if (!src_rgb || !dst_rgb) { aom_free(src_rgb); aom_free(dst_rgb); return 0; } // Variant 2: Explicit runtime check or abort() instead of assert syms = realloc(syms, sizeof(Symbol) * count); if (!syms) abort(); // Variant 3: Clean up acquired handle and return fallible status code JxlButteraugliApi *api = JxlButteraugliApiCreate(NULL); JxlButteraugliResult *res = JxlButteraugliCompute(api, ...); if (!res) { if (api) JxlButteraugliApiDestroy(api); return 0; }