gainmap: check avifImageCreateEmpty() result
avifRGBImageApplyGainMap() calls avifImageCreateEmpty() to back the
rescaledGainMap when the gain map image's dimensions differ from the
base image's, but does not check the return value. The next line
calls avifImageSetViewRect(rescaledGainMap, ...), which dereferences
the destination image via avifImageFreePlanes() before any rect
validation, so a NULL return from avifImageCreateEmpty() crashes the
caller under memory pressure.
This is the same pattern fixed for avifImageCopy() in PR #3201.
Bail to the existing cleanup label with AVIF_RESULT_OUT_OF_MEMORY
when the allocation fails.
diff --git a/src/gainmap.c b/src/gainmap.c
index f56484d..b35734f 100644
--- a/src/gainmap.c
+++ b/src/gainmap.c
@@ -187,6 +187,10 @@
if (gainMap->image->width != width || gainMap->image->height != height) {
rescaledGainMap = avifImageCreateEmpty();
+ if (rescaledGainMap == NULL) {
+ res = AVIF_RESULT_OUT_OF_MEMORY;
+ goto cleanup;
+ }
const avifCropRect rect = { 0, 0, gainMap->image->width, gainMap->image->height };
res = avifImageSetViewRect(rescaledGainMap, gainMap->image, &rect);
if (res != AVIF_RESULT_OK) {