fdct4x4_test: fix unsigned overflow

The difference between src and dst will be signed, the error will be
unsigned. The change quiets -fsanitize=integer:
    unsigned integer overflow: 4294967295 * 4294967295

Change-Id: I131cefcc9583ee8a5b98eb5182fd30e9c7237ea0
2 files changed