vp10_inv_txfm2d_test: normalize max_error type

quiets double -> int conversion warning

Change-Id: Ic860d187bc77e18b277eef28310feee1899cdbe6
1 file changed