Fix some rdopt bugs in ext-inter

* Refactor write_inter_mode and cost_mv_ref slightly

* In handle_inter_mode: When discount_newmv_test returns 1,
  ext-inter is enabled, and we're costing up a compound mode,
  we would previously call cost_mv_ref with mode=NEARESTMV and
  comp_pred=1. But this is inconsistent when ext-inter is enabled!

  Fix this by changing the behaviour of cost_mv_ref to (effectively)
  derive comp_pred from the mode passed in, and switch to
  mode=NEAREST_NEARESTMV in the above case.

* Fix incorrect motion vector costing for some compound modes

* Fix a case where compound blocks were mis-costed. This primarily
  affects ext-inter *without* wedge or compound-segment enabled,
  but should still have some small impact when those experiments
  are enabled.

  The cause of this is that we only spend bits to send the
  interinter compound type if is_any_masked_compound_used(bsize)
  returns true. But the rdopt loop forgot to check this condition
  in one case.

Change-Id: I8b54cb3552f9ec384fd5ea260e12685168fbccc1
1 file changed