Grindelwald's been trying to seduce Graves ever since he captured him. He doesn't rape him or threaten to withhold necessities for sex. Graves was actually given a fairly comfortable prison and wasn't tortured during his captivity. After the film, Grindelwald continues his pursuits, sending him letters from prison, then when he escapes, flowers and chocolates. Graves isn't budging. One day, though, Grindelwald makes his move on MACUSA. He has the whole building captive and is about to kill one of Graves's friends, and Graves tries to bargain with him, saying he'll sleep with Grindelwald if Grindelwald doesn't kill anyone.

This makes Grindelwald angry because coerced sex is just another phrase for rape and he doesn't want Percival that way. Hearing this might not make Graves want to give in to Grindelwald's advances, but it definitely makes Graves respect the man more. And who knows, maybe that respect ends up leading to....up to potential filler.