When Graves was Grindelwald's captive, Grindelwald would never physically force him to have sex with him. He might offer Graves special treats, like being allowed to sleep in a bed instead of on a cot, or obliviating instead of killing one of Graves's coworkers who was getting dangerously close to finding out, but he'd never use the imperius curse or beat Graves into submission. And when he did rape Graves, he'd be fairly gentle.

The wizarding community in America is backwards in several ways. The government reserves the right to arrange marriages in cases where it'll benefit the nation. With Grindelwald growing more and more powerful, they're in need of strong allies. When Graves is told he needs to marry some foreign diplomat, he's far from happy. Still, if it prevents Grindelwald from taking over...He goes through with it, only his new husband is a monster, who beats him and takes him violently, whether Graves resists or not. Graves tries to get a divorce on grounds of cruel treatment, but he's denied because it'll hurt America politically. One day, though, Grindelwald pays him a visit, with an offer to join him. Graves accepts, not because he wants to work for Grindelwald, but because he knows that at least Grindelwald won't physically hurt him as much as his husband and because, at this point, he's fed up with MACUSA.