There are chemicals that will turn red rust into something black that, unlike red rust, will not propagate. I believe that the active ingredient of Bar Keepers Friend, Oxalic acid, is one of them.
If the sandpaper still leaves pits showing red rust, I'd consider something like that. You don't want to leave it on there too long; it tends to discolor the metal.
Is that one of those knives with copper in the Damascus cladding, or is everything that looks orange just rust?