Not sure if guys got this handled or not but normally there is a little clip that holds them on, there is a special tool to remove them.
There is a trick with a shop towel where you basically "floss" between the door skin and the back of the handle. Get the towel down between the two and work it back and forth while working your way around the handle, hoping to catch the clip just right.
It's difficult to explain well, there may be a youtube video of it...like this one: