I always remove the calipers. It gives me a chance to spray down the inside of the calipers with brake parts cleaner and brush off the area around the pistons. It's also easier to make sure that the clip at the front of the caliper is in place. I replace the caliper pin when I change pads; you can clean off the old one, but they get a bit rough, and it's not much money and the result seems better to me.