Yes, the magnet will come off the arm. Most of them are glued on...get a flat head screw driver and something to hit it with...put the flathead screwdriver up against the side of the magnet and chisel away at the bottom where it meets the arm with a few good hits it should pop off and break the glue bond.
And yes, you need the magnet off the arm. It would be impossible to balance otherwise. I dont think you need to put them on a washer if its a harddrive magnet...some people are using much stronger rare earth magnets. Mine works fine without any modification