Yeah, Neodymium is no joke. A piece the size of a poker chip could injure you and would be very difficult to remove from steel. Something the size that OP is talking about needs to be in a box and you have to have a good plan before taking it out. That said, it wouldn't affect the car even if you did throw it in there loose, other than the dents, scratches, and severed fingers it collected along the way.