I had the same thing. Its not dirt or rust. It's stuff they put between the panel joins at the back. It seems to ooze out around the joins, presumably at joining time, and they just paint over it. It's soft and squishy and reminiscent of cork or neoprene, and has that odd brown color. It's easy to break or knock off on account of its softness, even when covered by paint.
Mine came like yours with some broken off exposing the brown. I complained about it at service along with the various other defects my car came with, and they just put touchup paint over it again.