So, would this residual dirt in the cloth be enough to scratch the paint?
Unfortunately IMO the answer is yes. I would not use a MF towel soiled like that. Too much of a risk of inducing small scratches.
Another option would be to put the dirty towels in a bucket to soak and just leave them soaking until you have enough to do a small load.