The mileage counts, so it can be a good way to increase your score. However, negative factors are based on the percentage of time they happen while you are manually driving. So if you have AP on 90% of the time and have a "hard brake" during your manual driving, it'll ding you more than if you manually drove the whole time.