I seem to see more problems when transitioning from wifi to cellular, but it does eventually stabilize. A correlation with connection issues of some kind seems likely.
When it fails, I find that the speech button doesn't always capture all of the audio. I get better results if I push the button, wait for a couple seconds, then talk, then wait, then push the button again to complete. If I don't, my first word often gets cut out, which throws off the whole command. Maybe the last word, too.
I've <push button> <wait> reported this issue <wait> <push button> several times.
Edit: v9 didn't make this worse for me, it's been like this for months.