> Plus all of those 1GB+ models won't fit into a phone anyway.

I don't think that's a requirement here. I've been playing with Whisper tonight, and even the tiny model drastically outperformed Siri dictation for me in my testing. YMMV, of course.