It might be a regional dialect. Growing up in the west coast, I have never heard "her-bal tea", but rather always heard "er-bull tea". Hearing the "h" sound in "herb" (as I do whenever I hear British (and Australian?) speakers speak) was always very jarring, until I realized that it was just that they pronounce that class of words differently than we do.
It's possible that in the eastern or southern US, they say the H in "herb".
I don't know that it's a regional variation, but about half of the people in my family pronounce the "h", including me most of the time. My sense is almost that dropping the "h" is an "educated" thing, and those who don't drop it are unconsciously tweaking the whole concept.
