It's not drift. Time passes more quickly on the moon, to such an extent that there are 56 microseconds more each day on the lunar surface than on Earth. This is a an effect of general relativity.
For everyday human-scale stuff, sure this doesn't matter. For precise scientific and industrial use cases, it does.
I repeat myself: What's the difference between taking a good clock to the moon, where it runs 56 microseconds fast each day, versus having a less-tuned oscillator that runs 56 microseconds fast each day?
Clock slew is unacceptable for many use cases. For keeping system time on a UNIX server or for astronaut watches, sure. For many other things you need a precise and accurate count of how many micro/nano/pico seconds have passed. You can’t just have one second suddenly be N nanoseconds longer and expect sensitive experiments to not be affected.
If you're just talking about measuring e.g. the duration of an experiment, you don't need to sync to TAI for that.
For matching with TAI timestamps, no two atomic clocks agree with each other exactly anyway -- TAI is an average of some 500 atomic clocks. If you care to that level, you have to correct for your elevation on Earth already. So no matter what you do, if you care enough, you'll end up with some correction factor.
For most practical uses, you don't use an atomic clock, just a good steady oscillator, apply a correction factor to its output if you want to match with some other clock source (convert to TAI, etc), and use feedback like NTP to keep your clock on time. The biggest difference between moon and Earth in that is the latency to authoritative clock.
For everyday human-scale stuff, sure this doesn't matter. For precise scientific and industrial use cases, it does.