You would think so, but buffers are actually a core part of regular router function. The problem is not inherently router buffers, but rather buffers that don't fully drain (bufferbloat) or that fill up (congestive drops).
Think about it this way: a 100Gbps link can process a 1500 byte packet in about 0.12 microsecond. If you have an average of 1,000 packets in that buffer at steady-state, that buffer is contributing a fraction of a millisecond to your overall latency. Meanwhile, if your home router's buffer has the same 1,000 packets for a 1Gbps link, that's 12 milliseconds of latency.
If you had a way to tell a router to never buffer packets, you'd encounter packet loss much sooner, and for no good reason. Buffers are great for smoothing out (small) bursts of traffic (although if you have large bursts on very short timescales, you can easily overwhelm these buffers -- this is why you typically have pacing on the OS level).
If instead you want the router to add no more than X amount of latency, that's suddenly much harder to dictate. (The "correct" value of X depends on your application requirements as well as the number of hops through the network between you and the server you're talking to.)
(Also, while you might think that this should be strictly beneficial for network operators since the user wants fewer resources, encoding exceptions like this into router policy ends up using more of a specialized and expensive kind of memory called TCAM. Assuming that it's even possible to do such a thing.)
Think about it this way: a 100Gbps link can process a 1500 byte packet in about 0.12 microsecond. If you have an average of 1,000 packets in that buffer at steady-state, that buffer is contributing a fraction of a millisecond to your overall latency. Meanwhile, if your home router's buffer has the same 1,000 packets for a 1Gbps link, that's 12 milliseconds of latency.
If you had a way to tell a router to never buffer packets, you'd encounter packet loss much sooner, and for no good reason. Buffers are great for smoothing out (small) bursts of traffic (although if you have large bursts on very short timescales, you can easily overwhelm these buffers -- this is why you typically have pacing on the OS level).
If instead you want the router to add no more than X amount of latency, that's suddenly much harder to dictate. (The "correct" value of X depends on your application requirements as well as the number of hops through the network between you and the server you're talking to.)
(Also, while you might think that this should be strictly beneficial for network operators since the user wants fewer resources, encoding exceptions like this into router policy ends up using more of a specialized and expensive kind of memory called TCAM. Assuming that it's even possible to do such a thing.)