Your statement that we will never get to the bottom may be true, but the reasoning is flawed.
Gödel's incompleteness theorems are about mathematical theories, not about physical theories. It may well be that for physical understanding a very limited arithmetic is sufficient. Also, there may be statements that are unprovable, but these might turn out to be irrelevant to sufficiently understanding the universe.
Also, an axiom, by definition, can not be proven. The things that are proven are theorems, statements, etc. An axiom is an assumption.
Eventually, it all depends on what level of understanding you want. But Gödel's theorems have little to do with that.
From the mathematical point of view, physics is about finding smallest set of axioms from which you can mathematically derive complete theory of universe's behavior. Validity of such axioms is in turn validated by experiments (by testing whether results match theory derived from given axioms)
It is true that they will never be "proven", but that doesn't mean we can't build a tremendous confidence in them through experimentation. (Which is to say "I agree", but I wanted to put a slightly different spin on it.)
The incompleteness theorem describes a tradeoff: A system of axioms cannot be both consistent and complete. But that seems irrelevant, since you could make a system that describes all of physics while not being either consistent or complete. https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
Scientific reasoning is inductive, not deductive. When doing inductive reasoning you are allowed to update your belief in your axioms based on the empirical evidence you see.
In fact, the theorem showing how to do so (Bayes' Theorem) is so utterly elementary we teach it in the first probability course everyone takes.
Gödel's incompleteness theorems are about mathematical theories, not about physical theories. It may well be that for physical understanding a very limited arithmetic is sufficient. Also, there may be statements that are unprovable, but these might turn out to be irrelevant to sufficiently understanding the universe.
Also, an axiom, by definition, can not be proven. The things that are proven are theorems, statements, etc. An axiom is an assumption.
Eventually, it all depends on what level of understanding you want. But Gödel's theorems have little to do with that.