Spelled out a little, the argument runs like this: a completely chaotic universe not only would not be amenable to mathematical description, but would not be such that any kind of mathematics could be practised in it, since the practice of mathematics itself requires that it be possible for certain kinds of stable entities and entailments (a stock of symbols, arrangements of those symbols according to a syntax, repeatable procedures of induction and verification) to exist.
The existence of mathematics does not prove that the universe is in essence mathematical, but it does prove that the universe is capable of at least local stability and regularity, of which the stability and regularity of mathematics itself is a demonstrable instance. Phenomena that are amenable to mathematical description are by that token also instances of at least local stability and regularity.
There is no good reason to suppose that the universe is fundamentally chaotic, any more than we are obliged to suppose that the universe is fundamentally stable and regular. Chaos is not necessarily degenerate order; order is not necessarily arrested (or misperceived) chaos.