That said, it’s misleading and inaccurate to state that neural networks are just statistics. In fact they are substantially more than just advanced statistics. Certainly statistics is a component—but so too is probability, calculus, network/graph theory, linear algebra, not to mention computer science to program, tune, and train and infer them. Information theory (hello, entropy) plays a part sometimes.

What I meant when I said that they are advanced statistics is that that is what they do. I know that a lot of disciplines play a part in creating them. I know it's incredible complicated, it took me quite a while to wrap my head around what the back-propagation algorithm.

I also know that neural networks can do some really cool stuff. Recognizing tumors, for example. But it's equally dangerous to overestimate them, so we have to be aware of their limitations.

Edit: All that being said, I do recognize that you have spent much more time learning about and working with neural networks than I have.