Biologically inspired neural networks are very promising in my opinion. Reverse engineering some of the algorithms that evolved naturally may be easier than trying to figure this stuff out from scratch.
Preprint available here https://arxiv.org/pdf/2509.05276v1