Fixed Filter Bank Neural Networks

Adjust the nonlinear functions rather than the filter bank:
https://github.com/S6Regen/Fixed-Filter-Bank-Neural-Networks

You can use the fast Walsh Hadamard transform (WHT), the FFT or other transforms for the fixed filter bank. The fast WHT does not intrinsically require multiply circuits and could be very power efficient on specialized hardware.