Fast Walsh Hadamard transform for global mixing in neural networks

Community Lounge
1

I suppose this is getting exhausting
https://archive.org/details/fast-walsh-hadamard-transform-in-neural-networks-clean-view
The idea is you can make local computations in a neural network layer global with a fast transform.
Improving the ability of sparse or local computations to operate globally at low cost.

Some example code then:
https://archive.org/details/sw-net-16-b

2

What happens if you insert a fast Walsh Hadamard transform between the layers of a ReLU neural network?
https://archive.org/details/inserting-a-fast-walsh-hadamard-transform-between-re-lu-layers