Subutai reviews the paper “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” and compares it to our dendrites paper “Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments”.
Paper: [1701.06538] Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Dendrites Paper: [2201.00042] Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments