After further messing around. KWinner is by far the best solution against noise without training against noisy data. Also, KWinner is effective against all sort of noise, set random pixels to random values, add random values to pixels, set random values to 0.5, etc… In fact in most cases the raw network without regularization performs better against noise than networks trained with regularizations besides KWinner + boosting.
For example, corrupting images with Gaussian noise:
image_new = clamp(image + 0.8*gaussian_noise, 0, 1)
|Raw||Dropout||Batch Norm||KWinner||KWinner + boosting|