The authors compare feedback alignment (FA)—a biologically plausible alternative to backpropagation that uses random backward weights—with backpropagation on convolutional networks trained on CIFAR-10. Standard FA fails on convnets, so they test modified FA variants and find that the versions that work do so because they converge on internal representations geometrically similar to those from backpropagation, despite using different weight update rules. The implication is that mimicking backprop's representational structure, not biological plausibility per se, drives the functional success.
Main takeaways:
- Standard feedback alignment doesn't scale to convolutional architectures; modified versions do
- Modified FA algorithms that succeed produce internal representations structurally similar to backpropagation
- Success appears rooted in mimicking backprop's representational geometry, not in the biological plausibility of the update rule
- Analysis covers biological plausibility, interpretability, and computational cost
- Suggests representational alignment with backprop is the key functional ingredient