When you backprop through a linear layer (a matrix W), you need to multiply with W.transpose which is impossible if connections are not bidirectional.
> Information absolutely can back- propagate. If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything?
Local error aggregation can have a similar effect with backprop but you can run each layer in parallel, you don't need to wait for the signal to reach the loss function and then gradients to come all the way back.
You're conflating the implementation with the principle. There is no matrix math with neurons; quite the opposite, we posit the existence of the matrix math from the behavior we've observed with neural systems governed by a sigmoid function. The equations we've derived are secondary to the initial implementation. Just as you tweak the error factor in backprop, so too do weights between intersecting neuron networks adjust until thought and intention falls into line with eventual perception and execution.
When you backprop through a linear layer (a matrix W), you need to multiply with W.transpose which is impossible if connections are not bidirectional.
> Information absolutely can back- propagate. If it couldn't, how does anyone think it'd be at all possible to learn how to get better at anything?
Local error aggregation can have a similar effect with backprop but you can run each layer in parallel, you don't need to wait for the signal to reach the loss function and then gradients to come all the way back.
An interesting read: Decoupled Neural Interfaces using Synthetic Gradients (DeepMind, 2017) http://proceedings.mlr.press/v70/jaderberg17a/jaderberg17a.p...