All of lumenwrites's Comments + Replies

Predictive Coding has been Unified with Backpropagation

You guys will probably find this Slate Star Codex post interesting:

Scott summarizes the Predictive Processing theory, explains it in a very accessible way (no math required), and uses it to explain a whole bunch of mental phenomena (attention, imagination, motor behavior, autism, schizophrenia, etc.)

Can someone ELI5/TLDR this paper for me, explain in a way more accessible to a non-technical person?

- How does backprop work if the information can't flow backwards?
- In Scotts post, he says ... (read more)

5samshap2moTLDR for this paper: There is a separate set of 'error' neurons that communicate backwards. Their values converge on the appropriate back propagation terms. A large error at the top levels corresponds to 'surprise', while a large error at the lower levels corresponds more to the 'override'.