I have never thought much about reversibility. Do humans make heavy use of reversible processes? And what is the relation between cross entropy and reversibility?
Do you count gradient descent as a blackbox optimization, and isn't backpropagation guided by gradient (at least in ANN)?
Care to elaborate how did you get to the formula O(1/sqrt(n))?
What is ACX?