Just to orient my exposition, which one do you prefer: the long or the short explanation?

The longer one:

"Artificial Neural Networks (ANN) are models of computation that simulate the way the neural tissue in animals computes.

Neural tissue is made up by a large number of interconnected neurons: each neuron is a cell which has numerous locations, called synapses, that receive the electrochemical signals of hundred or thousands of other neurons. When the combined potential of all those rises above a certain threshold, a chemical transmissions is initiated, propagating a signal through a long thin projection of the body of the cell, called axon, which in turn is connected to the body of other neurons. In this way, neurons make up complex biological circuitry, where chemical potentials travel back and forth: the paths that gets used the most strengthen, those that are less useful weakens over time.

Artificial neurons are abstractions that treats signals in a similar way: they are nonlinear functions that receives many weighted inputs, and when those adds to a value above the threshold, they output a value. Usually the hyperbolic tangent is taken to be the activation function.


Artificial neurons are connected in layers, each neurons receiving the inputs of every neurons in the layer before, outputting its value to every neurons in the layer after. This architecture is called fully connected, was the first to be considered and to be implemented."

The short one:

"Artificial Neural Networks aim at imitating the way neural tissue computes.

Neural tissue is made up by cells called neurons. They have a body littered with receptors where other neurons can connect (synapses), and have a long prolongation called axion, that can connect to other neurons. When combined the electrochemical potential in the synapses rises above a certain threshold, the axion transmit an impulse to the neurons it’s connected. Neurons are connected in complex ways, creating a biological circuit where signals can be strengthened or weakened based on their use.

Artificial neurons imitate this behaviour using a threshold function, usually a sigmoid function such as the hyperbolic tangent. Each neuron weights each of its input, add them together and pass the value to the threshold function, which will propagate the signal if the combined input is sufficient.

Artificial Neural Network usually are made up of neurons lined in layers, each neuron in a layer receiving the input of every neuron in the layer before and passing its output to each neuron in the layer after. This model is called fully connected and, while powerful, quickly becomes unmanageable for ANNs that have thousands of neurons per layer or have many layers to process the input (deep neural networks)."

Both versions need proofreading. That aside:

First paragraph - short version.

Second - "The paths that gets used the most strengthen, those that are less useful weakens over time" seems good to include, but you don't make any mention of how that translates into ANNs. Maybe that comes later? Apart from that, I prefer the short version.

Third - Both discussions of threshold functions are kind of awkward. Long version - the first part makes it sound like the activation function is Heaviside: either 0 or 1. You attempt to correct that impression, but yo... (read more)

Open Thread, Feb 8 - Feb 15, 2016

by Elo 1 min read8th Feb 2016224 comments


If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.