Quick recap of the previous post and the reactions, using an analogy: Post: I invented a new type of engine that run on water, based on a law of nature that has never been properly used so far. Here are all the details, and the plan to reproduce it yourself...
The piece of code bellow is indisputable, reproducible, proof of what we are going to say in this post. It implements a basic spiking neural network, with a simple learning method that introduces a new paradigm. The accuracy reaches state of the art (98.9%) on PI-MNIST with 750,000 low precision...
I am going to answer this comment because it is the first to address the analysis section. Thank you.
I close the paragraph saying that there is no functions anywhere and it will aggrieve some. The shift I am trying to suggest is for those who want to analyse the system using mathematics, and could be dismayed by the absence of functions to work with.
Distributions can be a place to start. The quantilisers are a place to restart mathematical analysis. I gave some links to an existing field of mathematical research that is working along those lines.
Check this out: they are looking for a multi-dimensional extension to the concept. Here it is, I suggest.
This introduces a new paradigm. Read T.Kuhn. You cannot compare different paradigms.
Everything that matters is in the post. Read it; really.
What is needed next is engineering, ingenuity and suitable ICs, not maths. The IT revolution came from IT (coders) and ICs, not CS.
As for your recommendation, I have tried so many things over the past four years… I posted here first to get to the source of one of the evidences; to no avail.
Good bye everyone
I am available through private messages
Quick recap of the previous post and the reactions, using an analogy:
Post:
I invented a new type of engine that run on water, based on a law of nature that has never been properly used so far. Here are all the details, and the plan to reproduce it yourself easily. It develops 990 hp with 5cm3. There are 2 parameters to adjust for various settings.
Reactions:
BP is Back-Propagation.
We are completely missing the plot here.
I had to use a dataset for my explorations and MNIST was simple; and I used PI-MNIST to show an 'impressive' result so that people have to look at it. I expected the 'PI' to be understood, and it is not. Note that I could readily answer the 'F-MNIST challenge'.
If I had just expressed an opinion on how to go about AI, the way I did in the roadmap, it would have been just, rightly, ignored. The point was to show it is not 'ridiculous' and the system fits with that roadmap.
I see that your last post is about complexity science. This is an... (read more)
Also,
No regularisation. I wrote about that in the analysis.
Without max-norm (or maxout, ladder, VAT: all forms of regularisation), BP/SGD only achieves 98.75% (from the dropout -2014- paper).
Regularisation must come from outside the system. - SO can be seen that way - or through local interactions (neighbors). Many papers clearly suggest that should improve the result.
That is yet to do.
... and it is in this description:
"The spiking network can adjust the weights of the active connections"
It is not a toolbox you will be using tomorrow.
I applied it to F-MNIST, in a couple of hours after being challenged, to show that is not just only MNIST. I will not do it again, that is not the point.
It is a completely different approach to AGI, that sounds so ridiculous that I had to demonstrate that it is not, by getting near SOTA on one widely used dataset (so PI-MNIST) and finding relevant mathematical evidence.
I am going after pure BP/SGD, so neural networks (no SVM), no convolution,...
No pre-processing either. That is changing the dataset.
It is just a POC, to make a point: you do not need mathematics for AGI. Our brain does not.
I will publish a follow-up post soon.
I doubt that this would be the best a MLP can achieve on F-MNIST.
I will put it this way: SONNs and MLPs do the same thing, in a different way. Therefore they should achieve the same accuracy. If this SONN can get near 90%, so should MLPs.
It is likely that nobody has bothered to try 'without convolutions' because it is so old-fashioned.
Convolutions are for repeated locally aggregated correlations.
Spot on.
I hope your explanation will be better understood than mine. Thank you.
It 'so happens' that MNIST (but not PI) can also be used for basic geometry. That is why I selected it for my exploration (easy switch between the two modes).
The piece of code bellow is indisputable, reproducible, proof of what we are going to say in this post.
It implements a basic spiking neural network, with a simple learning method that introduces a new paradigm.
The accuracy reaches state of the art (98.9%) on PI-MNIST with 750,000 low precision (binarisable) connections (>98.6% with 80,000 connections), one layer, no bias, no back-propagation and only additions. It is yet to be optimised. It works online, and is backed by existing mathematical theorems. It also relates to the real cortical structure and personal experience; and even popular wisdom.
So it cannot be just ignored or dismissed. It requires proper falsification, or further analysis and extension.
I will describe... (read 13030 more words →)
Welcome onboard this IT ship to baldly go where no one as gone before !
Indeed, I just wrote 'when it spikes' and further as the 'low threshold' and no more. I work in complete isolation and some things are so obvious inside my brain that I do not consider them as non obvious to others.
It is part of the 'when' aspect of learning, but uses an internal state of the neuron instead of an external information from the quantilisers.
If there is little reaction to a sample in a neuron (spiking does happen slowly, or not), it is meaningless and you should ignore it. If it comes too fast, it is already 'in'... (read more)