TL;DR:

This short post introduces EvoNet, a prototype of a novel neural architecture that maintains a persistent internal state, routes activations bidirectionally, groups nodes into clusters, and remains GPU-friendly through iterative updates. The core question: could such an architecture transport and refine information in a way that can compete with conventional ANNs?

A New Structure: Persistent State and Flowing Activations

Unlike typical ANNs (which clear their activations after each forward pass), EvoNet maintains a persistent internal state. Signals aren't just pushed forward and discarded — they continue to circulate and evolve with each iteration. This is more akin to how biological brains maintain working memory and adapt responses over time.

Information-Flow Diagrams: Conventional ANN vs EvoNet

In conventional networks, information moves in a single sweep. In EvoNet, activations loop back and cross paths, letting the network build a richer internal context. Whether this is truly viable remains an open question. Can such recursive flows reliably transport meaningful signals, or do they collapse into noise or a static state?

Nodes and Clusters: Scalable Building Blocks

EvoNet's micro-units are nodes, similar to ANN neurons but with local state updates every iteration. This lets signals ripple like waves, not just be processed once. Nodes group into clusters, forming natural units for scaling and specialization. This could later support dynamic growth by inserting nodes or clusters precisely where informational "strain" emerges.

These ideas for network growth will be discussed in later posts.

GPU-Friendly Design

Unlike typical spiking neural nets, EvoNet is built for GPUs by:

Iteratively updating the entire network in parallel via sparse matrix multiplications.
Exploiting sparsity to skip zero connections.
Distributing clusters across GPUs, with only light communication between them. Similar to model parallelism.

This should make it computationally plausible even on existing hardware.

Next Steps: Testing in DOOM

I plan to continue by testing EvoNet in DOOM or another simple RL environment. These games are perfect early testbeds: they’re complex enough to challenge learning, but controlled enough to clearly observe what the network is doing.

The main goal isn’t just to perform well, but to see whethe...

Posts

Wikitag Contributions

Comments