I was taught the more classical 'ideal' point of view on the structure of rings in school. I'm curious if [and why] you regard the annihilator point of view as possibly more fecund?

Nice post Cole. 

I think I'm sympathetic to your overall point. That said, I am less pessimistic than you that Neural Network computation can never be understood beyond the macroscopic level ' what does it do' .

The Turing machine paradigm is just one out of many paradigms to understand computation. It would be a mistake to be too pessimistic based on just the failure of the ur-classical TM paradigm. 

Computational learning theory's bounds are vacuous for realistic machine learning. I would guess, and I say this as a nonexpert, that this is chiefly due to 

(i) a general immaturity of the field of computational complexity, i.e. most of the field is conjectures, it's hard to prove much about time complexity even if we're quite confident the statements are likely true 

(ii) computational learning theory grew out of classical learning theory and has not fully incorporated the lessons of singular learning theory. Much of the field is working in the wrong 'worst-case/pessimistic' framework when they should be thinking in terms of Bayesian inference & simplicity/degeneracy bias. Additionally, there is perhaps too much focus on exact discrete bounds when instead one should be thinking in terms of smooth relaxation and geometry of loss landscapes. 

That said, I agree with you that the big questions are currently largely open. 

https://youtu.be/tgkP0W7OvMc?si=hoa0l2mu5B6aRbpy

Perhaps of interest, 16:33 the guy mentions the development of a new type of drone resistant "turtle" tank

Highly recommended video on drone development in the Ukraine-Russia war, interview with a Russian private military drone developer. 

some key takeaways

• Drones now account for >70% of kills on the battlefields.
• There are few to none effective counters to drones. The on
  • Electronic jamming is a rare exception but drones carrying 5-15km fiber optic cables are immune to jamming. In the future AI-controlled drones will be immune to jamming.
  • 'Laser is currently a joke. It works in theory, not in practice. Western demonstrations at expos are always in ideal conditions. ' but he also says that both Russia and Ukraine are actively working on the technology and he thinks it could be an effective weapon.
  • Nets can be effective but fiber-optic drones can fly very low and not lose connection are increasingly used to slip under the nets.
  • Soldiers are increasingly opting for bikes instead of vehicles as the latter don't offer much protection to drones.
• The big elephant in the room: AI drones.
  • It seems like the obvious next step - why hasn't it happened yet?
  • 'at Western military expos everybody is talking AI-controlled drones. This is nonsense of course' Apparently the limitation is that it's currently too expensive to run AI locally on a drone but this is rapidly changing with new nVidea chips. He expects chips to become small and cheap soon enough that AI drones will appear soon.
  • There is a line of 'Vampire' drones that are autonomous and deadly but use older pre-programmed tactics instead of modern AI
• One of the most lethal tactics is drone mining: let a drone lie in wait somewhere in the bushes until a human or vehicle passes by.
  • This tactic was pioneered by the Ukranians. " Early on, soldiers would try to scavenge fallen drones... then Boom" .
• Western drones are trash compared to Ukranian and Russian forces
  • Swishblade, Phoenix Ghost and a consortium of Boeing designed drones are ineffective, fragile and wildly overpriced. Apparently, the Ukranians have stopped using Western drones opting to focus solely on domestically produced variants.
  • The only western weapon the Russian drone dev was impressed by is Starlink.
    • Apparently, Starlink is the reason Ukraine has been succesful with its naval drones. 'we have the boats, we just don't have Starlink'
  • One point of optimism: " you know what's the difference between Russian and American military exercises? In the Russian exercises Russia always wins, in the American ones NATO occasionally loses.'
• The Russian drone dev admits Ukraine is ahead of Russia in drone innovation, mostly due to their ecosystem of private enterprise drone manufacturers outpacing slow government run Russian military contractors - but the difference is small and Russia is only a couple month behind.
  • For instance, at Kursk, Russian forces massively deployed the fiber-optic jammer-resistant drones surprising Ukranian forces.
• " At the start of the war a 1000 drones was a lot. Now a single assembly line can produce 40,000 drones a month"
  • A quick Google states that Ukraine produces about 4000 FPV drones a day. Alarmingly, the same Google search revealed that the total procurement of FPV drones at this moment by the British army is 450 FPV drones! Western militaries are completely asleep at the wheel.
  • ' The drones we produce today are slapped together. The first thing that will happen after this war is over is that drones will be redesigned from the ground up'
• China
  • Electronic components forming the drones come predominantly from China. China officially prohibits export of weapons to Russia-Ukraine war but unofficially condones the large-scale smuggling of material across the Chinese-Russian border.
  • China is producing their own variants of the Ukrainian and Russian drones, apparently much faster than the West.
  • ' Chinese chips are only a little worse than Western chips but a ten to hundred times cheaper'
  • ' Chinese has nearly caught up with the West technologically, maybe half a step behind' 

  • Chinese manufacturing and technological prowess is strong but they produce their military material like toys - they don't have the on-hands experience that Russia [and Ukraine] has.-

     

Perhaps of interest. 

Much appreciated Habryka-san!

You might be interested in my old shortform on the military balance of power between US and China too. It's a bit dated by now - the importance of drones has become much more clear by now [I think the evidence that we are in a military technological revolution on par with the introduction of guns] but you may find it of interest regardless. 

According to Ukraine drone operators western drones are often not even regarded as very good. Expensive, overengineered, fail often, haven't kept pace with rapid innovation during the Ukraine war. 

Note also in PPP China is already 50% ahead of the US

The US military has about 10k drones of all sizes. Ukraine alone builds 2-4 million drones a year, mostly smaller. Most of the production involves assembling chinese made components. China has something like ninety percent of the global market share for components of small drones.

There is not a single NATO country currently thathat is building drones at scale.

Circling back to this. I'm interested in your thoughts. 

I think the Algorithmic Statistics framework [including the K-structure function] is a good fit for what you want here in 2. 

to recall the central idea is that any object is ultimately just a binary string $s$ that we encode through a two-part code: a code $c$ encoding a finite set of strings $A_c$ such that $s\in A_c$ with a pointer to $s$

 within $A_c$. 

For example $s$ could encode a dataset while $c$ would encode the $ϵ-$typical data strings for a given model probability distribution in a set of hypotheses for some small $ϵ>0$. This is a way to talk completely deterministically about (probabilistic model), e.g. like a LLM trained in a transformer architecture. 

This framework is flexible enough to describe two codes $c_1,c_2$ encoding $A_{c_1},A_{c_2}$ such that $x\in A_{c_1}$

and $x\in A_{c_2}$ as required. One can e.g. easily find simple examples of this using mixtures of gaussians.  

I'd be curious what you think!