Alexander Gietelink Oldenziel

(...) the term technical is a red flag for me, as it is many times used not for the routine business of implementing ideas but for the parts, ideas and all, which are just hard to understand and many times contain the main novelties.
                                                                                                           - Saharon Shelah

 

As a true-born Dutchman I endorse  Crocker's rules.

For my most of my writing see my short-forms (new shortform, old shortform)

Twitter: @FellowHominid

Personal website: https://sites.google.com/view/afdago/home

Sequences

Singular Learning Theory

Wiki Contributions

Comments

I mostly regard LLMs = [scaling a feedforward network on large numbers of GPUs and data] as a single innovation.

arbitrary reference parameters .

what is an 'arbitrary reference parameter'? This is not in my vocabulary. 
(and why do we need it? can't we just take the log here). 

Why do people like big houses in the countryside /suburbs?

Empirically people move out to the suburbs/countryside when they get children and/or gain wealth. Having a big house with a large yard is the quintessential American dream. 

but why? Dense cities are economoically more productive, commuting is measurably one of the worst factors for happiness and productivity. Raising kids in small houses is totally possible and people have done so at far higher densities in the past. 

Yet people will spend vast amounts of money on living in a large house with lots of space - even if they rarely use most rooms. Having a big house is almost synonymous with wealth and status. 

Part of the reason may be an evolved disease response. In the past, the most common way to die was as a child dieing to a crowd-disease. There was no medicine that actually worked yet wealthier people had much longer lifespans and out reproduced the poor (see Gregory Clark). The best way to buy health was to move out of the city (which were population sinks until late modernity) and live in a large aired house. 

It seems like an appealing model. On the other hand, there are some obvious predicted regularities that aren't observed to my knowledge. 

Never ? That's quite a bold prediction. Seems more likely than not that AI companies will be effectively nationalized. I'm curious why you think it will never happen.

yes !! discovered this last week - seems very important the quantitative regret bounds for approximatiions is especially promising

You are absolutely right and I am of course absolutely and embarrasingly wrong. 

The minimal optimal predictor as a Hidden Markov Model of the simple nonunfilar is indeed infinite. This implies that any other architecture must be capable of expressing infinitely many states - but this is quite a weak statement - it's very easy for a machine to dynamically express finitely many states with finite memory. In particular, a transformer should absolutely be able to learn the MSP of the epsilon machine of the simple nonunifilar source - indeed it can even be solved analytically. 

 This was an embarrasing mistake I should not have made. I regret my rash overconfidence - I should have taken a moment to think it through since the statement was obviously wrong. Thank you for pointing it out. 

We intend to review end of the submit deadline June 30th but I wouldn't hold off on your application. 

You may be positively surprised to know I agree with you.  :)

For context, the dialogue feature just came out on LW. We gave it a try and this was the result. I think we mostly concluded that the dialogue feature wasn't quite worth the effort. Anyway

I like what you're suggesting and would be open to do a dialogue about it !

Compare also the central conceit of QM /Koopmania. Take a classical nonlinear finite-dimensional system X described by a say a PDE. This is a dynamical system with evolution operator X -> X. Now look at the space H(X) of C/R-valued functions on the phase space of X. After completion we obtain an Hilbert space H. Now the evolution operator on X induces a map on H= H(X). We have now turned a finite-dimensional nonlinear problem into an infinite-dimensional linear problem.

Load More