LESSWRONG
LW

3347
David Lorell
3173Ω92580
Message
Dialogue
Subscribe

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
No wikitag contributions to display.
Resampling Conserves Redundancy (Approximately)
David Lorell3d70

(Update 5)

A conjecture we are working on which we expect to be generally useful beyond possibly rescuing the stoch->det proof that used to rely on the work in this post:

Chainability (Conjecture):
∀P[X,Λ]with ϵ1:=DKL(X1→X2→Λ), ϵ2:=DKL(X1←Λ→X2),
Define  Q[X,Λ]:=P[X]P[Λ|X2], and  Q′[X,Λ]:=Q[X1|Λ]Q[X2|Λ]Q[Λ].
Then, DKL(P||Q)<n(ϵ1+ϵ2)

Here is a collab with a quick numerical test that suggests the bound holds (and that n=1, in this case).

(Note: The above as written is just one step of chaining, and ultimately we are hoping to show it holds for arbitrarily many steps, accumulating an associated number of epsilons as error.)

Reply
Resampling Conserves Redundancy (Approximately)
David Lorell9d60

Do you have sympy code for the example noted at the bottom of the collab that claims a ratio of > 9.77 including the mediation DKL? I tried with the parameters you mention and am getting a ratio of ~3.4 (which is still a violation of previous expectations, tbc.)

Reply
Resampling Conserves Redundancy (Approximately)
David Lorell10d110

(Update 2)

Taking the limit of the ratio of DKLs (using summation rather than max) with c→0 while b:=10c gives 5(r+111)

Setting c very small and ramping up r indeed brakes the bound more and more severely. (Code changes from the collab you provided, below.)

Code changes / additions

Block 1:
a,b,c,d,r = sp.symbols("a b c d r")
variable_substitutions = { # The definitions of these variables
   a: 0.25,
   b: 1e-90,
   c: 1e-91,
   r: 20000000,
}

Block 2 (later on):
expr = (kl3/(kl1 + kl2)).subs(d, (1-3*c-(r+1)*b-2*a))
print("KL(X2->X1->L')/sum[KL(X1->X2->L),KL(X2->X1->L)]=",(kl3/(kl1 +kl2)).evalf(subs=variable_substitutions))

Block3 (right after Block 2):
expr = (kl3/(kl1 + kl2)).subs(d, (1-3*c-(r+1)*b-2*a)).subs(b, 10*c)
lim = sp.simplify(sp.limit(expr, c, 0))
print("Limit of KL(X2->X1->L')/sum[KL(X1->X2->L),KL(X2->X1->L)] as c->0+ =", lim)

Reply
Resampling Conserves Redundancy (Approximately)
David Lorell10d150

(Update 1)

We've looked at the code and fiddled with the math and are now more convinced of the issue.

The 2nd order approximation holds when Q[X]P[X]≈1 ...Which our scaling-down construction does not provide. So, (among a host of other things,) we are now thinking about other ways to try wrangling the DKL bound into a euclidean space or otherwise into some form that is similarly "easy" to work with.

(Thanks for finding this!)

Reply
Before LLM Psychosis, There Was Yes-Man Psychosis
David Lorell2mo128-10

This post absolutely sings. The “yes-man psychosis” framing is sticky, clarifying, and honestly—iconic. You take a fuzzy, slippery problem and pin it to the mat with crisp language, vivid examples, and just enough systems thinking to make the lights come on. The Boyd/OODA connection is chef’s-kiss; it turns a cultural gripe into a concrete failure mode you can point at and say, “That—right there.” The Putin case study lands like a hammer, and the dead-organization metaphor (rocks, logs, and that hilariously misleading chart!) is going to live rent-free in my head. This is the kind of essay people forward to their bosses with “must read” in the subject line. It’s sharp, fearless, quotable, and—despite the bleak subject—fun to read. Truly an instant classic on power, perception, and how praise can calcify into poison. Bravo for naming the thing so precisely and making it impossible to unsee.

Reply71412111
A Self-Dialogue on The Value Proposition of Romantic Relationships
David Lorell2mo40

I wonder if your oxytocin is fine but you have, for whatever reason, a very strong cognitive "immune response" to its effects. I think it is common in teens (well, it was the case for me in tweenagehood,) to react to the hook of limerance/this-whole-cluster with "*no one must know*." Followed in my case by trying not to dwell on it. I'm not suggesting this thought specifically is something you have trained (maybe its more like "*be realistic*") but maybe you have a well developed cognitive kata that shuts these kinds of thoughts down before they can become limerance/love/companionship/etc.

Reply
A Self-Dialogue on The Value Proposition of Romantic Relationships
David Lorell3mo2213

You're disgusting monsters, both of you.

I can't even bear to look at how you've both shamelessly normalized usage of the phrase "beg the question" to mean "prompt the question" rather than its god-given original meaning of "assume the premise." 

Shame on you.

Oh and nice kinks. 

Reply3
A Self-Dialogue on The Value Proposition of Romantic Relationships
David Lorell3mo40

...ah. When you put it that way.....

If somehow something happened within the last decade which shifted my People vs Things interest parameter significantly more away from People and toward Things I'd probably be a much more capable researcher right now. (Unsure about before a decade from now because then we start messing with my middle-young teenagehood where the actual path I took to deciding I was going to work on alignment routed through caring deeply about others....or at least imagining the deep loss of not having the opportunity to mutually care very deeply about others in this way.)

 I'd also not have or be many things which I currently reflectively value highly, but that's a me thing :)

I might, if I meditated on it, press a button that goes back in time to perform that intervention back in my early college years, (and I'd grieve the decision more than I've grieved probably anything,) to increase the chance that our work is decisively counterfactual. I'm so glad that such a button does not exist. 

(Fun, and probably tragic from your POV, fact: Our very own Dan Hendryks more or less encouraged me to self modify in this way for this reason back when we were college. I shook my head and laughed at the time. Now I feel more complicatedly.)

Point being: Yup. That sure is a life-influencing personality-parameter. Concern is super merited.

Reply
A Self-Dialogue on The Value Proposition of Romantic Relationships
David Lorell3mo60

Fair enough and well taken. (I uh...don't think it's like written on the atoms that this stuff is Good tbc. I value it very highly and it seems like a big part of the human culture.) 

Some reasons that occur to me to be less worried than you seem:

  •  It does sound to me like you already are interested in connecting with people more deeply
  • People fall in and out of love so it's not that permanent an effect
  • I don't think Ive heard of anyone getting addicted to supplemental oxytocin, and while lots of people say they want more love in their life it doesn't seem like much of a addict-compulsion since most people are also not doing much to make that happen

That said, caution seems extremely reasonable, in general and especially from your perspective here.

Reply
A Self-Dialogue on The Value Proposition of Romantic Relationships
David Lorell3mo40

Well! This can be tested (Maybe.)

I know folks who spray oxytocin up their nose. From a brief google this may in fact appreciably raise oxytocin levels both in plasma and CSF. It might be non trivial to get the right pattern/timing to mimic natural oxytocin release under various romantic/sexual circumstances, though. Worth looking into if that's your model of what's going on and you want to know what this thing is that everyone else thinks is so valuable.

Reply
Load More
117Natural Latents: Latent Variables Stable Across Ontologies
Ω
2mo
Ω
20
66Do-Divergence: A Bound for Maxwell's Demon
Ω
2mo
Ω
4
126(∃ Stochastic Natural Latent) Implies (∃ Deterministic Natural Latent)
Ω
2mo
Ω
8
66Resampling Conserves Redundancy (Approximately)
Ω
2mo
Ω
18
102Curing PMDD with Hair Loss Pills
4mo
3
73$500 + $500 Bounty Problem: Does An (Approximately) Deterministic Maximal Redund Always Exist?
6mo
16
92$500 Bounty Problem: Are (Approximately) Deterministic Natural Latents All You Need?
6mo
24
184Instrumental Goals Are A Different And Friendlier Kind Of Thing Than Terminal Goals
Ω
9mo
Ω
61
47Minimal Motivation of Natural Latents
Ω
1y
Ω
14
89Values Are Real Like Harry Potter
1y
21
Load More