LESSWRONG
LW

247
Gunnar_Zarncke
10766Ω27143395932
Message
Dialogue
Subscribe

Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
8Gunnar_Zarncke's Shortform
5y
175
The Mom Test for AI Extinction Scenarios
Gunnar_Zarncke3d20

OK. That seems to require AI hacking out of a box, which is unbelievable as per rule 4 or 8. Or do more mundane cases like AI doing economic transactions or research count? 

Reply
The Mom Test for AI Extinction Scenarios
Gunnar_Zarncke3d20

That counterargument is unfortunately always available for all scenarios, including non-AI cases. "Just don't do the bad thing." I'm not sure what you think specifically in this scenario triggers it to be more salient. Is it "The Military" as a common adversary? If I think about a scenario where AI is used to optimize or "control" the energy grid of supply chain logistics, would that be different?

Reply
Cheap Labour Everywhere
Gunnar_Zarncke3d31

not sure about India, but disagree for many African countries. See my comment above.

Reply
Cheap Labour Everywhere
Gunnar_Zarncke3d110

My wife is from Kenya (as a single mom mid career government employee could afford a 24/7 household help last year) and even the poor have much better child care support than even middle class in eg Germany. That can take the form of communal or familial support and the quality may be lower, but it is definitely the case that it is in some sense easier or "normal" to care of esp. small children. 

Reply
Can LLMs Coordinate? A Simple Schelling Point Experiment
Gunnar_Zarncke4d30

Would be interesting to ask a Jeopardy egghead for comparison.

Reply
Cheap Labour Everywhere
Gunnar_Zarncke4d70

Cheap labor, or rather their absence, may also partly be a reason for the declining birthrates: In Kenya, most people can afford cheap child care. Raising kids with a full-time house help is easy. Except for school fees, but that is a different aspect.

Reply
The Mom Test for AI Extinction Scenarios
Gunnar_Zarncke4d0-2

Here is at least one scenario that should pass the mom-test, even though it is just boring old cold war with AI:

The Automated Cold War

Imagine the world’s great powers America, China, Russia, and/or Europe, always nervous about each other, always worried about being caught off guard. They used to rely on humans to make the big decisions about war and peace. Sometimes those humans have come terrifyingly close to pushing the nuclear button by accident.

Today, governments start automating these decisions with AI. AI is faster and they can sift through oceans of data. AI companies and the military will push for adoption and argue that "we cant fall behind." So one by one, nations roll out “AI decision-support systems” that track everything and recommend what to do in real time. First, the AIs suggest small things: move some submarines here, increase surveillance there. Over time, leaders start to rely on them more and more, especially when the advice turns out to be tactically smart. Soon, the AIs are recommending military deployments, cyber responses, even levels of nuclear alert.

At first, it works pretty well. Crises that would have taken weeks to analyze are now handled in hours. But these AIs aren’t programmed for caution. They’re programmed to “win.” 

So what happens when an American AI notices a Chinese military exercise and interprets it as the prelude to an invasion? It recommends raising the nuclear alert level. The Chinese AI, watching America’s moves, reads this as a sign that the U.S. is preparing to strike. It, too, recommends raising its alert. Each local AI is acting logically, but together they’re creating a spiral of tension that’s invisible to most citizens.

Human leaders still technically have the final say, but the AI’s recommendation lands on their desk stamped 99% confidence level. Imagine being the president at 3 a.m. when your advisors say, “Sir, the AI says China is about to launch. We have seven minutes to respond.” People stop second-guessing the AI because, frankly, they don’t have time. Decisions that used to take months of negotiation now happen in seconds. For the public, life goes on as usual. But under the surface, the world is walking on a hair-trigger.

Then one mistake or data error. The AI, following its programming, interprets it as imminent nuclear war and pushes the strongest possible recommendation: Fire now before they fire at you. The president hesitates. But their rival’s AI has already given the same instruction, and missiles are on the move. There’s no chance to undo it, no second thoughts. Cities vanish. Power grids fail. Survivors die in the aftermath, and society collapses.

It’s not that anyone wanted this outcome. It’s not that the AIs were “evil.” It’s just that the world delegated its most dangerous decisions to machines optimized for speed and winning, not for patience or human judgment.

Reply
The "Length" of "Horizons"
Gunnar_Zarncke5d7-2

Paleolithic canoeing records to forecast when humans will reach the moon

Not disagreeing with your main point, but Robin Hanson has tried this.

Reply
[Intuitive self-models] 3. The Homunculus
Gunnar_Zarncke5d20

What is the "I" in your reply "I have the same problem" referring to? What entity is doing the finding in "I can't find anything that..."? The first one can be answered with "the physical human entity currently speaking and called Dawn." But the second one is more tricky. At least it is not clear now that entity is doing the finding.

Reply
Tomás B.'s Shortform
Gunnar_Zarncke9d62

Describes me decently well:

  • don't drive ✅
  • work from home or close to home ✅
  • avoid office politics ✅
  • lots of reading alone ✅
  • only eat a few selected foods ✅
  • travel rarely ✅

I'd agree to a description of being "risk averse," but "anxious" doesn't feel fitting. I have a relatively high openness to experience. For example, on the last item, I didn't travel, estimating it to provide relatively little value of information per effort (or per negative stimuli?). Friends pointed out that I might be very wrong in my evaluation if I didn't travel even once. I accepted the challenge and visited India (for a friend's wedding; long story).

I guess people can be seen as imperfect Thompson samplers with different priors and weights.

Reply
Load More
21Is there a safe version of the common crawl?
Q
2mo
Q
6
11[Linkpost] How Am I Getting Along with AI?
3mo
0
9Hybrid model reveals people act less rationally in complex games, more predictably in simple ones
3mo
0
53Project Vend: Can Claude run a small shop?
4mo
8
13[Linkpost] The lethal trifecta for AI agents: private data, untrusted content, and external communication
4mo
3
34Unexpected Conscious Entities
6mo
7
13[Linkpost] The value of initiating a pursuit in temporal decision-making
7mo
0
81Mistral Large 2 (123B) seems to exhibit alignment faking
Ω
7mo
Ω
4
162Reducing LLM deception at scale with self-other overlap fine-tuning
Ω
7mo
Ω
46
63RL, but don't do anything I wouldn't do
10mo
5
Load More
Theory of Mind
a year ago
(+250)
Pareto Efficiency
a year ago
(+52/-52)
Pareto Efficiency
a year ago
(+52)
Pareto Efficiency
a year ago
(+392)
Babble and Prune
2 years ago
(+1264)
Has Diagram
2 years ago
(+163)
Simulation
2 years ago
(+9/-10)
Simulation
2 years ago
(+443/-24)
Simulation
2 years ago
(+174/-3)
Simulation
2 years ago
(+646)
Load More