LESSWRONG
LW

Iknownothing — LessWrong

Replying toEven briefer summary of ai-plans.com

Thank you! Changed it to that!

Yup, that's definitely something that can be argued by people Against during the Debate Stage!
And they might come to the same conclusion!

AI Law-a-Thon

Iknownothing

Calling All Lawyers

AI-Plans.com is holding a Law-a-Thon on which new policies can be made, thanks to the latest AI breakthroughs - e.g. how breakthroughs in machine unlearning enable a greater Right To Be Forgotten by AI models

Sign Up by February 1st to take part: https://lu.ma/vvpb5rtq

Schedule.

10am to 11:30pm - Proposals

Finding/searching for the latest breakthroughs which might open possibilities - will be posted in forum - will regularly post (every 15 mins) who has got the most.

Format:
Tech Breakthrough, [date released]
New Possible Policy
Closest Existing Policy

11:30am to 12pm - Break/Discussion

Discussion of things found- What might have been missed. What mistakes might have been made. Maybe more. Not discussing policy at this time- that’s coming soon!

12pm to

Iknownothing

These are the Critiques from the AI-Plans.com December Critique-a-Thon of 2023.
Prizes will be awarded to contestants based on collective judge feedback.

Critiques were anonymized when reviewed by the judges.

Judges:
- @So8res
- @Ramana Kumar
- @Peter S. Park
- @Charbel-Raphaël, Head of AI Unit at EffiSciences
- An anonymous judge (researcher at a major lab)

If you're interested in being a Judge for an upcoming Critique-a-Thon (held every 2 months), please email kabir03999@gmail.com or DM Kabir Kumar on Discord (kabs9744) or LessWrong!

Winners:

1st Place:

Congratulations to Lorenzo Venieri! 🥇

Lorenzo had the highest mean score, of 7.5, for his Critique of:
A General Theoretical Paradigm to Understand Learning from Human Preferences, the December 2023 paper by DeepMind.

2nd Place:

Congratulations to NicholasKees & Janus! 🥈

Nicholas... (read 7258 more words →)

Replying toE.T. Jaynes Probability Theory: The logic of Science I

Iknownothing2y

E.T. Jaynes Probability Theory: The logic of Science I

I'd also read Elementary Analysis before

Replying toE.T. Jaynes Probability Theory: The logic of Science I

Iknownothing2y

E.T. Jaynes Probability Theory: The logic of Science I

I'm not a grad physics student- I don't have a STEM degree, or the equivalent- I found the book very readable, nonetheless. It's by far my favourite textbook- feels like it was actually written by someone sane, unlike most.

Replying toCritical review of Christiano's disagreements with Yudkowsky

Iknownothing2y

Critical review of Christiano's disagreements with Yudkowsky

I'm really glad you wrote this!
I think you address an important distinction there, but I think there might be a further one to be made- in that how we measure/tell if a model is aligned in the first place.
There seems to be a growing voice which says that if a model's output seems to be the output we might expect from an aligned AI, then it's aligned.
I think it's important to distinguish that from the idea that the model is aligned if you actually have a strong idea of what it's values are, how it's gotten them, etc.

Replying toAI Safety Chatbot

Iknownothing2y

AI Safety Chatbot

I'm really excited to see this!!
I'd like it if this became embed-able so it could be used on ai-plans.com and on other sites!!
Goodness knows, I'd like to be able to get summaries and answers to obscure questions on some alignmentforum posts!

Replying toWhy aren't more people in AIS familiar with PDP?

Iknownothing2y

Why aren't more people in AIS familiar with PDP?

What do you think someone who knows about PDP knows that someone with a good knowledge of DL doesn't?
And why would it be useful?

Replying toWhy Is No One Trying To Align Profit Incentives With Alignment Research?

Iknownothing2y

Why Is No One Trying To Align Profit Incentives With Alignment Research?

I think folks in AI Safety tend to underestimate how powerful and useful liability and an established duty of care would be for this.

Replying toHere's the exit.

Iknownothing2y

Here's the exit.

I think calling things a 'game' makes sense to lesswrongers, but just seems unserious to non lesswrongers.

Replying toHow dath ilan coordinates around solving alignment

Iknownothing2y

How dath ilan coordinates around solving alignment

I don't think a lack of IQ is the reason we've been failing at making AI sensibly. Rather, it's a lack of good incentive making.
Making an AI recklessly is current much more profitable than not doing do- which imo, shows a flaw in the efforts which have gone towards making AI safe - as in, not accepting that some people have a very different mindset/beliefs/core values and figuring out a structure/argument that would incentivize people of a broad range of mindsets.

Critique-a-Thon of AI Alignment Plans

Iknownothing

AI-Plans.com will be hosting a Critique-a-Thon from December 16th to 18th, where participants will be proposing and discussing critiques of AI Alignment plans.

Judges include:

Nate Soares, President of MIRI @na
@Ramana Kumar, researcher at DeepMind
@Peter S. Park , MIT postdoc at the Tegmark lab
@Charbel-Raphaël Segerie head of AI unit at EffiSciences
@Robert Miles - who will be judging Communication

Benefits:

The Critique-a-Thon provides an excellent opportunity to gain critical insights into AI Safety by diving into what makes an alignment plan likely to work or fail.
We will highlight key elements of alignment plans, which will be fed back to the researchers and can be used to improve their plans.
It’s also a great opportunity to get feedback from... (read 270 more words →)

Proposal for improving state of alignment research

Iknownothing

Goal:

Increase the quality and quality of AI alignment research, such that there is less chance of human extinction- goal of 0%.
This relies on work being done to gain more time being fruitful- that shouldn't be neglected for this.

Proposed Method:

A way to improve the output level of ai alignment is to make it profitable.

With liability laws, maybe an evals based tax for models which break/risk laws/regulations, etc.

Such things let smaller companies compete with ones which have more compute access by having another vector of profitability- safety and meeting regulation standards.

Evidence it's worked elsewhere:

Companies like Matomo can compete with Google Analytics pretty much entirely due to GDPR- it's why I pay for their service,... (read more)

Looking for judges for critiques of Alignment Plans

Iknownothing

Hello!

AI-Plans.com recently held a "critique-a-thon," where participants submitted and refined 40+ critiques for AI Alignment plans. Here are the finalized critiques from the event: https://docs.google.com/document/d/1mW4SAxFN_aI6KyYXpl9qz5B9nVdeV9Xyc69GTNme5cA/edit?usp=sharing

We are looking for anyone who might be interested in helping to judge these final 11 critiques.

So far, we have gratefully had the assistance of Dr Peter S Park (MIT postdoc at the Tegmark lab, Harvard PhD) and Aishwarya G (AI Existential Safety Community Member at Future of Life Institute and Governance Course Facilitator at BlueDot Impact AI Safety Fundamentals), as well as some independent alignment researchers.

I would love to hear your thoughts!

Kabir Kumar (Founder, AI-Plans.com)

Specific Arguments against open source LLMs?

Iknownothing

https://www.msn.com/en-us/news/technology/github-urges-the-european-union-to-reconsider-its-ai-act/ar-AA1ey0nQ

https://github.blog/2023-07-26-how-to-get-ai-regulation-right-for-open-source/

"In a joint effort, GitHub, Hugging Face, Creative Commons, and other tech companies have penned an open letter appealing to European Union (EU) policymakers to revise certain aspects of the EU’s Artificial Intelligence Act. The letter expresses concerns that upcoming rules may inadvertently impede the development of open-source artificial intelligence (AI) models."

AI-Plans.com 10-day Critique-a-Thon

Iknownothing

Introduction

At AI-Plans.com, we are creating an accessible, centralized platform to discover and learn about the numerous alignment plans in existence. Our mission is to make the field of AI Alignment more transparent and accessible by providing a comprehensive compendium of alignment plans, along with criticisms against them.

As part of our effort to crowdsource knowledge and foster a critical, constructive discussion in the AI Alignment community, we are excited to announce the AI-Plans.com Critique-a-Thon, set to begin on the 1st of August and last for 10 days.

Proposal

We propose a prize fund of $500 to be awarded to the authors of the best critiques submitted during the Critique-a-Thon. Critiques will be judged based on... (read 494 more words →)

Simple alignment plan that maybe works

Iknownothing

I'll ask for feedback at the end of this post, please hold criticisms, judgements and criticisms for then.

Forget every single long complicated, theoretical mathsy alignment plan.
In my opinion, pretty much every single one of those is too complicated and isn't going to work.
Let's look at the one example we have of something dumb making something smart that isn't a complete disaster and at least try to emulate that first.
Evolution- again, hold judgements and criticisms until the end.

What if you trained a smart model on the level of say, GPT3 alongside a group of much dumber and slower models, in an environment like a game world or some virtual world?
Dumb models who, with... (read more)

Even briefer summary of ai-plans.com

Iknownothing

Currently, newcomers to the field of AI Alignment often struggle to understand the ongoing work and individuals involved, as well as the assumptions, strengths, and weaknesses of each plan.

We believe AI-plans.com will be an easy, centralized way to discover and learn more about the most promising alignment plans.

The site is currently in Stage 1, functioning purely as a compendium. We are in the process of adding up to 1000 plans and the criticisms made against them so far. Further plans and criticisms can be added by users.

Projected benefits of Stage 1:

- Easy discovery of proposed plans and better understanding of their prevalent challenges.

(This is already showing promise, with one researcher letting us... (read 300 more words →)

LeCun says making a utility function is intractable

Iknownothing

Has there been any post going over how LeCun's alignment plan is very fluffy and he hasn't responded to any criticism of it at all? LeCun's paper: https://openreview.net/forum?id=BZ5a1r-kVsf

One of the criticisms, which goes over how LeCun's idea of an Intrinsic Cost Module, his idea for how to motivate an ASI to want to do good things lacks any real substance: https://www.alignmentforum.org/posts/C5guLAx7ieQoowv3d/lecun-s-a-path-towards-autonomous-machine-intelligence-has-1#1__Background__the_paper_s_descriptions_of_the__Intrinsic_Cost_module_

LeCun himself says in the paper:
"In most RL settings the reward (or the cost, which is a negative reward) is fed to the agent by the environment. In other words, Intrinsic Cost module is the environment itself, and is therefore an unknown function. "
Essentially, what this means is that LeCun says there's no way to make a function to make an ASI want to do specific things and his plan is to give it as much information as possible and it'll pretty much automatically become good from that.
LeCun's quote is from the bottom of page 48.

Iknownothing

AI-Plans.com - a contributable compendium

Review of Alignment Plan Critiques- December AI-Plans Critique-a-Thon Results

An Ignorant View on Ineffectiveness of AI Safety

Even briefer summary of ai-plans.com

Iknownothing

Iknownothing

Review of Alignment Plan Critiques- December AI-Plans Critique-a-Thon Results

Proposal for improving state of alignment research

Looking for judges for critiques of Alignment Plans

Specific Arguments against open source LLMs?

AI-Plans.com 10-day Critique-a-Thon

Simple alignment plan that maybe works

Even briefer summary of ai-plans.com

Iknownothing

AI-Plans.com - a contributable compendium

Review of Alignment Plan Critiques- December AI-Plans Critique-a-Thon Results

An Ignorant View on Ineffectiveness of AI Safety

Even briefer summary of ai-plans.com

Iknownothing

Iknownothing

Review of Alignment Plan Critiques- December AI-Plans Critique-a-Thon Results

Proposal for improving state of alignment research

Looking for judges for critiques of Alignment Plans

Specific Arguments against open source LLMs?

AI-Plans.com 10-day Critique-a-Thon

Simple alignment plan that maybe works

Even briefer summary of ai-plans.com

Calling All Lawyers

Schedule.

​10am to 11:30pm - Proposals

​11:30am to 12pm - Break/Discussion

​12pm to

Winners:

1st Place:

Congratulations to Lorenzo Venieri! 🥇

2nd Place:

Congratulations to NicholasKees & Janus! 🥈

Benefits:

Goal:

Proposed Method:

Evidence it's worked elsewhere:

Introduction

Proposal

10am to 11:30pm - Proposals

11:30am to 12pm - Break/Discussion

12pm to