Software engineer from Ukraine, currently living and working in Estonia.
I mainly specialize in computer vison & robotics. https://grgv.xyz/.
If we don’t have a preliminary definition of human values
Another, possibly even larger problem is that the values that we know of are quite varying and even opposing among people.
For the example of pain avoidance -- maximizing pain avoidance might leave some people unhappy and even suffering. Sure that would be a minority, but are we ready to exclude minorities from the alignment, even small ones?
I would state that any defined set of values would leave a minority of people suffering. Who would be deciding which minorities are better or worse, what size of a minority is acceptable to leave behind to suffer, etc...?
I think that this makes the whole idea of alignment to some "human values" too ill-defined and incorrect.
One more contradiction -- are human values allowed to change, or are they frozen? I think they might change, as humanity evolves and changes. But then, as AI interacts with the humanity, it can be convincing enough to push the values shift to whatever direction, which might not be a desirable outcome.
People are known to value racial purity and supporting genocide. Given some good convincing rhetoric, we could start supporting paperclip-maximizing just as well.
Human enhancement is one approach.
I like this idea, combined with AI-self-limitation. Suppose that (aligned) AI has to self-limit it's growth so that it's capabilities are always below the capabilities of enhanced humans? This would allow for slow, safe and controllable takeoff.
Is this a good strategy for alignment? What if instead of trying to tame the inherently dangerous fast-taking-off AI, we make it more controllable, by making it self-limiting, with some built in "capability brakes"?
"I'm not working on X, because daydreaming about X gives me instant gratification (and rewards of actually working on X are far away)"
"I'm not working on X, because I don't have a strict deadline, so what harm is in working on it tomorrow, and relax now instead?"
No, thanks, I think your awards are fair )
I did not read the "Ethicophysics I" paper in details, only skimmed it. It looks to me very similar to "On purposeful systems" https://www.amazon.com/Purposeful-Systems-Interdisciplinary-Analysis-Individual/dp/0202307980 in it's approach to formalize things like feelings/emotions/ideals.
Have you read it? I think it would help your case a lot if you move to terms of system theory like in "On purposeful systems", rather than pseudo-theological terms.
One big issue is not that you are not respecting the format of LW -- add more context, either link to a document directly, or put the text inline. Resolving this would cover half of the most downvoted posts. You can ask people to review your posts for this before submitting.
Another big issue is that you are a prolific writer, but not a good editor. Just edit more, your writing could be like 5x shorter without losing anything meaningful. You have this overly academic style for your scientific writing, it's not good on the internet, and not even good in scientific papers. A good take here: https://archive.is/29hNC
From "The elements of Style": "Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell."
Also, you are trying to move too fast, pursuing too many fronts. Why don't you just focus on one thing for some time, clarify and polish it enough so that people can actually grasp clearly what you mean?
Regarding your example, I disagree. Supposed inconsistency is resolved by ruling that there is a hierarchy of values to consider: war and aggression are bad, but kidnapping and war crimes are worse.
I don't think that advanced tanks are needed for more efficient and more mobile warfare at that time. Just making an investment into transport for troops and supplies would be enough to hold better at the battle of Marne, or similar situations.
So I would:
Introducing stormtrooper tactics would be another impactful message.
I think the second part is bullshit anyway, I can't come up with a single example where compounding is possible to a whole year in a row, for something related to personal work/output/results.
A reference could be the cost of Estonian digital services which include e-signatures, and are reasonably efficient:
https://e-estonia.com/e-governance-saves-money-and-working-hours/ "Estonian public sector annual costs for IT systems are 100M Euros in upkeep and 81M Euros in investments"
So in Estonia it's ~1.3B spend for 7y. Switzerland is 7x larger population, and has higher salaries, let's say 2x larger. This puts the cost at 18B Eur.
Putting a cost on each signature does not make sense of course, it's probably just easier for the government to justify the spending this way, rather then discussing specifics of the budget.
The "sharp increase or risks" seems correct but is a bit misleading.
For paternal risks, there is indeed an big relative increase "14% higher odds of premature birth" (https://www.bmj.com/content/363/bmj.k4372). But in absolute terms, I would not think of the increase as huge: from ~6% ( based on quick googling) to ~6*1.14=6.84%.
IMO ~1% increase in risks is not something to be concerned about.
What about estimating LLM capabilities from the length of a sequence of numbers that it can reverse?
I used prompts like:
"please reverse 4 5 8 1 1 8 1 4 4 9 3 9 3 3 3 5 5 2 7 8"
"please reverse 1 9 4 8 6 1 3 2 2 5"
etc...
Some results:
- Llama2 starts making mistakes after 5 numbers
- Llama3 can do 10, but fails at 20
- GPT-4 can do 20 but fails at 40
The followup questions are:
- what should be the name of this metric?
- are the other top-scoring models like Claude similar? (I don't have access)
- any bets on how many numbers will GPT-5 be able to reverse?
- how many numbers should AGI be able to reverse? ASI? can this be a Turing test of sorts?