A /r/ithkuil user tests whether ChatGPT can perform translations from and to Ithkuil. It doesn't yet succeed at it yet, but it's apparently not completely missing the mark. So the list of things AI systems can't yet do still includes "translate from English to Ithkuil".
If it was human-level at Ithkuil translation that would be an imho very impressive generalization.
TIL that the expected path a new user of LW is expected to follow, according to https://www.lesswrong.com/posts/rEHLk9nC5TtrNoAKT/lw-2-0-strategic-overview, is to become comfortable with commenting regularly in 3-6 month, and comfortable with posting regularly in 6-9 month. I discovered the existence of shortforms. I (re)discovered the expectation that your posts should be treated as a personal blog medium style ?
As I'm typing this I'm still unsure whether I'm destroying the website with my bad shortform, even though the placeholder explicitly said... (\*r...
Just looked up Aligned AI (the Stuart Armstrong / Rebecca Gorman show) for a reference, and it looks like they're publishing blog posts:
E.g. https://www.aligned-ai.com/post/concept-extrapolation-for-hypothesis-generation
ChatGPT doesn't want to joke about science:
...As a machine learning model, I do not have the ability to create original jokes or humor. I can provide information and support to help answer your questions, but I am not able to generate jokes on my own.
In general, jokes are a form of humor that rely on wordplay, surprise, or incongruity to create a humorous effect. They often involve a setup that establishes a certain expectation, and a punchline that subverts that expectation in a surprising or unexpected way. Jokes can be difficult to create, as they require
Positive values seem more robust and lasting than prohibitions. Imagine we train an AI on realistic situations where it can kill people, and penalize it when it does so. Suppose that we successfully instill a strong and widely activated "If going to kill people, then don't" value shard.
Even assuming this much, the situation seems fragile. See, many value shards are self-chaining. In The shard theory of human values, I wrote about how:
I strongly agree that self-seeking mechanisms are more able to maintain themselves than self-avoiding mechanisms. Please post this as a top-level post.
Yet another ChatGPT sample. Posting to shortform because there are many of these. While searching for posts to share as prior work, I found the parable of predict-o-matic, and found it to be a very good post about self-fulfilling prophecies (tag). I thought it would be interesting to see what ChatGPT had to say when prompted with a reference to the post. It mostly didn't succeed. I highlighted key differences between each result. The prompt:
Describe the parable of predict-o-matic from memory.
samples (I hit retry several times):
1: the standard refusal: I'm ...
I had the "your work/organization seems bad for the world" conversation with three different people today. None of them pushed back on the core premise that AI-very-soon is lethal. I expect that before EAGx Berkeley is over, I'll have had this conversation 15x.
#1: I sit down next to a random unfamiliar person at the dinner table. They're a new grad freshly hired to work on TensorFlow. In this town, if you sit down next to a random person, they're probably connected to AI research *somehow*. No story about how this could possibly be good for the world, rece...
Also every one of the organizations you named is a capabilities company which brands itself based on the small team they have working on alignment off on the side.
I'm not sure whether OpenAI was one of the organizations named, but if so, this reminded me of something Scott Aaronson said on this topic in the Q&A of his recent talk "Scott Aaronson Talks AI Safety":
...Maybe the one useful thing I can say is that, in my experience, which is admittedly very limited—working at OpenAI for all of five months—I’ve found my colleagues there to be extremely serious
Master post for alignment protocols.
Other relevant shortforms:
The AGI is going to be based on an algorithm which, as a mathematical fact (if not proved then at least conjectured), converges to the correct hypothesis with high probability.
I understand now, that was the main misunderstanding motivating my worries. This and your other two points have driven home for me the role mathematical guarantees play in the protocol, which I wasn't contemplating. Thanks again for your kind answers!
This is a list of random, assorted AI safety ideas that I think somebody should try to write up and/or work on at some point. I have a lot more than this in my backlog, but these are some that I specifically selected to be relatively small, single-post-sized ideas that an independent person could plausibly work on without much oversight. That being said, I think it would be quite hard to do a good job on any of these without at least chatting with me first—though feel free to message me if you’d be interested.
Disclaimer: At the time of writing, this has not been endorsed by Evan.
I can give this a go.
Unpacking Evan's Comment:
My read of Evan's comment (the parent to yours) is that there are a bunch of learned high-level-goals ("strategies") with varying levels of influence on the tactical choices made, and that a well-functioning end-to-end credit-assignment mechanism would propagate through action selection ("thoughts directly related to the current action" or "tactics") all the way to strategy creation/selection/weighting. In such a system, strategies which dec...
"Prompt engineer" is a job that AI will wipe out before anyone even has it as a job.
After reading LW more consistently for a couple weeks, I started recognizing rationalists in other parts of The Internet and wondered what were common social medias. My guesses are Twitter, Hacker News, StackExchange, and Quora in about that order, and I will eventually attempt to confirm this more rigorously, be it by demographic survey or username correlation (much less reliable).
For now, I was particularly interested in finding LW users that are also on Hacker News, so I quickly queried both sites and found ~25% of active LW users had Hacker News ...
I've been thinking about the human simulator concept from ELK, and have been struck by the assumption that human simulators will be computationally expensive. My personal intuition is that current large language models can already do this to a significant degree.
Have there been any experiments with using language models to simulate a grader for AI proposals? I'd imagine you can use a prompt like this:
The following is a list of conversations between AIs of unknown alignment and a human evaluating their proposals.
Request: Provide a plan to cure c...
There are a series of math books that give a wide overview of a lot of math. In the spirit of comprehensive information gathering, I'm going to try to spend my "fun math time" reading these.
I theorize this is a good way to build mathematical maturity, at least the "parse advanced math" part. I remember when I became mathematically mature enough to read Math Wikipedia, I want to go further in this direction till I can read math-y papers like Wikipedia.
3 is my main reason for wanting to learn more pure math, but I use 1 and 2 to help motivate me
Feature suggestion. Using highlighting for higher-res up/downvotes and (dis)agreevotes.
Sometimes you want to indicate what part of a comment you like or dislike, but can't be bothered writing a comment response. In such cases, it would be nice if you could highlight the portion of text that you like/dislike, and for LW to "remember" that highlighting and show it to other users. Concretely, when you click the like/dislike button, the website would remember what text you had highlighted within that comment. Then, if anyone ever wants to see that highlighting...
Switching costs between different kinds of work can be significant. Give yourself permission to focus entirely on one kind of work per Schelling unit of time (per day), if that would help. Don't spend cognitive cycles feeling guilty about letting some projects sit on the backburner; the point is to get where you're going as quickly as possible, not to look like you're juggling a lot of projects at once.
This can be hard, because there's a conventional social expectation that you'll juggle a lot of projects simultaneously, maybe because that's more legible t...
Because your utility function is your utility function, the one true political ideology is clearly Extrapolated Volitionism.
Extrapolated Volitionist institutions are all characteristically "meta": they take as input what you currently want and then optimize for the outcomes a more epistemically idealized you would want, after more reflection and/or study.
Institutions that merely optimize for what you currently want the way you would with an idealized world-model are old hat by comparison!
A multiagent Extrapolated Volitionist institution is something that computes and optimizes for a Convergent Extrapolated Volition, if a CEV exists.
Really, though, the above Extrapolated Volitionist institutions do take other people into consideration. They either give everyone the Schelling weight of one vote in a moral parliament, or they take into consideration the epistemic credibility of other bettors as evinced by their staked wealth, or other things like that.
Sometimes the relevant interpersonal parameters can be varied, and the institutional designs...
Since I did not keep it in a drawer as much as I thought let me make a note here to have a time stamp.
Instead of going
(units sold * unit price) - productions costs => enterpreneour compensation
go
(production costs+ enterpreneour compensation)/units sold => unit price
you get a system where it is impossible to misprice items.
Combined with other stuff you also get not having to lie or be tactical about how much you are willing to pay for a product and a self-organising system with no profit motive.
I am interested in this direction but because I do not think the proof passes the musters it would need to, I am not pushy about it.
I am not bothered. Cool to have interaction even if it is just reveals that inferential distance / mistepping is large.
Patron is a customer. Because they have a more vested interest how the product they bought is doing, it might make sense to use a word to remind of that.
We pay customers retroactively the difference they would have saved if they shopped later, so that they do not have reason to lie about their willingness to pay or have a race to shop last. All customers at all times have lost equal amount to have access to the product and this trends down...
I'm writing a 1-year update for The Plan. Any particular questions people would like to see me answer in there?
I had a look at The Plan and noticed something I didn't notice before: You do not talk about people and organization in the plan. I probably wouldn't have noticed if I hadn't started a project too, and needed to think about it. Google seems to think that people and team function play a big role. Maybe your focus in that post wasn't on people, but I would be interested in your thoughts on that too: What role did people and organization play in the plan and its implementation? What worked, and what should be done better next time?
Branding: 3 reasons why I prefer "AGI safety" to "AI alignment"
When engineers, politicians, bureaucrats, military leaders, etc. hear the word "safety", they suddenly perk up and start nodding and smiling. Safety engineering—making sure that systems robustly do what you want them to do—is something that people across society can relate to and appreciate. By contrast, when people hear the term "AI alignment" for the first time, they just don't know what it means or how to contextualize it.
There are a lot of things that people are working on in this spa
I think if someone negatively reacts to 'Safety' thinking you mean 'try to ban all guns' instead of 'teach good firearm safety', you can rephrase as 'Control' in that context. I think Safety is more inclusive of various aspects of the problem than either 'Control' or 'Alignment', so I like it better as an encompassing term.
I'd like to make a quite systematic comparison of openAi's chatbot performances in French and English. After a couple days trying things I feel like it is much weaker in French - which seems logical as it has much less data in French. I would like to explore that theory, so if you have interesting prompts you would like me to test let me know !