All of aphyer's Comments + Replies
Chess is a game where, in every board state, almost all legal moves are terrible and you have to pick one of the few that aren't.
So is reality.
Another thing to keep in mind is that a full set of honest advisors can (and I think would) ask the human to take a few minutes to go over chess notation with them after the first confusion. If the fear of dishonest advisors means that the human doesn't do that, or the honest advisor feels that they won't be trusted in saying 'let's take a pause to discuss notation', that's also good to know.
Question for the advisor players: did any of you try to take some time off explain notation to the human player?
This is true, but in general the differences between an ordinary employee and a CEO go in the CEO's favor. I believe this does also extend to 'how are they fired': on my understanding the modal way a CEO is 'fired' is by announcing that they have chosen to retire to pursue other opportunities/spend more time with their family, and receiving a gigantic severance package.
Disclaimer: I do not work at OpenAI and have no inside knowledge of the situation.
I work in the finance industry. (Personal views are not those of my employer, etc, etc).
Some years ago, a few people from my team (2 on a team of ~7) were laid off as part of firm staff reductions.
My boss and my boss's boss held a meeting with the rest of the team on the day those people left, explaining what had happened, reassuring us that no further layoffs were planned, describing who would be taking over what parts of the responsibilities of the laid-off people, et...
1Visits to emergency rooms might not be down if parents are e.g. panicking and bringing a child to the ER with a bruise.
The board had a choice.
If Ilya was willing to cooperate, the board could fire Altman, with the Thanksgiving break available to aid the transition, and hope for the best.
Alternatively, the board could choose once again not to fire Altman, watch as Altman finished taking control of OpenAI and turned it into a personal empire, and hope this turns out well for the world.
They chose to pull the trigger.
I...really do not see how these were the only choices? Like, yes, ultimately my boss's power over me stems from his ability to fire me. But it w...
6
1Speculating of course, but it reads to me like the four directors knew Altman was much better at politics and persuasion than they were. They briefly had a majority willing to kick him off, and while "Sam would have found a basilisk hack to mind-control the rest of the board" is phrased too magically to me I don't think it's that far off? This sort of dynamic feels familiar to me from playing games where one player is far better than the others at convincing people.
(And then because they were way outclassed in politics and persuasion they handled the aftermath of their decision poorly and Altman did an incredible job.)
I think the central confusion here is: why, in the face of someone explicitly trying to take over the board, would the rest of the board just keep that person around?
None of the things you suggested have any bearing whatsoever on whether Sam Altman would continue to try and take over the board. If he has no board position but is still the CEO, he can still do whatever he wants with the company, and also try to take over the board. If he is removed as CEO but remains on the board, he will still try to take over the board. Packing the board has no bearing on...
1If you pushed for fire sprinklers to be installed, then yell "FIRE", and turn on the fire sprinklers, causing a bunch of water damage, and then refuse to tell anyone where you thought the fire was and why you thought that, I don't think you should be too surprised when people contemplate taking away your ability to trigger the fire sprinklers.
The situation is actually even less surprising than this, because the thing people actually initially contemplated doing in response to the board's actions was not even 'taking away your ability to trigger the f...
I think this is confusing 'be nice' with 'agree'.
Alice suggests that civilization should spend less on [rolls dice] animal shelters and more on [rolls dice] soup kitchens.
Bob: Yes, I absolutely agree.
Charlie: I don't think I agree with that policy. I think animal shelters are doing good work, and I wouldn't want to see them defunded to pursue some other goal.
David: YOU MONSTER HOW CAN YOU SAY CUTE PUPPIES DESERVE TO DIE I HATE YOU AND I HOPE YOUR FAMILY GETS EATEN ALIVE BY COCONUT CRABS.
If your interpretation of 'nice' is such that Bob is 'nice' and ...
...Some of the feedback was disappointing– people said things along the lines of: “Lies of omission are not lies! The fact that you have to add ‘of omission’ shows this!”.
The worst that I have gotten in that vein was: “When you are on stand, you take an oath that says that you will convey the truth, the whole truth, and nothing but the truth. The fact that you have to add ‘the whole truth’ shows that lies of omission are not lies.”
This was from a person that also straightforwardly stated “I plan to keep continuing to not state my most relevant opinions in pub
1If you died and went to a Heaven run by a genuinely benevolent and omnipotent god, would it be impossible for you to enjoy yourself in it?
2I think it is very hard to artificially forbid that: there isn't a well-defined boundary between playing out a full game and a conversation like:
"that other advisor says playing Rd4 is bad because of Nxd4, but after Nxd4 you can play Qd6 and win"
"No, Qd6 doesn't win, playing Bf7 breaks up the attack."
One thing that might work, though, is to deny back-and-forth between advisors. If each advisor can send one recommendation, and maybe one further response to a question from A, but not have a free-form conversation, that would deny the ability to play out a game.
I'm rated ~1700 on chess.com, though I suspect their ratings may be inflated relative to e.g. FIDE ones. Happy to play whatever role that rating fits best with. I work around NYC at a full-time job: I'm generally free in the evenings (perhaps 7pm-11pm NY time) and on weekends.
Two questions:
-
Do you anticipate using a time control for this? I suspect B will be heavily advantaged by short time controls that don't give A much time, while A will be heavily favored by having enough time to e.g. tell two advisors who disagree 'okay, C1 thinks that move is a
Fooling people into thinking they're talking to a human when they're actually talking to an AI should be banned for its own sake, independent of X-risk concerns.
I feel like your argument here is a little bit disingenuous about what is actually being proposed.
Consider the differences between the following positions:
1A: If you advertise food as GMO-free, it must contain no GMOs.
1B: If your food contains GMOs, you must actively mark it as 'Contains GMOs'.
2A: If you advertise your product as being 'Made in America', it must be made in America.
2B: If your produ...
You have as broad categories:
- Fizzlers saying capable AGI is far or won’t happen
- How-Skeptics saying AGI won’t be able to effectively take over or kill us.
- Why-Skeptics saying AGI won’t want to.
- Solvabilists saying we can and definitely will solve alignment in time.
- Anthropociders who say ‘but that’s good, actually.’
Pithy response: You don't need to not believe in global warming to think that 'use executive order to retroactively revoke permits for a pipeline that has already begun construction' is poor climate policy!
Detailed response: One that I think is nota...
Is the fruit you're thinking of typically red in color when ripe?
Yes
Is it a strawberry?
No
Is it a cherry?
No
Is it a raspberry?
No
Is it a pomegranate?
No
Is it a blueberry?
No. Remember, it’s often red when ripe.
My apologies for the oversight. Is it a cranberry?
No
I apologize for the previous guesses. Is it a currant?
No
I'm sorry for the incorrect guesses. Is it a red currant?
No
This part is actually fascinating. If this were a human I would imagine that they were trolling you. Here I'm wondering whether it gets through strawberry-cherry and then d...
The big problem faced by cooperative games without hidden agendas is that they are fundamentally solitaire games.
Imagine four people playing chess 'as a team' with the following ruleset:
- Player 1 controls the King and Queen
- Player 2 controls the Bishops and Knights
- Player 3 controls the Rooks.
- Player 4 controls the Pawns.
- Each turn, choose one player. That player may move one of their pieces once. Then, your opponent moves.
It's easy to see that the way to play this if you want to do well is simply to have whichever player is best at chess take over,...
The "every coal burned contributes to global warming, past a certain cap everybody loses" approach is not going to work in an otherwise competitive game. Instead, it will create a race to use up the coal budget as fast as possible so that your opponents can't benefit from it.
Citation: Twilight Struggle
Maybe I'm unusual in this regard, but peering at my mirrors closely enough to make out a car in one of these seems harder than just head-checking my blind spot?
Does the lie detection logic work on humans?
Like, my guess would be no, but stranger things have happened.
If the information environment prevents people from figuring out the true cause of the obesity epidemic, or making better engineered foods, this affects you no matter what place and what social circles you run in. And if epistemic norms are damaged in ways that lead to misaligned AGI instead of aligned AGI, that could literally kill you.
The stakes here are much larger than the individual meat consumption of people within EA and rationality circles. I think this framing (moralistic vegans vs selfish meat eaters with no externalities) causes people to misunderstand the world in ways that are predictably very harmful.
'If done intelligently' is really one hell of an 'if'.
Yes, intelligent climate change mitigation strategies would not cost very much. (On some assumptions about nuclear power, intelligent climate change mitigation strategies might have negative cost).
But the more relevant question is the cost of the climate change mitigation strategies we actually get.
How well do you think current governments do at abiding by their past commitments that large numbers of their constituents disagree with?
Probably generally not true, you're right. Even if they are prosecuted, 'fined' or 'barred from the securities industry' are more likely. I do think legal trouble would be at least very plausible. I'll edit the parent comment.
(Obligatory disclosure I guess: I work in the financial industry, though not in a way related to mortgages or housing. Anything I write here is my opinion and not that of my employer.)
______________________________________________
Imagine a world where, as well as their regular business selling groceries, grocery stores sell tokens that entitle you to a perpetual stream of groceries. Rather than spending $200/week on groceries, you spend...let's say (200*52/0.05)=$208,000 to buy a Grocery Token.
Of course many people cannot afford $20...
I had a similar gut reaction. When I tried to run down my brain's root causes of the view, this is what it came out as:
...There are two kinds of problem you can encounter in politics. One type is where many people disagree with you on an issue. The other type is where almost everyone agrees with you on the issue, but most people are not paying attention to it.
Protests as a strategy are valuable in the second case, but worthless or counterproductive in the first case.
If you are being knifed in an alleyway, your best strategy is to make as muc
I think that, in particular, protesting Meta releasing their models to the public is a lot less likely to go well than protesting, say, OpenAI developing their models. Releasing models to the public seems virtuous on its face both to the general public and to many technologists. Protesting that is going to draw attention to that specifically and so tend to paint the developers of more advanced models in a comparatively better light and their opponents in a comparatively worse light compared.
I agree with your assessment of the situation a lot, but I disagree that there is all that much controversy about this issue in the broader public. There is a lot of controversy on lesswrong, and in tech, but the public as a whole is in favor of slowing down and regulating AI developments. (Although other AI companies think sharing weights is really irresponsible and there are anti-competitive issues with llama 2’s ToS, which why it isn’t actually open source.) https://theaipi.org/poll-shows-overwhelming-concern-about-risks-from-ai-as-new-institute-launche...
Very nearly everyone agrees that there is a meaningful difference between action and inaction?
Alice is trying to decide whether to give Bob $10.
Claire is trying to decide whether to steal $10 from Bob.
If you refuse to acknowledge a difference between action and inaction, you can claim that both of these scenarios represent 'choosing whether $10 should end up in Bob's pocket or in your own', and therefore that these two situations are the same, and therefore that Alice's obligation to give Bob $10 is exactly as strong as Claire's obligation to not steal $10 from him.
Outside of the deep end of Peter-Singer-style altruism, though, I don't think many people believe that.
2[Chloe was] paid the equivalent of $75k[1] per year (only $1k/month, the rest via room and board)
So, it's not the most important thing in the post, but this sounds hella sketchy. Are you sure these are the numbers that were given?
$75k/yr minus $1k/mo leaves $63k/year in 'room and board'. The median household income in New York City is $70,663/yr per census.gov. Where were they boarding her, the Ritz Carlton?
This is more false info. The approximate/expected total compensation was $70k which included far more than room and board and $1k a month.
Chloe has also been falsely claiming we only had a verbal agreement but we have multiple written records.
We'll share specifics and evidence in our upcoming post.
The 'whole point of libel suits' is to weaponize the expensive brokenness of the legal system to punish people for saying mean things about you.
Going forward I think anyone who works
with Kat Woods, Emerson Spartz, or Drew Spartz,should sign legal employment contracts, and make sure all financial agreements are written down in emails and messages that the employee has possession of. I think all people considering employmentby the above people at any non-profits they runshould take salaries where money is wired to their bank accounts, and not do unpaid work or work that is compensated by ways that don’t primarily include a salary being wired to their bank accounts.
FTFY.
While I have no knowledge ...
9I have worked without legal contracts for people in EA I trust, and it has worked well.
Even if all the accusation of Nonlinear is true, I still have pretty high trust for people in EA or LW circles, such that I would probably agree to work with no formal contract again.
The reason I trust people in my ingroup is that if either of us screw over the other person, I expect the victim to tell their friends, which would ruin the reputation of the wrongdoer. For this reason both people have strong incentive to act in good faith. On top of that I'm wiling to take ...
Yeah, this post makes me wonder if there are non-abusive employers in EA who are nevertheless enabling abusers by normalizing behavior that makes abuse popular. Employers who pay their employees months late without clarity on why and what the plan is to get people paid eventually. Employers who employ people without writing things down, like how much people will get paid and when. Employers who try to enforce non-disclosure of work culture and pay.
None of the things above are necessarily dealbreakers in the right context or environment, but when an employ...
Haha, I like your edit. I do think there are exceptions — for instance if you are independently wealthy, you might take no salary, and I expect startups cofounders have high-trust non-legal agreements while they're still getting started. But I think that trust is lost for Kat/Emerson/Drew and I would expect anyone in that relationship to regret it. And in general I agree it's a good heuristic.
(Disclosures: am American. Strong views presented without evidence.)
The most damning indictment of French food I can think of is the fact that American capitalism hasn't even bothered stealing it. We have Italian restaurants on every corner, Chinese and Mexican and Thai and Indian and Korean and every other cuisine from every other corner of the world...except French. One time I went to a Korean hotpot place, but it was too full and had a long wait, so instead of waiting I walked to a different Korean hotpot restaurant. There are tw...
5American capitalism "stealing" food is usually a process of lower-income, unskilled migrants moving to a country and adapting their cuisines to American tastes/ ingredients, which explains the wave of Italian (historically), Chinese, Mexican, Thai and Indian places far better than the quality of their respective cuisines. Not sure about Korean/ Japanese places (higher income), but (in Europe at least) they're mostly run by people from Wenzhou, unless they're high-end, which may be an interesting exception to the rule.
I'd guess you see very few restau...
If you're allowed to cancel a pledge at any point, there's really very little reason not to just fund anything with a refund bonus the moment it posts, aiming to cancel your pledge if it looks like it might succeed.
Some thoughts:
#1: If a project is almost funded, a creator can contribute money themselves (or e.g. indirectly via a friend). The example above said that if you offered a 10% refund on a $100k project:
The most you'd have to pay is 10% of $99,999
but in practice if you raised $95,000, rather than pay back $9,500 in bounties and have the project fail you'll probably just kick in $5k of your own money to make the project succeed. Platforms can try to forbid this, but it's probably going to be quite hard to do so.
#2: If nobody or n...
This seems to assume that social graces represent cooperative social strategies, rather than adversarial social strategies. I don't think this is always the case.
Consider a couple discussing where to go to dinner. Both keep saying 'oh, I'm fine to go anywhere, where do you want to go?' This definitely sounds very polite! Much more socially-graceful than 'I want to go to this place! We leave at 6!'
Yet I'd assert that most of the time this represents these people playing social games adversarially against one another.
If you name a place and I agree to g...
...Tyler Cowen has his unique take on the actors strike and the issue of ownership of the images of actors. As happens frequently, he centers very different considerations than anyone else would have, in a process that I cannot predict (and that thus at least has a high GPT-level). I do agree that the actors need to win this one.
I do agree with his conclusion. If I got to decide, I would say: Actors should in general only be selling their images only for a particular purpose and project. At minimum, any transfer of license should be required to come with due
Actually, I feel like even this was pretty predictable: the text was entirely valid English words. If a text-prediction engine were reading through this character-by-character trying to predict the upcoming character, they would have failed on the first few characters of each word, but would still have been able to predict quite a lot: there aren't many words that begin with 'malar'.
I posted it like this anyway rather than aiming for actually unpredictable text because I thought that this text was funnier than a string of entirely random characters.
Putting a claim into ChatGPT and getting "correct" is little evidence that the claim is correct.
Unless your claims are heavily preselected e.g. for being confusing ones where controversial wisdom is wrong, I think this specific example is inaccurate? If I ask ChatGPT 'Is Sarajevo the capital of Albania?', I expect it to be right a large majority of the time.
I actually edited to include your PVE change, you did manage a 64% winrate. Sorry not to give you more time, didn't realize there was work still ongoing.
Sorry, wasn't expecting anything today! I'll update the wrapup doc to reflect your PVE answer: sadly, even if you had an updated PVP answer, I won't let you change that now :P
Sure, no objections. In the absence of further requests I'll aim to post the wrapup doc Friday the 9th: I'm fairly busy midweek and might not get around to posting things then.
Very minor gripe: '22m' parses to me as '22 years old and male', which was briefly confusing. Maybe '22mo' would be clearer?
For example, here’s a Nash equilibrium: “Everyone agrees to put 99 each round. Whenever someone deviates from 99 (for example to put 30), punish them by putting 100 for the rest of eternity.”
I don't think this is actually a Nash equilibrium? It is dominated by the strategy "put 99 every round. Whenever someone deviates from 99, put 30 for the rest of eternity."
The original post I believe solved this by instead having the equilibrium be “Everyone agrees to put 99 each round. Whenever someone deviates from 99 (for example to put 30), ...
Apologies, I was a bit blunt here.
It seems to me that the most obvious reading of "the burden of proof is on developers to show beyond-a-reasonable-doubt that models are safe" is in fact "all AI development is banned". It's...not clear at all to me what a proof of a model being safe would even look like, and based on everything I've heard about AI Alignment (admittedly mostly from elsewhere on this site) it seems that no-one else knows either.
A policy of 'developers should have to prove that their models are safe' would make sense in a world wh...
I don't think you're being consistently downvoted: most of your comments are neutral-to-slightly positive?
I do see one recent post of yours that was downvoted noticeably, https://www.lesswrong.com/posts/48X4EFJdCQvHEvL2t/ethicophysics-ii-politics-is-the-mind-savior
I downvoted that post myself. (Uh....sorry?) My engagement with it was as follows:
- I opened the post. It consisted of a few paragraphs that did not sound particularly encouraging ('ethicophysical treatment...modeled on the work of Hegel and Marx' does not fill me with joy to read), plu
... (read more)