All of sapphire's Comments + Replies

FWIW this is around the amount of progress I was expecting in 2016. For better or worse I updated very hard toward very short timelines once alphago was released in 2015. Amusingly at the time I figured we had 10-20 years until strong AGI, which gives an average of 2030. I prefer to say strong AGI since as far as I'm concerned publicly available AGI was released in late 2022. 

Arguably EA/Rationality needed much simpler and less nuanced messaging on how to deal with aI capabilities companies. We really should have gone with 'absolutely do not help or work for companies increasing ai capabilities. Only work directly on safety.' Nuance is cool and all but the nuanced messaging arguably just ended up enabling Anthropic and OpenAI.

FWIW Im very angry about what happened and I think the speedup was around five years in expectation.

If this goes through I would strongly advise you to buy MSFT stock. I bought a ton when the possible deal was announced.

2Jeroen Willems5mo
While this might be a great way to earn money (assuming competitors won't invest similarly in AI soon enough), but aren't there good reasons not to invest in AI capabilities, like reducing P(doom)? Also I assume it's wise to mention you're not a financial adviser and don't bear responsibility for actions people take because of your comment (same counts for me).

Any kind of contact sport is lunacy. Even non-contact sports like track can hurt your joints if you aren't careful. Basketball is bad for your knees, etc. I would tell my kids if they want to exercise: swim or jog on sand. If they want to have fun play video games or have sex (bang in your bedroom at any age). I don't believe in being coercive but I hope my child would trust me enough they wouldn't even consider playing football or heading a soccer ball. 

Updated my 'diversified' portfolio for this:


MSFT - 10%
INTEL - 10%

Nvidia - 15%
SMSN - 15%
Goog - 15%
ASML - 15%

TSMC - 20%

1milo6mo
This is diworsified portfolio! Equities, Tech and Semiconductors concentrated

It doesn't seem to be the safest to be too sure how the future will go. Id recommend hedging for whatever possibilities you can. 

7Adam Jermyn6mo
And: having a lot of capital could be very useful in the run up to TAI. Eg for pursuing/funding safety work.

I have been saying similar things and really wish we had gotten started sooner. It's a very promising approach. 

-1[comment deleted]1y

solana is now under 40 but well above 3!

As of 2022 the fire alarm is ringing. I think there is a decent chance I will be dead in ten years. Whatever I want to do I need to do it soon. It has been very motivational:


-- Restored my relationship of ~10 years (we broke up in 2021 and got back together in 2022)


-- Actually learned to work on crypto apps. Wrote some cool code. Previously I just did crypto finance. I've always thought crypto was sweet and wanted to contribute but didn't push myself to start writing dapp code.


-- Started ramping up animal activism (animals matter in the future. If I only h... (read more)

4Not Relevant1y
Not over yet, if I thought there was no hope I wouldn’t be trying. The game is not unwinnable, it just takes skill and luck.

My favorite explanation of what it means to be a friend is "Pigs on the Wing"

You know that I care
What happens to you

And I know that you care
For me too

So I don't feel alone on the weight of the stone
Now that I've found somewhere safe to bury my bone
And any fool knows a dog needs a home
A shelter from pigs on the wing

Pink Floyd, Animals

Possibly my favorite written passage is the last verse of "Wish You Here". I thought about it all the time when I was separated from my best friend.

How I wish, how I wish you were here
We're just two lost souls
Swimming in a fish

... (read more)

The second post is really quite good. But I am not sure it's the best fit for what I wrote. It's not like I wrote about racism or how much people suffer in solitary confinement. I am expressing some truly niche views that are only popular among a tiny minority. For fundamentally I think my emotional reaction is grounded in some very basic cruxes. As far as I can tell turkeys served for food really do suffer terribly. I think one is just Eulering themself if they start rejecting 'suffering is bad'. 

I think the first post is an interesting critique. Res... (read more)

I am expressing some truly niche views that are only popular among a tiny minority.

This isn't really evidence against the second post being applicable. Every mainstream morality today must have started as a niche view among a tiny minority. A lot of humans probably have a status-gaining strategy of trying to be a successful moral vanguard. There are lots of examples of people playing status games in a tiny group. The Status Game has a chapter on Heaven's Gate, for example. Or maybe a better comparison is to the anti-vaxx movement, which must have once b... (read more)

All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it. It's like being a native French speaker and dropping in on a high school French class in a South Carolina public school, or like being someone who just learned how to tell good kerning from bad keming.  I spend a lot of time wincing, and I spend a lot of time not being able to fix The Thing That's Happening because the inferential gaps are so large that I'd have

... (read more)
9agrippa2y
Maybe there is some norm everyone agrees with that you should not have to distance yourself from your friends if they turn out to be abusers, or not have to be open about the fact you were there friend, or something. Maybe people are worried about the chilling effects of that. If this norm is the case, then imo it is better enforced explicitly.  But to put it really simply it does seem like I should care about whether it is true that Duncan and Brent were close friends if I am gonna be taking advice from him about how to interpret and discuss accusations made in the community. So if we are not enforcing a norm that such relationships should not enter discussion then I am unclear about the basis of downvoting here.

This quoted material has increased my respect for Duncan. Thank you for posting it.

Duncan hosted 2+ long FB threads at the time where a lot of people shared their experiences with Brent, and I think were some of the main ways that Berkeley rationalists oriented to the situation and shared information, and overall I think it was far better than the counterfactual of Duncan not having hosted those threads. I recall, but cannot easily find, Kelsey Piper also saying she was surprised how well the threads went, and I think Duncan's contributions there and framing and moderation were a substantial part of what went well.

It is public record he

... (read more)

Humans don't swim very well wearing lots of clothing. Take off your suit before going into the water.

6Richard_Kennaway2y
An adult may wade where a child would drown. "The water, in this parable, didn't look like it would be over their own adult heads." (I don't know if Eliezer added that after you brought this up.) (And "heads" should be singular. The class are children, and there is one hypothetical adult.)
5Slider2y
Isn't this an additional example of "practicality module" having integrity and not caving in to the "look I am rushing in so valorantly" impulse?

I actually keep thinking this in the back of my own mind every time I run into this parable, so thank you for stating it out loud.  (I expect if a child brought it up, the Watcher credited them for noticing further consequences, then asked to assume the Less Convenient Possible World where this is not the case.)

Animal rights obsessed vegan checking in:

I am extremely worried gpt3 is concious! To be honest i am worried about whether my laptop is concious! A lot of people worried about animal suffering are also worried about algorithms suffering.

7Mart_Korz2y
It seems I am not as worried about gpt3 as you, but when listening to the  simulated interview with simulated Elon Musk by Lsusr [https://www.lesswrong.com/posts/oBPPFrMJ2aBK6a6sD/simulated-elon-musk-lives-in-a-simulation#fnref-DJwdDfxvjSoiBK6Ka-2] in the clearer thinking podcast episode 073 [https://clearerthinkingpodcast.com/?ep=073] (starts in minute 102), I was quite concerned

I have suffered the severe consequences of selling a chunk of my Solana around 40. 

2sapphire1y
solana is now under 40 but well above 3!

This reaction has been predictable for years IMO. As usual, a reasonable response required people to go public. There is no internal accountability process. Luckily things have been made public.

Releasing GPT-3 non-trivially increased the odds of doomsday. So yeah they are not good actors.

Can you elaborate on that? It seems non-obvious; I feel like I could tell a different story

... like, OpenAI has much fewer resources than Deepmind, so if Deepmind released GPT-3 first, and if it is the case that scaling up a language model far enough gives you AGI, there would have been less time in between knowing what AGI will look like and having AGI.

Many people have proposed this model with respect to animals but it isn't true. This old post covers a particular saliant time it came up in the rationalist community. https://agentyduck.blogspot.com/2014/11/how-i-feel-about-emotional-appeals.html

Vegans: If the meat eaters believed what you did about animal sentience, most of them would be vegans, and they would be horrified by their many previous murders. Your heart-wrenching videos aren't convincing to them because they aren't already convinced that animals can feel.

 

I think the people who changed m

... (read more)

In my experience, people seem to coordinate ok when they genuinely share the same goals. A lot of friends of mine have mostly shared the goal of 'make six to eight figures on crypto'. Truly enormous amounts of money was loaned on trust. Several deals were made when the price was still unclear (just get me 30K on Biden) and people never asked for receipts. No one was ever stiffed out of their money. As far as I know, there has not been a single serious dispute.

Many of the people involved don't even like each other! And yet huge amounts of money changed hand... (read more)

Good amount of RSVPs. Excited to meet people.

I don't know how it will all play out in the end. I hope kindness wins and I agree the effect you discuss is real. But it is not obvious that our empathy increases faster than our capacity to do harm. Right now, for each human there are about seven birds/mammals on farms. This is quite the catastrophe. Perhaps that problem will eventually be solved by lab meat. But right now animal product consumption is still going up worldwide. And many worse things can be created and maybe those will endure.

People can be shockingly cruel to their own family. Scott's Who... (read more)

I don't particularly blame humans for this world being full of suffering. We didn't invent parasitoid wasps. But we have certainly not used our current powers very responsibly. We did invent factory farms. And most of us do not particularly care. 

I am very afraid more powerful humans/human-aligned beings will invent even worse horrors. And if we tolerate factory farming it seems possible we will tolerate the new horrors. So I cannot be confident that humans gaining more power, even if it was equitably distributed among humans, would actually be a good... (read more)

But don't you share the impression that with increased wealth humans generally care more about the suffering of others? The story I tell myself is that humans have many basic needs (e.g. food, safety, housing) that historically conflicted with 'higher' desires like self-expression, helping others or improving the world. And with increased wealth, humans relatively universally become more caring. Or maybe more cynically, with increased wealth we can and do invest more resources into signalling that we are caring good reasonable people, i.e. the kinds of peo... (read more)

It is not obvious at all that 'AI aligned with its human creators' is actually better than Clippy. Even AI aligned with human CEV might not beat Clippy. I would much rather die than live forever in a world with untold numbers of tortured ems, suffering subroutines, or other mistreated digital beings. 

Few humans are actively sadistic. But most humans are quite indifferent to suffering. The best illustration of this is our attitude toward animals. If there is an economic or ideological reason to torment digital beings we will probably torture them. The ... (read more)

1Shiroe1y
Exactly your point is what has prevented me from adopting the orthodox LessWrong position. If I knew that in the future Clippy was going to kill me and everyone else, I would consider that a neutral outcome. If, however, I knew that in the future some group of humans were going to successfully align an AGI to their interests, I would be far more worried. If anyone knows of an Eliezer or SSC-level rebuttal to this, please let me know so that I can read it.
1MSRayne2y
The way I see it, the only correct value to align any AI to is not the arbitrary values of humans-in-general, assuming such a thing even exists, but rather the libertarian principle of self-ownership / non-aggression. The perfect super-AI would have no desire or purpose other than to be the "king" of an anarcho-monarchist world state and rigorously enforce contracts (probably with the aid of human, uplift, etc interpreters, judges, and juries stipulated in the contracts themselves, so that the AI does not have to make decisions about what is reasonable), including a basic social contract, binding on all sentient beings, that if they possess the capacity for moral reasoning, they are required to respect certain fundamental rights of all other sentient beings. (This would include obvious things like not eating them.) It would, essentially, be a sentient law court (and police force, so that it can recognize violations and take appropriate action), in which anything that has consciousness has its rights protected. For a super-AI to be anything other than that is asking for trouble.
2countingtoten2y
What do you believe would happen to a neurotypical [https://threadreaderapp.com/thread/1368966115804598275.html] forced to have self-awareness and a more accurate model of reality in general? The idea that they become allistic neurodivergents like me is, of course, a suspicious conclusion, but I'm not sure I see a credible alternative. CEV seems like an inherently neurodivergent idea, in the sense that forcing people (or their extrapolated selves) to engage in analysis is baked into the concept.

It is not obvious at all that 'AI aligned with its human creators' is actually better than Clippy.

It's pretty obvious to me, but then I am a human being. I would like to live in the sort of world that human beings would like to live in.

I think it's a bad use of time these days. 

Imagine thinking the intelligence community will be honest.

4ChristianKl2y
A lot of the individual facts are already out there from the NIH letter to the EcoHealth alliance and what the intelligence community leaked. It's really hard to tell a story given the facts that doesn't come out with the lab leak being the most likely explanation.  Additionally, the intelligence community itself won't be harmed by finding that there was a lab leak. They can leverage it into more funding by saying that they didn't have enough manpower in October 2019 to investigate the WIV when it started acting strange and call for more funding to surveil biosafety labs. The story is good for justying more NSA spying powers.  How often does the US intelligence community fail to find the story that helps them to justify their spying powers?

I usually cut the crust off my bread! It is way tastier that way. 

These leverage tokens do not behave how they should. Very few if any people should use them. But Binance did not lose use funds. If you were doing perp-spot arbitrage on Binance you were not at any risk.

note: the future-spot arb has indeed dried up.

People, by default, do not care very much about the suffering of the powerless. This is a very general pattern: De facto torture of the old and dying (Who by Very Slow Decay), Animal in factory farms, prisoners in solitary, Scott's rant about school being child prison, etc. 

There is no real need to explain a specific example of a very general trend. In fact it is the opposite that needs explanation. We actually have made progress in various ways (for example slavery is greatly reduced, though the USA still has prisoner slaves). But compassion toward t... (read more)

4Slider2y
From my Nordic perspective being ok to standby when other groups fail to powergrab decent conditions seems like an outlier.
9ChristianKl2y
There are a lot of people who care about animals in factory farms, a lot of grant money and many organizations. Prison reform is similar. Both causes are clearly part of the modern left coalition.  There aren't similarly powerful organizations that care about the rights of patients in psychiatric hospitals.

Update. I exited the oly bet 2 days ago at 87.2 percent. Just letting people know.

I am holding some eth. Hope this timeline happens. But color me skeptical. 20k eth seems doable though. I think we probably cross 20k if eth 2.0 and optimism both perform well.

Cardano has been very Too weak Too Slow. They have a giant valuation but they still haven't launched smart contracts! The coin launched in 2017, so they had plenty of time to work. Despite this they have the fourth-highest valuation! (excluding stablecoins). This situation seems crazy to me on fundamentals. Of course, they probably will pump once smart contracts launch. But despite having most of my net worth in crypto I am not buying Cardano.

2mako yass2y
My stance on how long it's taken for them to deliver smart contracts depends on when they said they would deliver smart contracts. Are they long past a previously stated deadline? Or did they expect the preliminary formal work to take this long? If they expected it to take this long (or like, 2.5 years), and still decided to do the formal work, that would be a credible signal that they really believe it's that important and I would be moved by it. Otherwise, cause for concern.

Explicit YOLO ALL IN numbers for my biggest bags. Until we hit these numbers I am not shoving everything:

Sol - 8 

Matic - 0.3 

FTT - 7.5 

ETH - 500 

BTC - 12K

Remember in 2018 ETH went 1400->89 and it wasn't all downhill. Lower highs, lower lows. Be ready for the opportunity.

The early Pro Tour seemed amazing. When I followed the pro your ten years ago it was still extremely interesting. But I started interest years before the MPL. They have continually de-emphasized draft. They killed Rochestor draft in ~2005. Then they got rid of the all draft pro tours in 2008. In 2020 they completely removed draft from 'players tours'. 

De-emphasizing draft was bad enough. But the real death kneel was when they moved the Pro/Player's Tours so far back from set release. I am not sure exactly when they started doing this but it feels like... (read more)

I have been saying this on other forums (where more people listen to me) for over a week but I might as well post it here too:

I have been pushing crypto quite hard. I do NOT recommend buying crypto until momentum reverses. If taxes and bankroll were not relevant I would say buying and selling are probably close to equivalent with a slight preference to selling. Of course, this means I put a decent probability on a quick recovery but I am not betting on it. Make sure you have at least some dry powder for the deep bear market if it comes. I hope you took som... (read more)

The people who make ad blockers have improved my life enormously. It's worth noting Facebook timeline ads are not particularly annoying. Maybe forcing sites to mask their ads as content is a good thing.

0meedstrom2y
Compare with a world where an omnipresent virus has been eating all adblockers since the beginning. Perhaps some of us would be in a better place, using the internet less. Perhaps it'd actually have become a norm to pay for ad-free content and many of us would stay on the parts of the web we paid for. If that's the alternative, I believe they have harmed your life. Remember that bending backwards for ads is not just making your site uglier -- it's doing everything to maximize pageviews and clicks, which affects everything, down to selecting the very kinds of content that get posted. The web as a whole is less honest. Are "ads" just visual clutter to you? To me it's a low signal-to-noise ratio, demonstrated by the filler content on a cooking recipe or how a 10-minute YouTube video is carved into 3 minutes of introduction, 3 minutes of pointless digressions, 1 minute of information, 3 minutes on why the information was great, and "please like and share and don't forget to hit that subscribe button". Masked ads are in the same vein, you have to realize you're reading "nutritionally empty" content and filter it out. It's immensely tiresome and I'd rather have simple visual clutter.

It only assumes there are a lot of domains in which you would be happy to make progress. In addition success is at least somewhat fungible across domains. And it is much easier to cut red tape once you already resources and a track record (possibly in a different domain). 

Don't start out in a red-tape domain unless you are ready to fight off the people trying to slow you down. This requires a lot of money, connections, and lawyers and you still might lose. Put your talents to work in an easier domain, at least to start.

If you are a smart person I suggest working in domains where the regulators have not yet shut down progress. In many domains, if you want to make progress most of your obstacles are going to be other humans. It is refreshingly easy to make progress if you only have to face the ice.

6Vladimir_Nesov2y
That's ambitious without an ambition. Switching domains stops your progress in the original domain completely, so doesn't make it easier to make progress. Unless domain doesn't matter, only fungible "progress".

Everyone shits on younger people (say 14-22). But the younger people I interact with have a lot of great ideas and good perspectives. I am always happy to have a chance to learn from them. Why so much hate? 

3Gunnar_Zarncke2y
Because what they do or believed to do - gaming and social - is seen as idling/sloth.

Crypto is not doing the hottest. But another part of the story is BTC is losing dominance. Though who knows what is about to happen.

Use Authy, not Google authenticator. GA not supporting any sort of backups is a huge problem.

The magnitudes are much smaller on both sides of a recent situation. My father died three weeks ago. The family somehow ended up spending over 40K on death expenses (funeral, mausoleum slot, flowers, etc). My mom ended up spending six hundred and fifty dollars on a single rose arrangement in the shape of a big heart. The time to convince my family of effective altruism is not right after my father died. But I found spending so much money on a dead man, or more charitably expensive rites for the living, horrifying to see up close. Six hundred and fifty doll... (read more)

2hookdump2y
I'm very sorry for your loss. I do agree with your view on the ineffectiveness of spending a lot of money on a dead person. However... I think people find meaning from a wide range of things, and rituals around birth and death are two big ones since the origins of humankind itself. So in a sense, I can empathize with your family spending so much money. It seems to mean something to them. Sacrificing material things in funerary rituals is as old as Ancient Egypt [https://en.wikipedia.org/wiki/Ancient_Egyptian_funerary_practices], and probably much older. The only new component in modern society is that now this sacrifice transforms into profit for someone else (the funerary industry, who operate under dubious ethical grounds). But from the perspective of the griefing loved ones of the deceased, I think it's all about meaning and closure.

Years of accumulated experience and metis point the other way, at least for this audience. Anyone who has spent time in rationalist or rationalist adjacent spaces knows that a huge percentage of rationalists* are trans. After many years of being an active rationalist* I literally know dozens of trans people in or adjacent to the community. If a rationalist is struggling with whether to transition they should try transitioning. A huge number of rationalists think trying hormones was the best decision they ever made, very few seriously regret it. If a ration... (read more)

The coronavirus response has been so bad I no longer doubt many Sci-Fi premises. Before I often said to myself "you have tech that can do X and you still have problem Y. Ridiculous!". But apparently, we can make a coronavirus vaccine in two days. But we still had over a year of lockdowns and millions of deaths. One in six hundred people in the USA have died of the virus but we blocked a vaccine because one in a million people who take it MIGHT develop treatable blood clots. 

My standards for 'realistic' dysfunction have gotten a lot lower.

4Raemon2y
On the flipside: WTF Star Trek? 
3eigen2y
The heads of Government and the FDA don't work like you do. Who knows the incentives that they have? It's entirely possible that for them this is just a political play(moving chess pieces) that make sense for them while the well-being of the people take secondary place to a particular political move. This wouldn't be the first thing that this happens in any kind of government agency, but, at any rate, it's too early to be skeptical. We need to see how this unfolds, may be the pausing don't last as much.
4niplav2y
I remember Yudkowsky asking for a realistic explanation [https://twitter.com/ESYudkowsky/status/1061336147744317440] for why the Empire in Star Wars is stuck in an equilibrium where it builds destroyable gigantic weapons.

There are several groups now because people wanted to keep some topics seperated. Sadly I don't think any of them are open to the public.

Which epistemic values? I am posting these trades in public (not always on lesswrong itself but in adjacent spaces which are easy enough to check if needed). If they blow up I will suffer the reputational consequences. The inverse should occur if they work out well. Do you want to bet on whether FTX.com losses a substantial amount of user funds in the next year? What odds do you give?

The opportunities don't seem gone to me. Though I doubt we see anything as good as 2011 bitcoin. Many rationalists bought Solana around January 7th when it unlocks. I posted a... (read more)

So update as of November 23rd, 2022... 

FTX is bankrupt, with at least an $8 billion hole on their balance sheet. Customer funds gone, many people lost 6 and even 7 figure balances.

So... what were those odds on "FTX.com losses a substantial amount of user funds" again?

The more info that comes out, the more it looks like the entire thing was fraudulent from the beginning. They even have a software back-door that allows them to transfer funds without alerting auditors, which it looks like they used many times with FTX customer deposits.

Additionally...

Cel... (read more)

1GeneSmith2y
I know I'm kind of late to the party here, but here is the best argument against the future value of Bitcoin I've yet read [https://elidourado.com/blog/bitcoin-fee-market] This is specifically about Bitcoin, and does not apply to other coins that don't have a fixed supply, or those that use a different consensus mechanism. For anyone who stumbles across this comment in the future, the argument is basically that there is no market mechanism to ensure transaction fees stay high enough to maintain block security. The argument hinges on the assumption that miners need to be rewarded at least 1% of daily transaction volumes for them to be willing to spend money on the ASICs needed to mine blocks.
1antanaclasis2y
You mention the EA investing group. Where is that? A cursory search didn’t seem to bring anything up. Also, more generally speaking, what would be your top few recommendations of places to keep up with the latest rationalist investment advice?

1 - The historical returns would obviously have been ludicrously high. I assume advice will be worse going forward. There has not been all that much investing advice on lesswrong and some of it was to buy crypto relatively early. If some of your investments 200-2000x you don't need to be right about much else. And it is not like all the other advice would have gone terribly. Tech stocks are doing great. Some of the advice was also boring stuff like 'buy index funds'. Do you think there are hundreds of examples of investment ideas on lesswrong that went to ... (read more)

How many LWers bought Bitcoin in 2011 and ended up with poor returns due to the demise of Mt. Gox?

I almost ended up losing a little of my money that way, but was stopped when my KYC evidence was rejected for no clear reason. I ended up doing well by buying Ripples a couple of years later, but I don't know whether I would have done that if I had lost money in my first attempt.

A good friend of mine, who I know is making absurd returns on altcoins, is still bummed about that losing his entire investment on that options trade. Losing 100% is pretty demoralizing I guess. 

Load More