it seems to me that disentangling beliefs and values are important part of being able to understand each other
and using words like "disagree" to mean both "different beliefs" and "different values" is really confusing in that regard
just a loose thought, probably obvious
some tree species self-selected themselves for height (ie. there's no point in being a tall tree unless taller trees are blocking your sunlight)
humans were not the first species to self-select (for humans, the trait being intelligence) (although humans can now do it intentionally, which is a qualitatively different level of "self-selection")
on human self-selection: https://www.researchgate.net/publication/309096532_Survival_of_the_Friendliest_Homo_sapiens_Evolved_via_Selection_for_Prosociality
Litany of Tarski for instrumental rationality 😊
If it's useful to know whether the box contains a diamond,
I desire to figure out whether the box contains a diamond;
If it's not useful to know whether the box contains a diamond,
I desire to not spend time figuring out whether the box contains a diamond;
Let me not become attached to curiosities I may not need.
here's my new fake-religion, taking just-world bias to its full extreme
the belief that we're simulations and we'll get transcended to Utopia in 1 second because future civilisation is creating many simulations of all possible people in all possible contexts and then uploading them to Utopia so that from anyone's perspective you have a very high probability of transcending to Utopia in 1 second
^^
There's the epistemic discount rate (ex.: probability of simulation shut down per year) and the value discount (ex.: you do the funner things first, so life is less valuable per year as you become older).
Asking "What value discount rate should be applied" is a category error. "should" statements are about actions done towards values, not about values themselves.
As for "What epistemic discount rate should be applied", it depends on things like "probability of death/extinction per year".
I'm helping Abram Demski with making the graphics for the AI Safety Game (https://www.greaterwrong.com/posts/Nex8EgEJPsn7dvoQB/the-ai-safety-game-updated)
We'll make a version using https://app.wombo.art/. We have generated multiple possible artwork for each card and made a pre-selection, but we would like your input for the final selection.
You can give your input through this survey: https://forms.gle/4d7Y2yv1EEXuMDqU7 Thanks!
In the book Superintelligence, box 8, Nick Bostrom says:
...How an AI would be affected by the simulation hypothesis depends on its values. [...] consider an AI that has a more modest final goal, one that could be satisfied with a small amount of resources, such as the goal of receiving some pre-produced cryptographic reward tokens, or the goal of causing the existence of forty-five virtual paperclips. Such an AI should not discount those possible worlds in which it inhabits a simulation. A substantial portion of the AI’s total expected utility might derive
Why do we have offices?
They seem expensive, and not useful for jobs that can apparently be done remotely.
Hypotheses:
status: to integrate
AI is improving exponentially with researchers having constant intelligence. Once the AI research workforce become itself composed of AIs, that constant will become exponential which would make AI improve even faster (superexponentially?)
it doesn't need to be the scenario of a singular AI agent self-improving its own self; it can be a large AI population participating in the economy and collectively improving AI as a whole, with various AI clans* focusing on different subdomains (EtA: for the main purpose of making money, and then using that money to buy t...
One model / framing / hypothesis of my preferences is that:
I wouldn't / don't value living in a loop multiple times* because there's nothing new experienced. So even an infinite life in the sense of looping an infinite amount of times has finite value. Actually, it has the same value as the size of the loop: after 1 loop, marginal loops have no value. (Intuition pump: from within the loop, you can’t tell how many times you’ve been going through the loop so far.)
*explanation of a loop: at some point in the future my life becomes indistinguishable from a pre...
In my mind, "the expert problem" means the problem of being able to recognize experts without being one, but I don't know where this idea comes from as the results from a Google search don't mention this. What name is used to refer to that problem (in the literature)?
x-post: https://www.facebook.com/mati.roy.09/posts/10159081618379579
Epistemic status: thinking outloud
The term "weirdness points" puts a different framing on the topic.
I'm thinking maybe I/we should also do this for "recommendation points".
The amount I'm willing to bet depends both on how important it seems to me and how likely I think the other person will appreciate it.
The way I currently try to spend my recommendation points is pretty fat tail, because I see someone's attention as scarce, so I want to keep it for things I think are really important, and the importance I assign to information is pretty fat tail. I'll som...
current intuitions for personal longevity interventions in order of priority (cryo-first for older people): sleep well, lifelogging, research mind-readers, investing to buy therapies in the future, staying home, sign up for cryo, paying a cook / maintain low weight, hiring Wei Dai to research modal immortality, paying a trainer, preserving stem cells, moving near a cryo facility, having some watch you to detect when you die, funding plastination research
EtA: maybe lucid dreaming to remember your dreams; some drugs (becopa?) to improve memory retention)
also not really important in the long run, but sleeping less to experience more
i want a better conceptual understanding of what "fundamental values" means, and how to disentangled that from beliefs (ex.: in an LLM). like, is there a meaningful way we can say that a "cat classifier" is valuing classifying cats even though it sometimes fail?
topic: AI
Lex Fridman:
I'm doing podcast with Sam Altman (@sama), CEO of OpenAI next week, about GPT-4, ChatGPT, and AI in general. Let me know if you have any questions/topic suggestions.
PS: I'll be in SF area next week. Let me know if there are other folks I should talk to, on and off the mic.
topic: lifelogging as life extension
if you think pivotal acts might require destroying a lot of hardware*; then this increases the probability that worlds in which lifelogging as life extension is useful are more likely to require EMP-proof lifelogging (and if you think think destroying a lot of hardware would increase x-risks, then it reduces the value of having EMP-proof lifelogs)
*there might be no point in destroying hard drives (vs GPUs), but some attacks, like EMPs, might not discriminate on that
#parenting, #schooling, #policies
40% of hockey players selected in top tier leagues are born in the first quarter of the year (compared to 10% in the 3rd quarter) (whereas you'd expect 25% if the time of the year didn't have an influence)
The reason for that is that the cut-off age to join a hockey league as a kid is January 1st; so people born in January are the oldest in their team, and at a young age that makes a bigger difference (in terms of abilities), so they tend to be the best in their team, and so their coaches tend to make them play more and pay ...
That suggest he has no idea about whether it's actually a good vote (as this is how the person differs from other candidates) and just advocates for someone on the basis that the person is his friend.
For the Cryonics Institute board elections, I recommend voting for Nicolas Lacombe.
I’ve been friends with Nicolas for over a decade. Ze’s very principled, well organized and hard working. I have high trust in zir, and high confidence ze would be a great addition to the CI's board.
I recommend you cast some or all of your votes for Nicolas (you can cast up to 4 votes total). If you’re signed up with CI, simply email info@cryonics.org with your votes.
see zir description here: https://www.facebook.com/mati.roy.09/posts/10159642542159579
i want to invest in companies that will increase in value if AI capabilities increases fast / faster than what the market predicts
do you have suggestions?
a feature i would like on content website like YouTube and LessWrong is an option to market a video/article as read as a note to self (x-post fb)
the usual story is that Governments provide public good because Markets can't, but maybe Markets can't because Governments have secured a monopoly on them?
x-post: https://www.facebook.com/mati.roy.09/posts/10159360438609579
Suggestion for retroactive prizes: Pay the most undervalued post on the topic for the prize, whenever it was written, assuming the writer is still alive or cryopreserved (given money is probably not worth much for most dead people). "undervalue" meaning amount the post is worth minus amount the writers received.
Topic: Can we compute back the Universe to revive everyone?
Quality / epistemic status: I'm just noting this here for now. Language might be a bit obscure, and I don't have a super robust/formal understanding of this. Extremely speculative.
This is a reply to: https://www.reddit.com/r/slatestarcodex/comments/itdggr/is_there_a_positive_counterpart_to_red_pill/g5g3y3a/?utm_source=reddit&utm_medium=web2x&context=3
The map can't be larger than the territory. So you need a larger territory to scan your region of interest: your scanner can't scan themselves...
Topic: AI adoption dynamic
GPT-3:
Human:
So an AI currently seems more expensive to train, but less expensive to use (as might be obvious for most of you).
Of course, trained humans are better than GPT-3. And this comparison has other limitations. But I still find it interesting.
...Acco
generalising from what a friend proposed me: don't aim at being motivated to do [desirable habit], aim at being addicted (/obsessed) at doing [desirable habit] (ie. having difficulty not to do it). I like this framing; relying on always being motivated feels harder to me
(I like that advice, but it probably doesn't work for everyone)
Philosophical zombies are creatures that are exactly like us, down to the atomic level, except they aren't conscious.
Complete philosophical zombies go further. They too are exactly like us, down to the atomic level, and aren't conscious. But they are also purple spheres (except we see them as if they weren't), they want to maximize paperclips (although they act and think as if they didn't), and they are very intelligent (except they act and think as if they weren't).
I'm just saying this because I find it funny ^^. I think consciousness is harder (for us) to reduce than shapes, preferences, and intelligence.
topic: lifelogging as life extension
which formats should we preserve our files in?
I think it should be:
- open source and popular (to increase chances it's still accessible in the future)
- resistant to data degradation: https://en.wikipedia.org/wiki/Data_degradation (thanks to Matthew Barnett for bringing this to my attention)
x-post: https://www.facebook.com/groups/LifeloggingAsLifeExtension/permalink/1337456839798929/
topic: lifelogging as life extension
epistemic status: idea
Backup Day. Day where you commit all your data to blu-rays in a secure location.
When could that be?
Aphelion is at the beginning of the year. But maybe would be better to have it on a day that commemorates some relevant events for us.
x-post: https://www.facebook.com/groups/LifeloggingAsLifeExtension/permalink/1336571059887507/
I feel like I have slack. I don't need to work much to be able to eat; if I don't work for a day, nothing viscerally bad happens in my immediate surrounding. This allows me to think longer term and take on altruistic projects. But on the other hand, I feel like every movement counts; that there's no loose in the system. Every lost move is costly. A recurrent thought I've had in the past weeks is that: there's no slack in the system.
epistemic status: a thought I just had
EtA: for those that are not familiar with the concept of moral trade, check out: https://concepts.effectivealtruism.org/concepts/moral-trade/
Steven Universe s1e5 is about a being that follows commands literally, and is a metaphor for some AI risks
epistemic status: speculative, probably simplistic and ill defined
Someone asked me "What will I do once we have AGI?"
I generally define the AGI-era starting at the point where all economically valuable tasks can be performed by AIs at a lower cost than a human (at subsistance level, including buying any available augmentations for the human). This notably excludes:
1) any tasks that humans can do that still provide value at the margin (ie. the caloric cost of feeding that human while they're working vs while they're not working rather than while they're not...
imagine (maybe all of a sudden) we're able to create barely superhuman-level AIs aligned to whatever values we want at a barely subhuman-level operation cost
we might decide to have anyone able to buy AI agents aligned with their values
or we might (generally) think this way to give access to that tech would be bad, but many companies are already incentivized to do that individually and can't all cooperate not to (and they actually reached this point gradually, previously selling near human-level AIs)
then it seems like everyone/most people would start to run...
topic: economics
idea: when building something with local negative externalities, have some mechanism to measure the externalities in terms of how much the surrounding property valuation changed (or are expected to change based, say, through a prediction market) and have the owner of that new structure pay the owners of the surrounding properties.
I wonder what fraction of people identify as "normies"
I wonder if most people have something niche they identify with and label people outside of that niche as "normies"
if so, then a term with a more objective perspective (and maybe better) would be non-<whatever your thing is>
like, athletic people could use "non-athletic" instead of "normies" for that class of people
topics: AI, sociology
thought/hypothesis: when tech is able to create brains/bodies as good or better than ours, it will change our perception of ourselves: we won't be in a separate magistra from our tools anymore. maybe people will see humans as less sacred, and value life less. if you're constantly using, modifying, copying, deleting, enslaving AI minds (even AI minds that have a human-like interface), maybe people will become more okay doing that to human minds as well.
(which seems like it would be harmful for the purpose of reducing death)
topic: intellectual discussion, ML tool, AI x-risks
Idea: Have a therapist present during intellectual debate to notice triggers, and help defuse them. Triggers activate a politics mindset where the goal becomes focused on status/self-preservation/appearances/looking smart/making the other person look stupid/etc. which makes it hard to think clearly.
Two people I follow will soon have a debate on AI x-risks which made me think of that. I can't really propose that intervention though because it will likely be perceived and responded as if it was a political m...
Topics: AI, forecasting, privacy
I wonder how much of a signature we leave in our writings. Like, how hard would it be for an AI to be rather confident I wrote this text? (say if it was trained on LessWrong writings, or all public writings, or maybe even private writings) What if I ask someone else to write an idea for me--how helpful is it in obfuscating the source?
Topic: AI strategy (policies, malicious use of AI, AGI misalignment)
Epistemic status: simplistic; simplified line of reasoning; thinking out loud; a proposed frame
A significant "warning shot" from a sovereign misaligned AI doesn't seem likely to me because a human-level (and plausibly a subhuman-level) intelligence can both 1) learn deception, yet 2) can't (generally) do a lot of damage (i.e. perceptible for humanity). So the last "warning shot" before AI learns deception won't be very big (if even really notable at all), and then a misaligned agent would ...
topic: AI alignment, video game | status: idea
Acknowledgement: Inspired from an idea I heard from Eliezer in zir podcast with Lex Friedman and the game Detroit: Become Human.
Video game where you're in an alternate universe where aliens create an artificial intelligence that's a human. The human has various properties typical of AI, such has running way faster than the aliens in that world and being able to duplicate themselves. The goal of the human is to take over the world to stop some atrocity happening in that world. The aliens are trying to stop the human from taking over the world.
✨ topic: AI timelines
Note: I'm not explaining my reasoning in this post, just recording my predictions and sharing how I feel.
I'll sound like a boring cliche at this point, but I just wanted to say it publicly: my AGI timelines have shorten earlier this year.
Without thinking about too much about quantifying my probabilities, I'd say the probabilities that we'll get AGI or AI strong enough to prevent AGI (including through omnicide) are:
But at this point I feel like not much...
topic: genetic engineering
'Revolutionary': Scientists create mice with two fathers
(I just read the title)
topic: genetic engineering
'Revolutionary': Scientists create mice with two fathers
(I just read the title)
Idea for a line of thinking: What if as a result of automation we could use the ~entire human population to control AI — any way we could meaningfully organize this large workforce towards that goal?
My assistant agency, Pantask, is looking to hire new remote assistants. We currently work only with effective altruist / LessWrong clients, and are looking to contract people in or adjacent to the network. If you’re interested in referring me people, I’ll give you a 100 USD finder’s fee for any assistant I contract for at least 2 weeks (I’m looking to contract a couple at the moment).
This is a part time gig / sideline. Tasks often include web searches, problem solving over the phone, and google sheet formatting. A full d...
a thought for a prediction aggregator
Problem with prediction market: People with the knowledge might still not want to risk money (plus complexity of markets, risk the market fails, tax implication, etc.).
But if you fully subsidize them, and make it free to participate, but still with potential for reward, then most people would probably make random predictions (at least, for predictions where most people don't have specialized knowledge about this) (because it's not worth investing the time to improve the prediction).
Maybe the best of both worlds is to do...
But if the brain is allowed to change, then the subject can eventually adapt to the torment. To
This doesn't follow. It seems very likely to me that it can allow it to change it ways it doesn't adapt to the pain, both from first principles and from observations.
How different would each loop need to be in order to be experienced separately?
That would be like having multiple different Everett branches experiencing suffering (parallel lives), which is different from 1 long continuous life.
Superintelligent Devil will take its victim brain's upgrade under its control and will invest in constant development of the victim's brain-parts which can feel pain. There is eternity to evolve in that direction and the victim will know that every next second will be worse. But it is really computationally intense way of punishment.
The question of the minimal unit of experience, which is enough to break the loop sameness is interesting. The need not only to be subjectively different, but the difference need to be meaningful. Not just one pixel.
Being immortal means you will one day be a Jupiter brain (if you think memories are part of one's identity, which I think they are)
x-post: https://twitter.com/matiroy9/status/1451816147909808131
Here's a way to measure (a proxy of) relative value of different years I just thought (again?); answer the question:
For which income I would you prefer to live in perpetual-2020 over living in perpetual-2021 with a median income? (maybe income is measured by fraction of the world owned multiplied by population size, or some other way) Then you can either chain those answers to go back to older years, or just compare them directly.
There are probably years where even Owning Everything wouldn't be enough. I prefer to live in perpetual-2021 with a median incom...
Hobby: serve so many bullets to sophisticated philosophers that they're missing half their teeth by the end of the discussion