Recent Discussion

Please excuse my naivety, would really like to hear from more knowledgable people about this.

I recently discovered a note-taking website called Roam which allows you to create pages, take bullet point notes in those pages, and use double brackets around phrases and words to create a new, doubly linked page. When you do this, you can see the all the connected pages in a visual; a graph where each node is a page. I think this tool is valuable because it allows me to externalize the connectedness of ideas and concepts with clarity.

I'm wondering why a tool like this hasn't been populari... (Read more)

I've been interested in this area for the last couple of years as well. Surprisingly I had not found https://arbital.com/ until very recently which has got to be the closest thing to what would be ideal.


The main problem seems to be the amount of refactoring/reframing that can be done. As mentioned by @__nobody 's answer, there is a fundamental problem in naming and concept drift. I say fundamental because it has become my belief that on a practical level defining words is essentially performed on a community level, not an individual. Coming up w... (read more)

4Answer by __nobody3hI haven't created an account on their page, so this is based purely on what I'm seeing in the example collection / demo videos. It looks broadly similar enough to what I've been building/using over the last years[1] that I think a summary of my experiences with my own tool and the features that you will need might be useful: In short: It looks awesome as long as you have only small amounts of content – and I think it may actually be awesome in those cases: For almost all my projects, I'm creating separate(!) maps/webs and collecting todos and their states to get a visual overview, and these are also useful to get back into the project if I come back months or years later – so they'd probably also help other people too. But… as things grow, it will get very confusing. (Unless you add lots of functionality into the tool and put in extra effort specifically to counter that.) All attempts to collect all my projects / ideas in a single big map have failed so far… That said, I haven't given up yet and am still trying to make it work. -------------------------------------------------------------------------------- Here's how things have been breaking down for me: naming things is hard and over time, you'll pick subtly different names (unless you have a fast way to look up what you called the thing a couple of months ago), and then spelling variants will point to different pages and the web breaks down… Also, there will be drift / shift in meaning or goals – you explore a new sub-topic and suddenly it looks like a good idea to rename / move a page or even a whole category, in sync, across all pages. Without tool support for both of these, things will fail. (p=1.0, N=3; after adding fast search (not just prefix-based like autocomplete!), the 4th map/web died of ontology instead of spelling variants and the 5th isn't dead yet…) Name changes are made simpler if you have page IDs that are independent from the name (and actually Roam has that, too – so these two shouldn't b
Plans for COVID-19?Q
51h1 min readShow Highlight

In places where COVID-19 is already widespread, lockdowns may not be fully lifted until we have widespread vaccinations. It is unlikely we will have widespread vaccinations before 2021. It is therefore prudent to contingency for 8 months or more of lockdown.

If you're financially secure then now could be a great time to begin long-term projects like gardening, writing a novel, physical training or learning a new skill.

Is anyone else starting projects like this?

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

HIGHLIGHTS

AI Alignment Podcast: An Overview of Technical AI Alignment in 2018 and 2019 (Lucas Perry, Buck Shlegeris and Rohin Shah) (summarized by Rohin): This podcast with Buck and me is loosely structured around the review I wrote (AN #84), but with a lot more debate and delving into specific points of pessimism and optimism. I suspe... (Read more)

Without having read the transcript either, this sounds like it's focused on near-term issues with autonomous weapons, and not meant to be a statement about the longer-term role autonomous weapons systems might play in increasing X-risk.

How uniform is the neocortex?Ω
654d11 min readΩ 23Show Highlight

How uniform is the neocortex?

The neocortex is the part of the human brain responsible for higher-order functions like sensory perception, cognition, and language, and has been hypothesized to be uniformly composed of general-purpose data-processing modules. What does the currently available evidence suggest about this hypothesis?

"How uniform is the neocortex?” is one of the background variables in my framework for AGI timelines. My aim for this post is not to present a complete argument for some view on this variable, so much as it is to:

  • present some considerations I’ve encountered that she
... (Read more)

And MuZero, which beats AlphaZero and which does not use symbolic search over a simulator of board states but internal search over hidden state and value estimates?

Neural networks, on the other hand, are famously bad at symbolic reasoning tasks, which may ultimately have some basis in the fact that probability does not extend logic.

Considering all the progress on graph and relational networks and inference and theorem-proving and whatnot, this statement is giving a lot of hostages to fortune.

1Ankesh Anand5hThe raw neural network does use search during training though, and does not rely on search only during evaluation.

Especially as your main account

Since my username contains a fish, I find it easier to not take myself too seriously.

If I can't start off a sequence about my research with a giant illustrated Balrog, am I even alive at that point?

2Answer by romeostevensit4hI decided that handwriting analysis would eventually uncover pseudonyms, so I may as well use my real name and be continuously reminded that security via obscurity is falsely comforting.
2Answer by jefftk4hI don't use a pseudonym, here or anywhere [1], but even though you're not asking I'll answer anyway to give some of the ideas that point in the other direction. Overall, I feel like there are a lot more advantages to a unified identity than disadvantages, and using a single identity has gone pretty well for me. * I'm in multiple communities that are mostly separate but still have a lot of overlap. And people in that overlap are often the people I most want to talk to about intersections (ex: automating a piece of being a contra dance musician) If I used different identities for LW, EA, contra dance, tech, etc then I would miss out on these connections. * Actually successfully keeping your identity private is hard. It's especially hard because technological advance makes things retroactively and unpredictably no longer private. So I generally act as if I don't have privacy, and get the advantages of making things public. * I'm not very worried about this making things hard for me in employment, but I'm also pretty established in my career as an engineer at this point. I put all my side-project code on github, even things [https://github.com/jeffkaufman/jammer/blob/master/jammermidilib.h] that I would absolutely not write in a work context. Though if I wanted to move into something EA-ish as my primary career, I think these years of public writing would really be very helpful. * In general I am really pro-transparency, to the point that things like "never associate anything with your real name unless it makes you look good and you can take it down later when the cultural tides change and that stops being true" are not at all a way I would like to live. If I do good things I want people to know that about me, but if I do bad things I want people to hold that against me. That keeps me honest. And history is really important: I have old blog posts that I definitely wouldn't write now,
2Answer by Dagon6hI like having a bit of deniability and search obfuscation to governments and employers. It's not very strong information hiding - truth is entangled, and there's enough of a trail that anyone who tried could find me. Using a handle/nickname/'nym/whatever is very comfortable for me - I've used this one (or variations, as the source got more popular and namespaces more crowded) for well over 3 decades, since before the Internet was around. I have friends from the early days who use my online name in person, because that's how they first knew me, and because my so-called "real" name is fairly common.

Background: I started meditating with the app Headspace in 2017, and started using the app Waking Up this past month at the same time I started meditating a lot more. (10h in the past month, vs 30h in the three years before that). I am not an expert, merely an amateur who's seen interesting improvements after relatively little effort.

Part I of this post is a comparison of the two apps, meant to justify why I think someone getting into meditation should start with Headspace. If you are not interested in meditation but enjoy thinking about the human mind, the description in Part I of what ... (Read more)

A book I like to recommend to people interested in getting started in meditation is "A Path With Heart" by Jack Kornfield. It's written by an author who is very decided not a rationalist and it's filled with lots of references to supernatural things, but it's also a very kind and gentle introduction to meditation and wider practice of the way. If you think of the supernatural stuff as metaphors rather than claims about physical reality, I think it can be quite helpful and teaches a lot of techniques and gives some good motivations for why and how they are useful.

The following is an informal exposition of some mathematical concepts from Topology via Logic, with special attention to philosophical implications. Those seeking more technical detail should simply read the book.

There are, roughly, two ways of doing topology:

  • Point-set topology: Start with a set of points. Consider a topology as a set of subsets of these points which are "open", where open sets must satisfy some laws.
  • Locale theory: Start with a set of opens (similar to propositions), which are closed under some logical operators (especially and and or), and satisfy logical relations.

What

... (Read more)

Since you are aiming towards philosophy with this one, I'll share something about my intuitions around emptiness (as opposed to form, in Buddhist Madhyamaka philosophy) as they relate to open sets in topology.

In my mind it has been fruitful to think of emptiness like openness and relate the two, specifically thinking of emptiness as describing the same aspect of reality that make "open" a good intuitive label for "open sets". This has helped me understand what is pointed at by "emptiness" by understanding it as "open... (read more)

Stop saying wrong things
696d5 min readShow Highlight

Lots has been said within these servers on finding ways to say new exciting correct things. Or correcting things many think are right, but didn't know were actually wrong. This is a relentlessly optimistic perspective. I have gotten grand mileage by merely trying to hold back from saying or acting on information or ideas I know are wrong, as opposed to worrying about thinking better, or not making mistakes, or being more rational when I am concerned with the truth. This other part is the meaty part, the intense part - the struggle to stop mouthing the words, to stop myself from willingly ... (Read more)

Lots has been said within these servers on finding ways to say new exciting correct things. Or correcting things many think are right, but didn't know were actually wrong. This is a relentlessly optimistic perspective. I have gotten grand mileage by merely trying to hold back from saying or acting on information or ideas I know are wrong, as opposed to worrying about thinking better, or not making mistakes, or being more rational when I am concerned with the truth. This other part is the meaty part, the intense part - the struggle to stop mouthing the
... (read more)

I've been optimizing various aspects of my investment setup recently, and will write up some tips and tricks that I've found in the form of "answers" here. Others are welcome to share their own here if they'd like. (Disclaimer: I’m not a lawyer, accountant, or investment advisor, and everything here is for general informational purposes purposes only.)

8Answer by steve215216hI've been using MaxMyInterest [https://www.maxmyinterest.com/] since 2015. They list out the highest-interest FDIC-insured savings accounts and make it easy to open them and transfer money between them. They'll also automatically track which of your savings accounts has the highest interest rate (if you have more than one) and move your money there. (Or if you have so much money that you exceed the FDIC limit ... which somehow has never been a problem for me! ... it can split your money into multiple accounts to get around that.) It also links with your low-interest everyday checking account, and will periodically transfer money back and forth to keep the latter balance at whatever amount you tell it. I really like that last feature, it saves me time and mental energy. They charge a fee of (currently) 0.08%/year × however much money you have in the high-interest savings accounts.
1Pongo5hThis only just occurred to me on reading your comment (and is probably obvious): many savings accounts have some limit of free withdrawals a year. But there are many savings accounts with close to the best rate -- so just by splitting your savings across multiple accounts allows you to have more of your money in higher interest accounts with little cost

I think MaxMyInterest only lists savings accounts that offer unlimited free online transfers. You might think that there must be a trade-off of that requirement against interest rate, but that doesn't seem to be the case; the rates are as good as anything on the market, even including CDs, as far as I've been able to tell the couple times I've quickly looked over the years. PM me if you want a screenshot of their current offerings.

When I was in high school, I noticed is that it was possible to score the top mark on an Advanced Placement (AP) Exam by answering a relatively small portion of the questions correctly.

During my junior year, I self-studied calculus, and took the AP Calculus AB exam. I was very surprised that I scored a 5 (the top mark), because at the time when I took the exam, I didn't know some very basic things that were on the syllabus.

The College Board gives the raw score to AP score conversions for the exams that have been most recently released. The percentages needed to get a 5 are as fo... (Read more)

I imagine the cutoffs are based on the percentage of students able to achieve a certain score. I wouldn't make a whole lot of sense to have the score for "5" set at 90% if only 1 in 200 students got above a 90%. For example, the national average on most biology written responses is well below 50% of the points, and considering that statistic, I think the lower cutoffs are quite reasonable.

If I were less concerned with looking sufficiently weird (cf https://en.wikipedia.org/wiki/Countersignaling) to the pinks/muggles/mehumes/mundanes/proles/masses, I'd likely enjoy popular media more, and probably be closer to correctness on topics where the crowd really is likely to show wisdom.

Oh, did you mean "less concerned that I'd look too weird"? That probably isn't possible. Cases where I conform to expectations are about trust-earning and making people happy, not about concern with appearances (well, not about my concern... (read more)

4shminux6hI can't answer that without looking weird!
A game designed to beat AI?Q
102mo1 min readShow Highlight

It's no surprise LW is dominated by COVID questions right now. My guess is that many of us are holding their ground at home and have more spare time than usual. So here is a question for you: if you are to design a 1v1-type board game and your purpose is to confuse the AIs, to make it as difficult for them as possible, to level the playground between humans & machines, or at least to prolong the period when human players have an upper hand over AIs; then what designs would you use, what elements would you introduce into the game?

Of course, the 2nd most important goal is to make an int... (Read more)

No, I'm not against that trading money for valuable stuffs part. And while the game can be digital, it does not hurt to have some physical sets for the human elements.

1Tetraspace Grouping7hThe AI Box game [http://yudkowsky.net/singularity/aibox], in contrast with the thing it's a metaphor for, is a two player game played over text chat by two humans where the goal is for Player A to persuade Player B to let them win (traditionally by getting them to say "I let you out of the box"), within a time limit.
Competitive safety via gradated curriculaΩ
292d4 min readΩ 13Show Highlight

Epistemic status: brainstorming some speculative research directions. Not trying to thoroughly justify the claims I’m making.

One way to think about the AI safety problem: there’s a spectrum of methods which each represent a different tradeoff between safety and ease of training an AGI, and unfortunately the two are anticorrelated. In particular, consider four regimes in which the bulk of training might occur (perhaps with additional fine-tuning afterwards):

  1. Training a language model to answer questions correctly.
  2. Training a RL agent on a range of limited tasks (e.g. games).
  3. Training
... (Read more)

I don't think that design (1) is particularly safe.

If your claim that design (1) is harder to get working is true, then you get a small amount of safety from the fact that a design that isn't doing anything is safe.

It depends on what the set of questions is, but if you want to be able to reliably answer questions like "how do I get from here to the bank?" then it needs to have a map, and some sort of pathfinding algorithm encoded in it somehow. If it can answer "what would a good advertising slogan be for product X" then it ... (read more)

tl;dr

Organizations that enforce rationality at the collective level can get very different voting outcomes than organizations that enforce rationality at the individual level, per known results in social choice theory. This has implications for real-world expert panels.

Here, "rationality" is logical consistency - it is possible for the majority of members to vote to reject a conclusion while also believing the necessary conditions to accept it hold, and vice-versa, even if they all independently evaluated the precepts and arrived at the conclusion logically. This arises because of how majorit

... (Read more)
1jonathanstray9ha) "Everyone does Bayesian updating according to the same hypothesis set, model, and measurement methods" strikes me as an extremely strong assumption, especially since we do not have strong theory that tells us the "right" way to select these hypothesis sets, models, and measurement instruments. I would argue that this makes Aumann agreement essentially useless in "open world" scenarios. b) Why should uniquely consistent aggregation methods exist at all? A long line of folks including Condorcet [https://en.wikipedia.org/wiki/Condorcet_paradox], Arrow [https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem], Sen [https://www.hup.harvard.edu/catalog.php?isbn=9780674919211&content=toc] and Parfit [https://en.wikipedia.org/wiki/Mere_addition_paradox] have pointed out that when you start aggregating beliefs, utility, or preferences, there do not exist methods that always give unambiguously "correct" answers. Sure, but finding the set of coefficients for comparing different people's utilities is a hard problem in AI alignment, or political economy generally. Not only are there tremendous normative uncertainties here ("how much inequality is too much?") but the problem of combining utilities a minefield of paradoxes even if you are just summing or averaging [https://arxiv.org/abs/1901.00064].

Yeah. I was more trying to argue that, compared to Bayesian ideas, voting doesn't win you all that much.

Coping and Cultures
391d3 min readShow Highlight

Coping and Cultures

(or, Acquiring Coping Mechanisms from the Culture of a Field of Study)

One of the useful pieces of implicit knowledge that college or mentorship gives people are mental adaptations that make the stressors particular to a field of study more tolerable. These often go unnoticed, and seem undervalued as a result.

Some adaptations like this seem to become a part of academic cultures, and get osmosed by most students. Others might only have living cultures in industry, with academics carrying a reputation for burnout compared to individuals who learn on the job.

I expect self-taugh

... (Read more)
2Raemon12hAh, gotcha.
2Spiracular14hTrue, and I've seen lab work cultivate something similar. (I'm pretty sure this particular skill is the inverse of programmer-style "laziness," funnily enough. In one field, seeing repetition is reassuring. In the other, it can be evidence that your code is not as elegant and modularized as it could be.) I always thought you'd automatically learn the gait if you just did the work often enough, though. It's definitely a coping skill, but I read its origins as more cultivated than culturally-induced or taught. It mostly follows the natural incentive gradients of the work. This can be in contrast to things like separation of self and client in psychology, which seems to feel actively un-natural for many people. Of course, there's something of a spectrum here, with heavy individual variation.

Yes. How do you identify the culturally-induced or taught CM in people who have started working? (In contrast to personal CM erected through self-teaching which was gained simultaneously [with the CM obtained by peer cross-pollination and between-generation communication].)

Announcing Web-TAISU, May 13-17Ω
191mo1 min readΩ 8Show Highlight

I am excited to announce Web-TAISU!
May 13-17, 2020

I was going to run a Technical AI Safety Unconference (TAISU) at CEEALAR (formerly EA Hotel), Blackpool, UK. Then there was a pandemic. So instead there will be a Web-TAISU on the Internet.

This is an unconference, which means that the program is participant driven. I do have some backup ideas to put in if there are not enough suggestions from you, but I don’t expect those to be necessary.

I thought a lot about how to adapt this event to be run online, and I am grateful to the participants who took time to discuss this with me. In the end ... (Read more)

Vika8h8Ω4

Thanks Linda for organizing, looking forward to it!

As far as I understand the prime advantage of surgical masks over N95 masks lies in the N95 masks being able to seal and filter all of the air while the surgical masks allow air to pass at their sides.

Given that there are many situations where people would ideally wear N95 masks but have only access to surgical masks, why can't we improve the surgical masks by using adhesive tape to seal their borders to the face?

Answer by ediMay 07, 20201

This article here suggests that N95 and surgical masks performances were similar while preventing to catch the flu.

https://smartairfilters.com/en/blog/n95-mask-surgical-prevent-transmission-coronavirus/

I'm guessing data is limited here, but a related-related question might be "how likely am I to catch the flu or a few other common diseases by interacting with a victim for an hour?"

Thanks. The thing I'm ultimately looking for though is more like "at humidity X, your likelihood is Y". I know roughly how the variables fit together, but not enough to decide "do I let a random person who might have covid into my house or not?"

Covid-19 5/7: Fighting Limbo
2211h10 min readShow Highlight

Last week: Covid-19 4/30: Stuck in Limbo

Recently: Covid-19: New York’s Antibody Tests 2On “COVID-19 Superspreader Events in 28 Countries: Critical Patterns and Lessons”

Background Assumptions: On R0Taking Initial Viral Load SeriouslyOn New York’s Antibody TestsMy Covid-19 Thinking: 4/23 pre-Cuomo Data

Spreadsheet where I do work: Access it here as read only

Deaths By Week in the 5 Big Regions:

WEST MIDWEST SOUTH NE ex-NY NY
Mar 26-Apr 1 164 450 182 143 364
Mar 19-Mar 25 424 1894 667 856 1988
Apr 2-8 764 3873 1331 2248 4694
Apr 9-15 890 4853 1596 3605 5318
Apr 16-22 10
... (Read more)

I think that, when most people use the term "herd immunity", they mean "herd immunity sufficient to get R<1 while everyone parties like it's 2019". That could require 75% to be infected, don't you think?

Introduction

Insight meditation, enlightenment, what’s that all about?

The sequence of posts starting from this one is my personal attempt at answering that question. It grew out of me being annoyed about so much of this material seeming to be straightforwardly explainable in non-mysterious terms, but me also being unable to find any book or article that would do this to my satisfaction. In particular, I wanted something that would:

  • Explain what kinds of implicit assumptions build up our default understanding of reality and how those assumptions are subtly flawed. It would then point out
... (Read more)
1Max Hodges13hWhat makes you call meditation a religious program? Where is the religion in this practice: PRACTICE: MINDFULNESS OF BREATHING 1. Find a quiet place, and sit on either a chair or cushion. Choose a chair with a firm, flat seat, and hold your back upright (although not stiffly so). Let the soles of your feet meet the ground, and bring your hands on to your lap. If you sit on a cushion, you can be cross-legged. Let your body be untensed, inviting openness and confidence. 2. Decide how long to practice for. Your session can be as short as five minutes, or longer. You may find it useful to set an alarm to tell you when to stop, so you don’t have to think about it. 3. Bring attention to the sensations of breath in your belly. Let go of thinking about or analyzing the breath. Just feel it. Follow its natural rhythms gently with attention: in and out, rising and falling. Let thoughts, emotions, body sensations, and sounds be as they are—you don’t need to follow them or push them away. Just allow them to happen, without interference, as you direct gentle attention to the breath. 4. When you notice that your mind has wandered, as it likely will often, acknowledge that this has happened, with kindness. Remember, as soon as you’re aware of the wandering, bring your attention back to the breath, and continue to follow it, in and out, moment by moment, with friendly interest. 5. Continue with steps three and four until it’s time to stop. === Clearly you have some interest since you're here reading and responding to a rather long series of articles on meditation. But it seems you may also harbor a lot of misunderstandings. What's meditation in your mind? And why are you convinced it's "a waste of time" when hundreds of millions of people are doing it? Out of curiosity, is there anything that might change your mind? Scientific papers? Meta-analysis studies? Perhaps testimonials of people's positive experience? Hund
is there anything that might change your mind? Scientific papers? Meta-analysis studies?

Yes, studies with good methodologies and decent sample sizes would make me question my stance. If they were replicated, that would completely change my mind. As I mentioned in my other comments, I have arrived at my present beliefs by doing a literature review few years ago.

I'm a bit more sceptical about meta-analyses since a lot of papers published on the subject are of terribly low quality (or at least were, when I looked into it).

1CheerfulWarrior13hGreat post. I can't wait to read the subsequent parts. Any particular reason to believe that? It's true that when googling for reports of meditators, many results seem consistent with each other. However, there are at least two strong biases at play: * Confirmation bias - because the meditator doesn't want to admit that they have wasted their investment and did not experience what was supposed to be the result of the practice. * Selection bias - because reports diverging from the common theme don't get as much attention.
2Kaj_Sotala12hThanks! Frequently people experience things which they never expected to happen. E.g. I recall that Scott Alexander had a post (which I failed to find) about how he as a psychiatrist occasionally gets patients who have had mystical experiences because they tried a clinical mindfulness program that was supposed to be just for stress relief. They had no idea that something like this could happen and are now totally freaked out by what's going on. Then Scott tells them that things will go back to normal if they just stop meditating, which seems to mostly work. Also, as I noted in another comment, a previous version of the post included the following: At least in my own experience, confirmation bias typically feels more like "aha, I knew it all along" than "huh, I guess I was thinking about this totally wrong before".
Load More