All Posts

Sorted by New

Week Of Sunday, November 17th 2019
Week Of Sun, Nov 17th 2019

No posts for this week
Shortform [Beta]
9TurnTrout8h I feel very excited by the AI alignment discussion group I'm running at Oregon State University. Three weeks ago, most attendees didn't know much about "AI security mindset"-ish considerations. This week, I asked the question "what, if anything, could go wrong with a superhuman reward maximizer which is rewarded for pictures of smiling people? Don't just fit a bad story to the reward function. Think carefully." There was some discussion and initial optimism, after which someone said "wait, those optimistic solutions are just the ones you'd prioritize! What's that called, again?" (It's called anthropomorphic optimism [https://www.readthesequences.com/Anthropomorphic-Optimism]) I'm so proud.
6crabman12h In my understanding, here are the main features of deep convolutional neural networks (DCNN) that make them work really well. (Disclaimer: I am not a specialist in CNNs, I have done one masters level deep learning course, and I have worked on accelerating DCNNs for 3 months.) For each feature, I give my probability, that having this feature is an important component of DCNN success, compared to having this feature to the extent that an average non-DCNN machine learning model has it (e.g. DCNN has weight sharing, an average model doesn't have weight sharing). 1. DCNNs heavily use transformations, which are the same for each window of the input - 95% 2. For any set of pixels of the input, large distances between pixels in the set make the DCNN model interactions between these pixels less accurately - 90% (perhaps usage of dilution in some DCNNs is a counterargument to this) 3. Large depth (together with the use of activation functions) lets us model complicated features, interactions, logic - 82% 4. Having a lot of parameters lets us model complicated features, interactions, logic - 60% 5. Given 3 and 4, SGD-like optimization works unexpectedly fast for some reason - 40% 6. Given 3 and 4, SGD-like optimization with early stopping doesn't overfit too much for some reason - 87% (I am not sure if S in SGD is important, and how important is early stopping) 7. Given 3 and 4, ReLu-like activation function works really well (compared to, for example, sigmoid). 8. Modern deep neural network libraries are easy to use compared to the baseline of not having specific well-developed libraries - 60% 9. Deep neural networks work really fast, when using modern deep neural network libraries and modern hardware - 33% 10. DCNNs find such features in photos, which are invisible to the human eye and to most ML algorithms - 20% 11. Dropout helps reducing overfitting a lot - 25% 12. Batch normalization improve
5strangepoop12h The expectations you do not know you have control your happiness more than you know. High expectations that you currently have don't look like high expectations from the inside, they just look like how the world is/would be. But "lower your expectations" can often be almost useless advice, kind of like "do the right thing". Trying to incorporate "lower expectations" often amounts to "be sad". How low should you go? It's not clear at all if you're using territory-free un-asymmetric simple rules like "lower". Like any other attempt at truth-finding, it is not magic. It requires thermodynamic work. The thing is, the payoff is rather amazing. You can just get down to work. As soon as you're free of a constant stream of abuse from beliefs previously housed in your head, you can Choose without Suffering. The problem is, I'm not sure how to strategically go about doing this, other than using my full brain with Constant Vigilance. Coda: A large portion of the LW project (or at least, more than a few offshoots) is about noticing you have beliefs that respond to incentives other than pure epistemic ones, and trying not to reload when shooting your foot off with those. So unsurprisingly, there's a failure mode here: when you publicly declare really low expectations (eg "everyone's an asshole"), it works to challenge people, urges them to prove you wrong. It's a cool trick to win games of Chicken but as usual, it works by handicapping you. So make sure you at least understand the costs and the contexts it works in.
5Matthew Barnett3d Bertrand Russell's advice to future generations, from 1959
4Chris_Leong3d Anti-induction and Self-Reinforcement Induction is the belief that the more often a pattern happens the more likely it is to continue. Anti-induction is the opposite claim: the more likely a pattern happens the less likely future events are to follow it. Somehow I seem to have gotten the idea in my head that anti-induction is self-reinforcing. The argument for it is as follows: Suppose we have a game where at each step a screen flashes an A or a B and we try to predict what it will show. Suppose that the screen always flashes A, but the agent initially thinks that the screen is more likely to display B. So it guesses B, observes that it guessed incorrectly and then, if it is an anti-inductive agent will increase it's likelihood that the next symbol will be B because of anti-induction. So in this scenario your confidence that the next symbol will be B, despite the long stream of As, will keep increasing. This particular anti-inductive belief is self-reinforcing. However, there is a sense in which anti-induction is contradictory - if you observe anti-induction working, then you should update towards it not working in the future. I suppose the distinction here is that we are using anti-induction to update our beliefs on anti-induction and not just our concrete beliefs. And each of these is a valid update rule: in the first we apply this update rule to everything including itself and in the other we apply this update rule to things other than itself. The idea of a rule applying to everything except itself feels suspicious, but is not invalid. Also, it's not that the anti-inductive belief that B will be next is self-reinforcing. After all, anti-induction given consistent As pushes you towards believing B more and more regardless of what you believe initially. In other words, it's more of an attractor state.
Load More (5/7)

Week Of Sunday, November 10th 2019
Week Of Sun, Nov 10th 2019

No posts for this week
Shortform [Beta]
12TurnTrout8d Yesterday, I put the finishing touches on my chef d'œuvre, a series of important safety-relevant proofs I've been striving for since early June. Strangely, I felt a great exhaustion come over me. These proofs had been my obsession for so long, and now - now, I'm done. I've had this feeling before; three years ago, I studied fervently for a Google interview. The literal moment the interview concluded, a fever overtook me. I was sick for days. All the stress and expectation and readiness-to-fight which had been pent up, released. I don't know why this happens. But right now, I'm still a little tired, even after getting a good night's sleep.
12Kaj_Sotala9d Here's a mistake which I've sometimes committed and gotten defensive as a result, and which I've seen make other people defensive when they've committed the same mistake. Take some vaguely defined, multidimensional thing that people could do or not do. In my case it was something like "trying to understand other people". Now there are different ways in which you can try to understand other people. For me, if someone opened up and told me of their experiences, I would put a lot of effort into really trying to understand their perspective, to try to understand how they thought and why they felt that way. At the same time, I thought that everyone was so unique that there wasn't much point in trying to understand them by any *other* way than hearing them explain their experience. So I wouldn't really, for example, try to make guesses about people based on what they seemed to have in common with other people I knew. Now someone comes and happens to mention that I "don't seem to try to understand other people". I get upset and defensive because I totally do, this person hasn't understood me at all! And in one sense, I'm right - it's true that there's a dimension of "trying to understand other people" that I've put a lot of effort into, in which I've probably invested more than other people have. And in another sense, the other person is right - while I was good at one dimension of "trying to understand other people", I was severely underinvested in others. And I had not really even properly acknowledged that "trying to understand other people" had other important dimensions too, because I was justifiably proud of my investment in one of them. But from the point of view of someone who *had* invested in those other dimensions, they could see the aspects in which I was deficient compared to them, or maybe even compared to the median person. (To some extent I thought that my underinvestment in those other dimensions was *virtuous*, because I was "not making assumption
11Ben Pace7d Trying to think about building some content organisations and filtering systems on LessWrong. I'm new to a bunch of the things I discuss below, so I'm interested in other people's models of these subjects, or links to sites that solve the problems in different ways. Two Problems So, one problem you might try to solve is that people want to see all of a thing on a site. You might want to see all the posts on reductionism on LessWrong, or all the practical how-to guides (e.g. how to beat procrastination, Alignment Research Field Guide, etc), or all the literature reviews on LessWrong. And so you want people to help build those pages. You might also want to see all the posts corresponding to a certain concept, so that you can find out what that concept refers to (e.g. what is the term "goodhart's law" or "slack" or "mesa-optimisers" etc). Another problem you might try to solve, is that while many users are interested in lots of the content on the site, they have varying levels of interest in the different topics. Some people are mostly interested in the posts on big picture historical narratives, and less so on models of one's own mind that help with dealing with emotions and trauma. Some people are very interested AI alignment, some are interested in only the best such posts, and some are interested in none. I think the first problem is supposed to be solved by Wikis, and the second problem is supposed to be solved by Tagging. Speaking generally, Wikis allow dedicated users to curated pages around certain types of content, highlighting the best examples, some side examples, writing some context for people arriving on the page to understand what the page is about. It's a canonical, update-able, highly editable page built around one idea. Tagging is much more about filtering than about curating. Tagging Let me describe some different styles of tagging. One the site lobste.rs there are about 100 tags in total. Most tags give a very broad description of an area o
10elityre7d new post: Metacognitive space [Part of my Psychological Principles of Personal Productivity, which I am writing mostly in my Roam, now.] Metacognitive space is a term of art that refers to a particular first person state / experience. In particular it refers to my propensity to be reflective about my urges and deliberate about the use of my resources. I think it might literally be having the broader context of my life, including my goals and values, and my personal resource constraints loaded up in peripheral awareness. Metacognitive space allows me to notice aversions and flinches, and take them as object, so that I can respond to them with Focusing or dialogue, instead of being swept around by them. Similarly, it seems to, in practice, to reduce my propensity to act on immediate urges and temptations. [Having MCS is the opposite of being [[{Urge-y-ness | reactivity | compulsiveness}]]?] It allows me to “absorb” and respond to happenings in my environment, including problems and opportunities, taking considered instead of semi-automatic, first response that occurred to me, action. [That sentence there feels a little fake, or maybe about something else, or maybe is just playing into a stereotype?] When I “run out” of meta cognitive space, I will tend to become ensnared in immediate urges or short term goals. Often this will entail spinning off into distractions, or becoming obsessed with some task (of high or low importance), for up to 10 hours at a time. Some activities that (I think) contribute to metacogntive space: * Rest days * Having a few free hours between the end of work for the day and going to bed * Weekly [[Scheduling]]. (In particular, weekly scheduling clarifies for me the resource constraints on my life.) * Daily [[Scheduling]] * [[meditation]], including short meditation. * Notably, I’m not sure if meditation is much more efficient than just taking the same time to go for a walk. I think it might be or might not be. * [[Exerc
9elityre9d New (short) post: Desires vs. Reflexes [https://musingsandroughdrafts.wordpress.com/2019/11/12/desires-vs-reflexes/] [Epistemic status: a quick thought that I had a minute ago.] There are goals / desires (I want to have sex, I want to stop working, I want to eat ice cream) and there are reflexes (anger, “wasted motions”, complaining about a problem, etc.). If you try and squash goals / desires, they will often (not always?) resurface around the side, or find some way to get met. (Why not always? What are the difference between those that do and those that don’t?) You need to bargain with them, or design outlet polices for them. Reflexes on the other hand are strategies / motions that are more or less habitual to you. These you train or untrain.
Load More (5/15)

Week Of Sunday, November 3rd 2019
Week Of Sun, Nov 3rd 2019

Shortform [Beta]
11TurnTrout17d With respect to the integers, 2 is prime. But with respect to the Gaussian integers [https://en.wikipedia.org/wiki/Gaussian_integer], it's not: it has factorization 2=(1−i)(1+i). Here's what's happening. You can view complex multiplication as scaling and rotating the complex plane. So, when we take our unit vector 1 and multiply by (1+i), we're scaling it by |1+i|=√2 and rotating it counterclockwise by 45∘: This gets us to the purple vector. Now, we multiply by (1−i), scaling it up by √2 again (in green), and rotating it clockwise again by the same amount. You can even deal with the scaling and rotations separately (scale twice by √2, with zero net rotation).
11Daniel Kokotajlo18d It seems to me that human society might go collectively insane sometime in the next few decades. I want to be able to succinctly articulate the possibility and why it is plausible, but I'm not happy with my current spiel. So I'm putting it up here in the hopes that someone can give me constructive criticism: I am aware of three mutually-reinforcing ways society could go collectively insane: 1. Echo chambers/filter bubbles/polarization: Arguably political polarization [https://en.wikipedia.org/wiki/Political_polarization] is increasing across the world of liberal democracies today. Perhaps the internet has something to do with this--it’s easy to self-select into a newsfeed and community that reinforces and extremizes your stances on issues. Arguably recommendation algorithms have contributed to this problem in various ways--see e.g. “Sort by controversial” [https://slatestarcodex.com/2018/10/30/sort-by-controversial/] and Stuart Russell’s claims in Human Compatible. At any rate, perhaps some combination of new technology and new cultural or political developments will turbocharge this phenomenon. This could lead to civil wars, or more mundanely, societal dysfunction. We can’t coordinate to solve collective action problems relating to AGI if we are all arguing bitterly with each other about culture war issues.Deepfakes/propaganda/persuasion tools: Already a significant portion of online content is deliberately shaped by powerful political agendas--e.g. Russia, China, and the US political tribes. Much of the rest is deliberately shaped by less powerful apolitical agendas, e.g. corporations managing their brands or teenagers in Estonia making money by spreading fake news during US elections. Perhaps this trend will continue; technology like chatbots, language models, deepfakes, etc. might make it cheaper and more effective to spew this sort of propaganda, to the point where most onlin
10toonalfrink12d Here's a faulty psychological pattern that I recently resolved for myself. It's a big one. I want to grow. So I seek out novelty. Try new things. For example I might buy high-lumen light bulbs to increase my mood. So I buy them, feel somewhat better, celebrate the win and move on. Problem is, I've bought high-lumen bulbs three times in my life now already, yet I sit here without any. So this pattern might happen all over again: I feel like upgrading my life, get this nice idea of buying light bulbs, buy them, celebrate my win and move on. So here's 4 life-upgrades, but did I grow 4 times? Obviously I only grew once. From not having high lumen light bulbs to having them. My instinct towards growth seems to think this: But in reality, it seems to be more like this: which I define as equal to The tap I installed that puts this preservation mindset into practice seems to be very helpful. It's as follows: if I wonder what to do, instead of starting over ("what seems like the best upgrade to add to my life?") I first check whether I'm on track with the implementation of past good ideas ("what did my past self intend to do with this moment again?") Funnily enough, so far the feeling I get from this mindset seems pretty similar to the feeling I get from meditation. And meditation can be seen as training yourself to put your attention on your past intentions too. I think this one goes a lot deeper than what I've written here. I'll be revisiting this idea.
10toonalfrink14d You may have heard of the poverty trap, where you have so little money that you're not able to spend any money on the things you need to make more. Being broke is an attractor state. You may have heard of the loneliness trap. You haven't had much social interaction lately, which makes you feel bad and anxious. This anxiety makes it harder to engage in social interaction. Being lonely is an attractor state. I think the latter is a close cousin of something that I'd like to call the irrelevance trap: * Lemma 1: having responsibilities is psychologically empowering. When others depend on your decisions, it is so much easier to make the right decision. * Lemma 2: being psychologically empowered makes it more likely for you to take on responsibility, and for others to give you responsibility, because you're more able to handle it. I speculate that some forms of depression (the dopaminergic type) are best understood as irrelevance traps. I'm pretty sure that that was the case for me. How do you escape such a trap? Well you escape a loneliness trap by going against your intuition and showing up at a party. You escape an irrelevance trap by going against your intuition and taking on more responsibility than you feel you can handle.
9TekhneMakre13d The hermeneutic spiral is the process of understanding a text (or more generally, anything big and confusing) by passing over it again and again, each time using what you've learned to more deeply understand the workings and roles of each part and grow a truer image of the whole. The hermeneutic spiral is not depth-first search; it's more like bread-first search, but it can also involve depth, and altering the ordering you use to search, and expanding the set you're searching over. The hermeneutic spiral involves noticing landmarks, principles, cruxes, and generators. It involves logogenesis [https://www.lesswrong.com/posts/2J5AsHPxxLGZ78Z7s/bios-brakhus?commentId=Kz4tFqnEPwYCiMmKe] . It's an aspect of how Alexander Grothendieck did math. It's the Unix philosophy [https://homepage.cs.uri.edu/~thenry/resources/unix_art/ch01s06.html] of programming. It's a way to make a pattern language [https://en.wikipedia.org/wiki/Pattern_language] (of rationality [https://unstableontology.com/2017/04/12/rationality-techniques-as-patterns/]). Compare the expectation–maximization algorithm [https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm]. GPT-2 uses the transformer architecture [https://nostalgebraist.tumblr.com/post/185326092369/the-transformer-explained], which is a sort of toy version of a hermeneutic spiral. "The sun was sinking in the sky, for Harry had been thinking for some hours now, thinking mostly the same thoughts over and over, but with key differences each time, like his thoughts were not going in circles, but climbing a spiral, or descending it." HPMOR, ch. 63.
Load More (5/17)

Week Of Sunday, October 27th 2019
Week Of Sun, Oct 27th 2019

No posts for this week
Shortform [Beta]
54orthonormal20d DeepMind released their AlphaStar paper a few days ago [https://deepmind.com/blog/article/AlphaStar-Grandmaster-level-in-StarCraft-II-using-multi-agent-reinforcement-learning] , having reached Grandmaster level at the partial-information real-time strategy game StarCraft II over the summer. This is very impressive, and yet less impressive than it sounds. I used to watch a lot of StarCraft II (I stopped interacting with Blizzard recently because of how they rolled over for China), and over the summer there were many breakdowns of AlphaStar games once players figured out how to identify the accounts. The impressive part is getting reinforcement learning to work at all in such a vast state space- that took breakthroughs beyond what was necessary to solve Go and beat Atari games. AlphaStar had to have a rich enough set of potential concepts (in the sense that e.g. a convolutional net ends up having concepts of different textures) that it could learn a concept like "construct building P" or "attack unit Q" or "stay out of the range of unit R" rather than just "select spot S and enter key T". This is new and worth celebrating. The overhyped part is that AlphaStar doesn't really do the "strategy" part of real-time strategy. Each race has a few solid builds that it executes at GM level, and the unit control is fantastic, but the replays don't look creative or even especially reactive to opponent strategies. That's because there's no representation of causal thinking - "if I did X then they could do Y, so I'd better do X' instead". Instead there are many agents evolving together, and if there's an agent evolving to try Y then the agents doing X will be replaced with agents that do X'. (This lack of causal reasoning especially shows up in building placement, where the consequences of locating any one building here or there are minor, but the consequences of your overall SimCity are major for how your units and your opponents' units would fare if they attacked you. In one
15Vaniver21d I've been thinking a lot about 'parallel economies' recently. One of the main differences between 'slow takeoff' and 'fast takeoff' predictions is whether AI is integrated into the 'human civilization' economy or constructing a separate 'AI civilization' economy. Maybe it's worth explaining a bit more what I mean by this: you can think of 'economies' as collections of agents who trade with each other. Often it will have a hierarchical structure, and where we draw the lines are sort of arbitrary. Imagine a person who works at a company and participates in its internal economy, and the company participates in national and global economies, and the person participates in those economies as well. A better picture has a very dense graph with lots of nodes and links between groups of nodes whose heaviness depends on the number of links between nodes in those groups. As Adam Smith argues, the ability of an economy to support specialization of labor depends on its size. If you have an island with a single inhabitant, it doesn't make sense to fully employ a farmer (since a full-time farmer can generate much more food than a single person could eat), for a village with 100 inhabitants it doesn't make sense to farm more than would feed a hundred mouths, and so on. But as you make more and more of a product, investments that have a small multiplicative payoff become better and better, to the point that a planet with ten billion people will have massive investment in farming specialization that make it vastly more efficient per unit than the village farming system. So for much of history, increased wealth has been driven by this increased specialization of labor, which was driven by the increased size of the economy (both through population growth and decreased trade barriers widening the links between economies until they effectively became one economy). One reason to think economies will remain integrated is because increased size benefits all actors in the economy on net; a
14Vaniver22d One challenge for theories of embedded agency over Cartesian theories is that the 'true dynamics' of optimization (where a function defined over a space points to a single global maximum, possibly achieved by multiple inputs) are replaced by the 'approximate dynamics'. But this means that by default we get the hassles associated with numerical approximations, like when integrating differential equations. If you tell me that you're doing Euler's Method on a particular system, I need to know lots about the system and about the particular hyperparameters you're using to know how well you'll approximate the true solution. This is the toy version of trying to figure out how a human reasons through a complicated cognitive task; you would need to know lots of details about the 'hyperparameters' of their process to replicate their final result. This makes getting guarantees hard. We might be able to establish what the 'sensible' solution range for a problem is, but establishing what algorithms can generate sensible solutions under what parameter settings seems much harder. Imagine trying to express what the set of deep neural network parameters are that will perform acceptably well on a particular task (first for a particular architecture, and then across all architectures!).
14jacobjacob22d Something interesting happens when one draws on a whiteboard ⬜ [https://emojipedia.org/white-large-square/]✍️ [https://emojipedia.org/writing-hand/]while talking. Even drawing 🌀 [https://emojipedia.org/cyclone/]an arbitrary squiggle while making a point makes me more likely to remember it, whereas points made without squiggles are more easily forgotten. This is a powerful observation. We can chunk complex ideas into simple pointers. This means I can use 2d surfaces as a thinking tool in a new way. I don't have to process content by extending strings over time, and forcibly feeding an exact trail of thought into my mind by navigating with my eyes. Instead I can distill the entire scenario into 🔭 [https://emojipedia.org/telescope/]a single, manageable, overviewable whole -- and do so in a way which leaves room for my own trails and 🕸️ [https://emojipedia.org/spider-web/]networks of thought. At a glance I remember what was said, without having to spend mental effort keeping track of that. This allows me to focus more fully on what's important. In the same way, I've started to like using emojis in 😃 [https://emojipedia.org/smiling-face-with-open-mouth/]📄 [https://emojipedia.org/page-facing-up/]essays and other documents. They feel like a spiritual counterpart of whiteboard squiggles. I'm quite excited about this. In future I intend to 🧪 [https://emojipedia.org/test-tube/]experiment more with it.
12toonalfrink20d Today I had some insight in what social justice really seems to be trying to do. I'll use neurodiversity as an example because it's less likely to lead to bad-faith arguments. Let's say you're in the (archetypical) position of a king. You're programming the rules that a group of people will live by, optimizing for the well-being of the group itself. You're going to shape environments for people. For example you might be running a supermarket and deciding what music it's going to play. Let's imagine that you're trying to create the optimal environment for people. The problem is, since there is more than one person that is affected by your decision, and these people are not exactly the same, you will not be able to make the decision that is optimal for each one of them. If only two of your customers have different favourite songs, you will not be able to play both of them. In some sense, making a decision over multiple people is inherently "aggressive". But what you can do, is reduce the amount of damage. My understanding is that this is usually done by splitting up the people as finely as possible. You might split up your audience into stereotypes for "men", "women", "youngsters", "elders", "autistic people", "neurotypicals", etc. In this case, you can make a decision that would be okay for each of these stereotypes, giving your model a lower error rate. The problem with this is that stereotypes are leaky generalizations. Some people might not conform to it. Your stereotypes might be mistaken. Alternatively, there might be some stereotypes that you're not aware of. Take these 2 models. Model A knows that some people are highly sensitive to sound. Model B is not aware of it. If your model of people is A, you will play much louder music in the supermarket. As a result, people that are highly sensitive to sound will be unable to shop there. This is what social justice means with "oppression". You're not actively pushing anyone down, but you are doing so passively,
Load More (5/31)

Load More Weeks