# All of gjm's Comments + Replies

Challenge: know everything that the best go bot knows about go

Robot arms and computer vision, at the level necessary for playing a game of go, are I think a sufficiently solved problem that there's no particular reason why AI researchers working on making a strong go-playing program would bother hooking them up. On its own I don't think doing that would add anything interesting; in particular, I don't think there's any sense in which it would make the program's thinking more human-like.

I don't know about the Alphas, but my laptop running KataGo uses an amount of power that's in the same ballpark as Ke Jie (more than ... (read more)

Containment Thread on the Motivation and Political Context for My Philosophy of Language Agenda

OK. I'm not sure to what extent I'm supposed to take the last comment as an insult ("you're very good at emitting sophistical bullshit" or whatever), but no matter :-).

I don't know that I was feeling optimistic, but I had had some hopes that you might be persuaded to engage with what seem like key criticisms rather than just dismissing them. But you certainly should feel obliged to engage with someone you aren't finding it worthwhile arguing with.

By the way, I see that at least one earlier comment of yours in this thread has been downvoted; it wasn't by me.

Containment Thread on the Motivation and Political Context for My Philosophy of Language Agenda

Definitions aren't generally arbitrary in communication for reasons similar to why they aren't arbitrary in cognition; if I define "woman" to mean "adult female human" (for some possibly-contentious definition of female" I will communicate more effectively than if I define it to mean "adult female human who is not called Jane, OR 4x2 lego brick" (same definition of "female"), even if everyone knows what definitions I am using. I think the distinction that's doing the actual work isn't between communication and cognition, but between proper nouns (where the... (read more)

2Zack_M_Davis18hOkay. I give up. I really liked your 11 May comment [https://www.lesswrong.com/posts/fRwdkop6tyhi3d22L/there-s-no-such-thing-as-a-tree-phylogenetically?commentId=3eZ8J94gXGoaDcL7F#3eZ8J94gXGoaDcL7F] , and it made me optimistic that this conversation would lead somewhere new and interesting, but I'm not feeling optimistic about that anymore. (You probably aren't, either.) This was fun, though: thanks! You're very good at what you do!
Challenge: know everything that the best go bot knows about go

I too am not sure whence cometh our disagreement, but I know the point at which I first thought we had one. There was some discussion of CNN-based go programs looking at "local patterns" and you said:

Does AlphaGo rely on local patterns? Possibly, but AlphaGoZero? Where humans see a 3 phase game with maybe 320 moves, which gets broken down into opening, middle and end game, ko's, threats, exchanges, and so on, it seems likely AlphaGoZero sees the whole game as one 'thing' (and in fact sees that one game as just one variation in the likely billions of millio

1Josh Smith-Brennan13hSorry if this goes a bit funny in places, I've been up all night. We had 4 cop cars and a helicopter taking an interest in the apartment complex I live in last night and I haven't been able to sleep since. Ok. I think we are on the same page now, which is good. I've had to readjust the parameters of my thinking a bit in order to look at similarities in our writing about our thinking. I consider myself to be a natural skeptic, so I tend to question things first before I find a way to agree with them. I blame my mom for this, so any complaints should be sent to her. :) I'm a little familiar with CNN's, although I didn't know the exact name. I've previously done a little research into Neural Nets as they relate to Machine Vision, namely just trying to familiarize myself with toy models of what they are, how they function, and a little on training them. I am/am not surprised they are used for Go playing Ai, but that is a slightly different topic for another time hopefully. As for the meaning of "local patterns", I think of them as a human concept, a 'short cut' of sorts to help humans divide the board up into smaller subsets as you mentioned. I think we naturally see the whole board when we play Go, and it is through training that we begin to see 'local patterns.' Every move in a physical game, uses the matter of a stone to create meaning to everyone watching. All observers as well as the players are all seeing the same matter, and so the meaning is shared even though some people are trained to see more, and more accurate information about the game. You cannot see the players brains working unless you put them in fMRI machines or something of that nature, but you can see the judgement of their contemplation in the matter of the placement of a stone on the board. The meaning is a by product of the matter, and vice versa. The meaning and the matter are entangled. In an instance of a Go playing AI, we can actually 'see' or try ti 'understand' what is going on inside
Challenge: know everything that the best go bot knows about go

As I said elsewhere in the thread, by "local" I mean "looking only at a smallish region of the board". A "local pattern" is one defined by reference to a small part of the board. A group is "locally alive" if there's nothing in the part of the board it occupies that would make it not-alive. A move is "the best move locally" if when you look at a given smallish region of the board it's the best move so far as you can judge from the configuration there. Etc. (There are uses of "local" that don't quite match that; e.g., a "local ko threat" is one that affects... (read more)

1Josh Smith-Brennan1dOnce again, I have to say then, I'm not sure where the disagreement stems from between you and I. Although I would say that the idea of 'locally alive' is a little confusing: a group is either 'alive' because it has 2 real eyes or has implied shape so that it cannot be killed (barring potential ko's which might force the player to sacrifice the group for a more important strategic play elsewhere) or it's 'possible to kill' at which point it would be considered 'not yet alive.' I think this is another way to describe 'locally alive' possibly? Maybe I don't understand what you mean by this, but I think that does match the same concept: i.e. white starts a ko battle by capturing a stone in blacks huge dragon, a stone which is necessary for blacks shape to live. So black must respond by making a ko threat elsewhere that is approx. of equal value to the loss of blacks dragon, otherwise white has no reason to continue the battle and can take the ko, thereby killing blacks huge group. If black makes such a threat, so that white must respond with another ko threat, it would be to whites advantage to be able to make a 'local ko' threat, meaning that the new ko threat by white would still effect the shape of concern - namely blacks dragon - so that now there are 2 points of importance at risk for blacks group to live instead of just the one. This is what I would consider to be a 'local ko' threat, because it builds directly on the first ko threat instead of forcing white to find another ko threat elsewhere, indirectly affecting blacks play, but not blacks dragon, the place where the original ko started.
Challenge: know everything that the best go bot knows about go

Perhaps you would like to clarify how you are intending to use the word "local"?

My usage here is as follows: a "local pattern" is something whose presence or absence you can evaluate by looking at a small region of the board. (The smaller, the more local; locality comes in degrees. Presence or absence of a pattern might do, too.) So e.g. an empty triangle is an extremely local pattern; you can tell whether it is present by looking at a very small region of the board. A ponnuki is slightly less local, a table-shape slightly less local again, but these are a... (read more)

1Josh Smith-Brennan1dThat is a lot to consider. I'll try to take my time to parse it apart a bit more before I try to respond.
Containment Thread on the Motivation and Political Context for My Philosophy of Language Agenda

The examples seem relevant to me because they illustrate that language is not used only to predict, that the merits of a particular language-using strategy are not determined only by its impact on predictive accuracy. If language in general has proper goals other than predictive accuracy, why should I think that category-boundary drawing has no proper goal other than predictive accuracy?

I'm not sure exactly what distinction you're appealing to, by the way. In particular, you say "the communicative function of proper names ... the cognitive function of cate... (read more)

1Slider12hI think the signficance of the symbols might not be so monolithic or atleast interesting dynamics can be discovered when differences onthem are studied and that concepts would benefit from being divided into smaller chunks for this discussion. Say that there is a community that has access to 3 kinds of berries and has 2 types of members. There is a "poisonous berry" that kills anybody that eats it. Then there is a "safe berry" that anybody can eat. Then third type kills one type of member while the other type can eat fine. Say that amout of people that are "allergic" is a small minority (say 1%). If the safe and the allergen berries are very close in coloration and identifiable characteristics I could understand why the hunter gatherers would lump them into the same concept. However using this concept in cooking etc will probably result in food that would be unreasonably risky for a allergic person to attempt to bite. Now anybody could be allergic to anything, it might no tbe feasible to be paranoid about the environemnt and make all the catalog all the tiniest differences. But people can also get lucky in that if the safe and allergen berry are saliently different then the allergics looking after their own survival would be relatively straightforward (gatherers might put them in different buckets to begin with etc). I am reading/guessing that the "information conveyance" property is important in that it is value/survival ambivalent while allowing people to maximise for value/survival. In a way because anybody could have a lot of weird idiosyncraties then critising that society does something inconvenient for you is not neccesarily a basis to change society per se. On the other hand if a little eye-squinting could prevent poisonings, not bothering to do so could be extremely cruel. If the squinting feels like pointless pedantry to the non-allergic gatherer, it can feel like inefficiency. I could also see that one way to deal with the situation is to forbid allerg
Challenge: know everything that the best go bot knows about go

Last I heard, Leela Zero still tended to play slack moves in highly unbalanced late-game situations.

Challenge: know everything that the best go bot knows about go

I think that at least some of the time you are using "local" and "global" temporally whereas I am using them spatially. (For short-term versus long-term I would tend to use distinctions like "tactics" versus "strategy" rather than "local" versus "global".) Aside from that, I cannot think of anything more local than wincing at an empty triangle.

If by "lowest levels" you mean the earliest layers of the network in a KataGo-like bot, they literally cannot take global position into account (except to whatever extent the non-neural-network bits of the program feed the network with more-global information, like "how many liberties does the group containing this stone have?").

1Josh Smith-Brennan2dI am using the terms 'locally' and 'globally' both temporally and spatially. Like all board games, time and space effect the outcomes, I don't really think you can have one without the other. Can you give me an example of what you are referring to specifically? I really don't know what you mean by this. Empty triangles get played all the time, it's just that they are not considered an efficient use of the stones. Placing one stone next to another is 'just as local' as creating an empty triangle; in terms of spatial consideration, local just refers to moves or shapes that are close by to each other. Is there another meaning for local you are thinking of? What I mean by globally in this instance has more to do with how the training the engine has already gone through has biased it's algorithms to consider 'good plays' over 'bad plays' by previously playing games and then analyzing those games from a global perspective in order to retrain the networks to maximize it's chances of success in subsequent games. The 'global' consideration has already been done prior to the game, it is simply expressed in the next game.
Challenge: know everything that the best go bot knows about go

I believe what DanielFilan is mostly interested in here is the general project of understanding what neural networks "know" or "understand" or "want".

(Because one day we may have AIs that are much much smarter than we are, and being much smarter than us may make them much more powerful than us in various senses, and in that case it could be tremendously important that we be able to avoid having them use that power in ways that would be disastrous for us. At present, the most impressive and most human-intelligence-like AI systems are neural networks, so getting a deep understanding of neural networks might turn out to be not just very interesting for its own sake but vital for the survival of the human race.)

1Josh Smith-Brennan2dIf he used the concept of a Go playing AI to inspire discussion along those lines, then Ok, I did get that. I guess I'm not sure where the misunderstanding came from then.
Zvi's Law of No Evidence

I agree with you that Zvi and Eliezer are both correct, but I am unconvinced by your analysis of why. I don't think "no evidence" is vague. I think it's false.

"There is no evidence of Bigfoot" is simply untrue. There are people who claim to have seen things-that-maybe-could-be-Bigfoot. There are people who claim to have found footprints and the like. All these things are evidence. Crappy evidence, which is why it doesn't do much to change my opinion that Bigfoot is almost certainly not real, but still evidence.

"Nobody looked for Bigfoot and nobody found hi... (read more)

9PeterMcCluskey2dI disagree. Much of what's going wrong is differing meanings of the word evidence [https://www.lesswrong.com/posts/fhojYBGGiYAFcryHZ/scientific-evidence-legal-evidence-rational-evidence] . Most people are oblivious to the Bayesian meaning of the word evidence. When I hear an ordinary person say "no evidence", I usually interpret it as "no evidence that's admissible in a court", or maybe even "no proof".
Deliberately Vague Language is Bullshit

I don't entirely agree. I think vague language is sometimes bullshit but sometimes honesty.

I am a human being rather than an idealized superintelligence. Sometimes my own knowledge and understanding are vague. E.g., to take one of the examples in the OP, I may say "you probably shouldn't try for a career in theoretical physics unless you are very smart"; I'm not giving an "IQ threshold" (or a mapping from IQ to probability of success, or whatever) because I don't have one to give. I could be more concrete: "you probably shouldn't try for a career in theore... (read more)

1Josh Smith-Brennan2dAnd sometimes honesty can be bullshit too. Meaning true facts can be communicated in a way that I would consider 'bullshit-ish', or severely nit-picky to the point of absurdity. I find this is the case alot, and I would add that I think Acrackedpot is on point with: I think all vagueness can be considered bullshit, but not all bullshit can be considered vague. It's how someone deals with their lack of knowledge about a particular subject in social or personal relationships that determines whether vagueness or even bullshit is negative or not though. For example in the social conversation about becoming a theoretical physicist, I could say to someone "you probably shouldn't try for a career in theoretical physics unless you are very smart", but I assume that most people would consider this to be common sense. So why do I really need to say it? If I don't have anything of value beyond common sense to contribute to the conversation because of my lack of knowledge or understanding, there are other motivating factors which make me say something so vague instead of just listening to people who might have more to contribute. Am I expected to say something, or expected to listen? Am I expected to know something beyond common sense, or expected to always say something stupid, funny or outrageous? Am I interested in being seen as a contributor or as a lurker? Novices I think are rightly forgiven for bullshitting when they don't have much to contribute initially, as long as they can learn and improve their understanding of a subject enough to bullshit less and less as time goes by. Or at the least bullshit in a deliberately entertaining way. You might say they are good bullshitters. Posers on the other hand are generally not forgiven for bullshitting because they don't learn and improve their understanding of a particular subject, and continue to bullshit as much or possibly even more than they did in the beginning. You might say they are bad bullshitters. On the oth
3AllAmericanBreakfast2dYou're right on a personal level. On the social level, I lean in lsusr's direction. Given the apparent utility of IQ as a measure of aptitude, we could be testing everybody and using those results to help people find careers that are good fits for them. The fact that we don't is at least suggestive of a massive social taboo, which is the cause of the lack of information on the part of individuals that necessitates vagueness. Taboo -> lack of information -> vagueness.
3wzp2dI think this example shows more that the difference between vagueness and simplicity is dependent on the context. If I talk to somebody who has not read LW/rationality/IQ studies they have no concrete mental model of what it means to have IQ 130 (other than that the higher IQs are better). So then saying "you need IQ 130" and explaining in a convoluted way what exactly did you mean by that conveys less information than just saying "you need to be smart" and makes a simple statement more complex. On the other hand, if I talk to somebody about whom I have reasonable expectation that they understand what the IQ actually means, saying smart is also simpler, but due to the abundant and less well-defined use outside precise conversations, it might make sense to default to more precise and quantitative statements to avoid confusion.
Challenge: know everything that the best go bot knows about go

Yes, KataGo trains entirely through self-play.

It's not "100% pure Zero" in that it doesn't only play entire games from the start. So e.g. it gets supplied with some starting positions that are ones in which some version of KataGo was known to have blindspots (in the hope that this helps it understand those positions better and lose the blindspots) or ones that occur in human games but not in KataGo self-play games (in the hope that this helps it play better against humans and makes it more useful for analysing human games). But I believe all its training i... (read more)

1Josh Smith-Brennan2dFrom the arxiv.org paper on KataGo [https://arxiv.org/abs/1902.10565]: This says enough to help me understand there were no human games as input involved in the initial training of the KataGo engine. Here the acknowledgement of the gains in greater efficiency from non-domain specific improvements to the software and hardware architecture are somewhat insightful. As I'm running a windows machine with less than an i5, an integrated graphics card, and a somewhat respectable 12gigs of ram, playing around with neural nets and attempting to train them is sort of out of the question at this point. I do have an interest in this area, although at this point my interests I think would be better served if I could work with someone already trained in these areas. I suppose this gets back to OP's desire to program a Go Bot in the most efficient manner possible. I think the domain of Go would still be too large for a human to 'know' Go the way even the most efficient Go Bot would/will eventually 'know' Go. Although I'm sure we agree about quite a bit in these regards, I wouldn't necessarily put an isolated instance of something like 'an empty triangle' under the heading of local play. Although at lower levels of consideration under circumstances of attempting to define the idea of 'shape' and the potential they have, it is closer to local play than to global play, especially if it's true that the earlier layers compute based on local play, instead of global play. I kind of doubt that though. Maybe the initial models did, but after some refinement it seems plausible that even the lowest levels take global position into account, and at the scale and speed AI neural nets can compute, it's difficult to compare human thinking of differences between local play and global play to AI thinking on the matter. It seems reasonable to assume a good Engine like Katago with the most up-to date models might function as if it plays globally the entire game. This is what human players stu
Challenge: know everything that the best go bot knows about go

KataGo was not trained on human games.

I wonder whether we are interpreting "local patterns" in different ways. What I mean is the sort of thing whose crudest and most elementary versions are things like "it's good to make table shapes" and "empty triangles are bad".

The earlier layers of a CNN-based go-playing network are necessarily identifying local patterns in some sense. (Though KataGo's network isn't a pure CNN and does some global things too; I forget the details.)

If you can predict the winner of a go game after two moves then it's because (1) one of ... (read more)

1Josh Smith-Brennan1dHere I don't think you're using the terms 'locally' and 'globally' in the standard sense that Go players use them. Seeing as how CNN based processing underlies much of image processing, analyzing the shapes on a GO board this way makes a lot of sense, it's also how humans understand the game. However, I don't understand what you mean by "KataGo's network isn't a pure CNN and does some global things too..." here the use of the word 'global' seems qualitatively different than how you use 'local'.
1DanielFilan2dThe 'global' things seem to be pooling operations that compute channel-wise means and maxes. Paper link [https://arxiv.org/abs/1902.10565].
1Josh Smith-Brennan3dI'm not a programmer, but have been trying to fit learning more about AI into my day, sort of using Go bots as an entry point into beginning to understand how neural nets and algorithms work in a more concrete, less conceptual way. So then was KataGo simply playing itself in order to train? I've spent 20 years or so playing casually on 19x19 boards mostly, and I think my concept of local play is less crude than the one your talking about. I tend to think of local play as play that is still in some sort of relationship to other shapes and smaller parts of the board, where what you are describing seems to imbue the shapes with an 'entity' of sorts apart from the actual game, if that makes sense. I think it's hard to describe a tree to someone who's never heard of one, without describing how it relates to it's environment: (A) "one type of tree is a 30ft tall cylinder of wood with bark and has roots, branches and leaves" versus (B) "many trees make up a forest, they use roots buried in the ground to stand up and pull nutrients from the ground and they use their leaves to photosynthesize and turn carbon dioxide into oxygen as well as providing shade". (A) describes certain characteristics of a particular species of tree as it relates to itself and what it means to be a tree, whereas (B) describes what trees do in relation to a local environment. If you take that further, you could talk globally (literally) about how all the trees in the world contribute to clearing pollution out of the air, protecting the integrity of soil and provide support for all kinds of wildlife, as well as provide timber for construction, fuel and paper industries. All the local situations around the world add up to complete the global situation.
Challenge: know everything that the best go bot knows about go

In case it wasn't clear, that sentence beginning "Stronger players do better" was not purporting to describe all the things that make stronger go players stronger, but to describe specifically how I think they are stronger in joseki.

I don't think joseki are the main reason why professional go players spend so much time studying, unless you define "studying" more narrowly than I would. But that's pure guesswork; I haven't actually talked to any go professionals and asked how much time they spend studying joseki.

(Professional chess players spend a lot of tim... (read more)

2DanielFilan2dThis is also my understanding.
1Josh Smith-Brennan3dI didn't take it as if it was all they did. With (1) it seems like your describing the skill of reading, but not necessarily reading with the understanding of how to play so that you have a good outcome, or reading and assessing the variations of a particular position, and with (2) your describing reading how local play affects global play. I think if they are truly strong players, they also (3) understand the importance of getting and maintaining sente, and (4) also see joseki (or standard sequences) from both sides, as white and as black. I was talking mostly about studying in preparation to become a professional, like daily study for 8 hours a day, the path from say 1k-9p, although Joseki are usually an important part of study at any level. I think the term also applies more loosely to 'sequences with a good outcome'. Coming up with new and personal 'proprietary' joseki I think consumes a lot of study time for professionals, while going over other peoples or AI games and exploring the different variations. There are other things to study, but I still maintain that Joseki make up fair amount of Professional knowledge. Some people study openings, others life and death problems, end game scenarios, but they all rely on learning set patterns and how to best to integrate them.
Challenge: know everything that the best go bot knows about go

I have the same sense that strong go bots play more "globally" than strong humans.

(Though I think what they do is in some useful sense a generalization of spotting local patterns; after all, in some sense that's what a convolutional neural network does. But as you add more layers the patterns that can be represented become more subtle and larger, and the networks of top bots are plenty deep enough that "larger" grows sufficiently to encompass the whole board.)

I think what's going on with different joseki choices between amateurs and very strong humans isn'... (read more)

1Josh Smith-Brennan3dVery much so. I have the same sense. From my understanding, Professional players (and stronger amateurs) still rely heavily on Joseki, it's just that they Joseki become longer and more complicated. In a lot of ways, the stronger you get I think the more reliant you become on patterns you know have succeeded for you or others in the past. It's the reason why Professionals spend so much time studying, and why most, if not all top ranked professionals started studying and playing as children. It takes that kind of dedication and that amount of time to learn to become a top player. It's possible to become a strong amateur Go player based on 'feeling' and positional judgement, but without being able to read your moves out to a decent degree - maybe 10 -15 moves ahead methodically - it's not easy to get very strong.
Challenge: know everything that the best go bot knows about go

It might be worth mentioning that the specific bot mentioned in the OP, David Wu's KataGo, doesn't make random-looking slack moves in the endgame because the figure of merit it's trying to maximize involves both win probability and (with a very small coefficient) final score.

This doesn't entirely negate Christian's point; some other strong bot might still have that behaviour, and KataGo itself may well have other features with similar implications.

On the other hand, there's at least one respect in which arguably chess bots are harder to learn from than go ... (read more)

1Josh Smith-Brennan3dThing is, the way you build shape in Go isn't a straightforward process; the 3 phases of a game, opening, middle game and end game usually involve different types of positional judgement, requiring different ratios of consideration between local position and global position. Shape building occurs as game play progresses simply because of the aggregation of moves on the board over time, with the development of 'good shape' being desirable because it's easy to defend and useful, and 'bad shape' being difficult to defend and a hindrance. Most of the opening of the game and part of the middle game, shape is implied, and it is the potential of a shape to support a specific approach or tactic which develops into strategy over the game. It is the ability of the human player to correctly see the potential for shape especially in the opening, and to read out how it is likely to grow over the course of game play which makes the difference between a good player and a mediocre one. Since a great endgame can never make up for a bad opening, especially when you consider many games between evenly matched players will result in a win with only a 0.5 point lead, a human has to be good at either the opening or the middle game in order to even have a chance of winning in the end game. In human terms, Go bots seem to contemplate all 3 phases, opening, middle, and end game at the same time - from the beginning of the game - while the human player is only thinking about the opening. It seems this long view of AI leads the bots to play moves which at times seem like bad moves. Sometimes a potential rational becomes clear 10 or 15 moves later, but at times it is just plain impossible to understand why a Go bot plays a different move than the one preferred by professionals. At times yes. Trying to read out all the potential variations of a developing position - over time - from a seemingly arbitrary or random move a Go bot makes results in diminishing returns for a human player. Especi
2ChristianKl3dEven when that's true, it suggests that it might also try to do at the end a bit to maximize final score and not be as unconcered about it as Alpha Go, the fact that Alpha Go behaves like it does suggest that what's important for being able to play very strong isn't about local patterns. I have another argument that's a bit more Go specific is that even for humans professional players follow patterns less then amateuers in 1-5 kyu. If you look at openings, the amateur openings look play the same series of moves for a longer time then the professionals. The amateurs are often playing patterns they learned while the professionals (and very strong amateur players) are less pattern focused. According to the analysis of the pros, Alpha Go seemed not playing according to local patterns and thinking more globally to another level (here I have to trust the professionals because that difference goes beyond my Go abilities while the other arguments I made are more about things that I can reason about without trusting others). In Go it's often not three moves later but 100 moves later. Go is not a game where in the early/midgame with the expection of life/dead situations the important consequences are not a handful of moves in the future but much further out.
There’s no such thing as a tree (phylogenetically)

[The following is rather long; I'd offer the usual Pascal quotation but actually I'm not sure how much shorter it could actually be. I hope it isn't too tedious to read. It is quite a bit shorter than "Unnatural Categories are Optimized for Deception".]

I don't really understand what in what I wrote you're interpreting as condescension, but for what it's worth none was intended.

No, I don't think I ever read UCAOFD in any detail. The "did you read ...?" seems, on the face of it, to be assuming a principle along the lines of "you should not say that someone i... (read more)

Challenge: know everything that the best go bot knows about go

(I am not one of the people who have expressed skepticism, but I find myself with what I take to be feelings somewhat similar to theirs.)

I agree with 1 if it success is defined rather strictly (e.g., requiring that one human brain contain all the information in a form that actually enables the person whose brain it is to play like the bot does) but not necessarily if it is defined more laxly (e.g., it's enough if for any given decision the bot makes we have a procedure that pretty much always gives us a human-comprehensible explanation of why it made that ... (read more)

2DanielFilan2dI think I basically agree with all of this.
There’s no such thing as a tree (phylogenetically)

You keep saying this (and other roughly-equivalent things) but I think it's just wrong.

If you pick a measure on your concept-space, you can use it to define a notion of entropy, and then you can ask what clusterings permit maximally efficient communication. It's not clear that communication efficiency is the thing we want to maximize, and if you permit approximate transmission of information then you may actually want to minimize something like cost of errors + cost of communication, and for that you need not merely a measure but a metric. Anyway, the poin... (read more)

2Zack_M_Davis5dI agree. Did ... did you read "Unnatural Categories Are Optimized for Deception" [https://www.lesswrong.com/posts/onwgTH6n8wxRSo2BJ/unnatural-categories-are-optimized-for-deception] ? The post says this very explicitly in quite a lot of detail with specific numerical examples! (Ctrl-F for "metric".) If you're going to condescend to me like this, I think I deserve an answer: did you read the post [https://www.lesswrong.com/posts/onwgTH6n8wxRSo2BJ/unnatural-categories-are-optimized-for-deception] , yes or no? I know, it's kind of long (just under 10,000 words). But ... if you're going to put in the effort to write 500 words allegedly disproving what I "keep saying", isn't it worth ... actually reading what I say?
There’s no such thing as a tree (phylogenetically)

No such thing as a tree, phylogenetically?

:-)

Is there a way to preview comments?

So the trouble seems to be in liststyleediting.js, function upcastListItemStyle, which has a line

const listStyle = listParent.getStyle( 'list-style-type' ) || DEFAULT_LIST_TYPE

and at this point listParent is null. (It's called H after minification.) This is a list that isn't inside another list, so maybe it's correct that listParent is null (in which case the code should say something more like listParent && listParent.getStyle(...) || ... or, to be more modern about it, listParent?.getStyle?.(...) || ...); or maybe listParent is meant to be the wh... (read more)

Is there a way to preview comments?

I have definitely encountered them in the last week or two.

I just did the following test. After the line above, I entered [asterisk] [space] (so I'm now in a bullet point), some text, then ctrl-4 to enter the LaTeX editor, entered some mathematics, hit enter to get out of the LaTeX editor, and pressed the space bar.

At that point, in the console I got this:

JavaScript error: https://www.lesswrong.com/js/bundle.js?hash=76bd6cfc5754522a5be764349f9bfc9d497abf9fe5ca29ce4eb640ef47e1976f, line 139: CKEditorError: H is null

2habryka6dOh, interesting. That is a totally different bug from what I had encountered previously. Will look into it. Somehow I've never run into this.
Do you think TDT/FDT imply magic is real?

TDT and FDT don't state any propositions, they prescribe behaviours. So they could only "imply magic is real" in three ways that I can see.

1. They could require you to believe some proposition close to "magic is real".
2. They could require you to do things that, if done by someone using a more "normal" decision theory, would be a clear indication that they think magic is real.
3. They could presuppose something close to "magic is real".

I am not an expert on either TDT or FDT, and my understanding is that neither is actually well enough defined to be very sure about ... (read more)

Is there a way to preview comments?

In your profile settings there's a checkbox that switches between WYSIWYG and Markdown, and another that switches between two different WYSIWYG comment editors, one older and one newer. I am using the "new" WYSIWYG one, and it shows LaTeX just fine while composing. Obviously the pure-Markdown comment editor doesn't do that. I don't remember what the "old" WYSIWYG editor does.

However, the "new" WYSIWYG editor has annoying bugs (or perhaps interactions with quirks of my browser setup?) which mean that every now and then it will decide that hitting the space ... (read more)

3habryka6dI think these bugs should be fixed now, but if you still run into them, please let me know! I thought we had fixed these ~2-3 months ago.
1Jan Christian Refsgaard6dNice Thanks, now My only issue is this: I would prefer pure markdown for writing posts and the new WYSIWYG for commenting. But after playing with it it seems that it remember your settings for posts, so you can switch back before clicking create post, and then you are golden :). Thanks a lot
Taking the outside view on code quality

In that example I see that the actual format is yyyy/mm/dd rather than yyyymmdd. I definitely don't like the name yyyymmdd in that case; to me it suggests no separators. (I might advocate for switching to yyyy-mm-dd and using a name like currentDate_iso8601, though that's a bit unwieldy.)

2adamzerner9dAh. I didn't even notice that but that's a great point. I also think that yyyymmdd suggests no separators.
Taking the outside view on code quality

I'm not sure inside/outside is what's mostly going on when you're on the fence about whether making a minor name improvement is worth it. It seems to me more like the following things:

• Looking at a single decision rather than the policy it implies. (Cf. "How I lost 100 pounds using TDT".)
• Changing things has costs as well as benefits; if you rename the variable there's a (hopefully small) chance that you screw it up somehow and break things. Note that this needs to be considered even when you zoom out, even when you consider policies as well as individual de
2adamzerner9dHm. So if you look at a single decision like "it isn't worth refactoring this", and then you extrapolate out into the policy it implies ("it isn't worth refactoring for the most part"), you're still left with the question of what to do with your macro-level conclusion of "it isn't worth refactoring for the most part". Is it a good conclusion or a bad one? You could just use a reducto ad absurdum argument of "of course that's a bad conclusion", but I feel like looking at other things in your reference class is (a big part of) the way to go. Yeah, great point. I agree that those are important things to consider.
Taking the outside view on code quality

I'm aware that the currentDate versus yyyymmdd thing is only an example, but I'm not sure it's a good example because it's not obvious to me that currentDate is necessarily better.

If this thing is a string describing the current date then there are at least two separate pieces of information you might want the name to communicate. One is that it's the current date rather than some other date. The other is that it's in yyyymmdd format rather than some other format.

Whether currentDate or yyyymmdd is more informative depends on (1) which of those two things i... (read more)

3adamzerner9dAgreed! FWIW, I did realize that there are those issues with my example and that the post would be improved by using a better one (in addition to using multiple examples instead of just a single one). But I had trouble thinking of good examples and knew of the current one from here [https://github.com/ryanmcdermott/clean-code-javascript#use-meaningful-and-pronounceable-variable-names] .
Open and Welcome Thread - May 2021

I'm not sure that I'm necessarily advocating taking the other chemical names out. After all, they play a necessary role right at the very end, and I don't know how that would work without all the previous use.

I didn't mean to imply that there was any doubt that pollution was a central topic! That would be hard to miss. But it's not so clear what you're trying to say about it. (Or whether you're neutrally refraining from saying anything in particular, and just showing it in its natural habitat, as it were.) Perhaps if I were less ignorant that last parenthesis would tell me a clearer story. (Though I guess googling the chemical names would probably have sufficed.)

Open and Welcome Thread - May 2021

Some rather scattered thoughts:

There are some very nice things here; I think the paragraph where you introduce the convention of treating chemical names like biological ones is particularly good, for instance, though the convention isn't as effective after that when the chemical names are no longer pairs of words. Generally, I like your writing style at the word/phrase/sentence level, at least as it manifests in this particular piece of writing.

2sybaritick10dThank you so much (for both your kind words and your constructive criticism)! The point was intended to be about pollution and I appreciate you pointing out that it wasn't strong/clear enough-- that's something I want to work on. In the same vein, the narrator's intention with the garbage fished out of the creek would be to throw it out so it isn't litter, but I agree I don't really make that clear, especially since they call it "treasures" and say that they don't see it as unnatural. This is one of a few pieces that I've written inspired by various Superfund sites [https://www.epa.gov/superfund/national-priorities-list-npl-sites-state] in New Jersey. The specific one in question, https://semspub.epa.gov/work/02/437463.pdf [https://semspub.epa.gov/work/02/437463.pdf], is not as serious as some of the other ones I've written about on the Passaic River, or the American Cyanamid site (here are some cool photos [https://www.google.com/maps/place/American+Cyanamid+Agricultural+Research/@40.2948708,-74.674024,3a,75y,90t/data=!3m8!1e2!3m6!1sAF1QipN-IHiDOD8Qfxzxps7WQ1Zz3KyhpyMmLTM8Lvrf!2e10!3e12!6shttps:%2F%2Flh5.googleusercontent.com%2Fp%2FAF1QipN-IHiDOD8Qfxzxps7WQ1Zz3KyhpyMmLTM8Lvrf%3Dw203-h135-k-no!7i1500!8i1000!4m5!3m4!1s0x89c3e19d46382bdf:0x54cbda197ea73a75!8m2!3d40.2945182!4d-74.6747502] ) near where I grew up. It was both a major fear and inspiration to me as a kid. I also really like all the suggestions you made about the oak, both avoiding the "I characterized him" and making sure that I continue to use "he" and not "it". That and the gimmick of the chemical names not being necessary throughout the whole piece-- I was on and off about that myself, whether I should keep them in just that one paragraph or leave them in the whole piece, but now that I have a second opinion it makes sense to take the extras out. Will make changes based on this and consider the ideas you describe here in my future writing-- I appreciate you taking the time to write this. :)
The Schelling Game (a.k.a. the Coordination Game)

Fair enough. In "my" version, a contacting attacker, or a defending defender, has to figure out the specific word the clue-making attacker has in mind (or "essentially" the same word; e.g., if what's known is that the defender's word begins GA and a clue is "Eppur si muove", clearly GALILEO and GALILEI and GALILEO GALILEI are all equally good answers).

Again, I think the game works about equally well with any convention for how close you have to be, so long as you apply the same convention to attackers and defender.

The Schelling Game (a.k.a. the Coordination Game)

What counts as "fitting" the clue? ("My" version permits clues to be literally anything and in actual play they may be very obscure or indirect; in particular, they are very often not straightforward definitions.)

1Measure11dIn my version, clues are normal sentences/definitions. In edge cases it's up to the group to decide whether a word fits the clue.
[timeboxed exercise] write me your model of AI human-existential safety and the alignment problems in 15 minutes

(This was about 14:30 of writing time. I think it would probably fit into a 15-minute chunk of interview time. I deliberately avoided deleting or changing much as I went. I don't think anything in it is in any way original.)

So, first of all, why believe in existential risk? We know that sometimes species, empires, and the like come to an end. The human race is ingenious and adaptable, but there's no reason to think we're magically immune to every catastrophe that could wipe us out. (Which wouldn't necessarily mean killing every single human being; I would ... (read more)

I feel the same way (and viscerally detest ads, and go to very great lengths to avoid exposure to them), but I'm not sure whether I actually agree.

Having an advertiser attempt to manipulate your brain so that you do a thing you otherwise wouldn't have done is, for sure, bad for you. But so is having less money, and at present the only available ways of getting Nice Things On The Internet that no one is choosing to supply out of sheer benevolence[1] are (a) that you pay them money and (b) that someone pays them for showing you ads.

So, how do the harm of bei... (read more)

8Richard_Ngo12d+1 for making the case for a side that's not the one your personal feelings lean towards.
The Schelling Game (a.k.a. the Coordination Game)

I also know a word game called Contact, different from both of those (but very similar to Yoav's) and not very suitable for road trips.

• One player picks a word and tells everyone the first letter. (Call this player the defender and the others the attackers.)
• Attackers make clues (a clue can be anything at all) for words (a word can be anything at all provided it begins with the same initial letters as are known for the target word). Many clues can be "active" at once.
• If another attacker thinks they know what word a clue is pointing at, they can declare "cont
1Measure11dIn the version I've played, any word that fits a clue is sufficient to rebut it (even if it's not the attacker's intended word). The attackers can then make a more specific clue for the intended word.
2jchan13dI've played a variant like this before, except that only one clue would be active at once - if the clue is neither defeated nor contacted within some amount of time, then we'd move on to another clue, but the first clue can be re-asked later. The amount of state seemed manageable for roadtrips/hikes/etc.

The link at the start of the post (the one here, not the one at the far end) has for its text the URL of the thing it's pointing to, but its actual link target is ... this post itself.

(At least, so it is for me right now on my browser; I haven't checked whether some peculiarity of my setup is at fault, and of course it may get fixed later.)

2adamShimi14dMy bad, I thought that just putting the link at the beginning of the post would make a linkpost, but that's not how it works. Also, apparently if you make a link without linked url, it goes back to the post containing the link (as you mentioned). This should now be fixed.
Is there a good software solution for mathematical questions?

Of course what you asked for is information about computer systems for solving this kind of thing, but the particular system of equations you have here isn't so hard to make sense of (but, caution, I make a lot of mistakes, so if you want to make any use of what follows I strongly advise you to check it).

So, let's begin with the quantities z,m,n,b,c. z is known; m,n are constrained by having to add up to z, and z,m,n interact with the other variables only via b,c. Oh, and we also know y=b+c, so we've got two constraints on b,c which means we should expect ... (read more)

D&D.Sci April 2021 Evaluation and Ruleset

I underestimated the crabmonsters and overestimated the (entirely out-of-sample) possibilities of super-water-elementals and super-pirates. Worst mistake was eyeballing the crabmonsters rather than making a serious attempt to model the distribution, which would have made it clear that they actually do quite a lot of sinking.

Open and Welcome Thread - April 2021

I searched for <<family of subsets closed under union "more than half">> and the Wikipedia page about the conjecture was the first result :-).

1Thomas1moIt's nothing wrong with the Googling method. Besides, one could search for <<pizzeria at the end of the world papa mamma>>. Should work now, too. Maybe, next year's solution to this problem will be "at least 13".
Open and Welcome Thread - April 2021

I'm having trouble figuring out what the question actually is. On the face of it (taking, for now, only things that Martha and Marco and their computer actually affirm and ignoring  one thing that seems at first like obvious hyperbole) it seems that what we're told is: there's a (presumably finite) set P of pizza-types and a set O of subsets of P ("orders"), whenever two sets are in O so is their union, and O contains no singleton sets. This obviously isn't enough to tell us anything interesting.

But then there's also this stuff about whether there's a... (read more)

1Thomas1moYou are right. It's 12 or more different kinds of pizza. If it was 1 kind of pizza served, he could be certain, that one kind of pizza was at the majority (in that case at all minus one) orders. Since somebody could order a salad. But only one without pizza, to avoid equal orders by pizza kind. Even if there were 11 different pizza kinds on the menu, Marco could be sure, there is the majority kind of pizza there. Since this Fraenkel conjecture has been proved up to 11 by now. But for 12 or more, no one really knows yet. Probably it's true, but who knows. Congratulation, you were rather quick. Despite the fact, the problem formulation looks vague to you.
What does vaccine effectiveness as a function of time look like?

There's Figure 3 in this article in the NEJM. It's only a single study, and it concerns only the Pfizer vaccine. You might have seen a crude version of it in this xkcd cartoon. Note that the y-axis is cumulative incidence of Covid-19; to get a measure of its effectiveness at a particular time you should really look at the gradients.

2capybaralet1moFrom that figure, it looks to me like roughly 0 protection until day 10 or 11, and then near perfect protection after that. Surprisingly non-smooth!
The consequentialist case for social conservatism, or “Against Cultural Superstimuli”

Oh, for sure dysphoria is not the kind of thing that can specifically make someone need to be called "xe". I think it can specifically make someone find it upsetting to be referred to with specifically-male or specifically-female pronouns, though, and my sense is that Jordan Peterson isn't any happier being asked to refer to someone as "they" than as "xe".

If both "he" and "she" give someone feelings of dysphoria, then I think it's rude to require that they pick one of those. Singular "they" works pretty well, it has a long history of use in English[1], and... (read more)

The consequentialist case for social conservatism, or “Against Cultural Superstimuli”

I agree that almost certainly a lot of suicides among trans people are neither the result of being treated badly nor the result of expecting to be treated badly: as you say, gender dysphoria is apparently extremely horrible for many who suffer it. (My guess is that a substantial part of the distress comes specifically from being treated by other people as being of a gender that feels wrong to you, in which case much of it is "being treated badly" in an extended sense, though not necessarily one that involves any element of malice or anything from the peopl... (read more)

Covid 4/15: Are We Seriously Doing This Again

I think you're answering a different question from the one agc is asking. Unless I misunderstood, agc was asking why the UK isn't yet vaccinating people younger than 45. Being able to relax restrictions more in the US wouldn't explain that.

I think the actual answer is that the UK very promptly secured a pretty decent quantity of vaccines (mostly AstraZeneca), enough to vaccinate quite a lot of its population, but that while the US was slower it then got hold of more relative to population size, and now the US has more plentiful supply than the UK.

3Cesare1moI didn't word it very well, the original thought got lost along the way, thanks for pointing it out! I meant to conclude that since the UK cannot afford to lift restrictions for vaccinated people then they follow the logical route of vaccinating by age cohorts: this will keep deaths as low as they can be. However, in the US they are actually enabling vaccinated people to do stuff... which will boost the economy significantly, and they can afford to do so due to the degree mRNA vaccines seem to block transmission. So it makes sense to allow anyone that wants to be vaccinated to do that, instead of the inevitable slowdown caused by using an age restrictive criterion.
The consequentialist case for social conservatism, or “Against Cultural Superstimuli”

I concede that it's possible that Jordan Peterson's objections were purely 100% about compelled speech, and didn't arise from any particular wish to behave in ways that trans people find unpleasant. My own impression is that other things he's said -- e.g., that people wanting to be referred to with neopronouns like "xe" are usually making a "narcissistic power grab" -- make that a bit difficult to believe.

I do, in fact, share Peterson's objections to compelled speech, and if I thought he was right that the bill he was complaining about proposed to make it ... (read more)

2Viliam1moFrankly, I can imagine someone having a "I wish I had a male/femaly body" dysphoria, but not someone having a "I wish people called me xe" dysphoria in similar sense. So, from my perspective, gender dysphoria is a legitimate thing, made up pronouns are not. I mean, without Twitter, the number of people feeling they were born in a wrong body would be about the same, but the number of people using "xe" would be much smaller. Unless you are Finnish [https://translate.google.com/?sl=en&tl=fi&text=he%2C%20she&op=translate] or Hungarian [https://translate.google.com/?sl=en&tl=hu&text=he%2C%20she&op=translate], the pronouns are "he" and "she", choose one. Anything else is a jargon no one outside your group is obliged to use. (It would be like, dunno, a Less Wrong user asking people to call Less Wrong users "sane" and everyone else "insane", because we like it so.)
Monastery and Throne

Oh, whoops. I meant to delete that.

(It had been going to be about Andrew Sabisky, who was one of Cummings's "weirdos and misfits" and resigned after it turned out that he had said a lot of politically very unpalatable things about race, eugenics, and the like. I'd thought I remembered that a lot of the complaints about Sabisky were attacking his weirdness and geekiness as much as his controversial opinions. But when I went back and checked the discussions I was thinking of, that didn't after all seem to be so, so I cut that bit out. Except that I somehow failed to cut all of it out.)

The consequentialist case for social conservatism, or “Against Cultural Superstimuli”

(Separate reply for last paragraph, which is about something entirely different from the rest of your comment.)

If you declare that you're a trans woman and demand to be called "she", then I expect I'll call you "she". If (as I think is actually the case) you're a cis man, I don't see that you're going to get much satisfaction from being referred to that way.

I am not generally in favour of laws that would require me to do that, even if you are in fact not a troll or an asshole but simply a trans woman. Or indeed a cis woman. In my view, misgendering someone... (read more)

The consequentialist case for social conservatism, or “Against Cultural Superstimuli”

I have not claimed that "allowing bathroom bills" are a good thing on balance, or that "forbidding bathroom bills" are a bad thing on balance; only that the former are good for trans people and the latter are bad for trans people.

However, I do on the whole think that FBBs are a bad thing overall and (less confidently) that ABBs are a good thing overall, so let me address your question. What I would actually prefer is for all public toilets to be unisex, and designed in such a way that no one using them need care much who else is using them. Failing that, I... (read more)