gwern

gwern's Comments

Why Do You Keep Having This Problem?

before word got to him that the layout was broken on mobile devices

Emphasizing the point even more - word didn't get to me. I just thought to myself, 'the layout might not be good on mobile. I ought to check.' (It was not good.)

Bay Solstice 2019 Retrospective

Yeah, after watching that, I can't see how anyone reasonable could dislike it. That was awesome.

human psycholinguists: a critical appraisal

When I noticed a reply from ‘gwern’, I admit was mildly concerned that there would be a link to a working webpage and a paypal link

Oh, well, if you want to pay for StyleGAN artwork, that can be arranged.

Do you think training a language model, whether it is GPT-2 or a near term successor entirely on math papers could have value?

No, but mostly because there are so many more direct approaches to using NNs in math, like (to cite just the NN math papers I happened to read yesterday) planning in latent space or seq2seq rewriting. (Just because you can solve math problems in natural language input/output format with Transformers doesn't mean you should try to solve it that way.)

human psycholinguists: a critical appraisal

Feeding in output as input is exactly what is iterative about DeepDream, and the scenario does not change the fact that GPT-2 and DeepDream are fundamentally different in many important ways and there is no sense in which they are 'fundamentally the same', not even close.

And let's consider the chutzpah of complaining about tone when you ended your own highly misleading comment with the snide

But by all means, spend your $1000 on it. Maybe you’ll learn something in the process.

human psycholinguists: a critical appraisal

I predict that you think artwork created with StyleGAN by definition cannot have artistic merit on its own.

Which is amusing because when people look at StyleGAN artwork and they don't realize it, like my anime faces, they often quite like it. Perhaps they just haven't seen anime faces drawn by a true Scotsman yet.

human psycholinguists: a critical appraisal

GPT-2 is best described IMHO as "DeepDream for text." They use different neural network architectures, but that's because analyzing images and natural language require different architectures. Fundamentally their complete-the-prompt-using-training-data design is the same.

If by 'fundamentally the same' you mean 'actually they're completely different and optimize completely different things and give completely different results on completely different modalities', then yeah, sure. (Also, a dog is an octopus.) DeepDream is a iterative optimization process which tries to maximize the class-ness of an image input (usually, dogs); a language model like GPT-2 is predicting the most likely next observation in a natural text dataset which can be fed its own guesses. They bear about as much relation as a propaganda poster and a political science paper.

A LessWrong Crypto Autopsy

There's something I should note that doesn't come through in this post: one of the reasons I was interested in Bitcoin in 2011 is because it was obvious to me that the 'experts' (economists, cryptographers, what have you) scoffing at it Just Did Not Get It.

The critics generally made blitheringly stupid criticisms which showed that they had not even read the (very short) whitepaper, saying things like 'what if the Bitcoin operator just rolls back transactions or gets hacked' or 'what stops miners from just rewriting the history' or 'the deflationary death spiral will kick in any day now' or 'what happens when someone uses a lot of computers to take over the network'. (There were much dumber ones than that, which I have mercifully forgotten.) Even the most basic reading comprehension was enough to reveal most of the criticisms were sheer nonsense, you didn't need to be a crypto expert (certainly I was not, and still am not, either a mathematician or C++ coder, and wouldn't know what to do with an exponent if you gave it to me). Many of them showed their ideological cards, like Paul Krugman or Charles Stross, and revealed that their objections were excuses because they disliked the potential reduction in state power - I mean, wow, talk about 'politics is the mind-killer'. I think I remarked on IRC back then that every time I read a blog post or op-ed 'debunking' Bitcoin, it made me want to buy even more Bitcoin. (I couldn't because I was pretty much bankrupt and wound up selling most of the Bitcoin I did have. But I sure did want to.)

Even cryptopunks often didn't seem to get it, and I wrote a whole essay in 2011 trying to explain their incomprehension and explain to them what the whole point was and why it worked in practice but not their theory ("Bitcoin is Worse is Better"). So, it was quite clear to me that Bitcoin was, objectively, misunderstood, and a 'secret' in the Thiel sense.

And in a market where a price is either too low or too high, 'reversed stupidity' is intelligence...

(If anyone was wondering: I don't think this argument really holds right now. Discussions of Bitcoin are far more sophisticated, and the critics generally avoid the dumbest old arguments. They often even manage to avoid making any factual errors - although I've wondered about some of the Tether criticisms, which rely on what looks like rather dubious statistics.)

What sort of luck or cognitive strategy did this require? I think it did require a good deal of luck simply to be in LW circles where we have enough cryptopunk influence to happen to hear about Bitcoin early on. Otherwise, it would be unreasonable to expect people to somehow pluck Bitcoin out of the entire universe of obscure niche products like 'all penny stocks'. But once you pass that filter, all you really needed was to, while reading about interesting fun developments online, simply not let your brains fall out and notice that the critics were not doing even the most basic due diligence or making logically valid arguments and had clear (bad) reasons for opposing Bitcoin, and understand that implied Bitcoin was undervalued and a financial opportunity. I definitely do not see anything negative about most people for not getting into Bitcoin in 2011, since there's no good ex ante reason for them to have been interested in or read up on it and doing so in general is probably a bad use of time - but for LWers, we had other reasons for being interested enough in Bitcoin to realize what an opportunity it was, so there it is a bit of a failure to not get involved.

Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal

You'll need a bunch in a single passage. If you don't need to disambiguate a large hairball of differently-timed people (like in My Best and Worst Mistake), then you probably shouldn't bother in general.

Would you say that about citations? "Oh, you only use one source in this paragraph, so just omit the author/year/title. The reader can probably figure it out from mentions elsewhere if they really need to anyway." That the use of subscripts is particularly clear when you have a hairball of references (in an example constructed to show benefits) doesn't mean solitary uses are useless.

I'm struggling to see how this is an improvement over "on FB" or "on Facebook" for either the reader or the writer, assuming you don't want to bury-but-still-mention the medium/audience.

It's a matter of emphasis. Yes, you can write it out longhand, much as you can write out any equation or number long hand as not but "twenty-two divided by two-hundred-and-thirty" if necessary. Natural language is Turing-complete, so to speak: anything you do in a typographic way or a DSL like equations can be done as English (and of course, prior to the invention of various notations, people did write out equations like that, as painful as it is trying to imagine doing algebra while writing everything out without the benefit of even equal-signs). But you usually shouldn't.

Is the mention of being Facebook in that example so important it must be called out like that? I didn't think so. It seemed like the kind of snark a husband might make in passing. Writing it out feels like 'explaining the joke'. Snark doesn't work if you need to surround it in flashing neon lights with arrows pointing inward saying "I am being sarcastic and cynical and ironic here". You can modify the example in your head to something which puts less emphasis on Facebook, if you feel strongly about it.

Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal

I don't think they're confusingly different. See the "A single unified notation..." part. Distinguishing the two typographically is codex chauvinism.

Load More