Don't feel bad about not knowing basic things

[-]Gary Beverungen5y210

Thanks for writing this. I struggle with imposter syndrome sometimes and it's reassuring to be told I don't need to know everything, even if that should be obvious.

It's also worth mentioning that making a mistake doesn't necessarily indicate lack of knowledge. You can know something well and still make a mistake out of absentmindedness, lack of sleep, etc.

[-]Adam Zerner5y80

Glad it helped. Good point about mistakes not necessarily indicating lack of knowledge, that is very true.

[-]cousin_it5y120

Not knowing some trivia is fine. But bouncing off something when I try to figure it out, if my peers have no trouble with it, doesn't feel fine to me. It feels terrible. It makes me double my effort, then double it again. And this reaction feels right to me, I wouldn't want to get rid of it.

[-]Adam Zerner5y60

Thank you for bringing up a good counterpoint to a core point of the post.

I agree with the sentiment that tsuyoku naritai is desirable, and that some amount of disappointment is required for it. But feeling "terrible" seems like it is taking things too far.

I'm not sure where the right point on the spectrum is. I lean towards thinking that intrinsic motivation + carrots (rather than sticks) go a long way and that it is appropriate to apply "the stick" in a similar manner as you would apply hot sauce to a dish. You'd shake a little on top, not soak the entire plate in it. And similar to hot sauce, it depends on your tolerance level.

It's also worth distinguishing between two things I've conflated a little bit: 1) the appropriate emotional reaction, and 2) how much one should update on the evidence. I think the more central point in the post was about (2), although (1) is definitely still worth discussing. Also, if the answer to (2) is in fact "very little", then I think that feeling terrible for (1) would probably necessitate a little bit of lying to yourself, which is a dark art.

[-]johnswentworth5y80

There's an important point which I think this misses.

Rather than imagining the bottom level of a 2D pyramid, imagine the bottom level of a 3D pyramid. As you fill in the bottom level of that 3D pyramid, at some point you go from "it's mostly space with a few islands filled in" to "it's mostly filled in with a few islands of space". There's this phase-transition-like-phenomenon where all the concepts/knowledge go from disconnected pieces to connected whole.

For instance, in studying mechanics, this transition came for me around the time I took a differential equations class (I'd already taken some physics and programming). I went from feeling like "I can only model the dynamics of certain systems with special, tractable forms" to "I can model most systems, at least numerically, except for certain systems with special, intractable weird stuff". This was still only level 1 of the pyramid - the higher levels still provided important tools for solving mechanics problems more efficiently - but it gave me a unified framework in which everything fit together, and in which I could generally see where the holes were.

[-]Adam Zerner4y30

I really like that point about connected vs disconnected pieces, but isn't that true in a 2D pyramid as well?

[-]johnswentworth4y30

In a 2D pyramid, the bottom layer is 1D, so any "hole" anywhere breaks it into two disconnected pieces.

[-]Adam Zerner4y20

Ah, gotcha.

[-]Randomized, Controlled5y70

I feel deeply inadequate because I don't know how to scale a web system properly, despite doing web programming for eight years. I'm not even entirely sure what "systems design" actually is or isn't.

I also recently fell badly on my face when given a basic algorithms problem. This was equal parts anxiety, being out of practice interviewing, and being out of practice with algo questions.

[-]Brendan Long5y50

If it helps, I know how to scale web systems and basically never do it because you can run almost anything on one server unless you're Google.

[-]Adam Zerner5y50

If it makes you feel any better, up until a few weeks ago I was in the exact same boat for systems design. For algorithms it was similar. And I wasn't able to pass a Google interview after studying almost full time for four months.

[-]lsusr5y60

I don't know the Chinese word for spatula, the chemical formula for sucrose or what % does in Vim. Last month a friend showed me a second set of adjuster screws on my bicycle breakpads I had never used before. I sleep just fine at night.

[-]Adam Zerner5y50

Looking at it from that perspective of "how many times do people bat an eye vs how many opportunities to bat an eye are there", I agree that the ratio is small, but I disagree that the ratio is (particularly) relevant. To me, the important point seems to be that left hand side of the ratio is problematic regardless of what number you use for the right hand side. (This isn't the best or most precise way of saying what I'm trying to say, but hopefully it's sufficient.)

[-]Lucas20005y50

Database normalization is just about avoiding duplication, right?

I think the thing here is that people who get database design can't really understand how it is possible to not get it, but there are a lot of people for whom it is extremely difficult to understand this topic. I sat through years of lectures wondering why we were taught things that were completely self-evident. Then I looked at a lot of other people's code, and it became clear that it wasn't self-evident at all.

[-]Astor5y50

It's always puzzling me why this is so hard to accept for me. Maybe one aspect could be that work relationships force you to present yourself as best as possible to your employer. And this leads to situations in which you try to signal competence instead of uncertainty, even to yourself.

[-]StyleOfDog4y30

I've noticed that the more high-level and complex the work you're doing, the sillier your bugs get. Perhaps because you focus so much on the complex parts since they're difficult to get right, so you gloss over the more basic parts.

I don't think your pyramid is a good conceptual framework to understand programming expertise. Expertise comes mostly from seeing common/overarching patterns (which would be all over the place on your pyramid) and from understanding the entire stack - having at least some sense of how each level functions, from the high-level abstraction of RoR's ORM to object lifetime and memory concerns to how database queries happen to how the db executes it (e.g. db indexes are also relevant to your example), down to at least having played around a little with assembly language.

I don't even know Ruby or RoR, but if I had to use it for your example, my first thought would be "ok, how do I do a WHERE query in their ORM", because every db abstraction in every language and framework has to solve this problem and I've seen a lot of those solutions. And I'll know to consider eager vs lazy evaluation (what if a campaign has 1M records after filtering, maybe I want to iterate over results instead of getting a plain list), and whether campaign_id has an index, because all of those are very common concerns that crop up.

So the expertise isn't knowing a factoid "don't use x.all.filter() in RoR", it's knowing that anything that queries a database has to deal with the concerns above somehow.

[-]Adam Zerner4y*20

I don't think your pyramid is a good conceptual framework to understand programming expertise.

Perhaps. But does it meet the bar of "all models are wrong, but some are useful"?

So the expertise isn't knowing a factoid "don't use x.all.filter() in RoR", it's knowing that anything that queries a database has to deal with the concerns above somehow.

Agreed. The issue was that I am lacking a little in backend experience and didn't think about those concerns.

[-]Gurkenglas5y30

In the middle are stuff in between.

What an Applicative is? :)

[-]Adam Zerner5y20

I actually wouldn't know!

[-]Gurkenglas5y20

You can map across any monad, but not everything you can map across is a monad. Applicatives are in between.

[-]Liron4y20

Re database normalization, it’s obviously good to do if you can afford the hit for speed and scalability. Unfortunately I believe the software industry currently has a big problem with a lack of capable databases to support elegant data denormalization patterns: https://lironshapira.medium.com/data-denormalization-is-broken-7b697352f405

[-]Adam Zerner4y40

Interesting, thanks for sharing. I really like the way you explained what a paradigm shift is. And the point you make in the It's Not Your Fault section:

On the day you’re implementing the User.numUnreadRooms field, you’ll rack your brain for all the places in your code that will need to update it, and I don’t blame you for overlooking the onDeleteRoom event handler.

On the day you’re implementing the onDeleteRoom event handler, you’ll rack your brain for all the denormalized fields that it might need to update, and I don’t blame you for overlooking the User.numUnreadRooms field.

I don’t blame you, the application developer, for any bug in your denormalization logic. I blame the paradigm of tightly coupling denormalization logic with application code.

I think that it is related to my Don't Feel Bad About Not Knowing Basic Things post. If the "basic thing" you don't understand is also something that plenty of other people don't understand, then maybe it's because it is unnecessarily complicated. (There are other possibilities too of course.)

[-]Liron4y20

NodeJS is mostly cool because you can use the same language and the same development tools across your whole stack. When it launched I think another selling point was that it’s reasonably good at handling multiple requests in parallel.

[-]Adam Zerner4y20

NodeJS is mostly cool because you can use the same language and the same development tools across your whole stack.

Oh ok that's good to hear. That's always been what felt most appealing about it to me.

When it launched I think another selling point was that it’s reasonably good at handling multiple requests in parallel.

What is it about Node that causes this to be true? I have a feeling it has to do with Node being single-threaded, but that never made sense to me. Being single-threaded seems like it'd be strictly worse than being multi-threaded. Anything you can do on a single-threaded system, you can also do on a multi-threaded system, right? Just don't create a second thread.

[-]Liron4y20

Ya I don’t know the details even though I use NodeJS almost every day :) Maybe it does run parallel requests in separate threads.

[-]Adam Zerner4y20

Glad to hear I'm not alone :)

[-]Dirichlet-to-Neumann5y10

I'm three years into my PhD. Still don't really understand what a semi-group is (something something exponential). I can't remember the formula for the exposants in Sobolev inequality but here it is just my fault for never learning it correctly.

[-]johnlawrenceaspden5y20

So the idea is that if you've got a linear differential equation, like dx/dt=2x, then all the solutions look like x=x0*exp(2t).

And you could imagine that there's an operator which says "Where will this point end up in 1 second?", call it O(1), which looks like x0->x*exp(2). I.e. it just gets multiplied.

If you know that operator, then you know the operator that represents "Where will the point end up in 2 seconds", because you can say "where will it end up one second later" then "where will that point end up in another second.

So this gives you a law that O(2) is O(1) done twice. And O(3) is O(2) followed by O(1). So these operators are a group (the identity is 'where will it be after I run the equation for no seconds').

In the one-dimensional ODE case, this is all so trivial that it's not worth mentioning. Almost a pun.

But it generalises nicely to dx/dt=Ax, where x is a vector and A is a matrix, and that gives us a way of defining exp(At), i.e., how to exponentiate a matrix, by saying, "well what if we run this matrix equation for a second and see where it puts all the vectors?"

And in fact we can generalize it further, to differential equations on linear operators on infinite-dimensional spaces. Which is another way of looking at partial differential equations. (A function is an infinite dimensional vector, a vector is a function on a finite set).

So you can talk by analogy about how to exponentiate the diffusion equation, and a time delay operator that takes wiggly functions to less wiggly functions.

But you can't always run pdes backwards, e.g. the diffusion equation won't run in reverse, so you don't always get a full group.

I think that's the intuition for semigroups, I hope that's what you were talking about! (And sorry if it's not clear or just wrong, I haven't been a mathematician for thirty years.)

If you think about this for long enough, you should suddenly understand why e to the i pi is minus one. In fact it's just a really obvious thing that has to be true. At that point you've probably got it.

[-]Adam Zerner5y50

If you think about this for long enough, you should suddenly understand why e to the i pi is minus one. In fact it's just a really obvious thing that has to be true. At that point you've probably got it.

Be careful about the illusion of transparency here. This all strikes me as the sort of stuff that is unlikely to be obvious.

[-]johnlawrenceaspden4y20

Sorry, obvious to a mathematician who thinks about dz/dt=iz and realises that "exponentiation is time-evolution". At that point it's just "if you rotate for just long enough to turn half-way round, you'll be pointing backwards".

[-]Dirichlet-to-Neumann5y30

That's a good layman description - a semi group is basically the exponential of some linear operator. The problem is that I'm supposed to be a bit more than a layman.

[-]johnlawrenceaspden4y20

Still don't really understand what a semi-group is (something something exponential).

Then perhaps this was a little hyperbolic?

[-]Dirichlet-to-Neumann4y10

It was, a little. Truth is I know the basic definition but I've yet to build up enough knowledge and intuition around them to really use them in my research. Thinks analytic vs bounded semigroup, L^\infty calculus, angular sectors and so on.

LESSWRONG
LW

LESSWRONG
LW

91

Don't feel bad about not knowing basic things

91

91