I want to strong-downvote this on principle for being AI writing but I also want to strong-upvote this on principle for admitting to being AI writing, so I'm writing this comment instead of doing either of those things.
This seems like the sort of thing best addressed by me adding a warning / attention-conservation-notice at the start of the article, though I'm not sure what would be appropriate. "Content Note: Trolling"?
ETA: This comment has been up for 24 hours and it has positive agreement karma and no-one's suggested a better warning to use, so I'm doing the thing. Hopefully this helps?
I'd say sample size is more important if any experiment can get any statistical significance with the right sample size but not any sample size can get any statistical significance with the right experiment. But you're right, I overstated my case; amended; thank you.
The original D-K papers also found different curves for different subject matter.
I can think of several explanations for this, all of which might be true are definitely at least a little true:
- Some subjects have higher variance in performance, resulting in steeper D-K curves.
- Some subjects have higher variance in test-ability-to-measure-performance, again resulting in steeper D-K curves.
- An actual D-K effect does exist, sometimes, superposed over the statistical mirage; and it's stronger for some subjects than others.
- An anti-D-K effect exists, and it's stronger for some subjects than others.
- Something else is happening I don't know about.
And they made the unusual choice of dividing their populations into quartiles, throwing away quite a bit of resolution.
Doesn't seem unusual to me ( . . . or suspicious, if that's what you're getting at). I get away with using deciles at my day job because I work on large datasets with low-variance data, and I get away with it here because I can just add zeroes to the number of elves simulated until my plots look as smooth as I want; Dunning & Kruger had a much smaller sample since they were studying college classes full of real round-eared human beings, and sensibly chose to bucket them into fewer buckets.
It could, for a game with an unusually small & clean dataset (I'm thinking in particular of On The Construction Of Impossible Structures and How The Grinch Pessimized Christmas) . . . but realistically a LWer solving a problem like that on paper would spend the entire time lamenting that they weren't using a computer, which doesn't seem like a mental state conducive to personal growth. So nvm.
(I do have other thoughts on potential epistemic grounding activities but they're all obvious: board games, 2-4-6 tests[1], pub quizzes with confidence intervals attached, etc.)
- ^
With different rules than the original 2-4-6 test, obviously.
Fwiw, the scenarios don't have to be solved collaboratively online, and in fact most players play most of them solo. For that matter, they don't need internet access: would-be players could make sure they have the problem description & the dataset & their favorite analysis tools downloaded, then cut the wifi.
(. . . unless "be fully present" rules out laptops too, in which case yeah nvm.)
At CFAR workshops, people often become conscious of new ways their minds can work, and new things they can try. But we don’t have enough “and now I’ll try to repair my beautiful electronic sculpture, which I need to do right now because the windstorm just blew it all apart, and which will incidentally give me a bunch of real-world grounding” mixed in.
I’d love suggestions here.
I'll try to make sure I'm running a D&D.Sci scenario over both of the spans you mentioned: data-science-y attendants would get a chance to test their data-science-y skills against small but tricky problems with knowable right answers, and non-data-science-y attendants would probably still get something out of spectating (especially if they make a point of trying to predict which participants are closest to said right answer).
(. . . and if anyone else has some kind of [inference|decision]-centric moderately-but-not-excessively-demanding public puzzle/challenge they've been meaning to run, those spans look like the time to do it.)
Today, I estimate a 30–50% chance of significantly reshaping education for nearly 700,000 students and 50,000 staff.
I'd be interested to hear how that pans out a year from now.[1]
The lesson:
Don’t spend energy forcing people into actions they’re not already motivated to take.
I guess that's a valid moral to this story? I think most LWers would see this as further evidence for "political stuff gets you a lot more impact per unit effort if you're making significant use of your comparative advantages and/or taking stances orthogonal to existing party lines ('pulling ropes sideways')".
Regardless, strong-upvoted for doing interesting things in the real world and then writing about them.
- ^
. . . how do we not already have a custom emoji for this sentiment?
1
I think my best advice is to be specific (with yourself at least, even if you can't be with us).
Your post is super vague. You say you're uncertain, but don't specify what you're uncertain about; you say you're in the shit, but don't specify what kind or how you got there. Details matter, especially if you're trying to solve your problems with intelligence. If you're keeping the details private for your own reasons I can respect that, but I hope you're at least focusing on them yourself: looking at the cliff face ahead of you usually has less alpha than looking for footholds in front of you.
(I'm leaning on this in particular because it's the main thing you'd get from a human you wouldn't get from a RLHF'd LLM. You posted a vague & general complaint and the machines played along, either giving you vague & general advice or trying to yesand you from there into specific actions; none of them said "be more specific and concrete")
The sensible thing for me to do would be to end this comment here, but I'm not all that sensible, and I've thought of some clever things to say, so I'm going to shotgun them below on the off-chance at least one of them helps:
Okay, so at least part of your problem is something at least somewhat like depression. If so, I think the relevant SSC post still mostly holds up.
If it's a sea of uncertainty, how can you be sure there's no way out?
(While I'm taking things too literally: there was a way out for the mice, fwiw. It probably seemed like their problems were insurmountable, but they just had to wait for the scientists to fish them out. It was actually almost impossible for them to drown! I wouldn't apply this 1:1 to your own life for obvious reasons . . . though I will remark that just waiting for things to get better has a >>0% success rate in humans.)
Therapy doesn't imply medicines. If you get a counsellor/shrink/whatever and say "no pills", they have to respect that; and if they don't, you can just get a new counsellor/shrink/whatever. (Or, at least, that's how it should work, and how it typically works in most first-world nations: you might want to check it works that way in practice where you live.)
Also, shot in the dark, but: I used to feel very similarly, because I'd read enough anecdotal horror stories that such feelings felt warranted. I then realized that all the pill-centric horror stories I'd heard were about people on antipsychotics realizing "in retrospect, these things were really bad for me and the only reason I kept taking them so long was that they impaired my decision-making process; I probably should have just talked things out with the voices in my head"; and the only anecdotes I'd heard about antidepressants tended to involve the opposite kind of feedback loop, i.e. "the antidepressants made me agentic and ambitious enough that I agentically and ambitiously stopped taking them way earlier than I should have". Like I said, specificity matters.
(None of the above is Medical Advice, obviously.)
If you can't believe in the possibility of Success, you can still reorient around (what our community has been calling) Dignity: "I'm going to conduct myself such that when the inevitable failure happens it's going to be as little my fault as possible". Your call as to whether that's better for you vs just straight-up trying to win.