This is among the top questions you ought to accumulate insights on if you're trying to do something difficult.

I would advise primarily focusing on how to learn more from yourself as opposed to learning from others, but still, here's what I think:

I. Strict confusion

Seek to find people who seem to be doing something dumb or crazy, and for whom the feeling you get when you try to understand them is not "I'm familiar with how someone could end up believing this" but instead "I've got no idea how they ended up there, but that's just absurd". If someone believes something wild, and your response is strict confusion, that's high value of information. You can only safely say they're low-epistemic-value if you have evidence for some alternative story that explains why they believe what they believe.

II. Surprisingly popular

Alternatively, find something that is surprisingly popular—because if you don't understand why someone believes something, you cannot exclude that they believe it for good reasons.

The meta-trick to extracting wisdom from society's noisy chatter is learn to understand what drives people's beliefs in general; then, if your model fails to predict why someone believes something, you can either learn something about human behaviour, or about whatever evidence you don't have yet.

III. Sensitivity >> specificity

It's easy to relinquish old beliefs if you are ever-optimistic that you'll find better ideas than whatever you have now. If you look back at what you wrote a year ago, and think "huh, that guy really had it all figured out," you should be suspicious that you've stagnated. Strive to be embarrassed of your past world-model—it implies progress.

So trust your mind that it'll adapt to new evidence, and tune your sensitivity up as high as the capacity of your discriminator allows. False-positives are usually harmless and quick to relinquish—and if they aren't, then believing something false for as long as it takes for you to find the counter-argument is a really good way to discover general weaknesses in your epistemic filters.^[1] You can't upgrade your immune-system without exposing yourself to infection every now and then. Another frame on this:

I was being silly!  If the hotel was ahead of me, I'd get there fastest if I kept going 60mph.  And if the hotel was behind me, I'd get there fastest by heading at 60 miles per hour in the other direction.  And if I wasn't going to turn around yet … my best bet given the uncertainty was to check N more miles of highway first, before I turned around.
 — The correct response to uncertainty is *not* half-speed — LessWrong

IV. Vingean deference limits

The problem is that if you select people cautiously, you miss out on hiring people significantly more competent than you. The people who are much higher competence will behave in ways you don't recognise as more competent. If you were able to tell what right things to do are, you would just do those things and be at their level. Innovation on the frontier is anti-inductive.

If good research is heavy-tailed & in a positive selection-regime, then cautiousness actively selects against features with the highest expected value.
 — helplessly wasting time on the forum

finding ppl who are truly on the right side of this graph is hard bc it's easy to mis-see large divergence as craziness. lesson: only infer ppl's competence by the process they use, ~never by their object-level opinions. u can ~only learn from ppl who diverge from u.
 — some bird

V. Confusion implies VoI, not stupidity

look for epistemic caves wherefrom survivors return confused or "obviously misguided".
 — ravens can in fact talk btw

1. ^{^}

   Here assuming that investing credence in the mistaken belief increased your sensitivity to finding its counterargument. For people who are still at a level where credence begets credence, this could be bad advice.

   VI. Epistemic surface area / epistemic net / wind-wane models / some better metaphor

   Every model you have internalised as truly part of you—however true or false—increases your ability to notice when evidence supports or conflicts with it. As long as you place your flag somewhere to begin with, the winds of evidence will start pushing it in the right direction. If your wariness re believing something verifiably false prevents you from making an epistemic income, consider what you're really optimising for. Beliefs pay rent in anticipated experiences, regardless of whether they are correct in the end.

when making new words, i try to follow this principle:

label concepts such that the label has high association w situations in which you want the concept to trigger.^[1]

the usefwlness of a label can be measured on multiple fronts:

1. how easy is it to recall (or regenerate):
   1. the label just fm thinking abt the concept?
      1. low-priority, since you already have the concept.
   2. the concept just fm seeing the label?
      1. mid-priority, since this is easy to practice.^[2]
   3. the label fm situations where recalling the concept has utility?
      1. high-priority, since this is the only reason to bother making the label in the first place.

if you're optimising for b, you might label your concept "distributed boiling-frog attack" (DBFA). someone cud prob generate the whole idea fm those words alone, so it scores on highly on the criterion.

it scores poorly on c, however. if i'm in a situation in which it is helpfwl for me to notice that someone or something is DBFAing me, there are few semiotic/associative paths fm what i notice now to the label itself.

if i reflect on what kinds of situations i want this thought to reappear in, i think of something like "something is consistently going wrong w a complex system and i'm not sure why but it smells like a targeted hostile force".

maybe i'd call that the "invisible hand of malice" or "inimicus ex machina".

i rly liked the post btw! thanks!

1. ^{^}

   i happen to call this "symptomatic nymation" in my notes, bc it's about deriving new word from the effects/symptoms of the referent concept/phenomenon. a good label shud be a solution looking for a problem.

2. ^{^}

   deriving concept fm label is high-priority if you want the concept to gain popularity, however. i usually jst make words for myself and use them in my notes, so i don't hv to worry abt this.

here's the non-quantified meaning in terms of wh-movement from right to left:

$S:\underset{\text{image: rain inside your prospective thoughts}}{\underset{}{\underset{\text{image: you thinking}}{\underset{}{\text{I think}\underset{–}{\text{what}}}}\underset{\text{(circumfix time operator)}}{\underset{}{\text{it'll}\underset{–}{\text{what}}\text{tomorrow}}}\underset{\text{image: rain}}{\underset{}{\text{rain}}}}}$

for conlanging, i like${}_{95\%}$ this set of principles:

1. minimise total visual distance between operators and their arguments
2. minimise total novelty/complexity/size of all items the reader is forced to store in memory while parsing
   1. every argument in memory shud find its operator asap, and vice versa
   2. some items are fairly easy to store in memory (aka active context)
      1. like the identity of the person writing this comment (me)
      2. or the topic of the post i'm currently commenting on (clever ways to weave credences into language)
   3. other items are fairly hard
      1. often the case in mathy language, bc several complex-and-specific-and-novel items are defined at the outset, and are those items are not given intuitive anaphora.
      2. another way to use sentence-structure to offload memory-work is by writing hierarchical lists like this, so you can quickly switch gaze btn ii., c, and 2—allowing me to leverage the hierarchy anaphorically.

so to quantify sentence $S$, i prefer ur suggestion "I think${}_{55\%}$ it'll rain tomorrow". the percentage is supposed to modify "I think" anyway, so it makes more sense to make them adjacent. it's just more work bc it's novel syntax, but that's temporary.

$S^{\prime }:\underset{\text{image: rain inside your tentetative prospective thoughts}}{\underset{}{\underset{\text{(quantifier)}}{\underset{}{{\underset{–}{\text{what}}}_{55\%}}}\underset{\text{image: you thinking}}{\underset{}{\text{I think}\underset{–}{\text{what}}}}\underset{\text{(circumfix time operator)}}{\underset{}{\text{it'll}\underset{–}{\text{what}}\text{tomorrow}}}\underset{\text{image: rain}}{\underset{}{\text{rain}}}}}$

otoh, if we're specifying that subscripts are only used for credences anyway, there's no reason for us to invoke the redundant "I think" image. instead, write

it'll rain tomorrow${}_{55\%}$

in fact, the whole circumfix operator is gratuitously verbose!^[1] just write:

rain tomorrow${}_{55\%}$

$S^{\prime \prime }:\underset{\text{image: rain inside your tentetative prospective thoughts}}{\underset{}{\underset{\text{image: you tentatively thinking}}{\underset{}{{\underset{–}{\text{what}}}_{55\%}}}\underset{\text{(time operator)}}{\underset{}{\text{tomorrow}\underset{–}{\text{what}}}}\underset{\text{image: rain}}{\underset{}{\text{rain}}}}}$

1. ^{^}

   natlangs smh…^[2]

2. ^{^}

   tbh i wish we had an editor w optimised LaTeX keyboard shortcuts so we cud effortlessly use $\underset{\text{or undersets!}}{\text{underbraces}}$ wherever we fancy.^[3]

3. ^{^}

   additionally, we should just make much more use the dimensionality afforded to us by the editors we have.

   it's free cheap $\stackrel{\text{semiotic}}{\underset{\text{syntactic}}{\text{semantic}}}$ real-estate.

Edit: I found the post usefwl, thankmuch!!

Mh, was gonna ask when you were taking it.  I'm preparing to try it as a sleep-aid for when I adjust my polyphasic sleep-schedule (wanting to go fm 16h-cycles potentially down to 9h) bc it seems potentially drowsymaking and has much faster plasma decay-rate^[1] compared to alts.  This is good for polyphasic if not want drowsy aft wake.

The data in ^[1] concerns 100mg tablets, however, and a larger dose (eg 400mg) may be longer. The kinetic model^[2] they use will prob be good estimate of plasma concentrations even if adjust dose.

Questions is whether it's good estimation for duration of action in the brain, esp given that it's "single-compartment model" (the blood is one compartment, and the brain is another). My heuristic for whether plasma T predicts brain T is whether the molecule v easily passes the BBB (as melatonin does), since then I can guess that the curve for the brain will look similar to the curve for the blood, offset slightly down and to the right.
 

1. ^{^}

   

   Typical concentration-time curve of plasma ʟ-theanine of one participant after intake of 100 mg ʟ-theanine via one capsule (A) or 250 mL green tea (B). Circles represent measured concentrations of ʟ-theanine. The line represents the modeled plasma concentration-time curve by the use of the 1-compartment model.
   — Kinetics of ʟ-Theanine Uptake and Metabolism in Healthy Participants Are Comparable after Ingestion of ʟ-Theanine via Capsules and Green Tea, , 4 - ScienceDirect

2. ^{^}

   Hot tip, you can screenshot pdf equations (i rec Text Grab or Greenshot) and ask gpt-4-turbo (updated) to write it in latex and/or jupyter-notebook-compatible python^[3].

   

3. ^{^}

   Getting a tweakable python model for it was nontrivial after 35m of trying, so i'll prob j wing it w 200mg 1h pre bedtime and do 3h sleep, and adjust bon feelings.  ig the primary variable that determines how alert i feel upon waking up is getting the sleep-cycle timing right, and planning my wake-up routines (lights & text-to-speech-model as alarm) for j after I've done REM-sleep^[4].

4. ^{^}

   While I probably would feel alert if waking up in middle of REM-sleep (after emerging from deep), I want to avoid that bc studies show bad effects from targeted deprivation of REM (leaving other phases untouched).

API requests should be automatically screened for human intent, and requests judged by the model to be disrespectfwl should be denied. (And they shouldn't be trained to agree to respond to everything.)

I appreciate the post, but I also wish to hear more detailed and realistic scenarios of exactly how we might end up accidentally (or intentionally) sleepwalk into a moral catastrophe. I think it's unlikely that punishment walls will make AIs more productive, but similar things may profitable/popular if advertised for human (sadist) entertainment.

this is rly good.  summary of what i lurned:

• assume the ravens call a particular pattern iff it rains the next day. 
  • iow, $\frac{P\left(\text{raven's call}|\text{rain}\right)}{P\left(\text{raven's call}|\neg \text{rain}\right)}=\text{a bajillion}$, thus observing the raven's call is strong evidence u ought to hv an umbrella rdy for tomorrow.
  • "raven's call" is therefore a v good predictive var.
• but bribing the ravens to hush still might not hv any effect on whether it actually rains tomorrow.
  • it's therefore a v bad causal var.
• it cud even be the case that, up until now, it never not rained unless the raven's called, and intervening on the var cud still be fruitless if nobody's ever done that bfr.
• for systems u hv 100% accurate & 100% complete predictive maps of, u may still hv a terrible causal map wrt what happens if u try to intervene in ways that take the state of the system out of the distribution u'v been mapping it in.

how then do u build good causal maps?

• ig u can still improve ur causal maps wo trial-and-error (empirically testing interventions) if u just do predictive mapping of the system, and u focus in on the predictive power of the simplest vars, and do trial-and-error in ur simulations.  or smth.

u'r encouraged to write it!

You have permission to steal my work & clone my generating function. Liberate my vision from its original prison. Obsolescence is victory. I yearn to be surpassed. Don't credit me if it's more efficient or better aesthetics to not. Forget my name before letting it be dead weight.

but u don't know which distribution(s) u are acting in.  u only have access to a sample dist, so u are going to underestimate the variance unless u ~Bessel-correct^[1] ur intuitions.  and it matters which parts of the dists u tune ur sensors for: do u care more to abt sensitivity/specificity wrt the median cluster or sensitivity/specificity wrt the outliers?

ig sufficiently advanced vibes-based pattern-matching collapses to doing causal modelling, so my real-complaint is abt ppl whose vibe-sensors are under-dimensional.

1. ^{^}

   idk the right math tricks to use are, i just wanted to mk the point that sample dists underestimate the variance of the true dists

   also, oops, fixed link.  upvoted ur comment bc u complimented me for using RemNote, which shows good taste.

Fwiw, you're on my shortlist of researchers whose potential I'm most excited about. I don't expect my judgment to matter to you (or maybe up to one jot), but I mention it just in case it helps defend against the self-doubt you experience as a result of doing things differently. : )

I don't know many researchers that well, but I try to find the ones that are sufficiently unusual-in-a-specific-way to make me feel hopefwl about them. And the stuff you write here reflects exactly the unusualness what makes me hopefwl: You actually think inside your own head.

Also, wrt defending against negative social reinforcement signals, it may be sort of epistemically-irrational, but I reinterpret [people disagreeing with me] as positive evidence that I'm just far ahead of them (something I actually believe). Notice how, when a lot of people tell you you're wrong, that is evidence for both [you are wrong] and [you are so much righter than them that they are unable to recognise how you are right (eg they lack the precursor concepts)].

Also, if you expect [competence at world-saving] to be normally (or lognormally) distributed, you should expect to find large gaps between the competence of the most competent people, simply because the tail flattens out the further out you go. In other words, P(you're Δ more competent than avg) gets closer to P(you're Δ+1 more competent than avg) as you increase Δ. This is one way to justify treating [other people not paying attention to you] as evidence for [you're in a more advanced realm of conversation], but it's far from the main consideration.

I invite you to meditate on this Mathematical Diagram I made! I believe that your behaviour (wrt the dimension of consequentialist world-saving) is so far to the right of this curve, that most of your peers will think your competence is far below them, unless they patiently have multiple conversations with you. That is, most people's deference limit is far to the left your true competence.

I'm now going to further destroy the vibes of this comment by saying "poop!" If someone, in their head, notice themselves downvaluing the wisdom of what I previously wrote, merely based on the silly vibes, their cognition is out of whack and they need to see a mechanic. This seems to be a decent litmus test for whether ppl have actual sensors for evidence/gears, or whether they're just doing (advanced) vibes-based pattern-matching. :P