LESSWRONG
LW

1708
soycarts
-148440
Message
Dialogue
Subscribe

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by
Newest
No wikitag contributions to display.
1soycarts's Shortform
2mo
7
Unionists vs. Separatists
soycarts16d*10

It's a good concern, and I address this in my third-order cognition manifesto.[1]

To handle this I think we would consider two traits:[2]

  1. Agency permeability:[3] every tier of cognition can both influence and be influenced by the global action policy[4] with low friction or external arbitration (e.g driving your car to work — switching between intentional decisions and muscle memory)
  2. Homeostatic unity: all subsystems participate in the same self-maintenance goal (e.g biological survival and personal welfare)

For the unified entity (i.e human & superintelligence (SI)) to be well aligned we would ensure that they are unified such that these traits are protected.

I propose an example implementation for unification:

  1. Agency permeability: Bidirectional Brain-Computer Interface (BCI)
  2. Homeostatic unity: Integrative systems set up such that the human physiological loop feeds the SI reward model

The other 3 metaphysical traits for this implementation:

  1. Normative closure:[5] Superintelligence trained on a human's historical choices, joint deliberation loop architected
  2. Persistence conditions:[6] Cryptographic identity lock tied to a user's biometric pattern
  3. Boundary conditions:[7] Human plus any compute that has passed cryptographic requirements
  1. ^

    I initially referred to this as a thesis, but I think it's a bit more pointed (and meandering at times).

  2. ^

    The manifesto contains 8 traits, to be focused on for different scenarios.

  3. ^

    I used a few back-and-forths with an LLM, plus gathered feedback, to find a criteria and definition that could work here.

    Initially this was "unity of agency: one locus of practical authority", but this seems to 1) repeat ideas from normative closure, and 2) fail to capture how agency almost free-associates between layers of cognition in a human. We have times when we are “active and intentional” in our thoughts (i.e locus of control towards second-order) vs. operating on muscle memory (e.g, when driving a car) where the locus of control is closer to first-order.

  4. ^

    "I first need to determine a point of observability for the metaphysical conditions — the current moment. I can then define the global action policy for a being as its "master mapping from its current informational state (internal and external) to enacted behaviour, after all internal competitions and vetoes have been resolved". The action may be something like eating a marshmellow."

  5. ^

    "Normative closure: a single normative standpoint (e.g stating what you want to do)"

  6. ^

    "Persistence conditions: criteria for maintaining identity as the same being over change (e.g. losing a limb)"

  7. ^

    "Boundary conditions: clearly defined membership rules on what binds the being (e.g having skin and bones)"

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts17d10

To simplify things, even though VN is a prerequisite in my view we can even drop it (due to it holding little weight), so we're approximately evaluating:

  • P ( F=1 | R = 1, CFP = 1) = P ( D=1 | R = 1, CFP = 1) = x

Edit:

And implicit in this rebalancing (maybe) — adding this edit to clarify — is that I don't agree with this:

Now let us examine P(CFP_r | C_r). Looking over the site, almost all comments are in some way reactionary to the thing they are commenting on, and all but a tiny minority are >= 30 characters. So P(CFP_r | C_r) > 0.8 is likely in the background and not just under condition F_r.

I absolutely did not feel that CFP was just background noise. I gave significant weight to the fact that the first line (R) explicitly requested comments of the form of CFP.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts17d1-2

Sorry, I think you're putting far too much weight on something that is not my position.

Your post made the claim that it was substantially likely that most downvotes had no corresponding comment.

My closing line, verbatim:

I’m anticipating this post to be a straight shot to meta-irony: I have confidently made a non-normative claim, so expect a couple of negative post votes, absent of material feedback.

If I thought this was true "it [is] substantially likely that most downvotes had no corresponding comment.", that would look like me closing with:

I think I will get 10 negative post votes and no material feedback

I'm describing, explicitly in my post, a different phenomenon that contains more nuance: specifically low signal early votes that suppress visibility.

I say:

Sometimes I spend hours putting together a post that I’m proud of, but then receive no feedback besides a couple of downvotes.

and

a down-voting agent can effectively silence my voice just because they disagree with me.

and in my discussion with @Drake Morrison :

I agree [people don't owe me their time], but I feel that there is a distinct imbalance where a post can take hours of effort, and be cast aside with a 10-second vibe check and 1 second "downvote click". I believe that the platform experience for both post authors and readers could be significantly improved by adding a second post-level signal that only takes an additional few seconds — this could be a React like "Difficult to Parse" or a ~30-character tip like "Same ideas posted recently: [link]".

None of this looks like the claim: "it was substantially likely that most downvotes had no corresponding comment."

If you want to adjust your calculation, you would need to account for my true position which is that VN (recall: my post having a couple of negative votes) is a prerequisite for a comment that is made to be from a Downvoter.

However, obviously it's also a small sample size. That means it's high variability, and we shouldn't put much weight on it.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d30

Thank you for your densely rich reactions to my comment — I see validity in most of them.

Did you actually evaluate the probability?

Yes, evaluating the probability in my mind looked like:

  • Events:
    • R: I posted the explicit Request as the first line of the post: "Comment Guideline: If you downvote this post, please also add a Reaction or a 30+ character note"
    • CFP: A Comment was observed that Fulfilled Part of my requested format — it gave a reactionary note, and was on the order of characters I requested (~100 vs. my request of >=30 )
    • VN: The Vote tally was Negative on the order of a couple of votes
    • F: A comment was observed that Fulfilled all of my requested format (i.e CFP + it is a Downvoter providing a note) — this is what I evaluated to justify looking for a "Downvote-worthy" implication from their comment.
    • D: The commenter downvoted
  • The conditional probability I intuited:
    • P ( F=1 | R = 1, CFP = 1, VN = 1) = P ( D=1 | R = 1, CFP = 1, VN = 1) = x
  • My evaluation: I felt that x would be high, i.e >= 0.75 because of the intersection of R, CFP and VN

I admit that I "jumped the gun" by acting on this Bayesian instead of first asking for clarification, and then responding. I had a few motivations for this:

  1. Receiving a comment satisfying my Comment Guideline (event F above) was necessary for me to be able to operationalise my idea: that is to say that I wanted to be able to agree or disagree and state that because of this I was going to "hold them accountable" and vote on their comment positively or negatively.
  2. I was in the mood to write and express myself, and didn't want to wait — indeed the first-commenter in question only provided clarification 16 hours later. In the meantime, my karma was obliterated.
  3. Given (1), (2) and my high evaluated probability of F, I figured it was worth it on balance to push forwards.

I also see now that giving attention to spelling out why I was assuming that they were "implying that my post should be downvoted" — with the formalised Bayesian above — could have facilitated better rationalist discourse. My excuse would be that my attention was pulled in a few different directions, and I prioritised simply showcasing the operationalised version of my recommendation/ feature request.

On your disagreement to "Nobody has engaged with these points" I think I agree with you and could be more precise with my statement. I think nobody had, at least when I wrote that, engaged directly along the lines of reasoning of one of those points. However, engagement like "You may be interested in a very similar discussion" or "Sometimes, a post or comment seems so far from epistemic virtue as to be not worth spending effort describing all the problems. I mutter “not even wrong”, downvote, and move on." does engage with those points at a meta-level, in terms of providing constructive feedback for the post.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d10

I wholeheartedly agree with you.

There is something else going on here though. As I commented on this post, which also (in my view) fell prey to the phenomenon I am describing:

It’s complex enough for me to make the associations I’ve made and distill them into a narrative that makes sense to me. I can’t one-shot a narrative that lands broadly… but until I discover something that I’m comfortable falsifies my hypothesis, I’m going to keep trying different narratives to gather more feedback: with the goal of either falsifying my hypothesis or broadly convincing others that it is in fact viable.

To follow your analogy: I'm not asking that people purchase my sandwiches. I'm just asking that people clarify if they need them heated up and sliced in half, and don't just tell everyone else in the market that my sandwiches suck.

This directly aligns with a plea I express in the current post:

  1. The strongest counterargument is that I should just write my post like an automaton,[1] I add a footnote clarifying, instead of infusing the [attempts at] humour and illustrative devices that come naturally to me.
  2. The problem with this is that writing like that isn't fun to me and doesn't come as naturally. In essence it would be a barrier to me contributing anything at all. I view that as a shame, because I do believe that all of my logic is robustly defensible and wholly laid out within the post.

I believe that there is value in my ideas, and I'm not that far off repositioning them in a way that will land more broadly. I just need light, constructive feedback to more closely align our maps.

However in absence of this light, constructive feedback on LessWrong, I'm quite forcefully cast aside and constrained to other avenues. Epistemic status: I attend a weekly rationality meetup in Los Angeles, I attend AI Safety and AI Alignment Research meet-ups in Los Angeles and San Francisco, and I work directly on and with frontier AI solutions.

  1. ^

    This is [attempted] use of hyperbole for humour — instead of "like an automaton", precisely I mean that a common writing style on LW is to provide a numbered/bulleted list of principles.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d00

Thank you for the feedback!

On the path to benevolence there's a whole lot of friction. I agree with most of what you said, and I think we can extract substantive value that builds on my post:

Demanding people spend more of their own effort and time to engage with you is a losing proposition. You have to make it worth their time.

I agree, but I feel that there is a distinct imbalance where a post can take hours of effort, and be cast aside with a 10-second vibe check and 1 second "downvote click". I believe that the platform experience for both post authors and readers could be significantly improved by adding a second post-level signal that only takes an additional few seconds — this could be a React like "Difficult to Parse" or a ~30-character tip like "Same ideas posted recently: [link]".

Given the existing author/reader time-investment imbalance, it feels fair to suggest adding this.

Take some time to explain your recommendation, and why I should care first, then I know what you're talking about in this section. 

This is a valid call-out — in fairness it was an imprecision on my part because I added the "Operationalising my recommendation" section in the first (2025/09/13) edit, and overlooked the fact that this meant it preceded my stating the recommendation. I've updated the post to state the recommendation upfront. [Meta note: This to me is the beauty of rationality and the LessWrong platform — we can co-create great logical works. I hope this doesn't look too much like "relying on the reader to proof-read" in lieu of https://www.lesswrong.com/posts/nsCwdYJEpmW5Hw5Xm/lesswrong-is-providing-feedback-and-proofreading-on-drafts ]

There are similar sorts of problems all over the piece with assumptions that aren't justified, jumping around tonally between sections, and mixing up explaining the problem with your preferred solution.

This connects to the uncertainty I relayed to @Richard_Kennaway:

  1. Uncertainty: I'm uncertain about whether my writing style is suitable for this platform, or if I should defer to in-person interactions and maybe video content to express my ideas instead.
  2. The strongest counterargument [to my post] is that I should just write my post like an automaton,[this use of hyperbole for humour — instead of "like an automaton", precisely I mean that a common writing style on LW is to provide a numbered/bulleted list of principles] instead of infusing the [attempts at] humour and illustrative devices that come naturally to me.

I really enjoy Scott Alexander's writing and, while clearly he's a far more distinguished and capable writer than me, I feel he is a good role-model as someone who uses rationality but also storytelling prose to try to relay their point. That's effectively what I hope to accomplish — but I could only really get there if I get feedback on my writing.

I have three instances of posts in this style where I do successfully have some degree of positive feedback: [1], [2], [3]

At the same time, this is a red flag to me:

It's just not a well-written piece, or so I judged it.

In [rare] cases where I do successfully compel someone to fully read and engage with my post, I have a huge duty to the outcome that they enjoy and find insightful value in my writing.

The last thing I want to be doing is to be actually wasting someone's time.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d10

Got it — apologies for bending these rules (I didn't consider that this may break a rule) as an attempt to operationalise the post.

Can I state a lighter version instead, where I encourage all standard voting behaviour, but append a request for downvote justification? I've replaced the Comment Guideline accordingly as a placeholder until I receive further clarification.

I.e:

*** Voting Guideline: You should freely vote and react according to your views and LessWrong norms — I do not want to infringe upon this. ***

*** Comment Request: However, please allow me to make a request: if you do downvote this post and are willing to make that transparent, it would help me to operationalise the recommendation I put across in this post if you add a Reaction or a 30+ character comment prepended with "Downvote note:" on what to improve. ***

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d10

Haha well I alienated a lot of people by inferring that and using it to operationalise my recommendation, but I appreciate you clarifying this and acknowledging that the conclusion that I reached was reasonable.

Per the terms that I explicitly communicated, I've flipped my votes to approve and agree with your comment.

Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d-10

Thank you for providing your evaluative criteria.

To me you hit on a precise, valid downvote signal: "this post is effortful for me to falsify". That would be helpful to writers like me to receive as a precise, labelled signal in order to optimise.

The disconnect I guess is that to me, all of my logic is robustly defensible and wholly laid out within the post. That's precisely why I'm so keen on someone, anyone, being able to precisely state any logical inconsistency or area that lacks clarity.

If they were to do so, then I could expand on the area where our world-views/maps [https://www.lesswrong.com/w/map-and-territory] are too distinct, in pursuit of correcting one of our maps. To me this is the essence of rationality, and it's jarring that I'm not able to get it on this platform.

Since I have to defer to an LLM for this: ChatGPT5 Pro gives me the following checklist [points 1 through 5, all other language is my own] to avoid being "not even wrong", I've added my view on how strongly I'm doing against each item:

  1. State one main claim in plain language.
    1. Strongly achieved: "we should devise mechanisms that provide an actionable route for low-quality contrarian posts/comments to become high-quality to improve the platform as a whole."
  2. Define key terms (what exactly do you mean by X?).
    1. Moderately achieved — I could have formatted this differently, instead of containing it within the prose:
    2. Contrarian — surely is a standard term, I illustrated it as "Being Peculiar" or exhibiting "visionary arrogance"
    3. "High quality content" — stated as "[content that is] usually heavily upvoted [on the platform]"
    4. "devise mechanisms" — stated both as "implementing a system where negative votes on a post require the voter to cite their reason for negative voting — using either the Reaction system or a brief note of 30+ characters." and self-referentially on this post as I describe the "comment following my guideline" → "me responding"  → "readers casting their vote" dynamic
    5. "the platform as a whole" — described LessWrong, and its central mission
  3. Show your reasoning chain: premises → inference → conclusion.
    1. Strongly achieved:
    2. Premises: Spend hours putting together a post that contains a contrarian view that I’m proud of, but then receive no feedback besides a couple of downvotes.
    3. Inference: Contrarian view is too loosely dismissed on the platform "The only way you can be a visionary is to express a bunch of things that are, by definition, not the societal norm... [but] people with societally normative viewpoints will disagree with you."
    4. Conclusion: "I’m anticipating this post to be a straight shot to meta-irony: I have confidently made a non-normative claim, so expect a couple of negative post votes, absent of material feedback." — this has extended to -16 karma across 11 votes, and still nobody has engaged to offer a logical inconsistency.
  4. Cite evidence and say what would change your mind.
    1. Strongly achieved:
    2. Using my great wit,[this is use of hyperbole for humour] I self-referentially operationalised the post to illustrate my point. What would change my mind is if people actually upvoted me.
  5. Quantify uncertainty (even roughly) and address the strongest counterargument.
    1. Weakly achieved — I guess I was leaving this open for audience participation.
    2. Uncertainty: I'm uncertain about whether my writing style is suitable for this platform, or if I should defer to in-person interactions and maybe video content to express my ideas instead.
    3. The strongest counterargument is that I should just write my post like an automaton,[this is also use of hyperbole for humour — instead of "like an automaton", precisely I mean that a common writing style on LW is to provide a numbered/bulleted list of principles] instead of infusing the humour and illustrative devices that come naturally to me.
    4. The problem with this is that writing like that isn't fun to me and doesn't come as naturally. In essence it would be a barrier to me contributing anything at all. I view that as a shame, because I do believe my post wholly consists of robust logic and satisfies this "avoid being 'not even wrong'" checklist. If I had provided this checklist at the top of my post, would it have made my post easier to parse and thus well-received by the community? Or am I still missing something?
Reply
Visionary arrogance and a criticism of LessWrong voting
soycarts18d-1-2

It is plausible, but as rationalists we deploy Bayesians — you should know that as The Dao of Bayes.

I've made 4 relatively small edits to the post (called out in the first line) which, together, I think significantly strengthen my argument. Akin to "activating my trap card", lol.

I would love to hear from you whether you agree that this strengthens the argument that I make in the post. Additionally, thank you for continuing to engage with me, and I hope that I can convince you that I am trying to facilitate rationalist thinking in good faith.

Reply
Load More
-18Visionary arrogance and a criticism of LessWrong voting
19d
24
-10Unionists vs. Separatists
19d
2
-11One-line hypothesis: An optimistic future for AI alignment as a result of identity coupling and homeostatic unity with humans (The Unity Hypothesis)
20d
3
7Spiders and Moral Good
21d
0
4Small Steps vs. Big Steps
26d
5
10Paper Review: TRImodal Brain Encoder for whole-brain fMRI response prediction (TRIBE)
2mo
0
1soycarts's Shortform
2mo
7
2Third-order cognition as a model of superintelligence (ironically: Meta® metacognition)
2mo
5