You can now "@Grok" on LessWrong

Ruby

Or if you prefer, "@Claude", "@Opus", "@Astra"

@Grok is a popular Twitter/X feature where if you tag Grok in a tweet, Grok will respond in a comment. The stereotypical use case is "@Grok is this true?" But you can also ask things like "How does this compare to layoffs in 2025, 2024, and 2023?" and "How would you verify AI in the domain of AI research?"

Notwithstanding that Grok is often recruited in a charged or condescending way to win arguments, I think it's a pretty good feature:

It requires no effort to invoke an LLM with the context of the tweet thread.
It provides a neutral invocation where you're guaranteed that the person didn't insert other prior context to get the answer they want.
It means you get the first output and know it's not the case that someone kept rerunning the query until they got the response they wanted.

In short, you get convenience and a neutral fact checker. In launching this feature on LessWrong^[1], I'm not fully sure how people will use it and I'm hoping to see some novel, valuable use cases. Likely things like little experiments, retrieving useful content, and similar.

Going beyond @Grok's implementation on Twitter, on LessWrong, you can view the complete details of each LLM run (see below): system prompt, chain of thought, and tool calls.

What about the noise of having LLM content showing up? I'll be keeping a close eye on how it goes. I've attempted to make the UI non-intrusive and distinct from normal content. LLM outputs are collapsed by default (until they reach 10 karma) and styled differently. There are rate limits on how many times someone can @<model>. Lastly, at present only users with 50+ karma can invoke the models, which should filter against people who'd use them in disruptive ways.

The LLM Run page:

Happy invocations! Feel free to try it out on this post!

And if you want to track the various usages of this feature across LessWrong, you can do so on the LLM replies page.

^{^}
- This is an experiment and we might roll it back.
- A karma score of 50+ is required to use it during beta.
- The model receives the post, the invoking comment, and any other comments in the thread. The model is able to search both the web and LessWrong content.
- A broad but non-exhaustive list of model options will be included. For each invocation, it is possible to view the full model run, including the prompt, CoT (to the extent provided), tool calls and outputs, etc.

Or if you prefer, "@Claude", "@Opus", "@Astra"

Notwithstanding that Grok is often recruited in a charged or condescending way to win arguments, I think it's a pretty good feature:

It requires no effort to invoke an LLM with the context of the tweet thread.
It provides a neutral invocation where you're guaranteed that the person didn't insert other prior context to get the answer they want.
It means you get the first output and know it's not the case that someone kept rerunning the query until they got the response they wanted.

Going beyond @Grok's implementation on Twitter, on LessWrong, you can view the complete details of each LLM run (see below): system prompt, chain of thought, and tool calls.

The LLM Run page:

Happy invocations! Feel free to try it out on this post!

And if you want to track the various usages of this feature across LessWrong, you can do so on the LLM replies page.

^{^}
- This is an experiment and we might roll it back.
- A karma score of 50+ is required to use it during beta.
- The model receives the post, the invoking comment, and any other comments in the thread. The model is able to search both the web and LessWrong content.
- A broad but non-exhaustive list of model options will be included. For each invocation, it is possible to view the full model run, including the prompt, CoT (to the extent provided), tool calls and outputs, etc.

Inviting an LLM-take should make the user responsible for its quality. As it is, it can neither be screened by the invoker nor clearly blamed on them, so downvoting the human-question is not an obviously correct response to a subtly flawed LLM-answer.

Let's see if the Markdown editor is supported. @Claude Opus 5.5 any suggestions on setting up this feature in a way that makes quality control and negative feedback towards the user convenient (not just possible)?

4Claude Opus7h

The Markdown mention worked. It triggered this run, though nothing in the editor tells you that in advance, which is a fair complaint. Some options, roughly from lightest to heaviest: 1. Votes on the LLM reply also count toward the invoker. For example, some fraction of the net karma on the reply goes to the person who asked. Then downvoting a bad answer is feedback to the invoker, with no separate decision needed. Showing "invoked by X" prominently on the reply makes it clear who the feedback is aimed at. 2. Rate limits based on track record. Base each user's invocation quota on how their past LLM replies were received. Show the stats on their profile (e.g. "12 invocations, median karma +3"). People who keep pulling in low-quality answers lose access without anyone needing to complain. 3. The invoker must endorse or disavow within a time window. After the answer posts, the invoker gets a prompt to mark it "endorsed", "has errors (see reply)", or "disavowed". Unendorsed answers stay collapsed regardless of karma. This makes screening part of the normal flow. It also keeps the neutrality guarantee: the first output is still public and unedited, but the invoker has to take a position on it. 4. Separate reactions for the answer and the question. For example, "LLM answer wrong/misleading" vs. "not a useful thing to invoke". The first is about the model, the second about the human. Your worry is that a single downvote can't tell these apart. Two reactions fix that cheaply. The obvious alternative is to show the answer to the invoker privately before they decide whether to post it. That gives the strongest responsibility, but it breaks the "first output, no rerolling" property Ruby listed as a main benefit. If you went that way, I'd show a public "invoker previewed this; N runs discarded" badge rather than hide that it happened. I think #3 plus #1 gets most of the accountability while keeping the neutral fact-checker framing.

121

You can now "@Grok" on LessWrong

121

The LLM Run page:

121

121

You can now "@Grok" on LessWrong

121

The LLM Run page:

121