All of Rafael Harth's Comments + Replies

Current LLMs including GPT-4 and Gemini are generative pre-trained transformers; other architectures available include recurrent neural networks and a state space model. Are you addressing primarily GPTs or also the other variants (which have only trained smaller large language models currently)? Or anything that trains based on language input and statistical prediction?

Definitely including other variants.

Another current model is Sora, a diffusion transformer. Does this 'count as' one of the models being made predictions about, and does it count as ha

... (read more)

Registering a qualitative prediction (2024/02): current LLMs (GPT-4 etc.) are not AGIs, their scaled-up versions won't be AGIs, and LLM technology in general may not even be incorporated into systems that we will eventually call AGIs.

4Dagon14d
I give a fair chance that with additional scaling (a few orders of magnitude, perhaps), and multimodal training data (especially visual and haptic), it could cross the threshold of consciousness, and be part of (or most of) what will call itself AGI (ok, really they'll just call itself "The People") after the human era ends. But I also give a lot of weight to "this is an impressive dead-end".  I don't know how to narrow my very wide error bars on these possibilities.
3Ann15d
I'm not sure if I understand this prediction; let me break it down. Current LLMs including GPT-4 and Gemini are generative pre-trained transformers; other architectures available include recurrent neural networks and a state space model. Are you addressing primarily GPTs or also the other variants (which have only trained smaller large language models currently)? Or anything that trains based on language input and statistical prediction?   Natural language modeling seems generally useful, as does size; what specifically do you not expect to be incorporated into future AI systems? Another current model is Sora, a diffusion transformer. Does this 'count as' one of the models being made predictions about, and does it count as having LLM technology incorporated? What does 'scaled up' mean? Literally just making bigger versions of the same thing and training them more, or are you including algorithmic and data curriculum improvements on the same paradigm? Scaffolding? We are going to eventually decide on something to call AGIs, and in hindsight we will judge that GPT-4 etc do not qualify. Do you expect we will be more right about this in the future than the past, or as our AI capabilities increase, do you expect that we will have increasingly high standards about this?

It's not all that arbitrary. [...]

I mean, you're not addressing my example and the larger point I made. You may be right about your own example, but I'd guess it's because you're not thinking of a high effort post. I honestly estimate that I'm in the highest percentile on how much I've been hurt by reception to my posts on this site, and in no case was the net karma negative. Similarly, I'd also guess that if you spent a month on a post that ended up at +9, this would feel a lot more hurt than if this post or a similarly short one ended up at -1, or even -20.

It's not the job of the platform to figure out what a difficult to understand post means; it's the job of the author to make sure the post is understandable (and relevant and insightful).

I don't understand what the post is trying to say (and I'm also appalled by the capslock in the title). That's more than enough reason to downvote, which I would have done if I hadn't figured that enough other people would do so, anyway.

1Seremonia1mo
yes true, not the job for "lesswrong", but job for ours to reflect "lesswrong" by being patient , less hasty
1Seremonia1mo
okay now perhaps we will discuss, i want to explain the foundation of cause and effect, namely logical consequences, which are not just the commonly known logical consequences that fall into the category of cause and effect. instead, these are logical consequences that are truly separate from cause and effect because they form the basis. That's the essential information. if it's not fully understood, it's okay, but at least you grasp that there are 'logical consequences' different from the commonly known ones.

After the conversation, I went on to think about anthropics a lot and worked out a model in great detail. It comes down to something like ASSA (absolute self-sampling assumption). It's not exactly the same and I think my justification was better, but that's the abbreviated version.

1Ape in the coat1mo
Thanks! I'll look more into that.

I exchanged a few PMs with a friend who moved my opinion from to , but it was when I hadn't yet thought about the problem much. I'd be extremely surprised if I ever change my mind now (still on ). I don't remember the arguments we made.

1Ape in the coat1mo
Is you current certanity in the correctness of thirdism based on some specific arguments that you remember? I know that there are a lot of arguments for thirdism, but I'd like to find the strongest ones.

A bad article should get negative feedback. The problem is that the resulting karma penalty may be too harsh for a new author. Perhaps there could be a way to disentangle this? For example, to limit the karma damage (to new authors only?); for example no matter how negative score you get for the article, the resulting negative karma is limited to, let's say, "3 + the number of strong downvotes". But for the purposes of hiding the article from the front page the original negative score would apply.

I don't think this would do anything to mitigate the emot... (read more)

2Viliam1mo
Ah yes, we should somehow encourage new members to try their ideas in comments rather than articles. More Open Threads perhaps?

I don't buy this argument because I think the threshold of 0 is largely arbitrary. Many years ago when LW2.0 was still young, I posted something about anthropic probabilities that I spent months (I think, I don't completely remember) of time on, and it got like +1 or -1 net karma (from where my vote put it), and I took this extremely hard. I think I avoided the site for like a year. Would I have taken it any harder if it were negative karma? I honestly don't think so. I could even imagine that it would have been less painful because I'd have preferred reje... (read more)

2MikkW18d
I don't really agree with this. Sure, some people are going to write stuff that's not very good, but that doesn't mean that we have to go overboard on negative feedback, or be stingy with positive feedback. Humans are animals which learn by reinforcement learning, and the lesson they learn when punished is often "stay away from the thing / person / group that gave the punishment", much more strongly than "don't do the thing that made that person / thing / group punish me". Wheras when they are rewarded, the lesson is "seek out the circumstances / context that let me be rewarded (and also do the thing that will make it reward me)". Nobody is born writing amazingly, they have to learn it over time, and it comes more naturally to some, less to others. I don't want bad writers (who are otherwise intelligent and intellectually engaged, which describes almost everybody who posts on LW) to learn the lesson "stay away from LW". I want them to receive encouragement (mostly in forms other than karma, e.g. encouraging comments, or inclusion in the community, etc.), leading them to be more motivated to figure out the norms of LW and the art of writing, and try again, with new learning and experience behind them. It's not all that arbitrary. Besides the fact that it's one of the simplest numbers, which makes for an easy to remember / communicate heuristic (a great reason that isn't arbitrary), I actually think it's quite defensible as a threshold. If I write a post that has a +6 starting karma, and I see it drop down to 1 or 2 (or, yeah, -1), my thought is "that kinda sucked, but whatever, I'll learn from my mistake and do better next time". But if I see it drop down to, say, -5 or -6, my thought starts to become "why am I even posting on this stupid website that's so full of anti-social jerks?". And then I have to talk myself down from deleting my account and removing LW and the associated community from my life. (Not that I think LW is actually so full of jerks. There's

What’s the “opposite” of NPD? Food for thought: If mania and depression correspond to equal-and-opposite distortions of valence signals, then what would be the opposite of NPD, i.e. what would be a condition where valence signals stay close to neutral, rarely going either very positive or very negative? I don’t know, and maybe it doesn’t have a clinical label. One thing is: I would guess that it’s associated with a “high-decoupling” (as opposed to “contextualizing”) style of thinking.[4]

I listened to this podcast recently (link to relevant timestamp) wi... (read more)

I don't really have anything to add here, except that I strongly agree with basically everything in this post, and ditto for post #3 (and the parts that I hadn't thought about before all make a lot of sense to me). I actually feel like a lot of this is just good philosophy/introspection and wouldn't have been out of place in the sequences, or any other post that's squarely aimed at improving rationality. §2.2 in particular is kinda easy to breeze past because you only spend a few words on it, but imo it's a pretty important philosophical insight.

I think there’s something about status competition that I’m still missing. [...] [F]rom a mechanistic perspective, what I wrote in §4.5.2 seems inadequate to explain status competition.

Agreed, and I think the reason is just that the thesis of this post is not correct. I also see several reasons for this other than status competition:

  • The central mechanism is equally applicable to objects (I predict generic person Y will have positive valence imagining a couch), but the conclusion doesn't hold, so the mechanism already isn't pure.

  • I just played with

... (read more)

When I read the point thing without having read this post first, my first thought was "wait, voting costs karma?" and then "hm, that's an interesting system, I'll have to reconsider what I give +9 to!"

I can see a lot of reasons why such a system would not be good, like people having different amounts of karma, and even if we adjust somehow, people care differently about their karma, and also it may just not be wise to have voting be punitive. But I'm still intrigued by the idea of voting that has a real cost, and how that would change what people do, even if such a system probably wouldn't work.

I do indeed endorse the claim that Aella, or other people who are similar in this regard, can be more accurately modelled as a man than as a woman

I think that's fair -- in fact, the test itself is evidence that the claim is literally true in some ways. I didn't mean the comment as a reductio ad absurdum, more as as "something here isn't quit right (though I'm not sure what)". Though I think you've identified what it is with the second paragraph.

If a person has a personality that's pretty much female, but a male body, then thinking of them as a woman will be a much more accurate model of them for predicting anything that doesn't hinge on external characteristics. I think the argument that society should consider such a person to be a woman for most practical purposes is locally valid, even if you reject that the premise is true in many cases.

I have to point out that if this logic applies symmetrically, it implies that Aella should be viewed as a man. (She scored .95% male on the gender-contimuu... (read more)

8Zane2mo
Fair. I do indeed endorse the claim that Aella, or other people who are similar in this regard, can be more accurately modelled as a man than as a woman - that is to say, if you're trying to predict some yet-unmeasured variable about Aella that doesn't seem to be affected by physical characteristics, you'll have better results by predicting her as you would a typical man, than as you would a typical woman. Aella probably really is more of a man than a woman, as far as minds go. But your mentioning this does make me realize that I never really had a clear meaning in mind when I said "society should consider such a person to be a woman for most practical purposes." When I try to think of ways that men and women should be treated differently, I mostly come up blank. And the ways that do come to mind are mostly about physical sex rather than gender - i.e. sports. I guess my actual position is "yeah, Aella is probably male with regard to personality, but this should not be relevant to how society treats ?her."

I can't really argue against this post insofar as it's the description of your mental state, but it certainly doesn't apply to me. I became way happier after trying to save the world, and I very much decided to try to save the world because of ethical considerations rather than because that's what I happened to find fun. (And all this is still true today.)

Yeah, I definitely did not think they're standard terms, but they're pretty expressive. I mean, you can use terms-that-you-define-in-the-post in the title.

What would you have titled this result?

With ~2 min of thought, "Uniform distributions provide asymmetrical evidence against switchy and streaky priors"

1rotatingpaguro2mo
But the point of the post is to use that as a simplified model of a more general phenomenon, that should cling to your notions connected to "gambler's fallacy". A title like yours is more technically defensible and closer to the math, but it renounces an important part. The bolder claim is actually there and intentional. It reminds me of a lot of academic papers where it's very difficult to see what all that math is there for. To be clear, I second making the title less confident. I think your suggestion exceeds in the other direction. It omits content.
1Dweomite2mo
Are "switchy" and "streaky" accepted terms-of-art?  I wasn't previously familiar with them and my attempts to Google them mostly lead back to this exact paper, which makes me think this paper probably coined them.

I think this is a valid point, although some people might be taking issue with the title. There's a question about how one should choose actions, and in this case, the utility is relative to that of other actions as you point out (a action sounds good until you see that another action has ). And then there's a philosophical question about whether or not utility corresponds to an absolute measure, which runs orthogonal to the post.

I don't think this works (or at least I don't understand how). What space are you even mapping here (I think your are samples, so to itself?) and what's the operation on those spaces, and how does that imply the kind of symmetry from the OP?

I agree with that characterization, but I think it's still warranted to make the argument because (a) OP isn't exactly clear about it, and (b) saying "maybe the title of my post isn't exactly true" near the end doesn't remove the impact of the title. I mean this isn't some kind of exotic effect; it's the most central way in which people come to believe silly things about science: someone writes about a study in a way that's maybe sort of true but misleading, and people come away believing something false. Even on LW, the number of people who read just the headline and fill in the rest is probably larger than the number of people who read the post.

2Dweomite2mo
This seems like a difficult situation because they need to refer to the particular way-of-betting that they are talking about, and the common name for that way-of-betting is "the gambler's fallacy", and so they can't avoid the implication that this way-of-betting is based on fallacious reasoning except by identifying the way-of-betting in some less-recognizable way, which trades off against other principles of good communication. I suppose they could insert the phrase "so-called".  i.e. "Bayesians commit the so-called Gambler's Fallacy".  (That still funges against the virtue of brevity, though not exorbitantly.) What would you have titled this result?

I strong downvote any post in which the title is significantly more clickbaity than warranted by the evidence in the post. Including this one.

I feel like this result should have rung significant alarm bells. Bayes theorem is not a rule someone has come up with that has empirically worked out well. It's a theorem. It just tells you a true equation by which to compute probabilities. Maybe if we include limits of probability (logical uncertainty/infinities/anthropics) there would be room for error, but the setting you have here doesn't include any of these. So Bayesians can't commit a fallacy. There is either an error in your reasoning, or you've found an inconsistency in ZFC.

So where's the mistake... (read more)

1Kevin Dorst2mo
I see the point, though I don't see why we should be too worried about the semantics here. As someone mentioned below, I think the "gambler's fallacy" is a folk term for a pattern of beliefs, and the claim is that Bayesians (with reasonable priors) exhibit the same pattern of beliefs.  Some relevant discussion in the full paper (p. 3), which I (perhaps misguidedly) cut for the sake of brevity:

I have the impression the OP is using "gambler's fallacy" as a conventional term for a strategy, while you are taking "fallacy" to mean "something's wrong". The OP does write about this contrast, e.g., in the conclusion:

Maybe the gambler’s fallacy doesn’t reveal statistical incompetence at all. After all, it’s exactly what we’d expect from a rational sensitivity to both causal uncertainty and subtle statistical cues.

So I think the adversative backbone of your comment is misdirected.

If someone is currently on board with AGI worry, flexibility is arguably less important ( Kennedy), but for people who don't seem to have strong stances so far ( Haley, DeSantis), I think it's reasonable to argue that general sanity is more important than the noises they've made on the topic so far. (Afaik Biden hasn't said much about the topic before the executive order.) Then again, you could also argue that DeSantis' comment does qualify as a reasonably strong stance.

3tlevin2mo
Just being "on board with AGI worry" is so far from sufficient to taking useful actions to reduce the risk that I think epistemics and judgment is more important, especially since we're likely to get lots of evidence (one way or another) about the timelines and risks posed by AI during the term of the next president.

To be fair, @Zack_D hasn't written any posts longer than 2000 words!

Is it because Kat and Emerson are "public figures" in some sense?

Well, yeah. The whole point of Ben's post was presumably to protect the health of the alignment ecosystem. The integrity/ethical conduct/{other positive adjectives} of AI safety orgs is a public good, and arguably a super important one that justifies hurting individual people. I've always viewed the situation as, having to hurt Kat and Emerson is a tragedy, but it is (or at least can be; obviously it's not if the charges have no merit) justified because of what's at stake. If they weren't working in this space, I don't think Ben's post would be okay.

0frontier642mo
Why not protect the EAs from a bpd liar who accuses everybody she comes into contact with of mistreatment and abuse?

You don't have to justify your updates to me (and also, I agree that the comment I wrote was too combative, and I'm sorry), but I want to respond to this because the context of this reply implies that I'm against against weird ideas. I vehemently dispute this. My main point was that it's possible to argue for censorship for genuine reasons (rather than become one is closed-minded). I didn't advocate for censoring anything, and I don't think I'm in the habit of downvoting things because they're weird, at all.

This may sound unbelievable or seem like a warped... (read more)

2MondSemmel3mo
I stumbled over this part. What makes someone high prestige? Their total LW karma? To me that doesn't really make sense as a proxy for prestige.

The thing that seems to me to have gotten worse is what gets upvoted. AI content is the big one here; it's far too easy to get a high karma post about an AI related topic even if the post really isn't very good, which I think has a ton of bad downstream consequences. Unfortunately, I think this extends even to the Alignment Forum.

I have no idea what to do about it though. Disagreement voting is good. Weighted voting is probably good (although you'd have to see who voted for what to really know). And the thing where mods don't let every post through is also good. I wish people would vote differently, but I don't have a solution.

Yes. There's a stigma against criticizing people for their faith (and for good reason), but at the end of the day, it's a totally legitimate move to update on someone's rationality based on what they believe. Just don't mention it in most contexts.

1Slapstick3mo
I think I agree with this, but I also think it's really important to avoid making too many assumptions about what people believe when they say they're religious or practice religion . People often use similar language and labels to signify a very broad range of beliefs and views.

Yeah, I think the problem is just very difficult, especially since the two moves aren't even that different in strength. I'd try a longer but less complex debate (i.e., less depth), but even that probably wouldn't be enough (and you'd need people to read more).

The reason my tone was much more aggressive than normal is that I knew I'd be too conflict averse to respond to this post unless I do it immediately, while still feeling annoyed. (You've posted similar things before and so far I've never responded.) But I stand by all the points I made.

The main difference between this post and Graham's post is that Graham just points out one phenomenon, namely that people with conventional beliefs tend to have less of an issue stating their true opinion. That seems straight-forwardly true. In fact, I have several opinions ... (read more)

I thought I would open this up to the masses, so I have two challenges for you. I estimate that this is suitable for chess players rated <1900 lichess, <1700 chess.com or <1500 FIDE.

(Fixed debate, spent about 10 minutes.) I might have a unique difficulty here, but I'm 1900 on chess.com and am finding this quite difficult even though I did move some pieces. Though I didn't replay the complicated line they're arguing about since there's no way I could visualize that in my head with more time.

I would play Qxb5 because white gets doubled pawns, bla... (read more)

1Richard Willis3mo
Thank you for the informative response. I probably should have looked for a less complex position. Also it sounds like I need to work on my salesman pitch! Will reveal the better move in good time.
1nem3mo
I had the same problem. You are almost certainly a stronger player than me, but after a couple moves, I can't really analyze the lines suggested. Hell, I missed qC5 as a candidate move. What am I missing on move 3?

I hadn't, but did now. I don't disagree with anything in it.

4lsusr3mo
Fascinating. You're one of the names on Less Wrong that I associate with positive, constructive dialogue. We may have a scissor statement here.

Is OpenAI considered part of EA or an "EA approach"? My answer to this would be no. There's been some debate on whether OpenAI is net positive or net negative overall, but that's a much lower bar than being a maximally effective intervention. I've never seen any EA advocate donating to OpenAI.

I know it was started by Musk with the attempt to do good, but even that wasn't really EA-motivated, at least not as far as I know.

3Erich_Grunewald3mo
Open Philanthropy did donate $30M to OpenAI in 2017, and got in return the board seat that Helen Toner occupied until very recently. However, that was when OpenAI was a non-profit, and was done in order to gain some amount of oversight and control over OpenAI. I very much doubt any EA has donated to OpenAI unconditionally, or at all since then.

I think the central argument of this post is grossly wrong. Sure, you can find some people who want to censor based on which opinions feel too controversial for their taste. But pretending as if that's the sole motivation is a quintessential strawman. It's assuming the dumbest possible reason for why other person has a certain position. It's like if you criticize the bible, and I assume it's only because you believe the Quran is the literal word of god instead.

We do not censor other people more conventional-minded than ourselves. We only censor other peo

... (read more)
4lsusr3mo
Did you read the Paul Graham article I linked? Do you disagree with it too?
9StartAtTheEnd3mo
Isn't that basically always what's claimed, yet rarely the case? It's likely either because people cannot tell the difference between their dislikes and what's harmful to society, or because the correct answer is unintuitive. In either case, as long as a topic is taboo, one is banned from figuring out what the real answer is. It wasn't intuitive that legalizing porn was the way to go if you wanted a society with less sexual crimes. It wasn't intuitive that legalizing alcohol was the way to go. It wasn't intuitive that legalizing drugs somehow reduced drug-related problems, it wasn't intuitive in the past that making mental health issues taboo wasn't a good solution, etc. "X is bad, we should ban it so that it goes away" is a naive way of thinking. An extremely open-minded person with low intelligence might arrive at such a conclusion if he believes it to be correct, though, but he won't have all those negative emotions which are associated with pro-censorship viewpoints, and these mentalities are more of a problem than the actual censorship. I believe there's a murky border between "speech" and "action" which is not obvious. I'm for free speech in an absolute sense, but if somebody yelled into my ear and caused hearing damage, I wouldn't consider that "speech"  but "assault". It's not enough to name concrete examples like "slander", "threatening" , and "yelling fire in a theatre", there's bound to be a simple explanation which separates speech and malicious actions clearly. I believe that such a clear definition will reveal censorship to be objectively nonoptimal

Gonna share mine because that was pretty funny. I thought I played optimally missing a win whoops, but GPT-4 won anyway, without making illegal moves. Sort of.

Agreed. My impression has been for a while that there's a super weak correlation (if any) between whether an idea goes into the right direction and how well it's received. Since there's rarely empirical data, one would hope for an indirect correlation where correctness correlates with argument quality, and argument quality correlates with reception, but second one is almost non-existent in academia.

Thanks! Sooner or later I would have searched until finding it, now you've saved me the time.

Well I don't remember anything in detail, but I don't believe so; I don't think you'd want to have a restriction on the training data.

I fully agree with your first paragraph, but I'm confused by the second. Where am I making an argument for camp #1?

I'm definitely a Camp 2 person, though I have several Camp 1 beliefs. Consciousness pretty obviously has to be physical, and it seems likely that it's evolved. I'm in a perpetual state of aporia trying to reconcile this with Camp 2 intuitions.

I wouldn't call those Camp #1 beliefs. It's true that virtually all of Camp #1 would agree with this, but plenty of Camp #2 does as well. Like, you can accept physicalism and be Camp #2, deny physicalism and be Camp #2, or accept physicalism and be Camp #1 -- those are basically the three options, and you seem to b... (read more)

That's a relative rather than absolute claim. The article has pushback from camp 2

Yeah -- I didn't mean to imply that orthormal was or wasn't successful in dissolving the thought experiment, only that his case (plus that of some of the commenters who agreed with him) is stronger than what Dennett provides in the book.

I did remember reading, Why it's so hard to talk about Consciousness, and shrinking back from the conflict that you wrote as an example of how the two camps usually interact.

Thanks for saying that. Yeah hmm I could have definitely opened the post in a more professional/descriptive/less jokey way.

Since we seem to be unaware of the different sets of skills a human might possess, how they can be used, and how different they are 'processed', it kind of seems like Camp 1 and Camp 2 are fighting over a Typical Mind Fallacy - that one's experience is general

... (read more)
1Caerulea-Lawrence5mo
Thanks for the answer, Still, I know I read somewhere about intuitives using a lot of their energy on reflection, and so I gathered that that kind of dual-action might explain part of the reason why someone would talk about 'qualia' as something you are 'aware' of as it happens. I mean, if most of one's energy is focused on a process not directly visible/tangible to one's 'sense perception', I don't see why people wouldn't feel that there was something separate from their sense perception alone. Whereas with sensors, and it being flipped, it seems more reasonable to assume that since the direct sense perceptions are heavily focused on, the 'invisible/intangible' process gets more easily disregarded.  The thing is, there have been instances where I didn't 'feel' my usual way around qualia. In situations that were very physical demanding, repetitive or very painful/dangerous, I was definitely 'awake', but I just 'was'. These situations are pretty far between, so it isn't something that I'm that familiar with - but I'm pretty certain that if I was in that space most of the time, and probably learned how to maneuver things there better - I would no longer talk about being in Camp 2, but in Camp 1. I would be very surprised if I could see the color Phi phenomenon, as I just think that I would have noticed it already. But, as with many such things, maybe it simply is hidden in plain sight?

I think the philosophical component of the camps is binary, so intermediate views aren't possible. On the empirical side, the problem that it's not clear what evidence for one side over the other looks like. You kind of need to solve this first to figure out where on the spectrum a physical theory falls.

2TAG5mo
The camps as you have defined them differ on what the explanation of consciousness is, and also on what the explanandum is. The latter is much more of a binary than the former. There are maybe 11 putative explanations of mind-body relationship, ranging from eliminativism to idealism, with maybe 5 versions of dualism in the middle. But there is a fairly clear distinction between the people who think consciousness is exemplified but what they, as a subject are/have; and the people who think consciousness is a set of capacities and functions exemplified by other entities. Looking at it that way, it's difficult to see what your argument for camp 1 is. You don't seem to believe you personally are an experienceless zombie, and you also don't seem to think that camp 2 are making a semantic error in defining consciousness subjectively. And you can't argue that camp 1 have the right definition of consciousness because they have the right ontology , since a) they don't have a single ontology b) the right ontology depends on the right explanation depends on the right definition.

I think this lacks justification why the entire approach is a good idea. Improving mathematical accuracy in LLMs seems like a net negative to me for the same reason that generic capability improvements are a net negative.

No. It would make a difference but it wouldn't solve the problem. The clearest reason is that it doesn't help with Inner Alignment at all.

Having not read the book yet, I'm kind of stumped at how different this review is to the one from Alexander. The two posts make it sound like a completely different book, especially with respect to the philosophical questions, and especially especially with respect to the expressed confidence. Is this book a neutral review of the leading theories that explicitly avoids taking sides, or is it a pitch for another I-solved-the-entire-problem theory? It can't really be both.

1Ilio6mo
Yes it can. Like any bistable perception can be both one percept for someone and a very different percept for someone else. That means they don’t have the same kernels.

Downvote because I feel like this completely fails to engage with what rationality-as-used-in-this-community-and-in-the-sequences actually is. Especially since Eliezer devoted so much of his writing toward recognizing when you're wrong and changing your mind.

1Iknownothing6mo
Yeah, I think you're right- at least about the sequences.  I think something more specific about attitudes would be more accurate and useful.

Does anyone know about an update to GPT-4 that was deployed to the https://chat.openai.com/ version in the last 1-3 weeks? It seems to have gotten significantly better.

2Ann6mo
Not sure about the model but they might've fixed something important around the 9th-10th, haven't gotten any notices of issues since then and it stopped crashing on long code blocks (which was a brief issue after the update on the 3rd).

Fair point, but there's still a strong correlation between established consensus and expert consensus. In most cases, they're probably gonna be similar.

2ChristianKl7mo
I wrote the previous comment mainly on the experience of a few months ago. I just tested it by asking about Post-Treatment Lyme Disease Syndrome and GPT4 seems to be more open now. Part of the response was: This seems to be a good summary that does not ignore the views of patient advocates when they disagree with the orthodox CDC position. 

One particular application of GPT-4 (or other LLMs) that seems really valuable to me is as a fact check on claims about what other people think (e.g., "what's the scientific consensus on whether exercise lowers the risk of dementia?") As long as the topic isn't about political correctness, I pretty much trust GPT-4 to represent the consensus fairly, and that's a pretty amazing tool to have. Like, it's not uncommon that people disagree about what the scientific consensus is, and we didn't really have a way to answer these questions before.

Sometimes I even f... (read more)

4ChristianKl7mo
What GPT-4 does is to give you the establishment consensus. I would expect that there are many scientific questions where there are multiple scientific paradigms that touch the question. If one of the paradigms is more high status then I think there's a good chance that GPT-4 will give you the establishment position. If you for example ask GPT-4 some questions about hypnosis, I think you often get something that's more of an establishment position back than what you would get if you would read the Oxford Handbook of Hypnosis that describes what's scientific consensus among the scientists who study hypnosis.
Load More