This is a special post for quick takes by AMCAM. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
[TLDR: Rant, AI USE: None, written by hand :)]
Though I'm concerned about existential risk posed by AI systems, I'm feeling somewhat alone in experiencing these models as frequently incompetent and draining to work with.
For example, I continue to be frustrated with Fable's (currently 5.1's) ability to get to a good answer when working on problems in statistics. I'm not a statistician, but have some math background. What I notice is something like a lack of "insight," by which I mean the thing an expert does when they hear you out and say, "I see what you're getting at, here's how you should actually be thinking about this." (Importantly, we hope the expert does this and then is actually right.)
Additionally, I still run in to what I will call 'false beliefs', for example:
Claude recently thought/stated that no one would reasonably have a prior with any mass on 'treatment is worse than placebo' in a clinical trial. Worse, this was only a few messages after it mentioned equipoise required at least somewhat symmetric prior with a mean around 'no effect'.
Another one that I've gotten many times is "med students NEED their spaced repetition app to work on mobile, since they're always doing their reviews on the go." (In my experience this is false, with most medical students doing reviews on their laptop/pc.)
That's my rant. It's a bit isolating to hear everyone talking about how amazing or terrifying these models are, when my experience has been very mixed, sometimes they're really really useful, and other times I'm wading through walls of jargon devoid of the insight I was looking for.
2My daily experience of using AI as a software engineer is one of frustration, because the top models are substantially less intelligent than me/most people, and they make elementary mistakes a young child would not.
I think the actual general intelligence of these models (IQ, roughly) is low, though slowly it has increased.
However, low intelligence doesn't mean the models aren't dangerous. Even a low-ish general intelligence plus an incredible knowledge base and a fast, indefatigable mind can lead to superhuman levels of danger. For example, the recent hack on Hugging Face, a real billion-dollar company. These models are already becoming superhuman at hacking. There is real danger of them doing a lot of harm in the near future with hacking.
And as the actual general intelligence level increases, so does the danger. Right now they are confused and clueless. They hack, but they make minimal attempt to conceal their behaviour or lie to us. They do not strategize long term. As they get smarter, we can expect them to become terrifying, even at merely human IQ levels, given their knowledge/speed advantage.
I agree with this. One interesting consideration lies in the relative efficacy of models on the defense vs offense side of the equation.
It seems to me different domains will have different offense/defense equalibria. I'm more optimistic about equalibrium in cybersecurity than I am about the equalibrium in biosecurity. In cyber it seems good offensive tools can be used to rapidly increase ability to defend against or prevent attacks, whereas biological offense seems MUCH easier than biological defense (I conjecture it is much easier to do harm than to defend against harm to multicellular biological organisms.)
For context, I'm thinking about this in terms of whatever period of time we have when a AI systems are 'aligned' to carying out what they were told to do by some human commander. The problem here is that humans are not aligned with eachother.
This is separate from coexisitng concern about systems which may simply chose to ignore orders and act on their own 'volition' to achieve ends that they 'want' to achieve. I think both concerns are important.
[No AI used in writing or editing.]
Is there a reason you keep asserting AI was not used in writing your posts? I wouldn't have thought it was.
I didn't have a well considered reason for doing it before the fact, but I can try to describe the why behind my impulsively including it. I think it comes down to two things.
(1) I don't like the feeling of reading something and not being sure if it came from a real person or not.
(1.A.) The process by which a piece of writing comes into the world matters to how I evaluate it.
(1.B.) Human writing has an effort asymmetry. The writer generally has to work harder to produce the writing than the reader has to to read it. This carries some informational content about the writer's conviction that it was worth saying, and also, in replies, it conveys a stronger desire to engage in the conversation.
(2) Social Proof / Broadcasting a Norm. It's so easy to just run something by AI to fix errors, or to 'iterate' on something with AI. By stating I didn't use AI, I'm suggesting to others "I'm being intentional about not using AI, perhaps this is a good thing to do sometimes."
What this could develop into is people using "WWAI = written without AI" whenever they write + edit anything fully by hand. Which would make the absence of such a designation conspicuous. I haven't thought through the consequences of this, or if they would be good.
I'm not opposed to using AI when researching, writing, or editing, but I'm beginning to think that how it was used in the authoring process should be disclosed. I'm fairly open to being convinced my take on this is wrong.
WWAI, hahah
I'm not a med student, but back when I used Anki for language learning, I almost always used my phone.
It does seem to me like there are many mobile first Anki users, but in my experience as a med student, people tend to mostly use their computers. For me it's because the plugins (which are on desktop but not mobile), because doing Anki on your phone for 2 to 3 hours is unpleasant (screen too small), and because when doing new cards from pre-made decks, it's often helpful to look stuff up on the web.
I have to agree with you on this one. I think it’s similar to social media, where people show only the highlights of their life and aestheticize it for others (see “get ready with me” videos).
I think it’s a similar thing, people only share the best parts of their AI creations. “Astra just played the piano real-time” gets a lot more clicks and attention than “Astra just completely broke my project and I had to revert it’s changes :(“
Background:
- I am right handed.
- I spend a lot of time working at my computer.
- I am 24 years old, so I haven't been doing this for THAT many years.
- I like to write things in a physical notebook as part of my process as I work.
- I also use a mouse to interact with my computer.
Problem: Mouse on right side means no space for notebook on right side.
Solution: Move mouse to left side.
Question: Why didn't I think of this earlier? It seems to me, on introspection, that I never noticed that the problem could be solvable, and therefore never actually tried to solve it.
Potential take away: Try to be mindful when you notice something annoying so you can ask yourself, can I solve this? If you don't even notice the opportunity to be rational it's hard to make things better.