If the thesis in Unlocking the Emotional Brain is even half-right, it may be one of the most important books that I have read. It claims to offer a neuroscience-grounded, comprehensive model of how effective therapy works. In so doing, it also happens to formulate its theory in terms of belief updating, helping explain how the brain models the world and what kinds of techniques allow us to actually change our minds.
Latest Posts
Popular Comments
Recent Discussion
Produced while being an affiliate at PIBBSS[1]. The work was done initially with funding from a Lightspeed Grant, and then continued while at PIBBSS. Work done in collaboration with @Paul Riechers, @Lucas Teixeira, @Alexander Gietelink Oldenziel, and Sarah Marzen. Paul was a MATS scholar during some portion of this work. Thanks to Paul, Lucas, Alexander, Sarah, and @Guillaume Corlouer for suggestions on this writeup.
Introduction
What computational structure are we building into LLMs when we train them on next-token prediction? In this post we present evidence that this structure is given by the meta-dynamics of belief updating over hidden states of the data-generating process. We'll explain exactly what this means in the post. We are excited by these results because
- We have a formalism that relates training data to internal
Given that the model eventually outputs the next token, shouldn't the final embedding matrix be exactly your linear fit matrix multiplied by the probability of each state to output a given token? Could you use that?
Can anyone here recommend particular tools to practice grammar? Or with strong opinions on the best workflow/tool to correct grammar on the fly? I already know Grammarly and LanguageTool, but Grammarly seems steep at $30 per month when I don’t know if it is any good. I have tried GPT-4 before, but the main problems I have there, is that it is too slow and changes my sentences more than I would like (I tried to make it do that less through prompting, which did not help that much).
I notice that feeling unconfident about my grammar/punctuation leads me to wri...
You have heard and perhaps even used the expression "observable universe", right? What is included in the purportedly observable universe? The moon? The whole of the moon? If you had heard the expression "observable universe" a century ago, would you have been including the far side of the moon in that category?
tl;dr: Recently reported GPT-J experiments [1 2 3 4] prompting for definitions of points in the so-called "semantic void" (token-free regions of embedding space) were extended to fifteen other open source base models from four families, producing many of the same bafflingly specific outputs. This points to an entirely unexpected kind of LLM universality (for which no explanation is offered, although a few highly speculative ideas are riffed upon).
Work supported by the Long Term Future Fund. Thanks to quila for suggesting the use of "empty string definition" prompts, and to janus for technical assistance.
Introduction
"Mapping the semantic void: Strange goings-on in GPT embedding spaces" presented a selection of recurrent themes (e.g., non-Mormons, the British Royal family, small round things, holes) in outputs produced by prompting GPT-J to define...
Hope so, yeah. I'm cautiously optimistic he's doing well by his standards at least.
...Daniel Dennett, professor emeritus of philosophy at Tufts University, well-known for his work in philosophy of mind and a wide range of other philosophical areas, has died.
Professor Dennett wrote extensively about issues related to philosophy of mind and cognitive science, especially consciousness. He is also recognized as having made significant contributions to the concept of intentionality and debates on free will. Some of Professor Dennett’s books include Content and Consciousness (1969), Brainstorms: Philosophical Essays on Mind and Psychology (1981), The Intentional Stance (1987), Consciousness Explained (1992), Darwin’s Dangerous Idea (1995), Breaking the Spell (2006), and From Bacteria to Bach and Back: The Evolution of Minds (2017). He published a memoir last year entitled I’ve Been Thinking. There are also several books about him and his ideas. You
A Great Man and an inspiration to me and to this community and to all thinking men.
God rest his soul in peace in Paradise.
| Value tracked | Effect size d (λ, p, σ change) | Effect size d (λ, p, σ change) |
|---|---|---|
| 200 mg Caffeine (n=1, m=50) | 500 mg L-theanine (n=1, m=50) | |
| Log-score substance prediction[1] | -0.6 | -0.7 |
| Absorption | 0.61 (λ=13.3, p=0.00017, -0.072) | 0.04 (λ=1.38, p=0.77, -0.07) |
| Mindfulness | 0.58 (λ=11.8, p=0.0007, 0.021) | 0.12 (λ=0.72, p=0.89, -0.018) |
| Productivity | 0.58 (λ=28.9, p=1.3-12, 0.11) | -0.28 (λ=5.51, p=0.109, 0.03) |
| Creativity | 0.45 (λ=51, p=4.6-27, 0.09) | -0.12 (λ=5.05, p=0.14, -0.04) |
| Happiness | 0.27 (λ=10.6, p=0.002, 0.3) | 0.16 (λ=3.98, p=0.27, -0.155) |
| Contentment | 0.13 (λ=7.66, p=0.02, 0.47) | 0.25 (λ=6.83, p=0.04, -0.04) |
| Relaxation | -0.11 (λ=5, p=0.15, 0.42) | 0.12 (λ=1.5, p=0.74, 0.02) |
| Chastity[2] | -0.14 (λ=1.9, p=0.64, 0.11) | -0.03 (λ=1.15, p=0.8, 0.25) |
| Flashcard ease | 0.003 (λ≈∞, p≈0, -0.009) | -0.072 (λ=∞, p≈0, -0.01) |
| Flashcard ease factor | -0.039 (λ≈∞, p≈0, -32.7) | 0.0026 (λ=∞, p≈0, -18.9) |
| Flashcard new interval | 0.011 (λ≈∞, p≈0, -1.88) | -0.016 (λ=∞, p≈0, 3.1) |
| Time per flashcard[3] | 0.006 (λ≈∞, p≈0, 273.4) | 0.003 (λ=∞, p≈0, 13.66) |
...L-Theanine is synergistic with caffeine in regards to attention switching[318] and alertness[319][320] and reduces susceptibility to distractions (focus).[320][321] However, alertness seems to be relatively subjective
Edit: I found the post usefwl, thankmuch!!
Mh, was gonna ask when you were taking it. I'm preparing to try it as a sleep-aid for when I adjust my polyphasic sleep-schedule (wanting to go fm 16h-cycles potentially down to 9h) bc it seems potentially drowsymaking and has much faster plasma decay-rate[1] compared to alts. This is good for polyphasic if not want drowsy aft wake.
The data in [1] concerns 100mg tablets, however, and a larger dose (eg 400mg) may be longer. The kinetic model[2] they use will prob be good estimate of p...
Link to my own article. I removed the explanation of EV since I assume on LW that's not necessary.
A group of friends and I occasionally like to get together to play Poker. Yet something keeps happening that I have observed time and again with these kinds of group gatherings: It is hard to find a suitable date and then on top people cancel last minute. This is demotivating for other participants, who in turn also become less committed and this often leads to such groups failing.
Here is one theory of why this happens and how to solve it, explained with Poker. This article will assume Texas Hold'em Poker, probably the most popular variant.
tl;dr People's incentives are not aligned. The solution is to create a social rule that makes folding (canceling attendance) have a bit...
I'm gonna guess that you actually wouldn't make people pay for drinks if they said they missed because they had COVID, there was a death in the family, etc.?
This is a tough call. How do you determine what is a "legitimately bad enough" case to miss the event? The examples you mention are clearly bad enough but there are other situation where it's much more personal. If I'm feeling low on energy is that a choice I am making or an unavoidable fact about my metabolism? You would have to set up some kind of tribunal or voting for deciding on these cases. Th...
