This is a special post for quick takes by shawnghu. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
shawnghu's Shortform
6shawnghu
2Viliam
1shawnghu
6shawnghu
4cubefox
1Milan W
5shawnghu
1shawnghu
2shawnghu
1CstineSublime
2shawnghu
1CstineSublime
1shawnghu
1shawnghu
I tend to think of myself as immune to rage-baiting/click-baiting/general toxicity from social media and politics. I generally don't engage in arguments on classic culture war topics on the internet, and I knowingly avoid consuming much news on the grounds that it will make me feel worse without inducing meaningful change.
But I recently realized that the phenomenon has slightly broader implications: presumably in any medium, outrage is just more attractive to the human brain, and conflicts are entertaining, especially ones where you can take a side or criticize both sides.
This made me realize that this issue isn't constrained to just the forms of media I'm more explicitly cynical about. In particular, some of the content I read about culture, even if it is more nuanced, and even if it reads as following a debate, is still essentially to me about observing conflict for entertainment.
Entertainment for entertainment's sake is fine, but I doubt if doing this kind of reading is on the Pareto frontier of "enjoy myself" and "inform myself about things in an actionable way".
On the far end of the spectrum, I am not sure if it's sensible to attempt to fully avoid engaging in any form of social derision/feeling outrage, even unproductive outrage. It does feel like some forms of outrage arise organically as a natural consequence of valuing things.
I think you are correct on a general level, but some conflicts are pure zero-sum waste of resources, and the more nuanced ones maybe are not? Like, if we debate about whether object-oriented programming is better than functional, perhaps as a result both sides get better at writing software. Or at least the observers get better at writing software, regardless of which style they choose.
I agree that there are internet conflicts worth participating in, for sure. This site contains a large number of them!
But the original post was mostly about the value of passively reading certain things vs certain other things for entertainment. (In the first paragraph, I separate out "arguments on classic culture war topics" as an example of the sorts of conflicts that are most likely a waste of resources.)
Is anyone else noticing that Claude (Sonnet 3.5 new, the default on claude.ai) is a lot worse at reasoning recently? In the past five days or so its rate of completely elementary reasoning mistakes, which persist despite repeated clarification in different ways, seems to have skyrocketed for me.
Maybe they are preparing for switching from merely encouraging their main model to do CoT (old technique) to a full RL-based reasoning model. I recently saw this, before the GUI aborted and said the model was over capacity:
Then it wouldn't make sense anymore to have the non-reasoning model attempt to do CoT.
it's hard to find definitive information about this basic fact about how modern RL on LLMs works:
are there any particularly clever ways of doing credit assignment for the tokens in a sequence S that resulted in high reward?
moreover, if you adopt the naive strategy of asserting that all of the tokens are equally responsible for the reward, is the actual gradient update to the model parameters mathematically equivalent to the one you'd get SFTing the model on S (possibly weighted by the reward, and possibly adjusted by GRPO)?
the followup is this: in this paper they claim that SFT'd models perform badly at something and RL'd models don't. i can't imagine what the difference between these things would even be, except that the RL'd models are affected by samples which are on-policy for them.
followup: after looking at the appendix i'm pretty sure the biggest distinction is that the SFT'd models in this paper are SFT'd on data that comes from entirely different models/datasets. so not only is the data not coming from a policy which adapts during training, it's coming from policies very different from the model's own. i think this by itself is enough to explain the results of the paper; i think this is a useful result but not the one i imagined upon reading the title.
i would still like to know the answer to the original question.
Does anyone have a rigorous reference or primer on computer ergonomics, or ergonomics in general? It's hard to find a reference that says with authority/solid reasoning what good ergonomics are and why, and solutions to common problems.
I'd be interested in this myself. Where/how have you looked so far, and which resources have you found wanting so far?
I didn't do a lot of thorough research, but maybe I simply don't know how to.
I googled around for resources, which usually leads to... I don't know how to describe this, but short-form articles which are not very information dense and mutually contradictory, and I looked for opinions on Reddit and for an FAQ-like thing on /r/Ergonomics, which also didn't tell me much definitive except that a) people have a variety of problems due to their variety of body shapes and b) it is a normal thing to want a desk that's significantly lower than most desks.
I must have done some amount of Claude-querying, but it's intensive to figure out what the root problems are here and whether there are canonical solutions to them, possibly because of the fact that the resources Claude would most easily reference are the same inadequate ones I've just described. I bet that it's possible to figure this out with Claude if I go slowly and specifically enough, though.
I don't think I found anything even approaching a central resource which claims to be comprehensive (however opinionated). Something like what they have at /r/bodyweightfitness, for example, would be excellent by the standards described here.
These are the first things I found on the first search result page of GoodReads, do these suite?
Applying Systemic-Structural Activity Theory to Design of Human-Computer Interaction Systems
"Human–Computer Interaction (HCI) is no longer limited to trained software users. Today people interact with various devices such as mobile phones, tablets, and laptops. How can such interaction be made more user friendly, even when user proficiency levels vary? This book explores methods for assessing the psychological complexity of computer-based tasks. It also presents methods of qualitative and quantitative analysis of exploratory activity during interaction with a computer."
Assessment of the Ergonomic Quality of Hand-Held Tools and Computer Input Devices
"The International Ergonomics Association (IEA) is currently developing standards for Ergonomic Quality in Design (EQUID) which primarily intends to promote ergonomics principles and the adaptation of a process approach for the development of products, work systems and services. It is important to assess the ergonomic quality of products, hand-held tools and computer input devices through working processes that represent reality. Well-designed working tools can be expected to reduce or eliminate fatigue, discomfort, accidents and health problems and they can lead to improvements in productivity and quality. Furthermore, absenteeism, job turnover and training costs can positively be influenced by the working tools and the environment. Not all these short-term and long-term issues of working tools can be quantified in pragmatically oriented ergonomic research approaches. But multi-channel electromyography, which enables the measurement of the physiological costs of the muscles involved in handling tools during standardized working tests, and subjective assessments of experienced subjects enable a reliable insight in the essential ergonomic criteria of working tools and products. In this respect it is advantageous to provide a test procedure, in which working tests can be carried out alternating both with test objects and reference models."
Does anyone have a sense of whether, qualitatively, RL stability has been solved for any practical domains?
This question is at least in part asking for qualitative speculation about how the post-training RL works at big labs, but I'm interested in any partial answer people can come up with.
My impression of RL is that there are a lot of tricks to "improve stability", but performance is path-dependent in pretty much any realistic/practical setting (where state space is huge and action space may be huge or continuous). Even for larger toy problems my sense is that various RL algorithms really only work like up to 70% of the time, and 30% of the time they randomly decline in reward.
One obvious way of getting around this is to just resample. If there are no more principled/reliable methods, this would be the default method of getting a good result from RL. It would follow that that's just what the big labs do. But of course they may have secretly solved some stuff, but it's hard to imagine what the form of that would be.
at the end of the somewhat famous blogpost about llm nondeterminism recently https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ they assert that the determinism is enough to make an rlvr run more stable without importance sampling.
is there something i'm missing here? my strong impression is that the scale of the nondeterminism of the result is quite small, and random in direction, so that it isn't likely to affect an aggregate-scale thing like the qualitative effect of an entire gradient update. (i can imagine that the accumulation of many random errors does bias the policy towards being generally less stable, which implies qualitatively worse, yes...)
without something that mitigates the statement above, my prior is instead that the graph is cherry-picked, intentionally or not, to increase the perceived importance of llm determinism.