This is a special post for quick takes by papetoast. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
papetoast's Shortforms
23papetoast
2habryka
8papetoast
4MondSemmel
2habryka
17papetoast
6kave
3papetoast
4Raemon
11papetoast
13Dagon
3papetoast
10Vladimir_Nesov
4papetoast
5Vladimir_Nesov
1papetoast
5Vladimir_Nesov
1papetoast
4Seth Herd
5papetoast
3Viliam
1papetoast
1papetoast
2Seth Herd
5papetoast
3papetoast
3papetoast
3papetoast
3Dagon
1papetoast
3papetoast
2Dagon
2papetoast
0ProgramCrafter
1papetoast
2papetoast
2papetoast
1papetoast
1papetoast
3Dagon
1papetoast
1papetoast
1papetoast
1papetoast
1papetoast
1papetoast
2Dagon
1papetoast
1papetoast
1papetoast
3Dagon
1papetoast
1papetoast
3Dagon
1papetoast
3Dagon
3papetoast
3Dagon
0papetoast
3Dagon
1papetoast
1papetoast
0papetoast
-1papetoast
2Dagon
0[comment deleted]
In my life I have never seen a good one-paragraph[1] explanation of backpropagation so I wrote one.
The most natural algorithms for calculating derivatives are done by going through the expression syntax tree[2]. There are two ends in the tree; starting the algorithm from the two ends corresponds to two good derivative algorithms, which are called forward propagation (starting from input variables) and backward propagation respectively. In both algorithms, calculating the derivative of one output variable with respect to one input variable actually creates a lot of intermediate artifacts. In the case of forward propagation, these artifacts means you get for ~free, and in backward propagation you get for ~free. Backpropagation is used in machine learning because usually there is only one output variable (the loss, a number representing difference between model prediction and reality) but a lot of input variables (parameters; in the scale of millions to billions).
- ^
This blogpost by Christopher Olah has the clearest multi-paragraph explanation. Credits for the image too.
- ^
Actually a directed acyclic graph for multivariable vector-valued functions like f(x,y)=(2x+y, y-x), or if you do Common Subexpression Elimination in the syntax tree (as is done for the b node in the image). For simplicity I pretend that it is a tree.
3
1In the case of forward propagation, these artifacts means you get for ~free, and in backwards propagation you get for ~free.
Presumably you meant to say something else here than to repeat twice?
Edit: Oops, I now see. There is a switched . I did really look quite carefully to spot any difference, but I apparently still wasn't good enough. This all makes sense now.
1
I could barely see that despite always using a zoom level of 150%. So I'm sometimes baffled at the default zoom levels of sites like LessWrong, wondering if everyone just has way better eyes than me. I can barely read anything at 100% zoom, and certainly not that tiny difference in the formulas!
Our post font is pretty big, but for many reasons it IMO makes sense for the comment font to be smaller. So that plus LaTeX is a bit of a dicey combination.
Raw feelings: I am kind of afraid of making reviews for LW. The writing prompt hints very high effort thinking. The vague memory of other people's reviews also feel high effort. The "write a short review" ask doesn't really counter this at all.
Thank you!
Would it help if the prompt read more like a menu?
Reviews should provide information that help evaluate a post. For example:
- What does this post add to the conversation?
- How did this post affect you, your thinking, and your actions?
- Does it make accurate claims? Does it carve reality at the joints? How do you know?
- Is there a subclaim of this post that you can test?
- What followup work would you like to see building on this post?
Yeah, and perhaps a couple examples of bare minimum / average / high quality review in the main post
On thing to note is that "short reviews" in the nomination phase are meant to be basically a different type of object than "effort reviews." Originally we actually had a whole different data-type for them ("nominations"), but it didn't seem worth the complexity cost.
And then, separately: one of the points of the review is just to track "did anyone find this actually helpful?" and a short review that's like "yep, I did in fact use this concept and it helped me, here's a few details about it" is valuable signal.
Drive by "this seems false, because [citation]" also good.
It is nice to do more effortful reviews, but I definitely care about those types of short reviews.
1It is sad and annoying that if you do a mediocre job (according to the receiver), doing things even for free (volunteer work/gifting) can sabotage the receiver along the dimension you're supposedly helping.
This is super vague the way I wrote it, so examples.
Example 1. Bob wants to upgrade and buy a new quality headphone. He has a $300 budget. His friend Tim not knowing his budget, bought a $100 headphone for Bob. (Suppose second-handed headphones are worthless) Now Bob cannot just spend $300 to get a quality headphone. He would also waste Tim's $100 which counterfactually could have been used to buy something else for Bob. So Bob is stuck with using the $100 headphone and spending the $300 somewhere else instead.
Example 2. Andy, Bob, and Chris are the only three people who translates Chinese books to English for free as a hobby. Because there are so many books out there, it is often not worth it to re-translate a book even if the previous one is bad, because spending that time to translate a different book is just more helpful to others. Andy and Bob are pretty good, but Chris absolutely sucks. It is not unreadable, but they are just barely better than machine translation. Now Chris has taken over to translate book X, which happens a pretty good book. The world is now stuck with Chris' poor translation on book X with Andy and Bob never touching it again because they have other books to work on.
Allocation of blame/causality is difficult, but I think you have it wrong.
ex. 1 ... He would also waste Tim's $100 which counterfactually could have been used to buy something else for Bob. So Bob is stuck with using the $100 headphone and spending the $300 somewhere else instead.
No. TIM wasted $100 on a headset that Bob did not want (because he planned to buy a better one). Bob can choose whether to to hide this waste (at a cost of the utility loss by having $300 and worse listening experience, but a "benefit" of misleading Tim about his misplaced altruism), or to discard the gift and buy the headphones like he'd already planned (for the benefit of being $300 poorer and having better sound, and the cost of making Tim feel bad but perhaps learning to ask before wasting money).
ex. 2 The world is now stuck with Chris' poor translation on book X with Andy and Bob never touching it again because they have other books to work on.
Umm, here I just disagree. The world is no worse off for having a bad translation than having no translation. If the bad translation is good enough that the incremental value of a good translation doesn't justify doing it, then that is your answer. If it's not valuable enough to change the marginal decision to translate, then Andy or Bob should re-translate it. Either way, Chris has improved the value of books, or has had no effect except wasting his own time.
Bob can choose whether to to hide this waste (at a cost of the utility loss by having $300 and worse listening experience, but a "benefit" of misleading Tim about his misplaced altruism)
True in my example. I acknowledge that my example is wrong and should have been more explicit about having an alternative. Quoting myself from the comment to Vladimir_Nesov:
Anyways, the unwritten thing is that Bob care about having a quality headphone and a good pair of shoes equally. So given that he already has an alright headphone, he would get more utility by buying a good pairs of shoes instead. It is essentially a choice between (a) getting a $300 headphone and (b) getting a $100 headphone and a $300 pair of shoes.
If the bad translation is good enough that the incremental value of a good translation doesn't justify doing it, then that is your answer.
I do accept this as the rational answer, doesn't mean it is not irritating. If A (skillful translator) cares about having a good translation of X slightly more than Y, and B (poor translator) cares about Y much more than X. If B can act first, he can work on X and "force" A (via expected utility) to work on Y. This is a failure of mine to not talk about difference in preference in my examples and expect people to extrapolate and infer it out.
Now Bob cannot just spend $300 to get a quality headphone. He would also waste Tim's $100
That's a form of sunk cost fallacy, a collective "we've sacrificed too much to stop now".
Andy and Bob never touching it again because they have other books to work on
That doesn't follow, the other books would've also been there without existence of this book's poor translation. If the poor translation eats some market share, so that competing with it is less appealing, that could be a valid reason.
This is a tangent, but Sunk cost fallacy is not really a fallacy most of the time, because spending more resources beforehand really increases the chance of "success" most of the time. For more: https://gwern.net/sunk-cost
I am trying to pinpoint the concept of "A doing a mediocre job of X will force B to rationally do Y instead of X, making the progress of X worse than if A had not done anything". The examples are just examples that hopefully helps you locate the thing I am handwaving at. I do not try to make them logically perfect because that would take too much time.
Anyways, the unwritten thing is that Bob care about having a quality headphone and a good pair of shoes equally. So given that he already has an alright headphone, he would get more utility by buying a good pairs of shoes instead. It is essentially a choice between (a) getting a $300 headphone and (b) getting a $100 headphone and a $300 pair of shoes. Of course there are some arguments about preference, utility != dollar amount or something along those lines. But (b) is the better option in my constructed example to show the point.
Let me know if I still need to explain example 2
spending more resources beforehand really increases the chance of "success" most of the time
The decision to go on with the now-easier rest-of-the-plan can be correct, it's not the case that all plans must always be abandoned on the grounds of "sunk cost fallacy". The fallacy is when the prior spending didn't actually secure the rest of the current plan as the best course of action going forward. Alternatives can emerge that are better than continuing and don't make any use of the sunk resources.
It sure can! I think we are in agreement on sunk cost fallacy. I just don't think it applies to example 1 because there exists alternatives that can keep the sunk resources. Btw this is why my example is on the order of $100, at this price point you probably have a couple alternative things to buy to spend the money.
I just don't think it applies to example 1 because there exists alternatives that can keep the sunk resources.
What matters is if those alternatives are better (and can be executed on, rather than being counterfactual). It doesn't matter why they are better. Being better because they made use of the sunk resources (and might've become cheaper as a result) is no different from being better for other reasons. The sunk cost fallacy is giving additional weight to the alternatives that specifically use sunk resources, instead of simply choosing based on which alternatives are now better.
Again, seems like we are in agreement lol. I agree with what you said and I meant that, but tried to compress it into one sentence and failed to communicate.
(I need to defend the sad and the annoying in two separate parts)
- Yes, and but sometimes that is already annoying on its own (Bob is not perfectly rational and sometimes he just really want the quality headphone, but now math tells Bob that Tim gifting him that headphone means he would have to wait e.g. ~2 years before it is worth buying a new one). Of course Bob can improve his life in other ways with his saved money, but still, would be nice if you can just ask Tim to buy something else if you had known.
- Sometimes increasing sum(projects) does not translate directly to increasing utility. This is more obvious in real life scenarios where actors are less rational and time is a real concept. The sad thing happens when someone with good intention but with poor skill (and you don't know they are that bad) signing up to a time-critical project and failing/doing sub-par
Seems like the problem is that in real life people are not perfectly rational, and also they have an instinct to reciprocate when they receive a gift (at least by saying "thank you" and not throwing the gift away).
In a world where Bob is perfectly rational and Tim has zero expectations about his gift, the situation is simple. Previously, Bob's choices were "spend $300 on good headphone", "spend $100 on bad headphone and $200 on something else", and "spend $300 on something else". Tim's action replaced the last two options with a superior alternative "use Tim's headphone and spend $300 on something else". Bob's options were not made worse.
But real people are not utility maximizers. We instinctively try to choose a locally better option, and how we feel about it depends on what we perceive as the baseline. Given the choice between 10 utilons and 3 utilons, we choose 10 and feel like we just "gained 7 utilons". Given the choice between 10 utilons and 9 utilons, we choose 10 again, but this time we feel like we just "gained 1 utilon". Given the choice between 10 utilons and 10 utilons of a different flavor, we might feel annoyed about having to choose.
Also, if Tim expects Bob to reciprocate in a certain way, the new options are not strictly better, because "spend $300 on good headphone" got replaced by "spend $300 on good headphone, but owe Tim a favor for giving me the $100 headphone I didn't use".
1Joe Rogero: Buying something more valuable with something less valuable should never feel like a terrible deal. If it does, something is wrong.
clone of saturn: It's completely normal to feel terrible about being forced to choose only one of two things you value very highly.
https://www.lesswrong.com/posts/dRTj2q4n8nmv46Xok/cost-not-sacrifice?commentId=zQPw7tnLzDysRcdQv
There are infinite things to be sad and annoyed by, should you choose to focus on those. :) I'd rather focus on the world as a whole being made better in your examples.
One of my pet peeves is that the dropcaps in gwern's articles are really, really offputting and most of the time unrecognizable, even though gwern's articles are so valuable that he has a lot of weirdness points in my head and I will still read his stuff regardless. Most of the time I just guess the first letter.
I hate dropcaps in general, but gwern's is the ugliest I have came by.
image source: https://gwern.net/everything
it is interesting how the AI agent prompts seem to have mostly converged to xml, but system prompts from the LLM companies are in markdown
Just as documentation here are a bunch of people on Hacker News complaining about rationality: https://news.ycombinator.com/item?id=44317180. I have not formed any strong opinion on whether these are true, feels like they are wrong on the object level, but perception is also important
I think people (myself included) really underestimated this rather trivial statement that people don't really learn about something when they don't spend the time doing it/thinking about it. People even measure mastery by hours practiced and not years practiced, but I still couldn't engrave this idea deep enough into my mind.
I currently don't have much writable evidence about why I think people underestimated this fact, but I think it is true. Below are some things that I have changed my mind/realised after noticing this fact.
- cached thoughts, on yourself
Personally, I am a huge procrastinator and I can really flinch away from doing something even when it is weeks overdue. I was trying out BaaS and Beeminder to build up some good habits, but even with the tools I still somethings have procrastination episodes. Only after quite a lot of cycles of the procrastination episodes, I realised that I basically completely wasted the time when I was procrastinating, and I was overall actually worse than before I started the procrastination episode.
Therefore, what I concluded is that you should expect yourself to be the exact same if you haven't put in the time to think about that topic, especially high-level topics like math. (I acknowledge that motor skills require less thinking) It is a mere wish to be a different person since the last time; You don't just learn a new theorem automatically.
- cached thoughts, on modelling human
There was this bias that you assume other people are at around the same level as you, that is obviously false, but it is quite hard to internalize this. People really don't automatically improve themselves either, there must be a push for that to happen. Also you can probably see many of those people that just stopped changing themselves.
- there's a very limited amount you can learn by just reading a few summaries:
Some texts are better than others, but even if you only read the best text on the topic you are trying to learn, with the text being paraphrased by a magical AI to maximize for your learning efficiency, there is still a maximum bandwidth on learning. Don't expect to replicate what other people is able to do in just a few hours. Though I should acknowledge that there are actually very short texts that can change your mind greatly, I suspect that growth mindset is one of them but I'm not sure.
Alternatively, if the questions you ask are specific enough, then you may just be able to somewhat master that concept in a short amount of time. This seems to be how the "Learn in <very short amount of time>" courses out there do.
There are at least a few different dimensions to "learning", and this idea applies more to some than to others. Sometimes a brief summary is enough to change some weights of your beliefs, and that will impact future thinking to a surprising degree. There's also a lot of non-legible thinking going on when just daydreaming or reading fiction.
I fully agree that this isn't enough, and both directed study and intentional reflection is also necessary to have clear models. But I wouldn't discount "lightweight thinking" entirely.
^the above is a reply to a slightly previous version
Agree with everything here, and all the points the first paragraph I have not thought about. I'm curious if you have a higher resolution model to different dimensions of learning though, feels like I can improve my post if I have a clearer picture.
Btw, your whole reply seem to be a great example of what do you mean by "it's probably best to acknowledge it and give the details that go into your beliefs, rather than the posterior belief itself."
[Draft] It is really hard to communicate the level/strength of basically anything on a sliding scale, but especially things that could not make any intuitive sense even if you stated a percentage. One recent example I encountered is expressing what is in my mind the optimal tradeoff between reading quickly and thinking deeply to achieve the best learning efficiency.
Not sure what is the best way to deal with the above example, and other situations where percentage doesn't make sense.
But where percentage makes sense, there are still two annoying problems. 1. sometimes you don't have (haven't generated) a fixed percentage in your mind. 2. You still need to express the uncertainty of that percentage. (vague example: 70% +- 20% vs 70% +- 5%).
I think it could be worth it to establish a common knowledge for what percentages do the uncertainty-hinting keywords represent. Say, "seems like" = 75% +- 30%
Unsorted Ideas Below
- A lot of people don't seem to realise this problem because their models are too black and white.
- It is more difficult to communicate that the other person is being slightly too confident, than to tell them that they are way too overconfident.
For most topics, it's probably not worth going very deep in the rabbit hole of "what does a probability mean in this context". Yes, there are multiple kinds of uncertainty, and multiple kinds of ratio that can be expressed by a percentage. Yes, almost everything is a distribution, most not normal, and even when normal it's not generally specified what the stddev is. Yes, probability is causally recursive (the probability that your model is appropriate causes uncertainty in the ground-level probability you hold). None of that matters, for most communication. When it does, then it's probably best to acknowledge it and give the details that go into your beliefs, rather than the posterior belief itself.
For your example, the tradeoff between fast and careful, I doubt it can be formalized that way, even if you give yourself 10 dimensions of tradeoff based on context. "Slow is smooth, smooth is fast" is the classic physical training adage, and I can't think of a numeric representation that helps.
The ERROR Project: https://error.reviews/
Quoting Malte Elson
The very short description of ERROR is that we pay experts to examine important and influential scientific publications for errors in order to strengthen the culture of error checking, error acceptance, and error correction in our field. As in other bug bounty programs, the payout scales with the magnitude of errors found. Less important errors pay a smaller fee, whereas more important errors that affect core conclusions yield a larger payout.
We expect most published research to contain at least some errors, and that we believe that the presence of minor errors does not negate an article’s scientific contribution. Indeed, our reward system pays bonuses to both authors and reviewers even when minor errors are found. We believe that our field would be strengthened by a culture of checking, accepting, and communicating errors.
We expect most published research to contain at least some errors
Have you set up the prediction markets on that? Not necessarily "is there an error in this paper", but "in this group of publications, what fraction has an issue of this kind" and so on.
Starting today I am going to collect a list of tricks that websites use to prevent you from copy and pasting text + how to circumvent them. In general, using ublock origin and allow right click properly fixes most issues.
1. Using href (https://lnk.to/LACA-15863s, archive)
behavior: https://streamable.com/sxeblz
solution: use remove-attr in ublock origin - lnk.to##.header__link:remove-attr(href)
2. Using a background image to cover the text (https://varium.jp/talent/ahiru/, archive)
Note: this example is probably just incompetence.
behavior: https://streamable.com/bw2wlv
solution: block the image with ublock origin
3. Using draggable=true (Spotify album titles, archive)
Note: Spotify does have a legit reason to use draggable. You can drag albums or tracks to your library, but I personally prefer having texts to be selectable.
behavior: https://streamable.com/cm0t6b
solution: use remove-attr in ublock origin - open.spotify.com##.encore-internal-color-text-base.encore-text-headline-large.encore-text:upward(1):remove-attr(draggable)
4. Using EventListeners to nullify selections (https://www.uta-net.com/song/2765/, archive)
behavior: https://streamable.com/2i1e9k
solution: locate the function using the browser debugger/ctrl+f, then do some ublock origin javascript filter stuff that I don't really understand. Seems to just be overriding functions and EventListeners. The Annoyances filters worked in this case.
5. <canvas> (https://petitlyrics.com/lyrics/3675825, archive)
Not sure if there is a good way to handle this
Many people don't seem to know when and how to invalidate the cached thoughts they have. I noticed an instance of being unable to cache invalidate the model of a person from my dad. He is probably still modelling >50% of me as who I am >5 years ago.
The Intelligent Social Web briefly talked about this for other reasons.
A lot of (but not all) people get a strong hit of this when they go back to visit their family. If you move away and then make new friends and sort of become a new person (!), you might at first think this is just who you are now. But then you visit your parents… and suddenly you feel and act a lot like you did before you moved away. You might even try to hold onto this “new you” with them… and they might respond to what they see as strange behavior by trying to nudge you into acting “normal”: ignoring surprising things you say, changing the topic to something familiar, starting an old fight, etc.
In most cases, I don’t think this is malice. It’s just that they need the scene to work. They don’t know how to interact with this “new you”, so they tug on their connection with you to pull you back into a role they recognize. If that fails, then they have to redefine who they are in relation to you — which often (but not always) happens eventually.
How I use AI for coding.
I wrote this in like 10 minutes for quick sharing.
- I am not a full time coder, I am a student who code like 15-20 hours a week.
- Investing too much time on writing good prompts make little sense. I go with the defaults and add pieces of nudges as needed. (See one of my AGENTS .md at the end)
- Mainly codex (cloud) and Cursor. Claude Code works, but being able to easily revert is helpful, so Cursor is better.
- I still try out claude code for small pieces of edits, but it doesnt feel worth it.
- I have no idea why people like claude code so much? CLI is inferior to GUI
- Using cursor means I don't need to have multiple git worktrees for each agent, as long as I get them to work on different parts of the codebase
- Mobile coding is real and very convenient with codex (cloud), but I still review and edit on desktop.
- Using multiple agents is possible, but usually one big feature and multiple smaller background edits.
- Or multiple big features using codex cloud, and delay review to a later time.
- Codex cloud is good but only generate one commit for PR, often I need to manually split them up. I am eyeing on other cloud agents solution but havent tried them seriously yet.
Current prompt for one of the python projects
## Code Style
- 120-character lines
- Type hints is a must
- **Don't use Python 3.8 typings**: Never import `List`, `Tuple` or other deprecated classes from `typing`, use `list`, `tuple` etc. instead, or import from `collections.abc`
- Do not use `from __future__ import annotations`, use forward references in type hints instead. `TYPE_CHECKING` should be used only for imports that would cause circular dependencies.
## Documentation and Comments
Add code comments sparingly. Focus on why something is done, especially for complex logic, rather than what is done. Only add high-value comments if necessary for clarity or if requested by the user. Do not edit comments that are separate from the code you are changing. NEVER talk to the user or describe your changes through comments.
### Using a new environmental variable
When using a new environmental variable, add it to `.env.example` with a placeholder value, and optionally a comment describing its purpose. Also add it to the `Environment Variables` section in `README.md`.
### Using deal
We only use the exception handling features of deal. Use `@deal.raises` to document expected exceptions for functions/methods. Do not use preconditions/postconditions/invariants.
Additionally, we assume `AssertionError` is never raised, so `@deal.raises(AssertionError)` is not allowed.
## Testing Guidelines
To be expanded.
Mocking is heavily discouraged. Use test databases, test files, and other real resources instead of mocks wherever possible.
Allowed pytest markers:
- `@pytest.mark.integration`
- `@pytest.mark.slow`
- `@pytest.mark.docker`
- builtin ones like `skip`, `xfail`, `parametrize`, etc.
We do not use
- `@pytest.mark.unit`: all tests are unit tests by default
- `@pytest.mark.asyncio`: we use `pytest-asyncio` which automatically handles async tests
- `@pytest.mark.anyio`: we do not use `anyio`
### Running Tests
Use `uv run pytest ...` instead of simply `pytest ...` so that the virtual environment is activated for you.
## Asking for Help
- Refactoring:
As a command-line only tool, you do not have access to helpful IDE features like "Refactor > Rename Symbol". Instead, you can ask the user to rename variables, functions, classes, or other symbols by providing the current name and the new name. It is important that you don't rename public variables yourself, as you might miss some occurrences of the symbol across the codebase.
## Information
Finding dependencies: we use `pyproject.toml`, not `requirements.txt`. Use `uv add <package>` to add new dependencies.
(Note that the Asking for Help is basically useless. It was experimental and I never got asked lol)
A common failure mode in group projects is that students will break up the work into non-overlapping parts, and proceed to stop giving a fuck about other's work afterwards because it is not their job anymore.
This especially causes problems at the final stage where they need to combine the work and make a coherent piece out of it.
- No one is responsible for merging the work
- Lack of mutual communication during the process means that the work pieces cannot be nicely connected without a lot of modifications (which no one is responsible for).
At this point the deadline is likely just a couple days (or hours) away, everyone is tired of this crap and don't want to work on it, but the combined work is still a piece of incoherent crap.
I wonder how I can do better at coordination while dealing with normal peers and while only doing a fair amount of work.
Not just students, but this is a common failure in large engineering projects. There are often FAR too few "glue" and "end-to-end" responsibilities tracked and assigned, so the people who care about the end result aren't engineers actually doing stuff (many managers and execs are former engineers, but have somehow forgotten how things actually get built).
The most common solution is project managers. Companies hate to pay these "extra" employees not actually producing code/designs/output, and classes almost never acknowledge their existence, let alone the necessity. But good ones really pull their weight in coordination and identification of mismatches.
There is probably no way to "do better at coordination while dealing with normal peers and while only doing a fair amount of work." Either do more work (of a different type than the class is based on), accept the pain and bad results, or find/force another student to do that coordination work. In the real world, in good companies, results get noticed and lead to better personal outcomes - not everyone is equal, and "fair" doesn't matter much. In school, you're kind of screwed.
Though there ARE some things you can do - extra work and coordinating/cajoling people, but often effective and feasible. Start with integration. Get the end-to-end WORKING MOCKUP going with hardcoded behaviors in each module, but working interfaces. This is often half or more of the work, and there's no way to avoid it - doing it at the end is painful and often fails. Doing it up front is painful but actually leads to completion. You may learn some things in this phase that make you split up the work differently. Depending on size and duration of the project, that may be fixable with different division, or may just be "I'll try to help once I've finished my part".
I do believe that projects in general often fail due to lack of glue responsibilities, but didn't want to generalize too much in what I wrote.
Start with integration. Get the end-to-end WORKING MOCKUP going with hardcoded behaviors in each module, but working interfaces. This is often half or more of the work, and there's no way to avoid it - doing it at the end is painful and often fails. Doing it up front is painful but actually leads to completion.
Being able to convince everyone to put in the time to do this upfront is already a challenge :/ Sometimes I feel quite hopeless?/sad? in that I couldn't realistically make some coordination techniques work because of everyone's difference of goals and hidden motivations, or the large upfront cost in building a new consensus away from the Schelling point of normal university projects.
Ranting about LangChain, a python library for building stuff on top of llm calls.
LangChain is a horrible pile of abstractions. There are many ways of doing the same thing. Every single function has a lot of gotchas (that doesn't even get mentioned in documentations). Common usage patterns are hidden behind unintuitive, hard to find locations (callbacks has to be implemented as an instance of a certain class in a config TypedDict). Community support is non-existent despite large number of users. Exceptions are often incredibly unhelpful with unreadable stack trace. Lots of stuff are impossible to type check because langchain allows for too much flexibility, they take in prompt templates as format strings (i.e. "strings with {variables}") and then allows you to fill in the template at runtime with a dict, so now nothing can be statically type checked :)
There are a few things I dislike about math textbooks and pdfs in general. For example, how math textbooks often use theorems that are from many pages ago and require switching back and forth. (Sometimes there isn't even a hyperlink!). I also don't like how proofs sometimes go way too deep into individual steps and sometimes being way too brief.
I wish something like this exists (Claude generated it for me, prompt: https://pastebin.com/Gnis891p)
4 reasons to talk about your problem with friends
This is an advice I would tell myself 5 years ago, just storing it somewhere public and forcing myself to write. Writing seems like an important skill but I always feel like I have nothing to say.
- It forces you to think. Sometimes you aren't actually thinking about solutions to a problem even though it has been bothering you for a long time.
- for certain problems: a psychological feeling of being understood. For some people, getting the idea that "what I'm feeling is normal" is also important. It can be a false sense of comfort, but sometimes you need it.
- a hidden "contract" that makes the stake of not fixing your problem higher, you now risk looking unimpressive to more people if you fail to fix your problem, since now your friend also knows you're struggling. This increases the motivation of solving the problem. Related: akrasia??
- also the slim possibility that even though you think your friend cant help you but actually they can.
Since we have meta search engines that aggregate search results from many search engines, is it time for us to get a meta language model* to get results from chatGPT, Bing, Bard, and Claude all at the same time, and then automatically rank them, perhaps even merging all of the replies into a single reply.
*meta language model is an extremely bad name because of the company Meta and the fact that the thing I am thinking of isn't really a language model, but ¯\_(ツ)_/¯
I always thought that the in-page redirects are fucking stupid, it should bring the text I want to see closer to eye level, not exactly at the top where even browser bars can block the text (happens when you go back from footnotes to article on LW).
For some screen size/shape, for some browser positioning, for some readers, this is probably true. It's fucking stupid to believe that's anywhere close to a majority. If that's YOUR reading area, why not just make your browser that size?
It should be pretty easy to write a tampermonkey or browser extension to make it work that way. Now that you point it out, I'm kind of surprised this doesn't seem to exist.
I admit that 30-50% is arbitrary and shouldn't be brought up like a fact, I have removed it. (I didn't mean to have such a strong tone there, but I did) What I really want to say is that the default location for the target text to be somewhere closer to the middle/wherever most people usually put their eyes on. (Perhaps exactly the height where you clicked the in-page redirect?)
I still stand by that it should not be exactly at the top for ease of reading (I hope this doesn't sound too motte-and-bailey). The reason that it is redirected to the top is probably because it is a very objective location and wouldn't get affected by device size. But it is very much not a standard location where the current line of text you are reading will be. I am willing to bet that <3% of people read articles where they scroll their currently reading line up to the top three visible lines.
Documenting a specific need of mine: LaTeX OCR software
tl;dr: use Mathpix if you use it <10 times per month or you are willing to pay $4.99 per month. Otherwise use SimpleTex
So I have been using Obsidian for note taking for a while now and I eventually decided to stop using screenshots but instead learn about LaTeX so the formulas look better. At first I was relying on the website to show the original LaTeX commands but some websites (wiki :/) doesn't do that, and also I started reading math textbooks as PDF. Thus started my adventure to find a good and ideally free LaTeX OCR software.
An initial google search takes me to Mathpix, it honestly is pretty good except for the limit of 10 free OCRs per month ($4.99/month subscription). But for the worry that I will exceed the 10 free snips soon (it actually haven't happened for 3 months now), I started finding free ones on github, including LaTeX-OCR and Pix2Text, but their GUI kind of suck. Then I also tried SimpleTex and this is the one I will probably use when I run out of free snips on Mathpix.
How likely are people actually clicking through links of related materials in a post, seems unlikely to me, actually unlikely to the point that I am thinking about whether it is actually useful.
related: https://www.lesswrong.com/posts/JZuqyfGYPDjPB9Lne/you-don-t-have-to-click-the-links
Depends on the post and the links. I click through about 15% of Zvi's links, for instance, but I appreciate the others as further information and willingness to cite, even if I don't personally use them. Other posts, I skim rather than really examining, and links still add value by indicating that the author has actually done a bit of research into the topic.
Thanks for the datapoint. Also links serving as indicator of effort rather than actually expanding on the amount of information on the passage is a good point. If links are mainly indicator of effort, I think this imply that people should not try as hard to make sure the relevance of the links.
FWIW: My click through rate is probably <5%.
[Draft] Are memorisation techniques still useful in this age where you can offload your memory to digital storage?
I am thinking about using anki for spaced repetition, and the memory palace thing also seem (from the surface level) interesting, but I am not sure whether the investments will be worth it. (2023/02/21: Trying out Anki)
I am increasingly finding it more useful to remember your previous work so that you don't need to repeat the effort. Remembering workflow is important. (This means remembering things somewhere is very important, but im still not sure if that somewhere being your brain is as important)
example 1: I download music from youtube and I have been using online downloader for years until one day I learned about yt-dlp, and I spend half an hour reading the document to type the correct command for my use and I didn't save it down somewhere else, so the next time that I want to download music again I had to spend another half an hour reading the doc. Now I have a txt file storing the command.
examples to be written: lyrics (css shit), writing down ideas immediately on the nearest place.
Agree with Dagon that indexing is very important.
trial of using github to host version history
Some certainly are. For many facts, memorized data is orders of magnitude faster than digitally-stored knowledge. This is enough of a difference to be qualitative - it's not worth looking up, but if you know it, you'll make use of it.
There's the additional level of internalizing some knowledge or techniques, where you don't even need to consciously expend effort to make use of it. For some things, that's worth a whole lot.
If you're a computer nerd, think of it as tiered storage. On-core registers are way faster than L1 cache, which is faster than L2/3 cache, which is again faster than RAM, which is faster than local SSD storage which is faster than remote network storage. It's not a perfect analogy, because the limits of each tier aren't as clearly defined, and it's highly variable how easy it is to move/copy knowledge across tiers.
Indexing and familiarity matters a lot too. Searching for something where you think it's partway through some video you saw 2 years ago is NOT the same as looking up a reminder in your personal notes a week ago.
[Draft]
Filter Information Harder (TODO: think of a good concept-handle)
Note: Long post usually mean the post is very well thought out and serious, but this comment is not quite there yet.
Many people are too unselective on what they read, causing them to waste time reading low value material[1].
2 Personal Examples: 1. I am reading through rationality: A-Z and there are way too many comments that are just bad, and even the average quality of the top comments may not even be worth it to read, because I can probably spend the time better with reading more EY posts. 2. I sometimes read the curated posts instead of finishing rationality: A-Z first which I also think is a suboptimal order.
Too unselective comes in two ways: Not skipping enough parts of what you're reading, and not choosing hard enough.
Not skipping enough parts of what you're reading
I have the tendency of trying to read everything in a textbook, even if it is quite low in information density, with many filler stories or sentences served as conjunctions. I probably should be trying to skip sentences, paragraphs and sections where I have sufficient confidence of either 1. I have already learned it and don't need a refresher, or 2. They are not important for me (filler material or unimportant knowledge)
Not choosing hard enough
Sometimes there are things that even stop reading halfway is giving it too much attention. there are many things that you should just not read[2]. You can filter a lot of content based on the title, the author, the website which hosts the material, the "genre" of the text (most novels, most social media).
The Best Textbook on Every Subject
Related Reading: Lessons I've Learned From Self Teaching (TODO: make a short summary here about why it is related)
Further Question: When you don't read all parts of textbook / posts, are you eligible[3] to comment on it?
Unsorted Ideas
- There is a few rationality: A-Z posts that I want to link here, which talks about how you would accidentally believe in things you heard from other people.
- Breadth first learning method seem to be better than depth first because it clears the unknown unknowns. Also in general it seems to be better to first have a vague understanding of something and then increase the resolution of the model, using a sort of iterative learning approach.
- This idea can generalize too, but it may start to become useless. First generalizarion would be to filter more on what you listen to, and then to choosing carefully what you do, but then it just becomes rationality.
- During highschool, one heuristic I learned is to ignore basically every question that people have asked because most of them are just stupid, as in thay are either irrelevant or deducible from previous public knowledge (like what the teacher just said)
- ^
This applies to this shortform: think about whether it is worth continue reading, or just close this page and read other stuff
- ^
History: feels like most of history only serves as interesting stories to add in conversations to show that you are knowledgeable, unless you are actively trying to learn psychology/human dynamics. One of my friend wanted to give me a Chinese History book and I really dont think reading that would help me.
- ^
this word is vague, I know
If you could turn this into advice or guidance, it'd be really helpful. Even sharing a metric so we could say "you should be more selective if X, less selective if Y" would be better than a direction with no anchor ("too unselective", no matter what). I don't know if I'm in your target audience, but I'm at least somewhat selective in what I read, and I'm quite willing to stop partway through a {book, article, post, thread} when I find it low-value for me.
Clarifications:
- What I had in mind when I say "people" is myself, and the average non-LW friends around me.
- Worthless is a bad word choice, I just mean that there are better things to read.
Additionally:
I also think I have the tendency of trying to read everything in a textbook, even if it is quite low in information density, with many filler stories or sentences served as conjunctions. I probably should be trying to skip sentences, paragraphs and sections where I have sufficient confidence of either 1. I have already learned it and don't need a refresher, or 2. They are not important for me (filler material or unimportant knowledge)
I will try to make a more quantitative metric, but I don't have one right now, just intuitions.
Thanks, "don't read everything in a textbook" is good practical advice. Learn to skim, and to stop reading any given segment when you cross the time/value threshold. Importantly, learn to NOTICE what value you expect from the next increment of time spent. Getting that meta-skill honed and habitual pays dividends in many many areas.
My comment at the point of time of his reply:
Many people are too unselective on what they read, causing them to spend a lot of time reading worthless material (This applies to this shortform).
I don't necessarily disagree generally, but I do somewhat disagree for myself. Since I don't have visibility into other people's reading habits or selectivity, I'm unsure if I'm an outlier or if I actually do disagree. What does "many people" mean, and more importantly how can an individual (specifically: me) tell if they are too unselective, on what dimensions?
Just read free will, really disappointed.
- not many interesting insights.
- a couple posts on determinism, ok but I already believed it
- some unrelated stuff: causality, thinking without notion of time... these are actually interesting but not needed
- moral consequence of 'no free will': I disregard the notion of moral responsibility
- EY having really strong sense of morality makes everything worse
- low quality discussions: people keep attacking strawmans
You should always include a summary when recommending anything
You are the one who is interested in that thing, the other person isn't (yet). It saves time for the other person to quickly determine whether they want to learn about it or not.
Related: include a tl;dr in posts?
A Chinese company did some AI-assisted reverse engineering on Claude Code and published their findings. After a brief look I don't think it is worth reading for me, but possibly interesting for someone actively working in claude code-like products
[This comment is no longer endorsed by its author]
I think i'm going to unite all my online identities. Starting to get tired of all my wasted efforts that only a single person or two will see.
[This comment is no longer endorsed by its author]
Do you think a united/reused identifier will change who sees which efforts? Or do you mean "I'm going to focus attention where I'm more widely read, and stop posting where I'm not known"?