Steveot

Maybe you're right and it's climate, perhaps I also meant something unusual with "stale" or it could also be a cultural difference between US and europe? I'm talking about bread like this (https://www.lazycatkitchen.com/sourdough-rye-bread-beginner-friendly/), stored in a bread box and perhaps wrapped in a cotton dish towel. Of course it does get a bit harder over time, but I can usually still eat it like completely normal bread even after 7 days. And note that I agree that pure wheat sourdough bread does get stale more quickly.

This comment seems to imply Nisan missed something, but normal rye sourdough bread without any preservatives easily lasts (edit: should have said "can easily last under the right circumstances") 7 days before going stale. Of course people can mean different things by "real old fashioned bread" but afaik sourdough bread was the standard method for most of human history.

In your first argument, it seems to me slightly like you are arguing against virtue based ethics under the assumption that consequentialism is true. So in your argument, the only real value may arise from good consequences (however those are defined), while for virtue based ethics (if I understand correctly) the value would arise from truly acting virtuously (whatever that means). In my mind, neither can really be true (it seems like a choice). However, framing it like this would allow for something like a reverse of your argument within the framework of v... (read more)

Thanks, I really like these concepts, especially Grice's maxims were new to me and they seem very useful. Your list also got me thinking and I feel like I also have some (obvious) concepts in mind which I often usefully apply but which may not be so well known:

1. Data-processing inequality
2. Public good games
3. Evolutionarily stable equilibrium
4. Don't be results oriented (when acting in stochastic environments)

The data-processing inequality is often useful, especially when thinking about automated tools, like LLMs. It states that for any fixed channel K, the mutual i... (read more)

Tangential comment, but one where I'd be interested in how people in this community feel: When you wrote about the part with the meetup and sign saying "I might be lying", I immediately thought how little fun it must have been, and even how badly it might have felt for others attending. In my mind, people attending a meetup don't want to be lied to, even if you semi-communicated it (I say "semi" because the statement on the sign was trivially true for every person. You did not clearly state you were definitely going to lie about certain things) and in the context of a "social experiment". To me it seems quite similar to people wearing signs saying "I might be rude" and then actually being rude.

Another intuition I often found useful: KL-divergence behaves more like the square of a metric than a metric.

The clearest indicator of this is that KL-divergence satisfies a kind of Pythagorean theorem established in a paper by Csiszár (1975), see https://www.jstor.org/stable/2959270#metadata_info_tab_contents . The intuition is exactly the same as for the euclidean case: If we project a point A onto a convex set S (say the projection is B), and if C is another point in the set S, then the standard Pythagorean theorem would tell us that the angle of the tr... (read more)

It's not a mathematical argument, but here I first came across such an analogy drawn between training of neural networks and evolution, and a potential interpretation of what it means in terms of sample-(in)efficiency.

I thought about Agency Q4 (counterargument to Pearl) recently, but couldn't come up with anything convincing. Does anyone have a strong view/argument here?

I like the idea a lot.

However, I really need simple systems in my work routine. Things like "hitting a stopwatch, dividing by three, and carrying over previous rest time" already feels like it's a lot. Even though it's just a few seconds, I prefer if these systems take as little energy as possible to maintain.

What I thought was using a simple shell script: Just start it at the beginning of work, and hit a random key whenever I switch from work to rest or vice versa. It automatically keeps track of my break times.

I don't have Linux at home, but what I tried... (read more)

Thanks, I finally got it. What I just now fully understood is that the final inequality holds with high ${\pi }_0^n$ probability (i.e., as you say, ${\pi }_0$ is the data), while the learning bound or loss reduction is given for $\pi$. 

Thanks, I was wondering what people referred to when mentioning PAC-Bayes bounds. I am still a bit confused. Could you explain how $L\left(\pi \right)$ and $\hat{L}\left(\pi \right)$ depend on ${\pi }_0$  (if they do) and how to interpret the final inequality in this light? Particularly I am wondering because the bound seems to be best when $\pi ={\pi }_0$. Minor comment: I think $n=m$?

The main thing that caught my attention was that random variables are often assumed to be independent. I am not sure if it is already included, but if one wants to allow for adding, multiplying, taking mixtures etc of random variables that are not independent, one way to do it is via copulas. For sampling based methods, working with copulas is a way of incorporating a moderate variety of possible dependence structures with little additional computational cost. 

The basic idea is to take a given dependence structure of some tractable multivariate random... (read more)