Sorted by New

Wiki Contributions


This is a very good and perfectly clear post! Great job on cleaning cutting to the heart of the matter in a way that is also a good intuition pump.

Sadly, I don’t have the time to properly engage with your arguments. However, I have been finding your recent posts to be of unusually high quality in the direction of truth-seeking and explaining clear thinking. Please keep it up and give Scott Alexander some competition.

Please take this compliment. Despite being old enough to be your parent, I think discussions over beer with you would be a delight.

This is a very good tip and one of Richard Feynman’s better known tricks in physics.

Without doing the math to check, nothing that you said seems wrong. However, I take a very different lesson from the idea of the Lindy Effect than you do. Specifically, the Lindy Effect tells us that when making predictions about future lifetimes of non-perishable things, we should assume a power law distribution. If you've never dealt with predictions under thick-tail assumptions, you might be surprised how little intuition you will have for it. (The 80-20 rule is another example of assuming thick-tails.)

For example, a Pareto distribution (the easiest of the power law distributions) will typically have a mean larger than the median, and the mode (most common single outcome) is essentially instant failure. If the constant of proportionality in the Lindy Effect is greater than one, this implies an infinite variance even with a finite mean. Also, the force of mortality (instantaneous rate of death, aka hazard function) is a decreasing function of time.

The reason that this is typically a good rule of thumb for making future estimates is a consequence of making predictive distributions (i.e. integrating out uncertainty in a fit parameter). As the most familiar example, remember that even if the maker of the Matrix came down and told that a random variable was normally distributed, you will still need to estimate the parameters from observation. This will lead to your mean estimates being distributed as a Student-t, which is a power law.

Let me try to point you in the direction that has been useful for me. I figured getting an answer to you was more important than getting a well-edited reply. Sorry for my logorrhea. If you would like to have a one-on-one conversation let me know.

10 second background: I spent most of my 20s in a good Ph.D. program in Applied Physics. I've spent the last 10 years in the corporate world and devoted a lot of time developing new statistical models. My answer is going to be colored by that past.

You correctly pointed out that having a good network to bounce ideas off of is the best first place to look. But since you came here and I'm the first to answer, that probably isn't enough. I have some tips on how to establish one, but that's a longer term goal and it sounds like you want to get started now. I can elaborate on what worked for me if you want.

My biggest tip came to me from my Ph.D. advisor. You probably aren't reading enough in your field. You should be reading (at least abstracts) a lot. I probably go through about 100 papers a year (maybe 10-20 read deeply), and I'm now mostly just part time research. If you want to do this seriously, you are going to need to know what's going on outside of your own head. First step, get the good text books in the field and ideally read them, but at least know the topics. After that, find out what the important journals are and where you can find the new and exciting papers. The arXiv is where I typically start looking, but google scholar is also a very strong contender for finding relevant papers. This should also get you familiar with who is doing similar work. Some (frequently most) of them will be friendly and approachable people. If you can find a contact of somebody local, hold on to it.

Sometimes it'll take a few weeks of research to find out that the great idea of yours has already been discovered and goes by some specific name. Clearly, start reading about that, but don't be discouraged. A new look at an old idea is frequently worthwhile, especially if you're just starting. It's also possible that you've found something entirely new. Either way, as you learn more, try to actually do something with your idea. e.g. take an easy special case of what you're thinking about and code it / do the math / figure out logical implications. This does a couple of things for you. First, if will tend to tell you if the idea has merit or is just junk. All of us have had it go both ways, don't get discouraged. Secondly, it'll open up more avenues of thought that will likely lead you to other sub-fields to start reading about. Do the reading. But finally, it'll give you something concrete to talk about. If you have an interest result, and here small and easily explained is ideal, write it down somewhere. It doesn't have to be much, a couple of pages is fine. Use Nassim Taleb's arXiv page as an example.

In my experience, doing the first bit of work on the new idea is critical. Once you have some result, and know what's going on with other researchers, you've set yourself up to be taken seriously when you ask for opinions or help. If your method of writing up the proposal is getting seen by somebody in your field, then you're already done and can start getting feedback about the validity of the general idea. If not, my tactic has been to write a short (can't stress this enough, keep it short) email asking an author of a related paper that you have read (I have had the best luck with post-docs and grad students) if they can recommend a good source of a survey of the field. With luck, this will start a conversation where you can ask what they think of your first result. If it isn't dismissed, that will probably open the door to start a conversation about the larger topic you are interested in.

TLDR: Read everything you can find in the area. Develop some small portion of your idea to completion to teach yourself and seed future conversations. Reach out to somebody that you found in your reading.

The typical answer is that this is a result of the Poincaré recurrence theorem

I took the survey and answered every question. As usual, I found my ability to correctly answer the calibration questions comically bad . . . but hopefully well calibrated.

I completed every question on the survey that I could.

I took the survey and answered everything through the political compass.

I took the survey and was annoyed to realize that I didn't have a strong enough background to have informed answers to several questions.

Load More