I think that a lot of the answers here are touching upon aspects of the same thing: feedback and reward. The idea-generating mechanisms in our subconscious respond to rewards. If you want to have lots of ideas about something, try to ensure that it either feels as rewarding as possible intrinsically, or that you also get social rewards for them, or both.

• Eli notes that writing up your thoughts is useful. As he notes, this is useful for iterating them: but a lot of people also find that writing down your ideas, causes you to have even more ideas. I think that a part of this is that when you write the ideas down, it can feel rewarding by itself, and if you also iterate them further and enjoy it, then that's also a form of reward that can be propagated to the initial act of having thought them up.
• Abram's response is that an intellectual community or receptive audience is a big factor. This also matches: if you get to have enjoyable conversations about your ideas, that's rewarding, and it also gives you leads for new ideas. E.g. if someone didn't understand your explanation, it feel rewarding to have an idea of how you could explain it better; even moreso if you do actually succeed in explaining it.
• John suggests reinforcing yourself for ideas regardless of their quality, and getting excited about having produced them. The connection here is presumably obvious. Also, "Most people are too judgemental of their ideas" is an important bit: avoiding negative reinforcement for generating ideas, can be as important as having positive reinforcement. If for each idea you generate you automatically think "but this idea sucks", then that's a bit of a negative valence associated with idea-generation, which can quickly stop you from coming up with anything at all.
• Cousin It, in the comments: "sometimes people end up in some activity because they accidentally had a defining moment of fun with that activity". Yup, if an activity is fun, then you are going to enjoy thinking about it, and will also generate ideas related to it.
• Personally I've noticed that if I've been getting rewarded for having ideas related to something, then my mind will automatically be scanning things for anything that could contribute to it. For example, right now I'm working on several essays by gradually sketching out what I want to say in them; I enjoy this phase, and have liked it whenever I come up with a new idea that I could use in one of the essays. As a consequence, I feel like my mind is very inclined to notice new instances of the things that I'm writing about, and suggest using that in my essay. ("Hey, the way the person in front of you at the grocery store did X, that's an example of the psychological phenomenon you're writing about.") When I write that down, that process gets a bit of a reward and reinforces the act of scanning everything for its usefulness in these particular essays. So it's not just that rewards can reinforce your creativity in general; they can also reinforce your creativity as related to a specific project.

Here's a quote from the author Lawrence Block, writing in his book "Writing the Novel from Plot to Print to Pixel":

... the reception one’s ideas receive has a good deal to do with the development of future ideas.

An example: my longtime friend and colleague, the late Donald E. Westlake, had a period in the mid-1960s when he kept getting ideas for short stories about relationships. (That his own relationships were in an uncertain state at the time may have had something to do with this, but never mind.) He wrote three or four stories, one right after the other, and he sent him to his agent, who admired them greatly and submitted them to markets like Redbook and Cosmopolitan and Playboy and the Saturday Evening Post. All of the editors who saw the stories professed admiration for them, but nobody liked any of them well enough to buy it, and the stories went unpublished.

And Don stopped having ideas. He didn’t regret having written the stories, and he would have been perfectly happy to write more even with no guarantee of success, but the idea factory in his unconscious mind added things up and decided the hell with it. It was clear to Don that, if one or two of those stories had sold, he’d have had ideas for more. But they hadn’t, and he didn’t.

On the other hand, consider Walter Mosley. Shortly after the very successful 1990 publication of his first crime novel, Devil in a Blue Dress, Walter appeared on a panel at a mystery convention in Philadelphia. He announced that he probably didn't belong there, that this book was an anomaly, that it was actually highly unlikely that he'd write any more books within the confines of the genre.

This was certainly not a pose. He very clearly believed what he was saying. Since then, however, he’s written and published a dozen more Easy Rawlins mysteries, three Fearless Jones mysteries, and five Leonid McGill mysteries-along with close to two dozen other books, most of them novels. While a cynic might simply contend that Walter has gone where the money is, I know the man too well to believe commercial considerations outweigh artistic ones for him.

The ongoing success of the Easy Rawlins books have made it almost inevitable that his unconscious would come up with a succession of ideas for additional books. They've been good ideas, engaging their author even as they've engaged an increasing audience of readers, and it would have been a great betrayal of self not to have gone on writing them.

I agree with many of the existing answers, in particular Kaj's, but wanted to point out another factor, which in my own experience, contributes to not publishing many ideas despite having many half-baked ideas.

I think, even among people who have a lot of ideas, where having ideas is defined as having them appear (or be produced) within your conscious awareness, actually formalizing and publishing ideas requires overcoming multiple hurdles.

In this blog post about researcher productivity, the author summarizes a paper by William Shockley, the inventor of the transistor, that posits and tries to explain why researcher productivity levels fit a log-normal distribution. I quote:

Shockley suggest that producing a paper is tantamount to clearing every one of a sequence of hurdles. He specifically lists:

1. ability to think of a good problem

2. ability to work on it

3. ability to recognize a worthwhile result

4. ability to make a decision as to when to stop
and write up the results

5. ability to write adequately

6. ability to profit constructively from criticism

7. determination to submit the paper to a journal

8. persistence in making changes (if necessary as a result of
journal action).

Shockley then posits, what if the odds of a person clearing hurdle #i from the list of 8 above is pi? Then the rate of publishing papers for this individual should be proportional to p1p2p3…p8. This gives the multiplication of random variables needed to explain the lognormal distribution of productivity (Shockley goes on to note that if one person is 50% above average in each of the 8 areas then they will be 2460% more productive than average at the total process).

In my own experience, the ideas to published piece of writing pipeline is similar. In order to go from idea to post, I have to:

• have a good idea;
• write it down;
• block out time to expand upon it;
• (in some cases) find data that supports it;
• survey literature to see if someone's had it or disproven it before;
• (in some cases) write a program or do some math to flesh it out; and
• write something coherent explaining it.

Reinforcement/rewards help individuals summon the extrinsic or intrinsic motivation to persist through these phases. That said, I also think it makes sense for individuals to figure out in which of these phases they typically fail.

In my own life, I've recently been experimenting with lowering my own expectations for my data-gathering, literature survey, and editing phases in order to get more of my ideas down in writing. My recent Babble, Learning, and the Typical Mind Fallacy is an example of my attempts at this. Given its low popularity on LessWrong, I may have bulldozed my way through hurdles I should've still jumped, but it's better than nothing.

I believe for some people it's very important to have a moment of realization that one can get to the frontier of knowledge in a given field of interest. It feels intimidating if others are making contributions that seem decisively out of your league. Because people might intuitively underestimate how far you can get with focused reading and learning, it could be good to give tailored advice to people newer to (e.g.) AI risk for how/where they can make contributions that will feel encouraging. For illustration, a few years ago I was playing a computer game for fun for quite a while until I was by chance matched up with the one of the better competitive players and I almost won against them, getting lucky. That experience showed me that I'd have a shot if I actually tried, and it encouraged me to immediately start practicing with the aim of becoming competitive at that game. It changed my mindset over night. Similarly, I think there's a difference in mindset between "reading and talking about research topics for fun" and "reading and talking about research topics with the intent of seriously contributing".

I agree with others that a rewarding social environment and people in a similar range of competence you can bounce ideas back-and-forth with are extremely important. If you collaborate with people who are similarly driven to figure things out and discuss ideas with you, that automatically forces you think about your ideas for much longer and in more detail. By yourself you might stop thinking about a topic once you reach a roadblock, but if every morning you wake up to new messages by a collaborator adding criticism or new bits to your thinking, you're going to keep working on the topic.

I also suspect that people are sometimes too modest (or in the wrong mindset) to develop the habit of "taking stances". Some people know about a lot of different considerations and can tell you in detail what others have written, but they don't invest effort coming up with their own opinion – presumably because they don't consider themselves to be experts. Some of the community norms about not being overconfident might contribute to this failure mode, but the two things are distinct because people can try practicing taking stances with personal "pre-Aumann opinions", which they are free to largely ignore when deferring to the experts for an all-things-considered judgment.

Speculation about personality traits conducive to generating ideas: OCD was mentioned in the comments. There's also OCPD and hyperfocus. Carl Shulman's advice for researchers among other things mentions something about having a strong emotional reaction to people being wrong on the internet (in communities you care about) – I think this might be a symptom of being very invested in the ideas, and it can help further clarify one's thinking while trying to articulate fervently why something is wrong. Need for closure also seems relevant to me. It has its dangers because it can lead to one-sided thinking. But in me at least I'm often driven by feeling deeply unsatisfied with not having answers to questions that seem strategically important. And, anecdotally, I know some people with low need for closure who I consider to be phenomenal researchers in most important respects, but these people are less creative than I would be with their skills and backgrounds, and their obsessive focus maybe goes into greater width of research rather than zooming in on making progress on the "construction sites". Finally, I strongly agree with John Maxwell's point that a "temporary delusion" for thinking that one's ideas are really good is a great reinforcement mechanism (even though it often leads to embarrassment later on).

Only a partial answer: In my personal experience, writing up whatever thoughts / ideas you have (and even better, sharing them with other people), in some form or another, allows for iteration on what otherwise would have been idle musing.

I think a big contributing factor is having some kind of intellectual community / receptive audience. Having a social context in which new ideas are expected, appreciated, and refined creates the affordance to really think about things.

The way I see it, contact with such a community only needs to happen initially. After that, many people will keep developing ideas on their own.

A school/work setting doesn't seem to count for as much as a less formal voluntary group. It puts thinking in the context of "for work" / "for school", which may even actively discourage developing one's own ideas later.

Also, it seems like attempts to start intellectual groups in order to provide the social context for developing ideas will often fail. People don't know how to start good groups by default, and there is a lot which can go wrong.

Editing to add:

Another important bottleneck is having a mental toolkit for working on hard problems. One reason why people don't go past the first answer which comes to mind is that they don't have any routines to follow which get them past their first thoughts. Even if you're asked to think more about a problem, you'll likely rehearse the same thoughts, and reach the same conclusions, unless you have a strategy for getting new thoughts. Johnswentworth's answer hints at this direction.

The best resource I know of for developing this kind of mental toolkit is Polya's book How to Solve It. He provides a set of questions to ask yourself while problem-solving. At first, these questions may seem like object-level tools to help you get unstuck when you are stuck, which is true. But over time, asking the questions will help you develop a toolkit for thinking about problems.

Certain types of medications can potentially make people more intellectually active.

There is a concept in neuroscience called neuroplasticity. One important fact about neuroplasticity is that it declines with age, particularly it undergoes a sharp decline around or after puberty. This decline is a major reason why people have trouble learning foreign languages after this age. People who learn the language after the neuroplasticity decline are much more likely to speak with an accent, no matter how much effort they put into it. In the last decade there were some major discoveries made in this field. Particularly, the protein that naturally inhibits neuroplasticity in humans with age has been identified. It is called PirB. Its inhibitor is also identified by the same group that published this article. Potentially, a PirB inhibitor would re-enable neuroplasticity and allow people to learn things easily, to form new neuronal connections like they did while they where children. It can bring back a lot of mental freshness and learning abilities that people normally lose with age. This likely can significantly boost person's mental abilities, and make people more intellectually active. I think they are trying to get the FDA approval for the PirB inhibitor to be used as a treatment for amblyopia and Alzheimers. But the plasticity decline itself isn't a disease, so it's unlikely that it will become an officially approved drug for this purpose.

My best guess: There's a difference between reviewing ideas and exploring them.
Reviewing ideas allows you to understand concepts, think about them and talk about them, but you're looking at material you already have. Consider someone preparing a lecture well-they'll make sure that they have no confusion about what they're covering, and write eloquently on the topic at hand.

On the other hand, this is thinking along pre-set pathways. It can be very useful for both learning and teaching, but you aren't likely to discover something new. Exploring ideas, by contrast, is looking at a part of idea space and then seeing what you can find. It's thinking about the implications of things you know, and looking to see if an unexpected result shows up, or simply considering a topic and hoping that something new on the subject occurs to you.

Adding an 'obvious' but important answer that I didn't see, and couldn't find with a ctrl+f: cultivating the environment that lets you have sustained mental focus on what you want to make intellectual progress on. I first called this 'avoiding distraction' and wrote the below:

Where 'distraction' does not mean 'something unexpected suddenly disrupting you', but rather something which is occupying your focus in a sustained way, and which may not be unexpected or random-feeling.

This explains why I have more ideas when I'm e.g laying in bed than when I'm at my computer: the latter has discord, fantasy card games, LessWrong, etc (most things that are done with computers, tbh).

This one has a simple solution, too: avoiding sustained-focus-takers. E.g., lay in bed thinking instead of being at your PC or doing anything else. (I've found sensory deprivation to additionally help when combined with this: keep lights and sound low, and try to find a position where you won't move, to avoid body sensory changes. If you need to get up temporarily, take it very slowly and seamlessly, closing your eyes when you don't need to be seeing.)

Also, what I mean by 'focus' is not really 'object-level attention', as in having something in front of you on your conscious mind, but more something that encompasses your whole state of mind, including cognition/processing happening in the background.

This is probably easier for some than others. If you, like me, have a mind that can easily be pushed from one focus to another, you might need to take more precautions like trying to decrease non-alignment stimuli in general, e.g trying to minimize social stimuli, so when you're laying and thinking your mind defaults to alignment instead of other things.

Now if you just read the above, maybe you will think, this 'quila' user has really optimized their process for entering an intellectually active mental state, wow! But actually, I don't even do this every day, but I should, and writing this might help self-fulfilling-prophecy that outcome. posts comment and turns off pc

Edit: did this for an hour and had a new idea (it was vulnerable to Probable Environment Hacking, but gave me some insight into the nature of embedded algorithms). It definitely works for my mind.

As Kaj pointed out, most of the answers so far focus on feedback and reward. As an answer, that feels correct, but incomplete. I know so many people who are clearly very smart, surrounded by friends who give them positive feedback on whatever they're doing, but it doesn't end up channeling into intellectual development. If every intellectually-active person were linked to an idea-focused community, then the feedback answer would make sense, but I doubt that's the case. So what's missing?

I don't have a complete answer, but I remember a quote (maybe from Feynman?) about keeping a stock of unsolved problems in your head. Whenever you learn some new trick or method, you try applying it to one of those unsolved problems. At least for me, that's mostly how my "sprawling intellectual framework" develops. Some of them are open technical problems, others are deficits in my current social or economic models of the world. This feels connected to what Martin talks about - some people notice holes in their understanding and then keep an eye out for solutions. You hear something that doesn't sound right, doesn't quite make sense, and you reflexively start digging. Maybe you find an answer quickly, otherwise you carry the problem around in the back of your head.

I don't know why some people do this and others don't, but as a causal factor, it feels orthogonal to social feedback. It still feels like I don't have all the puzzle pieces, though. This question will continue to sit in the back of my head.

I think the main factor would be a sense of confidence that the person has a genuine chance of contributing meaningful ideas to the subject. If you think that others are already miles ahead of you, then a combination of the bystander effect and Dunning Kruger effect may stop you from ever trying. After all, there is social risk for presenting ideas that you expect to be seen as foolish.

In my experience, the things that make humans intellectually-active-in-the-sense-of-developing-extensive-frameworks-of-ideas are [ while much grander in the realization ] basically isomorphic to the things that make zoo lions [ /parrots/gorillas/elephants/whatever ] express a rich behavioral repertoire that is fun [ other than maybe in some kind of sadistic capacity ] to watch.

Reading the books and correspondence of Darwin, Newton, and Einstein makes it clear that they were all set up to feel, from the time they were 20 or so, like their social-intellectual lives had high stakes -- like they had or at least potentially had important friends [ and enemies! ] who were likely to care about their opinions and could influence the future of the world.

Another thing they had in common is that they all felt guaranteed or entitled to an amount of slack or career leisure rate that most of today's young people would give body parts for, from the time they were very young.

In other words, they expected an affluent society, which was small and cared about them -- an environment like the ones humans evolved to think in. They expected an environment in which a human could safely express all their natural instincts.

In my experience, that's really it. It's really rare to find that in modernity, and rarer to see it collide by chance with the level of natural awareness-of-opportunity and level of intellectual strategic thinking that means fields change in response to you.

Kaj Solata touches on "maybe people need to expect that they will be repaid for what they do, and not punished" in his answer about feedback and reward, but I don't think the case he looks at [ the in-retrospect success of a single type of novel, leading to more of that same type of novel ] is quite as large-scale as the case you [ Demski ] are asking about, and think that's because Westlake was experiencing small, isolated reward that led to cautious, narrow "intellectual productivity" -- not the kind of rich, ongoing "reward" that leads to great intellectual productivity.

Lack of a 'recognition' state increases ambiguity in goal directed tasks i.e. no clear picture of what an organized cupboard looks like generates friction in beginning the task due to lack of a feedback mechanism. In contrast, a clear internal picture generates lots of intermediate states to compare to.

People able to work on X have reasonably good proxy measures so that they can feel their way through the ambiguous parts of problems.

Just an opinion: ideas do not come from nothing, so the larger the data pool (memories, experiences, interests) the more ideas are likely to be generated.

It very much seems like we live in an age of hyperspecialization; people know very much about relatively few things. Generally, these areas of knowledge are complimentary or related. Sometimes they overlap outright. Life is barely long enough to get good at one thing, so people often choose to specialize early and stay on one very fixed path.

From the outside looking in these collectives of experience are very tribal. They develop their own languages and symbols. They become these closed systems where ideas aren't created as much as they are simply refined; bounced back and forth among tribal members. But this does not seem to be a very good pattern for long term growth or sustainability. Homogenization leads to extinction.

What I mean by that can be understood by looking at the evolution of life on Earth as an example of the obverse. Evolution tends towards diversity. Diversity gives life its best possible chance of success. That way, when an asteroid slams into the planet, not everything dies. If evolution had tended towards homogenization (only making the best dinosaurs possible) instead of diversity, the K-T Event might have turned this planet into a floating rock.

It may be a bit of a weak analogy, but I feel like the same principles might apply fairly well to specific areas of knowledge. Ideas are the mutations that allow knowledge to change and evolve into something new. Exclusivity and specialization are a sort of homogenization that leads to stagnation and fewer truly new & good ideas. Not that ideas don't happen at all, just that maybe they happen less often than they should... or could. I don't know, really. This is mostly just speculation based on personal observation and opinion.

Colloquially, I can say that the people I have known in my life who seem to have the most ideas are the ones whose interests are all over the map, so to speak. They tend to be older, with a deeper well of experience to draw from. Their knowledge pools, being varied as opposed to complimentary, allow them to look outside these otherwise closed systems and make inferences, or to see patterns that people too mired within the subject matter might easily miss.

They may not always be good ideas, but they are often striking in their seeming originality and unexpectedness.

An example that comes to mind is of a family friend who worked for years in automotive manufacturing before going back to school to get his certification as a laboratory technician. He got a job as a lab assistant at a University research hospital. He would overhear the researchers in the break room talking about their current projects, and one of them that really grabbed his interest was the problem of infectious disease control measures, specifically, getting healthcare professionals to wash their hands between patient interactions. He had the idea, based on his experience in manufacturing, to apply Poka Yoke (a Japanese manufacturing term that roughly means error-proofing) to the problem of getting nurses and doctors to wash their hands between patient encounters. His idea was to install sink-locks at all the doors to patient rooms. These doors would only open from the outside if the sink was used for at least 20 seconds immediately prior to opening them, or if an emergency button was pushed. From the inside they open at will. He mentioned the idea in casual conversation with one of the senior researchers who was so excited by it that he wanted to design a study around the concept.

I feel like there is a potential benefit to be had by looking outside as opposed to focusing too intently within. Maybe spending some percentage of time learning about completely new things as opposed to only endeavoring to learn new details about things we already know might yield an increase in new ideas. There's nothing wrong with getting out of our comfort zone and challenging our perspectives.

The main difference is having a thorough map of the territory. The stage before having worthwhile ideas is going on a mapping exercise - finding out what is already known about a topic, and learning what existing workers in the field are able to do, and how they understand it. As you learn how the different aspects of the field are connected, it’s possible to start having your own useful ideas - and most of the time you’ll find your idea is already known and part of the field. But as you continue to map and explore, you may come across ideas that don’t seem to be in the literature. And of course this may be an original idea. There is a big difference between being intellectually curious enough to know something about a field, and having dug through that field sufficiently to have a decent map of it. Conversely there are often crackpot ideas in every field - the easiest test for them is often to ask the originator of the idea to explain the current mainstream theory. If they can’t, then you can save your time on their alternative. For me one motivator is this - there is no difference between having an idea that turns out to be known already, and having an original idea. It’s the same thing, except for having not yet reached the frontier of the known world. It was my idea, and it was original once….. And it’s much more likely to be correct than an idea that nobody else is talking about yet - it’s a sign of being on the right track.

Opportunity is the key. Put it into context, what is the difference between, a rockstar who goes all round the world making music and spreading ideas through their words, and someone who buys their records, thinks about the music, tries their level best to play music and write songs that are of equal importance to the rockstars music, and yet their ideas arent able to get off the ground? It isnt for lack of trying. Someone once told me that undiscovered genius does not exist and never has. He was a very eminent professor of philosophy, yet I didnt believe him. Maybe you think the world is a meritocracy of ideas? I do not think it is. In a world where those with the good ideas get the most rewards there is a lot of competition. Maybe some kid with a guitar tries to be famous but never meets the right contacts, never plays at the right venues, lacks a certain amount of support from those around him, yet his ideas are astounding and equal to those of any rockstar, and gives up after trying but getting nowhere.

If you mean something like this place then there is also discipline. I would not dare submit an idea here yet. Others seem so much more skilled at formulating ideas and yet I enjoy reading their ideas and hate writing my own down, I believe I am probably not as well educated as most people here and my ideas may not be as intellectually stimulating as those I read. It may be a false belief but it is, anyway a belief that stops me creating ideas that spread and instead enjoy those who are good at it

Another thought I have is that those who are good at thinking are usually only the best in one particular area. Correct me if I am wrong but Wittgenstein is not usually noted for his ability to think about structural engineering. David Bowie was not noteworthy because of his ability to think about thermodynamics. People are reluctant to speak on subjects upon which they are not experts. As Wittgenstein once said "Whereof one cannot speak, thereof one must be silent"