I couldn't tell you how many times I've seen a great idea killed because the project leader or someone else with veto power didn't want someone else's idea or someone else's area of expertise to appear important. I've been (unofficially) fired from two jobs because I refused when my bosses flat-out told me to stop proposing solutions for the important problems, because that was their territory.
You fell afoul of Putt's First Law of Decision Making: Managers make decisions.
...Successful managers know they are measured on their ability to make decisions and, whether right or wrong, take action. Thus, they readily grasp the significance of the First Law of Decision Making:
Managers make decisions.
Nonmanagers, by contrast, seldom concern themselves with this law. They may, therefore, commit the fatal error of making all decisions themselves. This is particularly true in technical hierarchies, where low-level technologists continually make decisions in their areas of expertise. Finding so few issues that the manager is qualified to decide, they are likely to forget to bring any issues to him at all. This was one of Roger Proofsworthy's many problems at the Ultima Corporation. He perfor
Someone who just left our company on Friday worked the magic of cranking out good research publications while working as a programmer, always taking on only projects that had good publication potential and never getting stuck with the horrible life-sucking, year-sucking drudgery tasks of, say, converting application X from using database Y to database Z. I just don't know how he did it.
Can you ask him?
I've asked him twice, but didn't get a clear answer beyond "Get out of dead-end departments that aren't intellectually sexy."
So, what I have noticed is that professors primarily work as salespeople (writing grant proposals), managers (as principal investigators), and educators (teaching classes), but they typically have formal training for zero of those three tasks. A number of professors I know have commented that they were happier as postdocs (read: spending lots of time in the lab) than they are now as professors, and I can't help but think that there are gains from specialization to had here. There's some overlap with the problems you describe, but those are the things I would emphasize to get support from scientists in the system when pushing institutional change.
What I've learned after years in the field is that what most scientists really want to do is prove how smart they are, and that this is a big reason why science is so unproductive today.
Implied premise here is that science is unproductive today. In physics we've found the Higgs boson. In math, we've proven the fundamental lemma of the Langlands program). In astronomy, we've found multiplanet star systems. In quantum computing, we've made major progress in practical implementation of factoring algorithms. This is all in the last three years and is essentially off the top of my head. Given that, the claim that science is "so unproductive" today seems at minimum to be a claim which shouldn't be made without some evidence to support it.
On a per-dollar basis, science today is many orders of magnitude less productive than it was a century ago. I have a paper in draft I can email you.
Frankly, a draft of that paper would be far more interesting than this post itself. I'm curious what your metric is. Denominators is dollars, and numerator is what?
except at the NSA
If you work at the NSA, aren't you supposed to pretend afterwards that you didn't? Explain the hole in your CV some other way? Or is that no longer the custom?
That was years ago. I think that changed in the eighties. You can't list classified publications on your CV, or explain what you did, which is still a problem, and a big reason why I left 20 years ago. Ironically, I probably would have more unclassified publications by now if I'd stayed.
Someone who just left our company on Friday worked the magic of cranking out good research publications while working as a programmer, always taking on only projects that had good publication potential and never getting stuck with the horrible life-sucking, year-sucking drudgery tasks of, say, converting application X from using database Y to database Z.
This seems like a really important question that a lot of people might benefit from knowing the answer to. How does one "manage upwards" and ensure that one gets put on the interesting, rewarding projects as opposed to the tedious stuff?
yet metronidazole is never prescribed for Lyme disease while the latter are.
Making a note of this; Lyme disease is common enough that this would be useful to remember. (Or, because I'm pretty sure I'll forget the specifics, to remember that I can search my LW comment history for it.) Thanks!
Here's an interesting test of various antibiotics: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3132871/
It looks like different agents are more or less effective for different forms of the disease. This study didn't look at Ceftriaxone as far as I can tell, which is what is usually used for later stage Lyme disease. Doxycycline is typically used for prevention, which makes sense because you're probably less likely to have cysts forming immediately after infection. It doesn't look like Metronidazole is as effective against non-cyst form Lyme disease.
Doxycycline treatment reduced spirochetes by ∼94%
Metronidazole treatment reduced spirochetes by ∼54%
and
Tinidazole treatment reduced round bodies by ∼94%
Metronidazole treatment reduced round bodies by ∼68%
Looks like Tinidazole might be even better than Metronidazole vs. cyst-form Lyme disease.
This all shows the huge advantage to math types interested in academics of becoming an economist over a scientist. We almost never do postdocs, can often do quality research without grant money, and don't need to publish with senior people if we don't want to. Plus, I believe the academic job market for economists is pretty good right now.
The author hasn't posed any scientific problems. Instead, they have made sweeping generalizations based off of their bad experiences in one field.
This referee cannot recommend the article for publication.
Can't tell if serious or ironically humorous. But the author's experience includes years of work in each of these fields: linguistics, biology, cryptanalysis, air traffic management, artificial intelligence, and animation. And the author described five specific problems with science. If you don't understand that the point of the article is that those five problems are the important scientific problems facing us now, then you missed the point entirely. If you're complaining that you want "scientific problems" instead of problems with how science is done, well, that's not my job here. I'm identifying the problem, not writing a grant proposal.
Good article. I would suggest not leading off with the current first two points (Egos + No-men), as I think the extent to which each of these is true varies between labs / disciplines, whereas the other three points seem to be more or less universal throughout science.
(I have personally been lucky enough to never run into the Ego / No-men problem with any of my advisors; for a while I thought everyone who complained about these things was being melodramatic, but as some point a critical mass of highly reasonable people made such claims that I now believe t...
The big problems facing science are management problems. We don't know how to identify important areas of study, or >people who can do good science, or good and important results.
Bostrom "Predictions from philosophy" makes similar advice, but not specific to scientists. In both cases the solution is focus cognitive resources on strategic analysis, I suppose. However, is really dificult to implement this on a large scale without hurting egos.
One question:
Do scientists know they need it? That's a big question, so I'll refine it:
Have you yet met any scientists who have expressed interest being more effectively managed?
Search is very important and my experiences have been similar to what you described.
But I've often found myself wondering: what's the solution?
I would love nothing more than to always be on top of current discoveries in my field. But it just seems impossible to do.
Google and similar tools are great but they only do part of the job. You have to know what to search for, and it is very easy to get caught up in your own biases and only search for ideas, keywords, etc. that you are most familiar with or currently most interested in, thereby easily missing many ...
.I couldn't tell you how many times I've seen a great idea killed because the project leader or someone else with veto power didn't want someone else's idea or someone else's area of expertise to appear important.
The TL;DR of why I stopped quit my university's atheist club.
What do you think about David Brin's "disputation arenas?"
Maybe we could get a group of scientists to try out some form of disputation arena (Delphi Method for example) and see if they can be more effectively managed that way?
We already have many times more brilliant scientists than we can fund.
Could you give some evidence for this assertion? My feeling from the "inside" is quite the reverse -- at least in the States, NSF funding has remained stagnant, and grant allocation is widely considered a "black box" with high uncertainty in outcomes. I've visited three other countries in the past two years, and the feeling there seemed much the same.
So in short, scientists should do science for profit at startups, and they'll become massively rich so this solution should enact itself.
Science does not need more scientists. It doesn't even need you, brilliant as you are. We already have many times more brilliant scientists than we can fund. Science could use a better understanding of the scientific method, but improving how individuals do science would not address most of the problems I've seen.
The big problems facing science are organizational problems. We don't know how to identify important areas of study, or people who can do good science, or good and important results. We don't know how to run a project in a way that makes correct results likely. Improving the quality of each person on the project is not the answer. The problem is the system. We have organizations and systems that take groups of brilliant scientists, and motivate them to produce garbage.
I haven't got it all figured out, but here are some of the most-important problems in science. I'd like to turn this into a front-page post eventually, but now I'm going to post it to discussion, and ask you to add new important problems in the comments.
A lot of LWers think they want to advance scientific understanding. But I've learned after years in the field that what most scientists want even more is prove how smart they are.
I couldn't tell you how many times I've seen a great idea killed because the project leader or someone else with veto power didn't want someone else's idea or someone else's area of expertise to appear important. I've been "let go" from two jobs because I refused when my bosses flat-out told me to stop proposing solutions for the important problems, because that was their territory.
I don't mean that you should try to stop people from acting that way. People act that way. I mean you should admit that people act that way, and structure contracts, projects, and rewards so that these petty ego-boosts aren't the biggest rewards people can hope to get.
The more people your project has who can say "no", the worse the results will be. This is one reason why Hollywood feature films are stupid, why start-ups do good work, and why scientific projects are so often a waste of money. Good ideas are inherently unpopular. Most of the projects that I've worked on have been crippled because every good idea ran into someone with veto power who didn't want to do things differently, or didn't want somebody else to get credit for solving the problem. See "Egos".
Saying "no" to bad projects is important, but once the project is underway, there is a bias to say "no" more than "yes", even after adjusting for the number of times you can say "yes" in total. Requiring consensus is especially pernicious. You can't get good results when everbody on the project has to say "yes" to new ideas.
Team members often disagree about whose expertise particular decisions fall under. Most people see how their expertise applies to a problem more easily than they can see how someone else's expertise applies to a problem. What usually happens is that territorial claims are honored from the top of the org chart on down, and by seniority. For example, I worked for a computer game company where the founder hired a scriptwriter, then came up with his own story ideas and told the scriptwriter to implement them. The implementation had no text; the scriptwriter took the story ideas and produced descriptions of scenes acted out with body language. The animators thought that body motion fell completely within their jurisdiction, so they felt free to rework whatever they saw differently. The scriptwriter had very little chance for creative input, no control over anything, and very little job satisfaction.
This is a common problem for computer scientists and mathematicians. Computer scientists and mathematicians see themselves as people who understand how to most-effectively take a set of data, and arrive at the desired results. This includes figuring out what data to look at, and in the best case, means being involved in the proposal writing to look at possible problems to address, and determine which problems are soluble and which ones are not based on information theory. This never happens. People in other specialties see computer scientists as a kind of lab technician to bring on after they've figured out what problem to address, and what data and general algorithm to use. They see statisticians as people to consult when the project is done and they're writing up the results. They aren't even aware that these other disciplines can do more than that.
A classic example is the Human Genome Project. Some people you never hear about, including my current boss, came up with algorithms to take whole-genome shotgun data and assemble it. Craig Venter went to the leaders of the Human Genome Project and explained to them that, using this approach, they could finish the project at a fraction of the cost. Anybody with a little mathematical expertise could look at the numbers and figure out on the back of a napkin that, yes, this could work. But all the decision-makers on the HGP were biologists. I presume that they didn't understand the math, and didn't believe that mathematicians could have useful insights into biological problems. So they declared it impossible—not difficult, but theoretically impossible—and plowed ahead, while Craig split off to use the shotgun approach. Billions of taxpayer dollars were wasted because a few people in leadership positions could not recognize that a problem in biology had a mathematical aspect.
"Thou shalt not muzzle the ox when he treadeth out the corn." — Deuteronomy 25:4
I believe that a large number of the problems with scientific research are tolerated only because nothing is at stake financially. Government agencies have tried very hard to ensure that people do work for their contracts. You have to say in the proposal what you're going to do, and itemize all your costs, and do what you said you would do, and write reports once a month or once a quarter showing that you're doing what you said you would do. This results in unfortunate obvious stupidities. We can spend $30,000 to have an employee write a piece of software that we could have bought for $500, or to solve a problem that a consultant could have solved for $500, but we can't buy the software or hire the consultant because they aren't listed in the contract and the employee is.
But the bigger problem is that the strict financial structure of scientific research makes it illegal to motivate scientists by giving them a percentage of resulting profits. You simply can't write up a budget proposal that way. So managers and team members indulge their prejudices and fantasies because the little bit of self-esteem boost they get from clinging to their favorite ideas is worth more to them than the extra money they would earn (zero) if the project produced better results. Examples of petty prejudices that I've seen people wreck good work to preserve: top-down over bottom-up design, emacs over vim (I was in a shop once where the founders forbade people from using vim, which had an astonishingly destructive effect on morale), rule-based over statistical grammars, symbolic logic over neural networks, linguistics expertise as more important than mathematical expertise, biological expertise as more important than mathematical expertise, and, always, human opinions gathered from a few hundred examples as more valid than statistical tests performed on millions of samples.
When I read about machine learning techniques being applied in the real world, half the time it's by trading firms. I haven't worked for one, so I don't know; but I would bet they are a lot more receptive to new ideas because, unlike scientists, they care about the results more than about their egos. Or at least, an appreciable fraction as much as they care about their egos.
Everybody in science relies on two metrics to decide who to hire and who to give grants to: What their recent publications are, and what school they went to. It is possible to go to a non-top-ranked school and then get on important projects and get publication credits. Someone who just left our company on Friday worked the magic of cranking out good research publications while working as a programmer, always taking on only projects that had good publication potential and never getting stuck with the horrible life-sucking, year-sucking drudgery tasks of, say, converting application X from using database Y to database Z. I just don't know how he did it.
For the most part, that doesn't happen. You don't become a researcher; you start out as a researcher. You need to stay in school, or stay on as a postdoc, until you have your own track record publications and have won your own grant. You need people to read those publications. You don't get to work on important projects and get your work read and get a grant because you're brilliant. You get these things because your advisor works the old-boy network for you. Whatever your field is, there is a network of universities that are recognized as leaders in that field, and you are more-or-less assured of failure in your career (especially in academia or research) unless you go to one of those universities, because you won't get published in good journals, you won't get read much, and you won't get a big grant.
There are exceptions. Fiction writers and computer programmers don't need to go to a fancy university; they need credits and experience. (Computer programmers. But don't get a Ph.D. in computer science from a non-elite university and imagine you're going to do research; it won't happen.) Good stories can sort of be recognized; basic knowledge about Enterprise Java can be measured. Companies have recognized the monetary value of doing so. But grant review panels and companies don't really know how to rate scientists or managers, so they try to get somebody from MIT or from Wharton, because nobody ever got fired for buying a Xerox.
The value of scientists to their companies may or may not be reflected in their salaries, but the value of those select universities is certainly reflected in the price of tuition. If your college of choice costs you less than $55,000/yr to attend, including room and board, it will not lead you to success. Unfortunately, the U.S. government won't loan you more than $10,000/yr for tuition.
(One interesting exception is in cosmology. I did a study of successful physicists, as measured by their winning the Nobel or being on the faculty at Harvard. I found that after 1970, no one was successful in physics unless they went to an elite undergraduate college, with a few exceptions. The exceptions were astrophysicists who went to college in Arizona or Hawaii, where there are inexpensive colleges that are recognized as leading institutions in astronomy because they have big telescopes.)
The single-biggest problem with science today is finding relevant results. I have had numerous discussions with experts in a field who were unaware of recent (and not-so-recent) important results in their field because they relied on word-of-mouth and a small set of authoritative journals, while I spent half an hour with Google before our meeting. To take a spectacularly bad example, the literature showing that metronidazole kills Borrelia burgdorferi cysts, while penicillin, doxycycline, amoxicillin, and ceftriaxone do not, is over ten years old; yet metronidazole is never prescribed for Lyme disease while the latter are.
Attention is the most-valuable resource in the twenty-first century. Producing a significant result is not hard. Getting people to pay attention to it is. Scientometric analysis of scientific publications shows that producing more and more papers in a field has very little impact on the number of papers cited (a proxy for number of results used), probably because scientists basically read up to one paper per day chosen from one or two leading journals, and that's it. They aren't in the habit of regularly, actively searching for things relevant to their work; and frankly, there isn't much motivation to do that, since using Google to answer a specific question is like using excavation equipment to search for a needle in a haystack.