I'm concerned that this is one of those "I don't see the value in this thing so it must be useless" situations. I'm not a fan of the advertising industry and use ad blockers on principle, but it's a big stretch to assume that the entire sales department is engaged in a useless zero-sum game. If I can be snarky for a minute:
> If we can make a system such that the consumers would choose their products strictly on the basis of how much does the product fit to their needs, and make it easier for consumers to find these products, we can change the incentive system in such a way that the manufacturers would focus more on making better products instead of competing in the non-productive marketing dimension.
I propose that our company hire people whose job is to find customers whose needs are met by our product (or tell the engineering department how our product could better meet their needs), and then inform them of how our products meet their needs. I will call this department "marketing and sales".
I do like your idea of a prediction market for how much I'll like a product though. Having some way to get good things without having to do my own research would be nice.
The amount of money spent worldwide on advertisements is astronomical, reaching about 550 billion dollars annually and is expected to continue growing rapidly. In 2016, the top 20 companies with the biggest annual advertising budget spend between 2.7 to 8.3 billion dollars annually.
Is this a result of a race-to-the-bottom-type market failure? If so, can we solve it?
A nice story I heard was that when the united states acted to ban cigarette advertising, which was an enormous industry, the cigarette companies were actually in favor! The reason for that is that the cost of advertisement is huge and that the main motivation for advertising was to get more market share but that the advertisements did not cause more people to smoke more, so they were stuck in a prisoner's dilemma situation. Sadly, this story seems to be wrong, but it is meant to show that a big share of the money spent on marketing seems to go toward convincing that your product is better then the alternatives. Even if 1% of the marketing cost could be reduced without intrinsically harming the economy, it would still amount to a total of 1 billion dollars which could be spent on R&D, higher salaries etc.
By the way, it is very interesting to see that on average about 10% of revenue goes into sales and marketing, which for some sectors is more then the percentage of revenue that goes into R&D. This is a good indicator that it is not enough to have a good product, or even the best product, if you want people to buy it. More precisely, it means that this is what these companies believe. Perhaps some sectors are in a marketing bubble, and that is not actually the case, as I suspect might be the case in some industries. For example, in the laptop industry a crux that would imply whether or not we are in such a bubble is whether people today choose their laptop solely based on how much this laptop fits their needs (quality, price, size,..) as opposed to buying a computer based on emotional connection to the brand (macbook, dell,..). It would be a good thing, if we are in such a bubble, as it means that it will eventually burst and more money will be spent on improving the products as a way to generate more consumers. An alternative formulation of this process is how much does a company adapt the product to fit the consumer as opposed to influencing the consumers to fit the product.
To solve this market failure, I see two options. Regulations or a change in the incentive system itself. Since I know nothing about regulations, I'll say a few words about how the incentive system can change.
If we can make a system such that the consumers would choose their products strictly on the basis of how much does the product fit to their needs, and make it easier for consumers to find these products, we can change the incentive system in such a way that the manufacturers would focus more on making better products instead of competing in the non-productive marketing dimension.
One thing which could make such a change is a program which makes the choices for the consumer. Consider buying a new phone. If you are like me, you should feel a hint of agony from just imagining the process of going over the endless lists of available phones, reading their specs, comparing prices... An easier way to make such a purchase would be to be presented with some questions and then being presented with the top 2-3 options that satisfy your needs. It could be augmented with a rating system, a reward system for first discoveries of good products, a prediction market on what products would turn out the best (or predicting some other properties), and much more. This kind of technology is probably under development somewhere and it seems likely to be pretty good in a few years. Such a technology, applied broadly, could make the consumers better equipped to buy products which align better with their needs, and by that it would shift the incentives of the companies to be focused more on developing better products which would be discovered by smart algorithms or by brave consumers and would gain a big market share if they are simply better products.
There bound to be many better ideas out there, solving this and related problems. It may also solve only a fraction of the problem as, for example, the above example only works for B2C products, and not for B2B.
The important point is that it might be possible to defeat this market failure by a more careful analysis and better technologies or platforms. I find it an important exercise to seek market failures and to have some ideas about why it is the case and how it may be possible to combat it. This is because the market is a very powerful force, but nevertheless it is possible to shift it in better ways by inventing new platforms or technologies that change the incentive system.