To repeat the context, I presented “Is Parallel Programming Hard, And, If So, Why?” at the Multicore and Parallel Computing Miniconf, and had the good fortune of having Linus in the audience. As you will quickly learn if you post to LKML, Linus often gives very insightful (and sometimes quite pointed) feedback. My presentation was no exception.
Linus noted that many parallel programming experts are unwilling to admit that there are algorithms that do not parallelize nicely.
As you might expect, I responded by saying that parallelism is an optimization, and like other optimizations has places where it works well and places where it does not.
What else could I have said?
One additional point is that parallelism is but one potential optimization of many, and therefore people should apply parallelism to those algorithms that parallelize nicely. There is a large body of sequential optimizations that might apply to other algorithms. Some related advicee is to recast your algorithm into an “embarrassingly parallel” form.
Another point is that the sharply decreasing cost of multicore systems (to say nothing of their increasing numbers of CPUs) is making parallelism more attractive. It is therefore worthwhile putting significant time and effort into parallelism. Of course, putting additional time and effort into sequential optimizations might also be valuable. That said, given that the effort put into sequential optimizations has long overshadowed that for parallel optimizations, it seems likely that, all else being equal, effort invested in parallel optimizations will yield greater benefits.
In addition, parallel tooling is in much need of improvement. Better parallel tools should make parallelism more attractive.
Finally, because the Linux kernel is a low-level layer of software with a great many users,, it makes sense to invest quite a bit effort on performance and scalability in the Linux kernel. In contrast, the further you move up the software stack, the more productivity concerns outweigh performance and generality concerns.
But how would you have responded?