There is a saying “trees do not grow to the sky.” Infinity is a very large number indeed, and it seems quite reasonable to presume that something will stop the exponential growth of the number of cores (and the number of hardware threads) per chip long before infinity is reached. But what could possibly stop this exponential growth?
Battery life can certainly impose limits for mobile devices. Beyond a certain point, extra battery life becomes more valuable than do the extra transistors.
The atomic nature of matter presumably imposes some limitations, as famously noted by Stephen Hawking. But 3D electronics might well push this limit off for some time to come.
But even if the technological limits turn out to permit decades of exponential growth, at some point it is worth asking whether additional cores are really the best use for all those transistors. Beyond a certain point, mightn't addition of special-purpose hardware, for example, for encryption, be more useful? Perhaps the system-on-a-chip concept will migrate from the embedded space to the server space, so that a few tens of cores and a terabyte or two of DRAM makes the perfect server chip. Similar recipes might serve the desktop, laptop, and palmtop.
The limit to the number of cores on a chip is out there, somewhere. Exactly where, no one can yet say.