Noting that a 2012 Core i7 3770K is only a tad faster than 2x a 2006 Core 2 Q6600, it seems that speed increases are slowing down (becoming more difficult to achieve perhaps).
Although Moore's Law still seems to be operating, speedups don't seem to be proportional.  I decided to plot some speeds of Intel x86 CPUs taken from this wiki page:



I've kept to single core and quad core chips to make comparisons a bit easier, and snuck in the current consumer-grade top-of-the-line 6-core i7 3960X to give a feel of where the top CPU is at.

Wee've pretty much hit the "GHz wall" years ago, and have pretty much picked all the low hanging fruit in terms of instructions-per-clock optimisations (As far as I know).  Cache takes up like around 80% of modern CPU die space.  22nm is like 50? atoms wide.  Multicore has its limitations in the amount of speedup that can be achieved.
How much faster can wee get?