There's going to be an announcement on Feb 20th, so more details to come.
From source, currently known stuff:

Controller
Redesigned, similar size to existing DualShock, but has a touchpad which replaces select/start/PS button.
There will also be a Share button, so that they can join with all this social craze stuff.

Specs

• AMD 8 core (4 module?) 1.6GHz 'Liverpool' APU (Steamroller arch?)
  
• 4GB GDDR5 (Sony is going to try to match the next Xbox's 8GB)
  
• AMD GPU, quoted as "R10XX architecture", probably based off a RadeonHD 7/8xxx (it sounds like a discrete GPU as opposed to the integrated one)
  

Interesting that both the PS4 and next Xbox (note that the next Xbox is expected to have similar specifications) will be using x86 CPUs and possibly commodity GPUs.  I wonder whether this will be leveraged in some way.
I'd say that it's unlikely the PS4 will run PS3 games, unless they include hardware to specifically do this.


Expected to be released around Christmas this year in Japan/US, and probably later elsewhere.