The old NDS had two screens, one of them a touchscreen.  Something the PSP never had, and some games using the stylus were kinda interesting.  Also smaller form factor, and I think a bit cheaper.
New PSP adds the touchscreen, so somewhat patches up that flaw.  3DS adds 3D, which, IMO, is a complete gimmick, just like the lack of 3D TVs being sold etc.  It's probably a bit better, but since its aim is to improve realism, I'm not sure how far it can go on such a small screen.
3DS still has smaller form factor as an advantage.  Also a 3D camera which doesn't seem to be widely sold yet, much less amongst AIO devices.

I do wonder about dual cameras though - for the PSP2 - to make video calls or put your face in some virtual chat?  Why two cameras and not one which can be turned around?  I'd personally rather better screen than a second camera.