Different tools for different purposes, I'd say. But I want to answer with an objective - well, sort of - and a personal aspect.
Objectively, it seems useful to assume that the sensor is the heart of a camera, all else is secondary. The D4 sensor is very good, but both the d800(e) and the d600(10) are graded higher on DxO. Plenty more pixels, of course, but also a good stop more dynamic range - and high ISO performance is also not bad at all, particularly if you consider that you may downsize them to the same size as the D4.
Personally spoken, for me far more important, I think a photographer has to see, and know, her or his preferences. Resolution and detail matter for me, as I come from - and still use - analog view cameras, as does dynamic range - I don't think there can ever be enough, and digital still has a long way to go to meet monochrome film. I also very much like night photography, but whenever possible, I use a tripod, and even otherwise the D600 and D800 still give excellent results.
Frankly, if I wanted to buy a 16 MP camera, I would go for the Fuji xpro1: also very low noise on high ISOs, a sensor which doe not need anti-aliasing and still does not produce moire, excellent lenses available.
Also, even leaving the price difference out of consideration, the Fuji is very much lighter, looks unassuming and not like a DLSR on steroids.
That's my view, others may differ. However, I still hope this helps.
Lukas