Theme customizer which allows you to try various looks and settings for your current theme before making those changes live. If your theme support it, you can change colors, backgrounds, and add custom image headers. Updates are planned for this feature.
A Twitter permalink feature now has its own line in the post editor.
Image captions have been updated to allow HTML, such as links, in them.
Improvements have been made to XML-RPC, themes, and custom header APIs, and significant performance improvements in WP_Query and the translation system.
Developer features are detailed at: http://codex.wordpress.org/Version_3.4
More info and download at: http://wordpress.org/