In most cases the dexer step is the slowest one during compilation. Giving the dexer enough memory makes it much faster.
The only other step that can be slow is the deployment itself however there is nothing that can be done about it, except of using debug (rapid) mode which doesn't always require full deployments.
Compilation speed is at least two times faster with JDK x64 with my configuration, also you can add over 2GB ram to dexer (4GB in my case) with 64bit JDK.