@AHilton :
I worked years with Dell PowerEdge servers, they are a dream to work with and extremely fast.
But indeed, if you start counting what comes next to your server it goes up quite fast (datacenter rack & power cost, provider cost, maybe license costs, ...)
You're better of with a VPS with good virtual specs.
re: PowerEdge server ... I agree. And rock-solid machines that just keep on running. They're easy to work on/repair and parts are (still) quite easy to find (although pricey sometimes). The only downside is that they are power-hungry. If people look around, especially students building a lab or programmers needing some dedicated server to test/play with, these older servers can be found cheap and often just need a hard drive. Companies and governments are, basically, throwing them away to upgrade in order to get better hyperthreading and VPS densities for each physical server.
I started using hosted VPS many years ago but quickly outgrew them due to flexibility, costs (they were quite a bit more expensive then, especially for bandwidth) and, more importantly, my having control issues
about my clients' data, ownership and some clients' demands of responsibility. So, it became cost-effective to build my own. Even more so, now I hope, because of how B4J servers can scale. But, the vast majority of programmers/developers don't need that and, I agree, a hosted VPS is the way to go.