It's easy.
As you write applications not UI it is a console application. The console application can be run on servers without windows, without the desktop only text. Web servers do not require a graphical desktop, just like mail servers, ftp, etc. Servers without graphical desktops are lighter, faster, require less hardware resources and on such servers console applications run faster than UI applications because they are not loaded on UI.
You can write server applications with UI but you need to have a graphical machine. However, some server resources (CPU, RAM, HDD) will be used to maintain the desktop. The more stuff on the server, the more privileges you have. So why care about such a machine?
Economically: It is cheaper to maintain console servers than with graphics overlay.
As a programmer: I'm leaving to worry about the graphical part, doing only the back-end and not the front-end.