OS X Server systems have a build-in VNC Server.

But if you are not willing to pay the 299$ that costs Apple Remote Desktop, you will find that it is not evident how to enable it for generic VNC clients.

If you happen to get in front of a headless server (being you reading this, you may right now…), this may come handly:

• Connect to the server using ssh

$ ssh server -l username

• Issue the following script to encode the VNC connection password to the proper format:

$ perl -nwe ‘BEGIN { @k = unpack «C*», pack «H*», «1734516E8BA8C5E2FF1C39567390ADCA»}; chomp; s/^(.{8}).*/$1/; @p = unpack «C*», $_; foreach (@k) { printf «%02X», $_ ^ (shift @p || 0) }; print «\n»‘

It receives the plain text password as an argument and returns the encoded string you can paste in the following command. CTRL-D to end the script.

• Issue the following commands to activate the ARD service with the proper settings:

$ cd /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/
$ sudo kickstart -activate -configure -access -on -privs -all -restart -agent -menu -clientopts -setvnclegacy -vnclegacy yes -setvncpw -vncpw encoded_password

This command enables VNC access to all users. The parameter encoded_password is the string resulting from the previous password encoding script.

For full command reference see:

$ cd /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/
$ kickstart -help

This procedure will allow you to use your VNC client of choice from any other computer on the network. 

Adapted from here.