Web Site Project is deployed with source code to the server and all compilation takes place at runtime.

Web Application Projects, the code behind classes are compiled to dll. That dll is deployed and at runtime, the compiled code in the dll and the markup is combined to create a class which is used by the server to render output.

Update:

Good post by anthony on this

post by Stephen on this topic