How to optimize WordPress site performance?

Use a CDN
Use a caching plugin or edit .htaccess to cache different file types for different lengths of time and use gzip, etc
Use a simple theme/framework
Split comments, products and long posts into smaller pages
Use less plugins and only use well established ones
Keep WordPress updated
Host images on a subdomain

A separate domain prevents your domain's cookies from being sent for every image request, where they're not needed. Reduces the bandwidth overhead for each request.
Most browsers will only make a certain number of HTTP requests at one time to a particular domain, so serving images and other static content off a second domain potentially doubles the number of HTTP requests at one time.
You can conceivably use a different, lighter weight webserver like nginx/lighttpd to serve the static content.

1. Why host images on a separate subdomain?