Here is a list of my favorite quick-and-easy optimizations which will result in faster load times for your websites:
The fastest communication is that which doesn't occur, and sending appropriate content expiration headers with your data can accomplish just that - browsers and intermediate caches will hold on to your data after transmission and not bother your server with future requests. On the server side, you can speed things up by making use of any application-supported key/value stores such as memcached or redis, or even enable simple file-based caching to return data without bothering your database. If using PHP, enabling an opcode cache such as APC or Zend Optimizer will make sure that PHP doesn't have to parse your scripts every time they're needed, resulting in significantly faster scripts. A reverse proxy installed on the server can drastically reduce the number of requests that even make it to the web service stack by serving data straight out of memory. If you do have to talk to the database, make sure it is caching query results so that common queries are returned swiftly.
Use a CDN for static assets
Not only will this reduce the number of requests that are made of your web server, but static assets will be served more quickly as they will have to travel less to get to your user. There are lots of solutions for this, but we've been really liking Amazon's Cloudfront of late.
Run a recent Linux kernel
Make sure the server and receiver's TCP stacks communicate as efficiently as possible by keeping your server kernel up to date. Among other things, this will ensure that as much data can get out as early as possible in the communication process, that the max receive window size is large (resulting in greater throughput), and that it gets to that size as quickly as possible.
Combine/Minify CSS and JS
An oldie but a goodie, and one that is surprisingly often overlooked. This practice adheres nicely to the two tenets of sending as little as possible in as few round trips as possible between your server and your user's browser.
Instant and significant bandwidth savings for non-binary assets when sending to browsers that support gzip compression, which is pretty much all of them. Enable mod deflate in Apache or simply make sure the gzip directive is set if using Nginx.
Resolving hostnames to IP addresses is the first part of the connection process, and can itself go awry. Use a quality DNS service with good performance and set a reasonably high (but not too high, in case you need to make changes) TTL on your zone records. This will ensure that results will be usually cached nearby, with resolution happening as quickly as possible for your users. Reducing the use of unnecessary Cname records will decrease the number of lookups that need to happen and get your user an IP to connect to more quickly.
This will especially help your mobile users. By not loading non-visible images until they're needed, your time to paint can be reduced. And those images - you're optimizing them, right?
A fantastic tool that will accomplish a lot of these optimizations very easily is mod_pagespeed, which is available for both Apache and Nginx web servers. If you use this, start with a small number of optimizations and enable new ones one at a time, testing thoroughly after enabling each - they can sometimes have unintended consequences