Since you mentioned that changing DNS servers seemed to speed things up, it just could be that name resolution is having a problem.
Random 404 errors are a part of life on the Internet. We have two DS3s and we still get 404 errors.
Can you go to
http://www.speetest.net and pick a server near you and do a speed test from both a wired and a wireless?
With 54Mbps wireless and doing encryption, the best you will most likely get is about 15-20 Mbps.
You may want to install something like wireshark (
http://www.wireshark.org)
on one of the wireless computers and do a packet capture or two to see if you can tell if it is all Internet traffic or just some of it. You mentioned that changing DNS server improved things, it just could be for some reason the wireless devices are having problems doing name resolution, which will make everything slow.