By mistake I have had Wifi running as well as my mobile broadband adapter - and that makes Vista choose one of them only. As both of them are heading toward the Internet the problem is only the speed. If you are comfortable with IP addressing, the command route print under the command prompt would give you some information on what is actually going on from your computer in different situations. The rightmost column would give you the relative speed of different channels.
However, as you have admitted, you switch off the wifi in your office - that would of course be the only straightforward thing to do in your situation. If you use wifi you should use one preferred SSID/pass phrase combination only and for security never use any open wifi networks ever.