Question : terminal Server 2003 freezing for remote users

I have a server and term server at location A. The terminal server is licenced for 35 terminal server connections. I have office B and Office C and Office D all remote offices at seperate locations with upto 10 users per remote office.
The connections for users at these location may freeze at odd intervals. For example. A user at Office B their remote session freezes for 5 - 15 seconds but this may not occur to other users at the other offices.
 Another intermittent issue is Word and Excel "disappear" from the remote users screen. They have no warning and loose their work. There are no error in the LOG. I also have seen outlook take upto 4 mins to type 3 words in an email for a remote user.

Broadband in Office A ( The HQ ) = shdsl 4MB upload and 4MB download unconntended
Broadband at Office B = ADSL2+ 12MB down 1MB up
Broadband at office C= ADSL2+ 8MB down and 512 Up
Office D is a varying quantity of home workers that seem never to have any issues with remote.

I have spent night after night reviewing the server logs, firewall, Memory use, CPU use and all seem OK. I have downloaded all KB's and am uptodate with the MS downloads. I am running out things to lok at. The server did fall over some months ago and had to be repaired but it is working. Office B may work fine for 2 or 3 weeks than have an issue. Office C are having problems daily.

It is a bit long winded but where would you start to see if a server is configured correct for remote. Please note the remote users are not using VPN to link to term server all is through RDP

Help................... I would not see afault now if it was in front of my nose.

Answer : terminal Server 2003 freezing for remote users

The administrator thing is not good.  You can just add them to the terminal service group?  That is allot of instances of outlook running on one server.  Did you actually look under task manager and under processes to see the amount of ram etc.  that each user is using?  The 350 G5 doesn't seem like it would be made to handle that.  Another thing is...  If you go to folder options, then go under view, and make sure the Automatically search for network folders and printers is unchecked.  I had a problem with another network that i was responsible for...  And it made things run horrible.  (it is a setting on the client machine)