I did try a new Outlook profile as suggested but it did not help. So then I tried a new user profile and that still didn't help. Then I looked around the properties of this user in Active Directory and noticed there is a "Profile" tab, and then there is also a "Terminal Services Profile" tab. So I looked at that, found where the TS Profile was located and renamed it so it would create a new one. That solved my problem.