We're unable to test the Java client switch due to restriction issues, however I was able to get it working by deleting these Keys from the registry then importing them from another profile:
HKEY_CURRENT_USER\Software\Citrix
HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings
I didn't reboot initially when I re-opened IE to test it. But I've rebooted since and tested it again successfully.