You can install any number of .net framework versions in any order. They are backward compatible.
Yes, removing .net 4 will break the projects you've created in .net 4.0. But other than that it should not have an impact anywhere else.Arun