From youir vSphere client attach to the server and go to configuration, then under software select Licensed Featurs. Click the Edit.. link and enter you key in the dialog.
If you have a diagnostic memory test with your server you may want to run that on the new memory. I would first plug the new RAM back in and see if it works. Sometimes you don't get a DIMM seated correctly and it can cause all kinds of problems.
Good Luck