hmmm, that should have done the trick. To delete an address object/group, you have to delete the firewall/nat rules associated with the address objects. Click on Firewall > Settings. On the right, you'll see a matrix. You want WAN > LAN. Delete the rule there. Then, go to LAN > WAN and confirm there isn't a rule there. Then got to Networking > NAT. You'll have three rules. An ingress, an egress and a loopback. Delete those respective rules, then you should be able to deleted the address objects.
I added a manual that should get you through the public server wizard. check out page 503...Using the Public Server Wizard.