The Public Folder (PF) info in AD will be in the Configuration Container, under Services -> Microsoft Exchange -> [OrgName] -> Administrative Groups -> Exchange Administrative Group (FYDIBOHF23SPDLT) -> Databases -> [PF Database].
If you highlight and delete the entire object under the Databases node that corresponds to the dead server Exchange will forget all about it. Good luck.