Look at the email headers and check to see where the delays are coming from. The times should be apparent on the header as to when the mail was sent and when it arrived. Check to see where the obvious delay is.
If you get stuck - post a header and I'll analyze it for you.