This is really strange, I created a fresh Joomla installation and this feature seems to work fine.
I then deleted the fresh installation and tried the old one again and it now appears to be working fine. I do not think that it is cache related because I had cleared the cache in the Joomla back end.
What could have cause this problem?