The previous 'sessions' are remembered through cookies set in IE.
To change the default cookie behavior, go into the privacy settings:
Tools-->Settings-->Privacy-->Advanced-->
Check the box 'Override automatic cookie handling
Block first party cookies, block 3rd party cookies.
This will ensure that no sessions are remembered.
Also - to clear out existing session memory (existing cookies), use the Temporary Internet Files --> Settings as mentioned by the user above