You need to understand how Oracle networking works.
At a high level:
The listener ONLY listens for connections, validates the request, creates a new port, hands off the connection and goes back to listening.
It randomly picks a high level port between 1021 and 65535 (I think). You need to have ALL those open in the firewall.
There are two ways around this and neither is all that great:
1: Configure Multi-Threaded-Server (MTS). MTS allows you to define a fixed set of ports to be used.
2: Force ALL connections to use the listeners port with USE_SHARED_SOCKET.
The problem here is port contention and if you ever stop the listener, you KILL all active database connections.