Is this a large table or has large fields?
I would do a test with a smaller, simple table or view and validate your connection.
It sounds like it might be specific to the table. Does it have a primary key, etc? There are some conventions which InfoPath prefers.