Deeper dive:
Custom Security Metadata Endpoint
http://msdn.microsoft.com/en-us/library/aa395212.aspxThere are different binding mechanisms: metadata exchange (mex) or non-metadata exchange.
Non-metadata exchange endpoint:
<endpoint address=""
binding="wsHttpBinding"
bindingConfiguration="Bind
ing2"
contract="Microsoft.Servic
eModel.Sam
ples.ICalc
ulator" />
Metadata exchange endpoint:
<endpoint address="mex"
binding="wsHttpBinding"
bindingConfiguration="Bind
ing1"
contract="IMetadataExchang
e" />
I don't have a good enough understanding of your environment to be able to determine why using a custom binding, with higher quotas didn't help. The one thing that I can think of is that the configuration was not correct for the custom binding, and it was ignored.