1) Consumer = a customer of a merchant (e.g. you or me)
2) Merchant = a customer of both a bank and Visa (e.g. WalMart, etc)
3) Bank = a bank that issues credit and debit cards.
4) Visa = a (publicly traded) company that controls the electronic payment infrastructure (e.g. Visa, Mastercard)
5) Card = a credit or debit card
Now for some simple inference.
By (1), merchants want low fees, banks want high fees. By (1) and (2) Visa wants as many consumers
using Visa cards as possible. By (3) Visa's customers are really the banks, not the consumers. Visa wants banks to issue more Visa cards. Therefore, Visa has an incentive to raise their fees, thus attracting banks that want to make those higher fees.
Visa can't set the fees too high or merchants will revolt and refuse to accept cards. But now Visa and Mastercard are in some kind of reverse competition: who can set their rates the highest without a revolt from merchants.
If this were a free market, one could imagine creating a new Visa-like company (with the associated infrastructure) and drop the bank fee to cover the cost to the banks of supplying the cards. Here the consumers and merchants benefit from the competition, rather than the banks and Visa. Unfortunately, the card must be attached somehow to the bank account, and only the bank can do that. So it seems there is a big problem with this market. Perhaps this could be circumvented in the future if everyone has a PayPal-like account and all the transactions take place directly over the internet. I can imagine some kind of authentication device being used, part of a phone say. Merchants could give a small discount to use the PayPal (non-Visa) system. The banks could fight this of course, but it seems like a possibility for the future.