I never use caiman. Whenever I've ordered from them, they've had it not in stock (without telling you) and you have to wait til the A-Z claim can be filed to get your money back.
They usually have the lowest price, so I go to the next one up. Yikes. Its good to hear that someone had success with them.
Amazon's the best, but of course overstock.com has a decent selection and I get some from deepdiscount, it just depends what your looking for. Arkiv is good, but a bit expensive IMO.
Out of maybe 50 orders from Caiman over the last year, only one has resulted in trouble, but yes, they do not credit you very promptly. If you contact them directly, you can seldom make sense of their response.

It is very easy through Amazon to get your account credited, however, so I would not exclude them as they do seem to be on top of things for the most part.
As for ordering in general, it really depends on what it is you are looking for. For most new releases, Amazon (including Amazon Marketplace sellers) tend to be the easiest and often the best prices--it is a good idea to factor in shipping costs into the cost of item though as the free shipping from Amazon for orders over 25 USD generally makes for a better deal than ordering handfuls of CDs from separate resellers with 2.98 postage. For historic recordings,
Berkshire Records Outlet can provide a great bargain. They have a low minimum for US orders, and it is worth your while to order many at a time to reduce shipping costs.
eBay can sometimes provide decent deals on releases from a few years back and even some new releases can be had for much less, but shipping costs from eBay sellers tend to be pretty "out there" in terms of cost.... shipping times are not favorable either as I am still waiting on an auction won June 21st. Fortunately for the seller, I am patient.

While I try to order as few things as possible internationally unless it is many items at once--due to environmental concerns--
HMV,
Buywell and
JPC can often provide you with items you do not often see for sale stateside (I am assuming you are buying in the US of A).
For more obscure items
Norbeck, Peters & Ford can sometimes be useful, but their prices tend to be on the higher side. I have also had some success finding better prices at
Suncoast Movies (they sell music too

). And there is also
CD Universe which I have only used sparingly... they can be a better priced alternative to Amazon with some items, but again, consider the additional postage costs.