Yep, have to look at the sellers point of view as well. Verbal agreements over the phone don't mean much. Can't tell you how many times I've had calls, "I'll buy it" and you wait, and wait, and wait, and no one shows up, all kinds of excuses. Or they try to negotiate over the phone before they have even seen what they are buying, funny.
Honestly I don't waste much time on people like that. Money talks, and I'm up front about it. If you're serious and want to send a deposit, that works fine. Asking me to hold something over the phone for days or weeks until you can come see it, with no money down, not likely. Just have to be honest about it and tell the buyer, can't hold it without a deposit, if you want it and there are other interested parties, better hurry up.