Also consider using the electrum client. It uses remote servers for the blockchain, so you don't need to download and sync with the network, while you keep the private keys for your wallet, so you don't need to trust a remote service:
http://electrum.ecdsa.org/
It also generates new addresses deterministically from an initial seed, which you can write down and keep securely or remember in your brain, allowing you to recover your entire wallet if you lose access to your computer.
I was curious if this was secure. Apparently you are trusting the server you retrieve blockchain information from, and could be defrauded if that server is compromised:
Yes it is slightly less secure. But you can't really "lose" money so much. What can happen is the server can report that you have money that you don't actually have, so if you were expecting to receive some coins, and the attacker had control over one of the servers, they could make you believe you had received them.
There are several stratum servers however, and you can disconnect from one and check with another. I think the plan is to eventually connect to multiple and make sure that they agree. This doesn't prevent a possible man in the middle though if all your communication is being tapped.
Personally I think this slight risk is worth it for the instant start times and the deterministic wallet. And it's still considerably more secure than the online hosted wallets, where you are not in control of your coins at all.
With security I'm interested in the situation right now, not what the plan is to do eventually. If I were to get ripped off, it's no consolation knowing that someone was planning to fix the bug later when they got around to it.
The attack described is a serious one. If someone compromises (or colludes with) the server, they could “buy” 100 BTC worth of stuff from you without really paying. You send the stuff, and you're 100 BTC poorer. In the end it's as if you'd been robbed. Same result.
With an online hosting service, like Coinbase, I think it's rather obvious you're trusting them, just as you would trust PayPal or a bank. You can make an informed decision whether you trust them (and their security) based on reputation.
The Electrum website, in contrast, hides the fact that I'm trusting a third party and makes no mention of who that third party is or why they deserve my trust. IMHO, they should disclose exactly what's going on very prominently. But they don't. As a person new to Bitcoin, based on what I know so far, I would sooner go with Coinbase than Electrum.
It also generates new addresses deterministically from an initial seed, which you can write down and keep securely or remember in your brain, allowing you to recover your entire wallet if you lose access to your computer.