Now that you have a pretty OpenID with the full freedom to change providers on a whim, here are a few tips for what to look for when choosing one.
In my opinion the most important issue is phishing protection. At least to me an SSO login is much more valuable than a throwaway password for a random site.
Since the site you log into redirects you to your OpenID server, the authentication sequence is vulnerable to phishing attacks. The malicious site could actually send you to a proxy page which will attempt to steal your credentials.
There are ways of protecting yourself as a user, simply never type your credentials into that page (always open a new window, login there, and then reload the other), but it's easy to forget when you see a familiar looking page.
Some OpenID providers can display an image or text banner that depends on a cookie in your browser. That way if you don't see the image something suspicious is going on (the proxy will not receive the cookie from your browser).
Most providers can provide better authentication methods. My current favourite is an SSL certificate. This means no sensitive information is sent over the wire at all. It's not only more secure, but also quicker and more convenient.
If you're concerned about security, make sure your provider has decent logging for all activity in your account.
The next thing to look for is support for multiple persona support. When you log into a website your provider will some profile information along with the authentication token. If you want to use separate emails or languages settings for a certain website then your OpenID provider will need to allow you to pick which set of values to send.
Lastly, some OpenID providers and consumers are broken/out of date. It seems that ideally you'd want one that supports OpenID 2.0, but also version 1. I had quite a bit of trouble with Movable Type as an OpenID consumer, until I finally settled on myOpenID. I'm happy to say that it fulfills all my other requirements too.