Should you? It's certainly an interesting language. It is a community-driven project, so it's a terrible mess at times. Object Oriented features have been added as an afterthought, and although they work passably well in the latest versions (5.2 & 5.3), they are still not what you expect from OO. Many functions pollute the namespace with aliases, sometimes the naming conventions differ...etc.
Still, I am in love with PHP for the exact same reason. I can easily switch from procedural to object-oriented at any given time, and mix at my heart's content. This allows me to prototype extremely quickly using procedural code, and then organize in classes when everything works.
Another reason for PHP is that it is completely open-source, and that it works out of the box on Apache, which is arguably the most flexible and robust server out there. The dev & the server box are extremely easy to set-up (under linux, it's a matter of one command-line, and under windows, installing xampp is enough). Oh and also both are completely and totally free.
So my main argument for PHP is ethics. I believe in open-source, and free, and I like that I can set up a working env on any PC without worrying about the OS, the license, or if it is going to work.
On a side note, Ruby is open-source too, albeit harder to set-up.
So how to start?
Pick up a simple project that does not use a database (text editor, calculator...), and try to build it. the PHP.net community is a wonderful source of information.
tizag.com taught me my first steps. If you are an experienced dev, you will find the lecture somewhat boring, but skimming through it will give you the basics.
The various tutorials on devshed taught me almost everything that I could learn by reading.