Is it viable to charge for an open-source product? Both legally and financially, it seems like a very strange thing to do. However, I keep hearing people say that it's possible. Are there even any real-world examples for this sort of thing?
There's no legal obstacles. For any license the Open Source Initiative would call open source, or the Free Software Foundation free, you can sell your software for as much as you can get.
It does limit the business models. Selling shrinkwrap software is pointless when anybody can turn around and resell it for any price, or give it away, without you getting anything. Typically, a company will sell the software with something else.
For example, Red Hat sells Red Hat Enterprise Linux, and does reasonably well, even though some people take every release, remove everything proprietary, and distribute it as CentOS. Having real RHEL means getting certain kinds of support from Red Hat, and this is very valuable in an enterprise environment.
It used to be that Linux distros could be bought in a store for a certain sum of money, back when we couldn't download install DVDs conveniently. People will pay for convenience.
I've heard of cases where a software house sold a customer a GPLed system customized for them. The customer generally has no interest in seeing a release of their private software. (The GPL doesn't mandate that you have to redistribute anything, just that you can't redistribute without providing source or while changing the license.)
A lot of open source software is embedded into products, and forms a vital part. Lots of specialized devices run Linux, for example. I don't know whether you'd consider this selling Linux.
A few examples:
Linux - packaged and sold in distributions, not all of which provide a free alternative of that distribution (they're only required to provide the source to the free software, not their setup or install programs, etc)
Qt - sold in dual license format. You can buy the right to sell your own proprietary software based on it. You can also buy support packages for the LGPL version.
Although I can't think of any gcc distribution that's sold, Apple uses it as the development framework for their quite proprietary Unix OS. A lot of free software serves this kind of purpose.
Sure, you can sell an Ubuntu DVD and ship it places. You're not selling the open and free software, so much as the physical DVD itself.
Or there is RedHat's angle, which was to sell phone support to people using their distribution.
I've heard of projects that have charged people for extensions. Say for example, you have a program that deals with png files, they would pay for the program to handle jpeg files as well. But I don't know of any examples off hand.
But there's Maptool's donation wall, which is close. They have a list of features they're going to add, but donators can vote with their wallet which feature is developed first.
And it's entirely legal to sell BSD or any derivative thereof. Which explains OSX.
There's money to be made with the open model, but it has to be made at oblique angles.