From what I understand first you need to find the dominant colors in the images and store them inside a database. There's a script called "color thief" that does that beautifully. But chances are all the colors will be different! The color search form can't display all of them. Google search only allows you to pick from 12 colors. If you pick red it will show images that have all shades of red, not just the one you picked
I would guess that it is handled by some kind of range. Meaning you click "red" and Google internally knows "people think it's red when the color is between value x and value y". So when you are clicking "give me all pictures with red", internally you search "give me all pictures (=datasets) ranging from predominant value x to predominant value y". Some databases and index engines allow for fuzzy search params too. All this combined together: Color search.