Csdrhrt

Our sensory organs for light and sound work quite differently on a physiological level. The eardrum directly reacts to pressure waves while the photoreceptors on the retina are only senstive to a narrow range around the frequencies associated with red, green and blue. All light frequencies in between partly excite these receptors and the impression of seeing for example yellow arises due to the green and red receptors being exited with certain relative intensities. That's why you can fake out the color spectrum with only 3 different colors at each pixel of the display.

Seeing color in this sense is also more of a useful illusion than direct sensing of physical properties. Mixing colors in the middle of the visible spectrum retains a good approximation of the average frequency of the light mix. If colors from the edges of the spectrum are mixed, i.e. red and blue, the brain invents the color purple or pink to make sense of that sensory input. This however doesn't correspond to the average of the frequencies (which would result in a greenish color) nor does it correspond to any physical frequency of light. Same goes for seeing white or any shade of grey, as these correspond to all receptors being activated with equal intensity.

Mammal eyes also evolved in a way to distinguish intensity rather than color, since most mammals are nocturnal creatures. But I'm not sure if the ability to see in color was only established recently, that would be question for a biologist.

This is due mostly to physiology. There is a fundamental difference in the way we perceive sound vs. light: For sound we can sense actual waveform, whereas for light we can sense only the intensity. To elaborate:

• Sound waves entering your ear cause synchronous vibrations in your cochlea. Different regions of the cochlea have tiny hairs which vibrate in a frequency-selective way. The vibrations of these hairs are turned into electrical signals which are passed on to the brain. Due to the frequency selectivity of the hairs, the cochlea essentially performs a Fourier transform, which is why we can perceive superpositions of waves.


• 
  

  Light has such a high frequency that almost nothing can resolve the actual waveform (even state of the art electronics nowadays cannot do this). All we can effectively measure is the intensity of the light, and this is all that the eyes can perceive as well. Knowing the intensity of a light beam is not sufficient to determine its spectral content. E.g. a superposition of two monochromatic waves can have the same intensity as a pure monochromatic wave of a different frequency.

  
  
  

  We can differentiate superpositions of light in a limited way, due to the fact that eyes perceive three separate color channels (roughly RGB). This is why we can distinguish equal intensities of red and blue light. People with colorblindness have a defective receptor, and so color combinations that most humans can distinguish appear identical to them.

  
  
  

  Not all colors that we perceive correspond to a color of a monochromatic light wave. Famously, there is an entire "line of purples" which do not represent any monochromatic light wave. So for people trained in distinguishing purple colors, they can actually differentiate superpositions of light waves in a limited way.

  
  
  

  

  
  

Rod (1 type) plus cone (3 types) neurons in the eye give you the potential for 4-D sensation.
Since the rod signal is nearly redundant to the totality of cone signals, this is effectively a 3-D sensation.

Cochlear (roughly 3500 "types" simply due to 3500 different inner hair positions) neurons
in the ear give you the potential for 3500-D sensation, so trained ears can potentially
recognize the simulatenous amplitudes from thousands of frequencies.

So, to answer your question, eyes simply didn't evolve to have many cone types. An improvement, however, is seen through the eyes of mantis shrimp (with the potential for 16-D sensation). Notice the trade-off between spatial image resolution and color perception (and that audio spatial resolution was less important in evolution, and more difficult due to the longer wavelength).

The hairs form a 1D-array along the frequency axis, while rods and rods and cones form a spatial 2D array. In addition, that 2D array has 4 channels (rods and 3 types of cones). So the 2 ears have a poor spatial resolution, while the eyes have poor frequency resolution.

You could imagine an eye with many more types of cones, giving you a better frequency resolution. However, that would mean that the cones for a single color would be spaced further apart, limiting spatial resolution. In the end, that's an evolutionary trade-off. Physics tells us you can't have both at the same time, but biology is why we end up with this particular outcome.

Actually you can discern the different combinations to some extent. I figured this out after taking many art classes, particularly color theory and painting. I really started noticing the effect after the painting class. The reason is I was taught how to mix colors and needed to learn to see and think about the combinations while looking at the subject. This is the same for the note discernment you mentioned. The people who are usually able to do it, and especially well, are people who have studied music and have had to learn to hear and think about the combinations while listening.

So, now when I look at something "green" I see that it is actually blue with a little yellow and sometimes a bit of another color. Nothing is ever "white" or "black" to me anymore either, there is always a reflected color or a texture of some sort which catches the light from the surrounding sources.

Edit:
After further consideration I am inclined to say the question posted here is backwards. We actually discern many more separate colors in combination and at the same time than we do sounds. Look around and you will see a plethora of color combinations. We are actually perceiving many billions of separate photons coming from many billions of separate locations in our field of view. In contrast, listening to so many separate sounds would overwhelm our ears and perception of the sounds to the point of hearing only discord, static, or hissing like noises that have no meaning. It is only the combination of very few separate noises which can be discerned.