Neil Harbisson has never seen color. Born with achromatopsia he lives in a grayscale world. Technology has allowed him to detect color through sound. The camera on the end of the antenna permanently attached to his head reads the color frequencies of visible light and translates them into sound frequencies. But what this technology can detect does not have to be limited to visible light.

Today the technology in Neil Harbisson’s chip can now detect infrared, UV rays and Bluetooth. In part 5 of our 6 part series, Munsell interviewed Neil Harbisson about what it’s like to hear these other wavelengths, the societal implications when living with a camera attached to your head and privacy concerns. We also discuss what cyborg related projects he’s working on for the future.

Hearing Infrared

Munsell: So does infrared fall inside or outside of that octave with the rest of the colors?

NH: It’s still within, because just below F sharp is where infrared starts. So, F would be infrared but then I have only near infrared and near ultraviolet, so they all fit within one octave.

There are 3 different types of infrared. There’s the near infrared, the middle infrared and then the far infrared. It is like 3 different colors, but we give them the same name, but they are different frequencies. And then there’s also near ultraviolet, and then you can go too far ultraviolet. So now the lens can detect the near spectrums infrareds, which are usually found in remote controls or hand dryers. So when you dry your hands, automatic doors have color, also alarms. [Looks around] There isn’t one here…

Munsell: Do you hear UV when you’re in strong sunlight?

NH: Yes, if there’s ultraviolet and sometimes it’s sunny, you can hear the ultraviolet. Ultraviolet can be constant and annoying.

Munsell: You can’t turn it off?

NH: No, and ultraviolet is the highest frequency one, so it’s high pitched. But it’s actually a dangerous colour, a colour that can kill us. It can damage our skin and can actually be dangerous. So it’s good that it makes sounds.

Munsell: If you had an umbrella or a hat, would that stop the UV sounds?

NH: Depends. Sometimes if you go in a car or a bus, or a building that has special windows, then suddenly it stops.

Bluetooth on the Brain

Munsell: We recently read that your chip now has Bluetooth and that someone called the chip embedded in your skull on the phone. So you actually heard the sound vibration of the voice go directly into you, like a voice in your head? Was that somewhat creepy?

NH: Yeah, it was.

Munsell: What are the applications of having Bluetooth detection? They can’t hear you talk back unless you have a phone, right? So you’re only receiving.

NH: Exactly. So I was at Moogfest, in Asheville, North Carolina. There is a painter in Australia. He was painting and he was sending me the colours of the painting to my head. I was creating music from the colours I was receiving. So he was just painting in Australia, but his painting became a music performance in America, basically.

Munsell: What if the Bluetooth was connected to a computer so whatever sound frequencies you’re hearing could automatically get transferred to the computer? So, for example, if you were looking at someone’s face to make a sound portrait you wouldn’t have to write down the colors that you’re hearing, it would just automatically get transmitted into a computer. That could be a lot faster than writing it down.

NH: I don’t want to record in any way, because I feel that I should not use my body part to record images or sound because I feel this is something that will definitely give me problems with privacy. So, one of my rules is that it will never film or record. I don’t want to get into the world of privacy [issues]. That would interfere with a lot of people that wouldn’t like me to use this [technology]. I don’t want to be anything similar to Google Glass or anything similar to any other recording devices. In fact I don’t really need to [record anything]. I can write it down. My brain will record it, I will remember the sound. So I can use my brain for memory.

The Sound Palette

Munsell: What new projects are you working on now?

NH: I am doing what I call the sound palette for the last few months. I’m up to 600 now, but it’s different chords with each corresponding colour. So I’m creating a colour palette that is based on the sound of the colour. It’s very slow because there are so many different chords. For example, the colour C Major is the combination of the colour C, the colour E and the colour G, so if you combine these three colours you get C major colour and that’s one, basically a colour which is called C major basically.

Munsell: So you are mapping chords made up of 3 colors?

NH: There’s a colour for each note and then for each microtone because there are 360 notes in an octave, so in the end it will be a palette with, I guess, over a thousand sounds.

Munsell: So are the chords getting mapped back to the colors you are hearing?

NH: Yes, it’s a visual, there’s a colour on the paper and below it says C Major, and then there’s a little notation so you can play the colour. So it has the note and the exact number of the colour in the RGB and then there’s also a number with the color HEX [the six digit hexadecimal number used in computing applications to represent colors.]

C Major Chord

C diminished triad

C major 6th chord

In our last post of this series on Neil Harbisson, we get personal and talk to him about what it’s like to live life as a human cyborg and how the technology works.

Posted by Neil Harbisson.