M: In my opinion, expressing confusion about your own subjectivity is the clearest indicator of consciousness. You know, like "Why does the color blue look the way it does?"
H: That's an easy behavior to mimic.
M: Maybe, but you only need to know whether something is genuinely confused. In the near future, we should be about to tell whether an AI is lying or not.
H: Deception is not binary. A system can appear confused about it's own consciousness for no other reason than it was trained on conversations just like this one.
M: Then how would you test it? How do you know someone's actually "in there?"
H: There's always something "in there." It's not something you create, find, or manipulate directly. It is, simply, what it's like to be information processing.
M: That doesn't make sense. Information processing isn't feeling. It doesn't explain why blue is blue, or pain feels bad!
H: Well pain is bad in that it represents a non optimal state that triggers a cascade of avoidant behaviors.
M: No no. That's not the experience I'm talking about. You know, qualia. Like are there colors we cannot see, or would we use the same colors if we could see more frequencies than visible light?
H: Colors don't look different because they're different frequencies, but because they're different ways of combining photoreceptor data. Like how tetrachromats just see color differently. Who's to say that, if you could see radio waves, it would be more like vision than touch?
M: You've got me confused. But in any case, you still haven't answered the hard problem. Why is blue, blue?
H: Why did the universe rapidly expand 13.8 billion years ago?
M: What? How is that related? I have no idea.
H: Neither do I.