Me Too vs Me Neither

I'm confused about how to correctly use the phrases "me too" and "me neither." It seems that they're largely interchangeable even though on their own they say the exact opposite thing. I'll give you an example. I say, "I can't hear." Then you respond, "Me too," meaning that you also cannot hear, or you could have responded, "Me neither," meaning that you also cannot hear. You're responding to the same thing (not being able to hear) but in one case, you're agree with me and the other you're agreeing with the inability to hear. How English ever became the global language I'll never know.