Yeah? or What?

I realized a long time ago that often in conversation when I say, "Yeah," people think I'm saying, "What?" Then they repeat what they just said, so I say "Yeah" again. You'd think that this would spiral in an endless loop but most people figure it out after the first time around. Though it does occur to me that perhaps my "yeahs" sound more like "whats" because when someone says something idiotic to me I say "yeah?" like a question instead of a statement. As in, "Yeah, you sure that's the story you want to go with?"