Moving away from wrong company

When I'm with people and they talk too much about dunya, like what's on sale, or politics, what is latest etc, why do I feel that my heart is kinda in pain? Like I don't like hearing it and I wish I could just close off and leave. I can't tolerate it at all. And this happens with my relatives a lot. Shouldn't I have sabr? I just sit there and just keep feeling sad inside. It's like I really hate people doing that. I feel like my heart burns or something. I'm scared of being absorbed into dunya by listening to people's talk.