Audio plays back fine on some devices, not others

A really strange thing is happening with an audio track I'm editing for a video, using Audition and Premiere. It involves audio that is pulled from a .h264/AAC video file. Everything seems fine on my machine, but when some people view the video (or listen to just the audio track), the audio becomes glitchy and barely audible. It plays for some people fine, not for others. I only knew there was a problem because someone told me what they heard on their end. I cannot reproduce the problem on my computer. 

Here's where it gets strange: On mobile phones, it's almost always a problem. However, if you use headphones or speakers, the audio is great. Unplug the headphones/speakers and the audio is glitchy again.

I've tried several things: editing and exporting from Quicktime 7, Audacity, Audition, Final Cut Pro 7 and always with the same results. Doesn't matter what format/codec, compressed or uncompressed: it's the same if it's an aif, wav, mp3.

Here's a sample of the video on Vimeo:

https://vimeo.com/134962721

And just the audio on Soundcloud:

SonkaInterviewEdit by AgentGrifray | Free Listening on SoundCloud

Has anyone ever encountered anything like this? What could cause an audio file to work on some devices and not others? Why would a device not be able to play a file through its internal speakers, but can when connected to speakers/headphones?

Thanks for any help!