In many way, IELTS Listening is as much a test of attention and multitasking as of aural comprehension. Regardless of accent, the recorded voices on the tape speak more slowly and clearly than people generally do in real life, and they do not use the kinds of slang* and idiomatic language that can be confusing to many non-native English speakers.
The difficulties of the Listening portion lie elsewhere, in processing the written information on the page and the spoken information on the recording at the same time, and in connecting one to the other. In principle, this is simple because the two convey the same information. However, they may not always be identical: the written portion may restate the recording word-for-word or, at certain points, it may paraphrase/condense the spoken part. When the latter occurs, it is easy to become confused and miss essential information simply because it takes you a few extra seconds to figure out which phrase or sentence on the page corresponds to what you are hearing. And in Listening, every second counts.
An additional challenge is that multiple-choice questions accompanied by a long list of answer choices are physically separated from the rest of the text, making it necessary to continually jump back and forth between different parts of the page while you listen. There’s a lot of content to manage, and if you spend just a couple of seconds too many re-reading an answer choice and stop paying close attention to the recording, you may miss the information you need to answer the following question.
The other issue is that when you’re accustomed to treating electronic media as background noise (you might listen to podcasts while trying to fall asleep, for example), the kind of constant active listening required by the IELTS can feel unfamiliar and uncomfortable. Even as a native speaker, I find this challenging, not to mention very tiring.
The best strategy I know of for ensuring that you do not unintentionally stop paying attention or get confused by differences in the recording versus the words is to put your finger on the text and move it along each line as you listen. Obviously this is much easier if you are taking the paper-based exam, but if you’re doing the online version, you can try putting your finger on the screen. Yes, seriously. You may find this distracting, but it’s worth testing out at least once.
Yes, this is a lot more work, and it might feel unnatural at first, but it will keep you extremely focused and prevent those momentary lapses of attention that can cause errors entirely unrelated to your ability to understand English. Your score should reflect your actual level of comprehension, not the fact that you happened to get distracted by someone tapping their foot next to you and missed a key phrase. Assuming you have a good level of comprehension, doing well on the Listening portion of the IELTS is largely a matter of self-management and ensuring that you do not make careless errors.
And if you’re worried about looking silly… well, all I can say is that everyone else will be so focused on their own exam that it’s unlikely they’ll even notice.
*Note that slang is singular and uncountable; always slang, never slangs
