It happen when I use headphones too. But only when it's used as a PC.
I try to unplug it as well, doesn't work.
It does the same when I turn off the speaker in steam deck, you right it's probably a interference, but couldn't find from where.
I will try more test to see if I can somehow stop it.
To conclude, I got the ivoler one, work perfectly !