I have to assume that the AC typically passed to ground via the bypass cap does two things when you don't use a bypass cap. One thing is local NFB since AC is now present at the cathode. I don't imagine this is the problem. But the other thing is that AC is now present in that big resistor and it's leads. If modulation in the resistor and it's leads are creating any EM it could be picked up by other leads and components in the amp. Sooo... Try keeping sensitive leads away from that resistor or it's leads or perhaps even relocate some components. Since the offending EM is probably of a fairly high frequency (AF wise) the easiest solution may be to use a small bypass cap. Maybe a film cap with a value below 1uf would fix it. But I'm just guessing. Easy enough to try though.