Could be an oscillation due to lead dress. It's not "swirling" so I don't think it's crossover distortion. Can you provide a high res pic of the innards? Maybe we can pic something out visibly. If you have a scope you could put a high input signal through the front end (maybe 200mV or so) to induce clipping and trace the signal to see if and where it shows up in the chain.
And, FWIW, I think the basic clipping character of the amp is pretty damned good Just get rid of that background buzz and I think you have a winner.