Paul McGowan is no dummy, but I disagree with many of his points. Sure, surround sound is more expensive and takes a different setup than stereo (which can be with earbuds and a phone these days), but there's little doubt that it's a niche market - one that I'm fully a member of (although there are plenty of people who are far and away more involved than I am). I have purchased some gear from his company, and the "Copper" on-line magazine is usually full of interesting articles, mostly about music, including surround.
I don't know if I fit anyone's definition of "audiophile," and I don't really care if you or my neighbor considers me one. From the time my dad built a hi-fi in the early 1960s, though, I've loved being in the middle of the music. I adopted quad (at least SQ) when it first came out in the early 1970s, and have always bought what I consider to be good equipment. I'm not one of those with more bucks than brains, so those $1,000 power cables won't find their way into my system - ever. I'm in the process of building a home theater (I doubt if the process will ever be done, although I have some definite goals) where the emphasis is far more on the sound than the picture. I have 5.1 right now with Emotiva Pro Stealth 8 studio monitors across the front (but at this point I'm severely disappointed in their product support). I have installed ceiling speakers for eventual Atmos, and I have a decent collection of surround music discs and downloads. My avatar shows my vinyl collection, with a hundred or so quad LPs (and one EP).
So the real reason so few people are into surround music is that it's simply not all that appealing to the vast majority of people who listen to music. Everyone gets to like what they like, and nobody needs an excuse for it.