Best Universal Format for Digital Archiving of Multi-Channel Audio

[STLJLC]

A few years ago I abandoned a project to create a master multi-channel music server. I am revisiting the project. I have several folders of archived multichannel music, from various sources. Including but not limited to tons of .iso files from Oxforddikkie's dreaming spires site, multi-channel recordings from other sources. Rips of my own quad vinyl that are decoded. Etc.

Over the years, I've used a variety of PCs to serve up music. I am currently using a recently-purchased nvidia Shield, run into my AVR, and onto a 7.1 configuration. I make the assumption that going forward, the trend is going to be to use devices such as the shield (or firestick, roku, etc. etc.), rather than a PC or rasberry pi, because the processors of those devices continue to evolve. I understand that there is an order of magnitude more control over configuration when one is decoding the digital music on a computer, versus some device. However, I've found that keeping the interface between the computer and the AV system (whether an internal or external soundcard) can be cumbersome. In any case...

I'm thinking about converting the various formats I have to a consistent format. I'm not a purist about the container, however I would like to preserve lossless music where appropriate. I'm also not married to any one player, and while Foobar2000 seems to be the best, it also seems the least likely to keep up with evolving hardware options. Perhaps I'm wrong about that. So I've tended toward Kodi. Also, tagging info to the files - through .xml or otherwise - is a consideration.

Does anyone have recommendations about the best uniform format to use for archiving multi-channel audio? I will also probably begin to archive over time various quad vinyl and SACDs in the same format. Is it some form of .iso container? Is it in flac? Is it muxing it to a video container like .mkv? Or is there simply no answer? Thanks.

 

[ubertrout]

I'm a fan of FLAC. The format is open-source and patent free - GitHub - xiph/flac: Free Lossless Audio Codec. If you're working with ISOs you might want to use a MKV file as a container to replicate the multiple tracks, which is also open-source: Legal aspects.

 

[LuvMyQuad]

I use FLAC for everything. Its compatible with pretty much all modern equipment. I can even play it in the car. Its lossless. Its very taggable.

I use Kodi. I play Kodi from a Firestick on my bedroom stereo only rig (although the firestick is fully multichannel capable). I also play Kodi from a mini HTPC (like a NUC) in my main rig for stereo and multichannel. Kodi will also pass through Atmos using a proper container like MKV.

I make the assumption that going forward, the trend is going to be to use devices such as the shield (or firestick, roku, etc. etc.), rather than a PC or rasberry pi, because the processors of those devices continue to evolve.

I think all processors will evolve, regardless of the device they are in.

I understand that there is an order of magnitude more control over configuration when one is decoding the digital music on a computer, versus some device. However, I've found that keeping the interface between the computer and the AV system (whether an internal or external soundcard) can be cumbersome.

I'm not sure what you mean by decoding on a computer having more control. There has to be a multichannel DAC somewhere in the system. There is also DSP, but you can get that from lots of devices. The interface is no more cumbersome than an HDMI or USB connection isn't it?

I will also probably begin to archive over time various quad vinyl and SACDs in the same format. Is it some form of .iso container? Is it in flac? Is it muxing it to a video container like .mkv? Or is there simply no answer? Thanks.

You can keep SACD as DSD if you wish. Many do. The upside is the purity aspect. The downside is the fact that you can't use any DSP with it. I convert mine to FLAC after ripping, mainly because of the DSP issue. Both Foobar and JRiver will play SACD and other ISOs I believe. Kodi may be evolving to that as well.

If it has to be one format, Id go FLAC.

 

[STLJLC]

Thanks all for the feedback. Seems like flac is the way to go, if I want to try to standardize everything.

I'm not sure what you mean by decoding on a computer having more control. There has to be a multichannel DAC somewhere in the system. There is also DSP, but you can get that from lots of devices. The interface is no more cumbersome than an HDMI or USB connection isn't it?

I didn't phrase my point very artfully. I meant that when working on an HTPC, in my experience, one has much more ability to deal with configuration issues. Two basic reasons. (a) it is simply more comfortable to sit at a keyboard and mouse and work through the software, as opposed to some remote and (b) it has always seemed to me that windows applications are more configurable than versions meant for an android system, etc. As for it being cumbersome, I meant on a very basic level that having an HTPC, with its large box, connected by a wire (sometimes through an intermediate external sound processor) into the AVR is just more bulky and 'wirey' than a small HDMI dongle in the back. (I've never had much luck with a networked server, and always try to hide as many wires and boxes as possible. Over the years I've built multiple behind-the-wall hidee holes.)

But anyway that's beside the point. I think it seems like - if I choose to standardize - flac is the way to go. I like the idea because I've worked with flac, and I know how to get all the info correct manually, through the use of xml files. Takes a little more time, but ultimately one complete, and uniform interface, with cover art, and track lists and some technical info, makes for a very slick presentation. Also, I find that I listen to stuff more when I have a good presentation like that. And as I get older, I've been telling myself I have to stop geeking out on the tech stuff so much, and start to enjoy these terabytes of stuff I've collected.

Thanks.

 

[LuvMyQuad]

well said.

I run a wireless keyboard on my MCH rig. I use a 4k TV Amazon remote for my bedroom system. My son uses both at the same time. The keyboard for work like stuff and maintenance (rarely). The tiny remote or his phone app for playing. The amazon remote on a firestick is the best way to go for playing.

 

[DuncanS]

Thanks all for the feedback. Seems like flac is the way to go, if I want to try to standardize everything.



I didn't phrase my point very artfully. I meant that when working on an HTPC, in my experience, one has much more ability to deal with configuration issues. Two basic reasons. (a) it is simply more comfortable to sit at a keyboard and mouse and work through the software, as opposed to some remote and (b) it has always seemed to me that windows applications are more configurable than versions meant for an android system, etc. As for it being cumbersome, I meant on a very basic level that having an HTPC, with its large box, connected by a wire (sometimes through an intermediate external sound processor) into the AVR is just more bulky and 'wirey' than a small HDMI dongle in the back. (I've never had much luck with a networked server, and always try to hide as many wires and boxes as possible. Over the years I've built multiple behind-the-wall hidee holes.)

But anyway that's beside the point. I think it seems like - if I choose to standardize - flac is the way to go. I like the idea because I've worked with flac, and I know how to get all the info correct manually, through the use of xml files. Takes a little more time, but ultimately one complete, and uniform interface, with cover art, and track lists and some technical info, makes for a very slick presentation. Also, I find that I listen to stuff more when I have a good presentation like that. And as I get older, I've been telling myself I have to stop geeking out on the tech stuff so much, and start to enjoy these terabytes of stuff I've collected.

Thanks.

I'm standardising on flacs and use mp3tag to easily add all the file info, artwork etc. Mp3tag - the universal Tag Editor (ID3v2, MP4, OGG, FLAC, ...)

 

[STLJLC]

I'm standardising on flacs and use mp3tag to easily add all the file info, artwork etc. Mp3tag - the universal Tag Editor (ID3v2, MP4, OGG, FLAC, ...)

Thanks. I've used mp3tag in the past to tag stuff, and it is super easy. I loved it. I stopped after a while though for a couple of reasons. First, even though I say I'm not a "purist," let's face it, I'm on this website. So, the idea of embedding a jpeg into the audio file - even though I know it doesn't really change the audio - started to bother me. Second, I generally like to create .md5 fingerprints to keep in teh folder, for use to detect corruption of files over time. I have on occasion changed cover art to something more authentic, or added a back cover, etc. I usually set my xml to look for a standard front, back, etc., and I can just drop a new jpeg into the folder, and rename the old one. Its more work to re-tag and recreate a fingerprint. But I do like that program.

 

[MidiMagic]

QS, so anyone can play it.

 

[jimfisheye]

What I aim to avoid:
Any lossy format that will remove data and reduce quality.
(eg. mp3, lossy dolby, lossy dts, the old analog stuff like QS, SQ, CD-4, etc)
Any proprietary format that requires paid for 3rd party decoders to access the content.
(eg. dolby, dts)

That lands me on FLAC for audio (up to 7.1 channel format) and MKV for video+audio.

I use FLAC for all wav file compression including studio projects like most people. The more official WAVPACK sort of came along too late. Except... Wavepack files support channel counts higher than 8 channels where FLAC stops. This will likely be a good choice for the extended 3D formats like Atmos. (Assuming the Atmos playback codec is ever released and the format sticks around, that is!)

 

[atrocity]

+1 for FLAC with the reminder that it won't work for Atmos.

 

[STLJLC]

Except... Wavepack files support channel counts higher than 8 channels where FLAC stops. This will likely be a good choice for the extended 3D formats like Atmos. (Assuming the Atmos playback codec is ever released and the format sticks around, that is!)

Interesting, Wavepack had somehow eluded me. I'll have to investigate. But sounds like flac is for sure the way to go.

 

[Mr. Afternoon]

FLAC for any PCM.
WavePack for any DSD. ISO may be more space-effective if you need both stereo and multichannel.

Any spatial audio is tricky because there's no 'best' universal format for it, so it's up to you on that.

 

[atrocity]

Wavepack files support channel counts higher than 8 channels where FLAC stops. This will likely be a good choice for the extended 3D formats like Atmos. (Assuming the Atmos playback codec is ever released and the format sticks around, that is!)

Keeping in mind that I'm not Atmos-equipped and can only guess based on what little I know, I don't see how ANY format other than Dolby's proprietary one is ever going to work for Atmos. The issue is that by definition there doesn't have to be a direct 1:1 relationship between the number of channels in the file and the number of speakers. A container keeping the unaltered original Dolby TrueHD will have all the necessary metadata to send [whatever] to [wherever], but it seems unlikely that any other container will keep it.

 

[jimfisheye]

Wavpack supports up to 256 channels. 7.1.4 + the full capability of 64 object channels would be no problem.

It's not so much any dislike of the container formats that lend themselves to downmixing and supporting lesser playback systems. That part makes perfect sense. It's the blinded by greed business of using those formats as proprietary copy protection. Like currently making anyone who want to play Atmos content buy new DAC and amplifier hardware to get access to hidden software.

 

[atrocity]

Wavpack supports up to 256 channels. 7.1.4 + the full capability of 64 object channels would be no problem.

OK...so let's say you have a Wavepack file with 64 completely separate, individual, discrete channels in it and you have an Atmos system with fewer than 64 speakers.

1. How do you send that 64 channel file to your pre-amp or receiver or whatever you use? Does HDMI support 64 channels? (I'm not being a wiseass, I honestly don't know.)

2. Since you have a Wavepack file and not Dolby TrueHD *and* you have fewer speakers than channels, what is making the decisions about what speakers to send those channels to and *how* is it making those decisions?

It's not so much any dislike of the container formats that lend themselves to downmixing and supporting lesser playback systems. That part makes perfect sense. It's the blinded by greed business of using those formats as proprietary copy protection. Like currently making anyone who want to play Atmos content buy new DAC and amplifier hardware to get access to hidden software.

Well, I certainly don't want to claim that Dolby isn't trying to monetize this as much as they can, but what I'm trying to get at is that I just don't understand how any other option *could* work. I assume that any normal PCM container, whatever that may be, is simply unable to steer the 64 channels into the appropriate 12 (or whatever) speakers.

I'm not trying to say that I think you're wrong, I'm just genuinely trying to understand how a non-Dolby container could make it work when we're talking about more channels than speakers and no way to handle the steering.

I hope I'm making sense! I'm not confident that my words are properly expressing what my brain is struggling with.

 

[Mr. Afternoon]

@atrocity Like I was saying there is no true standard for Spatial yet.
With PCM, you at least can convert any proprietary codecs to an open-sourced one, such as FLAC.
However, with spatial, there is no true open-sourced codec, and one codec cannot be losslessly converted to the other because each codec has different ways of spatializing.

 

[STLJLC]

@atrocity Like I was saying there is no true standard for Spatial yet.
With PCM, you at least can convert any proprietary codecs to an open-sourced one, such as FLAC.
However, with spatial, there is no true open-sourced codec, and one codec cannot be losslessly converted to the other because each codec has different ways of spatializing.

I recently ran up against this issue, I think. I had various .mkv videos with Atmos audio streams. And my hardware had a quirk. It would not play Atmos, when delivered from an mkv. I tried every way I could find to convert the mkv files to .mt2s. With every attempt, the audio stream was transmitted as TrueHD 7.1, but lost the spatial metadata, or otherwise was not recognized as Atmos.

So my solution ultimately was simply to change hardware.

 

[jimfisheye]

All good questions @atrocity!

Playback aside, you could archive the full content of all discrete channels to Wavpack. That's what I was thinking about.

The playback codec is given the speaker array. If the full compliment of speaker channels are available, all the channels are sent 1:1.
After that, the object channels are mixed on the fly into the available array. If that array is just a 7.1.4 system with no additional object channels, so be it. If you don't have height channels available, those are also downmixed into the available channels. The core 7.1 content is output 1:1. If you have less channels than that (eg, 5.1), it's all even further downmixed.

So there's good reason for intelligence in the playback codec!

Back to DIY land with Wavpack...
You could do something like downmix for your system and make Wavpack directly in the channel format you have. I wonder how much use of the object channels will happen beyond film use? Most music mixes might only ever be 7.1.4 and for those examples you could archive a 12 channel file.
That's what I was thinking anyway.

Again, I think the intelligence in the dolby container formats is a fine thing! My saltiness comes from the lockout right now. I'm sitting here with all the hardware I need and no way to decode their file. The true audio and master audio formats would honestly be convenient to keep that way too. That lossless compression is about the same size as flac or mlp. It's just leariness over getting locked out.

 

[atrocity]

Playback aside, you could archive the full content of all discrete channels to Wavpack. That's what I was thinking about.

Yep, I can see how it would work for archiving. But I don't see how it would ever be useful for playback.

The playback codec is given the speaker array. If the full compliment of speaker channels are available, all the channels are sent 1:1.
After that, the object channels are mixed on the fly into the available array.

That's pretty much what I had assumed. But it seems to me that once you separate that out into 64 archived channels, you have a weird reverse of the old matrix problem: Matrix decoding has a hard time fully unscrambling the egg, while what we're talking about needs to re-scramble the egg but has no guide for how to do it.

Or does it? Maybe, come to think of it, there's a an official, consistent way to re-assign the "extra" channels. Something like "In the absence of Speaker 32, the audio from Channel 32 is always added to to the audio from Channel 16." Maybe that's exactly how it works now...in fact, could it reliably/consistently work any other way?

You could do something like downmix for your system and make Wavpack directly in the channel format you have. I wonder how much use of the object channels will happen beyond film use? Most music mixes might only ever be 7.1.4 and for those examples you could archive a 12 channel file.

But at that point you're again up against HDMI constraints. Though A Simple Guide to HDMI Cables and Connections says that HDMI 2.0 supports up to 32 channels, so we're halfway to 64! I'd like to know if there's any hardware out there on the market that will accept more than 8 channels coming in via HDMI and, if so, if there's a standard for the layout similar to today's WAVEFORMATEXTENSIBLE_CHANNEL_MASK.

And of course if you downmix/remix something for your current hardware, you have to do it again if you later find yourself with more or fewer speakers.

Now I'm having some weird fantasy about having a 64 channel Wavpack file on disk that gets encoded to TrueHD/Atmos on the fly during playback...

I just can't shake this nagging feeling that there's something obvious and easy that I'm missing.

 

[MidiMagic]

The question is: Who is going to use this archive?

If it is going to be used only by experts, then all that is needed is a channel assignment and speaker location map included with the recording.

If anyone is to buy and use it, QS makes much more sense.