New Opensource tool "Spleeter" for extracting stems

[zeerround]

FYI the readme file with the source points to this video:

 

[zeerround]

and this page: https://makenweb.com/spleeter_help.php

Really implies that they are using the GPU version of spleeter. Which could explain crashing etc. if you don't have a big GPU.

Also the readme in the source refers to an "expert mode" where you can choose your own python location, but I haven't yet figured out how to invoke that.

 

[mandrix]

This one uses it's own python installed under %APPDATA%\SpleeterGUI\python and doesn't add any environment varibles, etc. so "should" be standalone. However I think it could get messy if you do have another python DOES add itself to PATH, etc.

I don't think the SpleeterGUI\python is a static part of the msi, or else we wouldn't be seeing these issues with the latest version of Numba, however. So, my theory is either that 1) You installed it BEFORE the broken for windows numba release (I think was on the 5th) or 2) Yeah your other Python is being used, because it's in your PATH environment variable. Guess "we" should look at the source code to lean more about how the GUI (and msi?) work, but I think the msi is separate and build instructions not shared. The author told me he doesn't know much about Python and his friend packaged it for him.

@boondocks; did you uninstall Anacoda before trying this one? Windows 10? How old is your computer?

For all these GUIs and Batch Scripts there has to be a working python and spleeter underneath, and unfortunately if you are not familiar with command line and some python commands it could be rough going if it doesn't work the first time/every time. Certainly for debug it would help to run spleeter on the command line and report any errors.

e.g.

1) Open a command window and cd %APPDATA%\SpleeterGUI\python
2) Type python -m spleeter separate -i "full path to a stereo wav file"
3) Capture text output (errors or success)

No, I had already installed Anaconda & ffmpeg and added them to the path manually. The GUI will install everything?
I'm running Win 10 Pro x64 on an Asus Maximus MB with an overclocked/delidded - liquid metal TIM - relidded/water cooled 6700K i7, 16GB DRAM. C drive is a Samsung 840 Pro 256K SSD. DRAM is overclocked as well, but not stupidly so. Will try the command prompt instructions now.

 

[mandrix]

zeeround here is the output from your instructions:
E:\Downloads>cd %APPDATA%\SpleeterGUI\python

E:\Downloads>python -m spleeter separate -i e:\downloads tunnel of love.flac
ERROR:spleeter:An error occurs with ffprobe (see ffprobe output below)

ffprobe version git-2020-05-04-5767a2e Copyright (c) 2007-2020 the FFmpeg developers
built with gcc 9.3.1 (GCC) 20200328
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 43.100 / 56. 43.100
libavcodec 58. 82.100 / 58. 82.100
libavformat 58. 42.102 / 58. 42.102
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 80.100 / 7. 80.100
libswscale 5. 6.101 / 5. 6.101
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
e:\downloads: Permission denied

 

[zeerround]

Guess your computer should be beefy enough...

Yes SpleeterGUI has it's own python, etc.

What happens if you add .\ in front of python, so:

.\python -m spleeter separate -i e:\downloads tunnel of love.flac​

​

That should force the version of python in that directory

Can't help much with the ffmpeg error. The permission denied error must be some problem however. Oh it looks like maybe you are missing some quotes or slashes or both. did you mean "e:\downloads\tunnel of love.flac" maybe?

A good way to get paths/filenames into the command line is to go to them in an explorer, then hold down the shift key while while right clicking on the file and select "copy as path". Then back in the command window you right click to paste (or in later versions ow windows 10 you can also use control v).

 

[Marplot]

If you are processing in directory, you will not need the e:\downloads\. Depending on the O/S and interpreter, the spaces and special characters may also give the CLI fits.
"e:\downloads\tunnel of love.flac"
'e:\downloads\tunnel of love.flac'
"e\:\\downloads\\tunnel\ of\ love.flac"
Easiest thing to do, rename it love.flac and just run python -m spleeter separate -i love.flac to make sure it is doing what you want. Then play with the CLI issue that you are having.

But, the quotes and missing slash that zeerround provided should actually do the job.

 

[zcftr29]

You don't download anything. You should have an "Anaconda Prompt" icon on your desktop. double click that. Then in there you enter:

conda install -c conda-forge spleeter

for the cpu version or

conda install -c conda-forge spleeter-gpu

if you have a big Nvidia GPU (but I don't know if zcftr29's script supports that)

The batch script should work with both the gpu and non-gpu versions of Spleeter as the usage commands are the same, just the installation packages are different. RE the need for a beefy computer, mine is at leas 8 years old - it has an Nvidia Quadro video card, and 16GB ram, but it's certainly nothing special (an old one from the office which I re-purposed. I think the problems I was having are due to graphic card memory limitations - hence my script splits into 20-second chunks and then stitches back together which gets over the issue.

 

[mandrix]

Guess your computer should be beefy enough...

Yes SpleeterGUI has it's own python, etc.

What happens if you add .\ in front of python, so:

.\python -m spleeter separate -i e:\downloads tunnel of love.flac​

​

That should force the version of python in that directory

Can't help much with the ffmpeg error. The permission denied error must be some problem however. Oh it looks like maybe you are missing some quotes or slashes or both. did you mean "e:\downloads\tunnel of love.flac" maybe?

A good way to get paths/filenames into the command line is to go to them in an explorer, then hold down the shift key while while right clicking on the file and select "copy as path". Then back in the command window you right click to paste (or in later versions ow windows 10 you can also use control v).

ok i got 2 stems. The gooey isn't working for me though. So I guess I need to find out how to get more than 2 stems?
BTW I keep seeing people say they have a command line on the desktop for cobra? I don't have that. Wondering if I ought to start over and let the GUI install everything?

 

[mandrix]

Man o man age has caiught up. I can't type a sentence anymore w/o hitting the spacebar once or twice.
Anyway I did a backup before installing all this so I think I will start over tonight and let the GUI ins tall it all.
I've done all this 3 or 4 times because one link says do this, but then doesn't tell you anything else. e.g. I had no idea the GUI also would install anaconda and ffpmpeg. You guys should keep in mind that some of us are out of our element , lol.
I thank Zeeround for always giving precise instructions, even if I don't understand them.

 

[zeerround]

OK you are mixing up some names and what they do.

Re all the "snake" terms:

Python is a scripting computer language. Spleeter is built on python and hundreds of inter dependent python "packages", to the point where you need to manage python for spleeter specifically vs. other stuff you might be using python for. Fortunately there are ways to handle this (see below). Python comes with a package management system called pip. You would use pip to install/unistall, upgrade and downgrade Python packages. It also gets the packages for you. Everything is command line.

"Cobra" is a flavor of Ford Mustang ;0)

Anaconda is a python distribution. It, by itself, includes most of what you need to do scientific type stuff with python, and has it's own package management system, conda, to handle updates. Note that you really shouldn't mix conda and pip, because they keep track of things differently and you'll likely mess something up in your Anaconda setup.

Anconda also has the concept of environments, so you could have a python environment just for spleeter, and another one for astronomy, or whatever your other hobbies/interests are that use python. That way you don't have to worry about the above mentioned problem of installing a needed update for some package and having it break spleeter. But you do have to set up and mange the environments yourself (see below on spleeter and environments). Any tool that you use on top of command line spleeter would have to know about, or be launched from, your spleeter environment, to keep the advantages of a spleeter environment.

Spleeter folks suggest you use Anaconda, and they have their own conda-forge channel to get you spleeter under Anaconda. My experience is that they are still assuming you are familiar with Python and Anaconda Environments, so they leave instructions out and or don't fully address the environment thing.

You probably don't want to have generic python (python for windows) AND Anaconda python, etc. because they both want to setup environment variables so python itself, and python scripts etc. get found when you issue a command (without a full path name). As I mentioned before, the SpleeterGUI has it's own python, and doesn't set any environment variables, so that should co-exist with either Anaconda or native python for windows (but not both) AFAIK. But if all you are trying to do is use spleeter you should pick just one and stick with it.

Say Anaconda Python, with the conda command to keep spleeter updated, and a batch script or just command line.

OR

SpleeterGUI with its own python as a one time install that is going to stay fixed forever, in terms of spleeter and python, unless the SpleeterGUI author comes up with his own update.

I mentioned the ReadMe for SpleeterGUI mentioned and expert mode where you could use another python/spleeter install, but I see no evidence of that being implemented in the source code. The path to Python is hard coded.

Doesn't mean we couldn't hack the source code, but that's how it's made to work right now.

I also have no idea about the SpleeterGUI msi installer. I'm not an expert in C# or msi, etc. but I don't think any of that is currently part of the source. Maybe others can figure it out (in terms of is python in the msi or does the msi go an get some version of python at run time).

Back to spleeter for a moment, in chatting in email with my upmixer friend today he said:

"I am on the Spleeter email list & can not believe all the issues people are having with installing & using it. They are making too many changes that are causing issues IMO. I really like using it in the Acoustica program where it works fast & mostly a full spectrum with pretty low artifacts.​

​

I am still waiting for the MIT release of Open-Unmix Pro version with a much lager training base that they said will take over a year to do & then will require copyright releases also so likely be a pay version. I need to do a follow up email now to see how it is going."​

Open-Unmix is similar to spleeter in it approach and tool set, and pre-dates spleeter (but spleeter is supposed to be mo' better) but your mileage may vary on any given song, etc.

As I posted earlier in the thread spleeter is only one of the new "unmix" or music separation tools, and not necessarily the best (or essayist to use).

Hope this clears up the snake references, etc.

 

[zeerround]

"BTW I keep seeing people say they have a command line on the desktop for cobra Anaconda? I don't have that. "

Yeah it might be an option in the Anaconda installer. Don't recall. If you have an icon on the desktop it should be named "Anaconda Prompt".

"So I guess I need to find out how to get more than 2 stems? "

python -m spleeter separate
usage: spleeter separate [-h] [-a AUDIO_ADAPTER] [-p CONFIGURATION]
[--verbose] -i INPUTS [INPUTS ...] [-o OUTPUT_PATH]
[-f FILENAME_FORMAT] [-d DURATION] [-s OFFSET]
[-c {wav,mp3,ogg,m4a,wma,flac}] [-b BITRATE] [-m]
[-B {tensorflow,librosa,auto}]

CONFIGURATION can be:

spleeter:2stems
spleeter:4stems
spleeter:5stems
spleeter:2stems-16kHz
spleeter:4stems-16kHz
spleeter:5stems-16kHz

See here: deezer/spleeter

There are also commands for bit depth...

 

[mandrix]

Sorry, I was probably being sarcastic about the snake thing but there's an even chance it was just confusion. Part of it is I haven't been able to (until a little bit ago) to actually sit down for any period of time.
So anyway back to square one twice to see if anything installs correctly. The spleeter GUI would NOT install anything but itself. period. I did a system restore from Macrium backup and started from scratch. twice.
Since I'm now confused that nothing would run I wiped it all out and I installed separately Anaconda & ffpmpeg. I have confirmed the anaconda install in the cmd environment by a short tutorial I found. I did find in the start menu a listing for Anaconda Powershell prompt which I'm using to try and install the Spleeter package. The package is taking a long time to install, as it's now looking for incompatible packages. go figure.

 

[zeerround]

ugh, Windows 10 is now hiding cmd in favor of powershell (and they are getting close to a whole new "terminal" app anyway, which looks like... Linux ;0)

I did some registry hack or other setting (I forget which) to get the cmd menu items back. Anyway I guess Anaconda Powershell would work too. FYI I have Anaconda prompt(anaconda 3) and Anocanda Powershell prompt in my start menu.
...
"it's now looking for incompatible packages. "

Yeah that's part of the job of conda (or pip), to dependency map everything, and there are so many packages...

 

[zcftr29]

ugh, Windows 10 is now hiding cmd in favor of powershell (and they are getting close to a whole new "terminal" app anyway, which looks like... Linux ;0)

I did some registry hack or other setting (I forget which) to get the cmd menu items back. Anyway I guess Anaconda Powershell would work too. FYI I have Anaconda prompt(anaconda 3) and Anocanda Powershell prompt in my start menu.
...
"it's now looking for incompatible packages. "

Yeah that's part of the job of conda (or pip), to dependency map everything, and there are so many packages...

Whilst I know my way around the windows command line (with some occasional help from the excellent SS64 Command line reference website), I get lost very quickly with stuff like python, so relied heavily on forums etc to get Spleeter up and running. I had some difficulty when I first installed - I tried Anaconda with no luck then uninstalled and tried miniconda instead. I think I first got permissions errors relating to the environment - and tried to follow some instructions to create a dedicated Spleeter environment, but got out of my depth pretty quick so persevered with the default environment. If I recall correctly, I got a few messages about stuff being missing, so I took a punt and just typed "conda install -c conda-forge XXXXX" for whatever was missing. By luck or judgment it worked eventually. Once I'd established that Spleeter did work, it took me a little while to work out that to use via a batch script (or any CMD terminal) you first need to make sure that the Anaconda/Miniconda folder is in your PATH and then activate the Anaconda/Miniconda environment by typing "call activate C:\ProgramData\Miniconda3". From here on, you can just use the Spleeter commands as per Zeerround's posts above.

 

[mandrix]

Whilst I know my way around the windows command line (with some occasional help from the excellent SS64 Command line reference website), I get lost very quickly with stuff like python, so relied heavily on forums etc to get Spleeter up and running. I had some difficulty when I first installed - I tried Anaconda with no luck then uninstalled and tried miniconda instead. I think I first got permissions errors relating to the environment - and tried to follow some instructions to create a dedicated Spleeter environment, but got out of my depth pretty quick so persevered with the default environment. If I recall correctly, I got a few messages about stuff being missing, so I took a punt and just typed "conda install -c conda-forge XXXXX" for whatever was missing. By luck or judgment it worked eventually. Once I'd established that Spleeter did work, it took me a little while to work out that to use via a batch script (or any CMD terminal) you first need to make sure that the Anaconda/Miniconda folder is in your PATH and then activate the Anaconda/Miniconda environment by typing "call activate C:\ProgramData\Miniconda3". From here on, you can just use the Spleeter commands as per Zeerround's posts above.

Sweet, glad you got it working. Strange that I have now installed and/or rolled back to a saved C drive image several times now and only managed to get Spleeter to work a few times...and then only because Zeeround had some good suggestions and I got it working via command line. But although I got the Conda Powershell link in my start menu, I never got the command line on the desktop. Also the SpleeterGUI would never install Conda or ffmpeg. Last go-around I once again installed Anaconda then ffmpeg individually and tried to get the Spleeter installed it kept throwing errors.

I'm encouraged by your stick-to-it and I'll try and follow suit today if time permits.

 

[winopener]

I have found a weird thing and can't figure it out if it's as a missing option/argument on the cli or stems or whatever.
That's the situation: same PC with Anaconda + Spleeter, then installed also Acoustica demo with its own Spleeter. Using the same audio track source (44100 16bit 2 channel WAV 1411 bitrate - as on any CD) i have VERY different results between the two systems (Anaconda/Acoustica) not about the spleeting itself but on the resulting audio file.
1) Anaconda: spectral shows heavy artifacts as if it was a 128Kb mp3; valid for both 11k and 16k option. Any kind of file format, lossless or lossy, shows the same
2) Acoustica: spectral looks much better and artifacts are WAY LESS pronounced. Also, the extraction goes beyond the 16k limits.

Question are:
1) conda cli is missing an argument, or there is a forced mp3-lowbitrate condition, regardless of any audio format or bitrate passed on the cli?
2) the training set of spleeter for acoustica is different from the one of spleeter for anaconda, and improved?

 

[winopener]

 

[zcftr29]

View attachment 50809

My guess is that Acoustica have tweaked the config json file as per the FAQ here: Why are there no high frequencies in the generated output files?

According to this, the Spleeter models are only trained up to 11kHz, but work ok to 16kHz with some artifacts above 11kHz. It does warn that unexpected results may occur in the higher frequency ranges when separating to 22kHz and also that it doubles RAM usage - so it would probably be even less stable.

I was wondering if it might be possible to create my own model - trained using the HQ uncompressed MUSBD here (MUSDB18-HQ Uncompressed WAV) but I haven't got my head around how to do this (or how long it would take). This would be a great collaborative project to do and then share the model here so that we can all get better results. I'm sure if we put our heads together we could work it all out...

 

[mandrix]

But do we know if they already maxed out with that file, and would another one in addition be better? I t hought I read that they were encouraging people to take it a step farther and I might have assumed a new t raining source.

 

[zeerround]

Looking through the issues and project files yesterday on spleeter, it sounds like a lot of training options will be in the next version.

I also wondered about training, as "we" wouldn't have the constraints around only using open rights music, etc. but I also wasn't sure if "we" could put together more/enough high quality stereo, with matching stems/mulitracks or not (and keep in mind that rockband/oog files don't count as high quality).

Re: Acoustica, from what my re-mixer friend has said, and what has been mentioned here, I think that (maybe) extracting one source at a time is going to yield cleaner results. However, if the goal is to put back all the separations and end up with the original (and I have done those listening tests) splitting them all at once may be the more valid approach. AFAIK the "other" is generated by summing the separations back together and subtracting from the stereo.

Anyway, for the purposes of remixing/upmixing the normal artifact reduction stuff applies.

1) Don't stress (to much) if you hear it only in isolation​

2) mixing a little of the adjacent channel and/or the original stereo is a trade off between artifacts and separation.​

3) But keep in mind that separation helps things stand out in the mix (surround "WOW" factor)​

All that said, and all the issues with this one tool aside I'm still curious to see what people will do, in terms up upmixing, with the current toolsets.

It would be different story, for me, if you got each drum/cymbal, or each guitar, etc. as separate stems or multitracks, but as it is I'm not sure of the added value for stereo to 5.1

One could take each stem, and upmix it with specweb, using different image widths and rotations, vs. just assigning a stem to C, fronts, or rears, and I haven't done a lot of that, so curious with what others will come up with.

Rotation would let you, say, flip the drum upmix around so the "center" is virtual rear center, or pan an upmixed stem left or right in the surround field.​

I have been thinking about some sort of visualization tool, that shows you the LR angle of energy throughout the song, then having that as a panner/mixing tool for the (upmixed) stems. Hard to describe in words.