I'm going to dissent mildly from all the discussion of SSDs. They're nice - they give a faster loading time and startup and applications load faster. But the cost per GB is much higher, and a 5400 RPM HDD has the transfer rate to play 24/192 PCM audio - provided there's enough memory for a buffer and playback. In a laptop with only one hard drive, I'd get a 1TB (or more) physical disc so you don't need to plug in external drives constantly, and deal with the fast that startup takes 30-40 seconds longer.
Western Digital made an excellent hybrid drive called the Black2, which was a combination of a 120GB SSD and a 1TB HDD in one laptop-sized hard drive. I use one of them in my laptop and it runs like a dream - the best $100 I spent on an upgrade that I know of. However, they don't seem to be making that model anymore. Seagate makes a hybrid drive with 32gb of flash...it's not the same as a true SSD, but it will get you most of the way there:
https://www.amazon.com/Seagate-Gaming-2-5-Inch-Internal-ST1000LX001/
Of course, the ideal is to have a SSD with the programs and OS, and a HDD with all your music. One thing you might want to consider if (and this might be a big if) you're a little technically inclined is to do what I did and to build your own mini PC. It will cost far less and give you something far more versatile and powerful. It will mean using the tv as your monitor - and it will mean you won't need to unplug or move anything around.
The story of how I built my HTPC is here:
http://forums.anandtech.com/showthread.php?t=2155549. It's 5 years old and it runs like a pro - it plays every surround audio format I throw at it over HDMI to my receiver, and with a 40GB SSD and a 2TB HDD it boots fast and stores all my music. And it's the size of a large shoebox.
Just as a checklist, these are the parts you'll need:
MiniITX motherboard - this is the core of the system, and I use one of these in mine. It's a tiny form factor, but it gives you everything you'll need. If you're willing to spend $140 or so on the motherboard you can get one with dual HDMI (not necessary, but a nice option), integrated wireless/bluetooth, 6-8 USB ports, and the latest chipset from Intel. It's fine to use the Intel integrated graphics - the key is that they support HDMI 1.3 or above (I believe all recent Intel integrated graphics system support HDMI 1.4). This means that they can stream lossless high-res audio over HDMI.
CPU - The Core I5-6400 would be a good choice - it gives you serious performance and lower power, meaning it will be easier to keep cool in a small case. Beware (for this part and others) that a lot of review online are focused on gaming performance, which requires much more power than high-res audio.
For a case, this would be a great choice (including a small power supply that should be fine for your purposes) for $50:
http://www.newegg.com/Product/Product.aspx?Item=N82E16811147131. After that you'd get whatever memory your motherboard/CPU calls for (probably DDR3/3L/4) - 16 GB ideally - an internal blu-ray drive, and the hard drive/ssd. All said and done it should cost you less than $700, even including a OEM copy of Windows 10. I recognize this is getting a little techie, but it's not really that complicated to do.
For a keyboard/mouse, you'd use this:
http://www.logitech.com/en-us/product/wireless-touch-keyboard-k400r
As yet another option, has anyone tried one of these as a surround music PC? It has a HDMI port that's supposed to be HDMI 1.4a compliant:
http://www.newegg.com/Product/Product.aspx?Item=N82E16856102096