If the interest is in the experience of the old tech (all the interesting parts along with any and all the shortcomings), then that's that.
But if the interest is in the actual music and seeking out and preserving it with the least generational damage...
Our modern 24 bit HD digital format and available equipment is your friend!
Computer = access to ALL digital formats in the highest quality the format is capable of. 24/96 is pretty much what comes off the mixing desk and mastering desk now. You can deliver and preserve this 1:1
The critical parts are the AD conversion going in and the DA conversion going out. Spend high dollar on those bits!
The rest in-between is shuttling ones and zeros around. Learn the system enough to do this losslessly (ie avoid unnecessary conversions or digital reductions).
Class A AD stages WILL preserve an analog signal fully and completely at 24 bit HD. Invest time up front framing and capturing that signal and then preserve it digitally with 24 bit HD.
This part obviously gets into working with older analog formats (for older music), so you'll have plenty of fun and old tech experiences along the way here!
You can kind of have everything now. HD digital can be your baseline and preservation format and you can add old analog devices to the mix for digging up old buried treasure.
It's not really analog vs digital either! Analog never went away. The AD and DA stages are exceeding critical and require precision in their analog stages. Garbage in, garbage out applies more than ever. A so-so digitization of something with consumer grade converters leaves a lot on the cutting room floor by an incomplete capture. This isn't the fault of the digital system when this happens. It's the analog stage in the AD converter lacking. On playback, a cheap DA stage means not fully hearing what's in the audio data.
Invest in a good audio interface with high quality AD and DA stages first.
Get comfortable with the computer. It really makes even just playback simple. Especially with access to literally every digital format.
Then go after older analog devices to get access to old recordings. Now you can actually hear what's on them to the highest level and preserve them.
That's what I recommend.
But again, if the aim is truly for the user experience of old tech and one is not shooting for audiophile or at least fullest fidelity possible, then carry on with that.