We backed up Spotify (metadata and music files). It’s distributed in bulk torrents (~300TB), grouped by popularity.

This release includes the largest publicly available music metadata database with 256 million tracks and 186 million unique ISRCs.

It’s the world’s first “preservation archive” for music which is fully open (meaning it can easily be mirrored by anyone with enough disk space), with 86 million music files, representing around 99.6% of listens.

you are viewing a single comment's thread
view the rest of the comments

[–] studentofarkad@lemmy.zip 41 points 3 days ago (4 children)

Are these the actual music files you can use to play as well?

[–] notgold@aussie.zone 43 points 3 days ago (2 children)

Would be amazing if it was. I would love to just have Spotify's music on my nas

[–] AstralPath@lemmy.ca 51 points 3 days ago (3 children)

I'd wager 70% of what's on Spotify is not worth preserving since its AI slop.

[–] nagaram@startrek.website 54 points 3 days ago (1 children)

I'm not convinced AI slop can compete with the back log of organic slop personally.

But yeah a fuckton is probably slop either way

[–] bear@lemmy.blahaj.zone 3 points 3 days ago

AI slop is accelerating exponentially for the foreseeable future. It won't take long for world data storage to be a limiting factor.

[–] skisnow@lemmy.ca 17 points 3 days ago (1 children)

Yeah as with most of the internet, it’s only worth downloading anything uploaded before 2023.

So far, LLMs have done so much more harm than help.

[–] cassandrafatigue@lemmy.dbzer0.com 1 points 3 days ago

That's what they're for.

[–] Hawk@lemmy.dbzer0.com 13 points 3 days ago

Interestingly enough, with the data they provide, figuring out how much of it is AI slop wouldn't be that hard I think

[–] DavidDoesLemmy@aussie.zone 25 points 3 days ago (2 children)

If your nas has 300tb spare, you could.

[–] sopularity_fax@sopuli.xyz 14 points 3 days ago (3 children)

How much could it cost $10,000

[–] No_Eponym@lemmy.ca 17 points 3 days ago (1 children)

[–] oyo@lemmy.zip 6 points 3 days ago

This gif is going to completely lose its punch in a couple years.

[–] infinitesunrise@slrpnk.net 11 points 3 days ago* (last edited 3 days ago) (1 children)

A RAID6 of 24 * 20TB drives could contain that with both parity and hotswap, with room to spare. Let's say $400 per refurb drive, $2500 rackmount SAS enclosure, $2000 SAS RAID card, $14,100 total. Assuming you already have the server and power and SAS cables.

[–] wheezy@lemmy.ml 4 points 3 days ago (2 children)

You could budget this way down. I run 10+2 12TB with Unraid. No reason for a raid card if it's for archive and personal use.

[–] infinitesunrise@slrpnk.net 1 points 3 days ago

Oh totally, could do SATA instead of SAS too. I used to build out servers and render farms for motion graphics studios that needed the ability for multiple people to be doing high-bandwidth operations on the same network drive, and the above was just kind of the default offering.

[–] brognak@lemmy.dbzer0.com 1 points 3 days ago (1 children)

100% this. People who store easily replaceable media on RAID are just throwing away money (unless you have a need for faster read/write). If it's your family photos, copy of your in progress thesis, or other irreplaceable piece of info/content go for it.

I have like 40tb Unraid NAS and I get asked pretty much every time I talk to someone about it how I do backups. Easy, I backup my *arr stack databases and in case of a failure I restore them and let it pull down everything over time. Which I have done in the past when I wanted to upgrade quality, easier for me to scrub it all and start over than make upgrade profiles and such.

Or that's what I would have done, now I mostly use DebridService du jour and Stremio :-)

[–] wheezy@lemmy.ml 1 points 3 days ago* (last edited 3 days ago)

Nice sounds similar to what I do. I bought Unraid because everyone always suggested it. No regrets. It really makes things simple and have only had minor problems (usually related to docker). I had a HDD fail and recovery was easy (besides the 18 hours of rebuild). But the drive is emulated during repair so no real downtime.

Family photos I use cloud storage for. I could probably get away with just doing local because I do hate that I'm sure they are being used to train AI. But I'd have to do off-site storage somewhere anyway. Parity drives don't really do well in the "fire" scenario. Don't want to have any risk of losing photos of my daughter growing up.

[–] BlueRingedOctopus@lemmy.dbzer0.com 2 points 3 days ago (2 children)

Considering the Price per TB is 10-11 US dollars, it's gonna cost $3500 max

[–] N0x0n@lemmy.ml 18 points 3 days ago (1 children)

10US dollar per TB?? 🤣🤣 More like 30/35€ per TB for a good graded HDD!

Let's not talk about SSDs or nvme which are more in the 120€/TB.

I always hear people say that storage comes cheap nowaday... I'm still looking for that cheap HDD on amazon... It has been 10 years 🤣🤣

[–] rainwall@piefed.social 9 points 3 days ago* (last edited 3 days ago) (1 children)

$10/TB is a bit low, but not far off. Serverpartdeals has refurbed enterprise/NAS drives at about $15/TB right now, and thats with the AI pressure driving up prices. I recall seeing 18TB drives around $12/TB a few months back.

The above is well loved vendor in the IT space. Much better place to buy from than Amazon, as they actually guarantee their inventory is legitimate.

[–] HelloRoot@lemy.lol 5 points 3 days ago

US of A often has way lower hdd prices compared to Europe.

Take the serverpartdeals price and add shipping and import tax.

[–] sopularity_fax@sopuli.xyz 6 points 3 days ago* (last edited 3 days ago) (2 children)

Is it that cheap now?? I would kill for 10tb

[–] floofloof@lemmy.ca 7 points 3 days ago

It's nowhere near that cheap.

[–] rainwall@piefed.social 4 points 3 days ago* (last edited 3 days ago) (2 children)

Here you go mate. They dont have 10TB in stock, but the do have 20TB refurbs at around $15/TB.

[–] sopularity_fax@sopuli.xyz 1 points 3 days ago

Thx

[–] notgold@aussie.zone 6 points 3 days ago

I read it as 30tb in my head. 300tb is a bit more than i can manage

[–] AnarchistArtificer@slrpnk.net 10 points 3 days ago

They've released torrents of the metadata, and they plan to release the music files, but they haven't yet. They intend to start by offering the downloads as bulk torrents, but they're open to considering implementing the ability to download single songs in the future.

So in short, yes, but you can't download them yet

[–] Jerkface@piefed.social 22 points 3 days ago* (last edited 3 days ago)

The article claims that they are:

We backed up Spotify (metadata and music files).

[–] mic_check_one_two@lemmy.dbzer0.com 5 points 3 days ago

Not yet, but that’s the end goal. The tricky part is that they’re only offering bulk downloads for now, which means downloading a single artist or album would be difficult/impossible. You’d need to download the entire compressed file of like 300GB of music, then extract the specific songs/artists/albums you wanted. The goal for now is preservation, meaning they want to make the bulk download as easy as possible, to make sure people can preserve it. Once they’ve got that in a pretty good spot, they may look into allowing more granular downloads.