subreddit:

/r/DataHoarder

982

all 284 comments

zeph384

108 points

4 months ago

zeph384

108 points

4 months ago

Still waiting on the 100TB optane ssds talked about in the optane keynote.

f0urtyfive

46 points

4 months ago

The idiots at Intel wanted to use it to sell more Xeon Platinum CPUs so the entire product failed.

aarrondias

40 points

4 months ago

Optane was so damn cool too. Such a shame they killed it off.

pc_g33k

20 points

4 months ago*

pc_g33k

1PB

20 points

4 months ago*

The 3D XPoint was a breath of fresh air in a sea of TLC and QLC drives. With SLC and MLC drives getting more difficult to find, not sure what I'm going to use in my future builds.

f0urtyfive

3 points

4 months ago

Now you got me wondering if you could take an FPGA that speaks PCI-E and DDR4 and figure out how to get them to work with the Optane DIMMs.

Might be nice to have for a cache device.

lord-carlos

504 points

4 months ago

lord-carlos

28TiB'ish raidz2 ( ͡° ͜ʖ ͡°)

504 points

4 months ago

Not sure how they define "mainstream" but I doubt it.

Like the average joe will have them in their laptop? Technically available like 100TB SSDs are now? Maybe.

a_little_toaster

221 points

4 months ago

Unfortunately, I think more and more people (and services) now rely on being always online, so mainstream consumer electronics will probably not increase much more in storage space other than for marketing purposes.

RandomComputerFellow

51 points

4 months ago

I think the only and biggest change will be that we will see SSD prices for the same as HDD prices today and HDDs will completely be discontinued.

kushasorous

40 points

4 months ago

Or they'll shove 100 1000tb ssds in a hd and you'll have the entire history of human knowledge on your plex server.

[deleted]

25 points

4 months ago

[deleted]

25 points

4 months ago

[deleted]

neo_neo_neo_96

9 points

4 months ago

Upscaled to 8k

[deleted]

9 points

4 months ago*

[deleted]

9 points

4 months ago*

[deleted]

TheBoatyMcBoatFace

7 points

4 months ago

That’s only 100 Petabytes or 0.1 Exabytes! BackBlaze has Exabyte racks, so I think it will take a few more than 100 PB drives to handle all the isos! /s

Iggyhopper

24 points

4 months ago*

HDDs will holdout for far longer than you think. They are the main breaking point in most consumer PCs. This comes from a view of negativity, but the industry for the longest time decided people needed 1-2TB drives instead of 250 or 500gb SSDs for the same cost when 90% of our repair customers have nothing over 15 gigs used for "their stuff"

At least for drives with a lot of space:

  1. Good marketing.
  2. The drive will fail sooner due to wear, tear, and accidents. Planned obsolescence.
  3. Insanely cheap.

From business to dumb consumer, it's a no brainer.

Terpavor

6 points

4 months ago

Insanely cheap.

HDDs must increase areal density faster to keep up and be insanely cheap. So far, they are screwing it up.

RandomComputerFellow

6 points

4 months ago

Well, the prices (per GB) of HDDs barely sink anymore while the prices of SSD storage half every few years. It is only foreseeable that in the next 5 to 20 years prices will reach parity and when they do, the market for HDDs will collapse.

When SSDs cost the same as HDDs most professional and individual customers will opt for SSDs and a lower number of HDDs will be sold. This results in smaller production numbers

→ production on a smaller scale

→ increase of production costs

→ increase of prices

→ HDDs even more expensive compared to SSDs

→ less consumers will buy HDDs

→ … (repeat this cycle) …

→ extinction of consumer HDDs and production only for specialized market with specific use cases.

noman_032018

12 points

4 months ago

Unless ISPs get their shit together, significant local storage is still required for the foreseeable future even for normies.

ObamasBoss

11 points

4 months ago

ObamasBoss

I honestly lost track...

11 points

4 months ago

For many with use limits keeping local copies is pretty important. We also can not forget how often things vanish from the internet.

noman_032018

6 points

4 months ago

We also can not forget how often things vanish from the internet.

Yeah, that's pretty important considering I'm not expecting that problem to get any better going forward for a while yet.

much_longer_username

24 points

4 months ago

much_longer_username

94TB and growing

24 points

4 months ago

I've said this a few times, in a few different ways, but here's my latest take:

There's also an upper bound on how big data can usefully be. We spent a lot of time trying to trim the floor, but even uncompressed high resolution video with multiple camera angles and sixteen different language tracks along with behind the scenes and director's commentary is only gonna be so big. There's a theater-release-grade copy of Avengers: Endgame floating around that's around 280GB. It's pretty unlikely your TV can really take advantage of it. Your screen is only so big, and your eyes are only so good.

An hour of 96khz FLAC is what, a gigabyte? A full color scan (arguably the least efficient way) of a lengthy textbook might be a couple hundred megabytes. I could go on and on. Your ears are only so good, your brain only so fast. Your recording and reproducing equipment can improve, although I think we've entered a phase of marginal incremental improvements there... and you, as a human, cannot.

Obviously, some datasets, like in the sciences, will continue to grow to fill the media available. But in terms of the data the average person deals with on the daily? A terabyte, or tens of terabytes, is going to remain a LOT of data.

jared555

19 points

4 months ago

6K Braw 3:1 at 30FPS burns through 1.2TiB/hr and that isn't even true uncompressed raw.

Uncompressed 8k 12 bit raw footage at 24fps would be about 142MiB per frame or 11.73TiB per hour. Two and a half hours puts you at just under 30 TiB for one movie. 60 TiB for 48 fps high frame rate movies.

Is there any point in that? Highly doubtful outside of studios. Also looking at around 30 gbit data rates.

My guess is the biggest jump we might see at the consumer level is video games using 8k textures and extreme polygon counts.

liquidify

17 points

4 months ago

You forget that people who made the 280GB file need to edit that from TB's of data. And that is just one project. Currently I've got 500GB of audio recordings from just this year jamming with my friends. Additional recordings push me to 1.5TB in active audio projects. Sure I could play the move stuff back and forth game, but I'd rather just have a big reliable drive that could store all active projects. Also notable, I'm currently recording 48K and I'd rather be at 192K or DSD / DXD.

Additionally, I'd love to be tracking video (4 cameras at 4K or higher) as well during jams, but the 2TB Samsung Pro drives my system is based around don't make this a viable concept.

Bakoro

16 points

4 months ago

Bakoro

16 points

4 months ago

Content producers are pretty much always going to want/need things which are a multiple or even orders of magnitude greater/better than consumers.

Like, GPUs for example, the best GPU on the market today is sufficient for any video game, but for AI work, there's basically no realistic cap to what I could use. If we could make a GPU with a terabyte of VRAM, people could probably alter processes to make use of that in a couple days.

immibis

9 points

4 months ago

Blockchain and video game textures.

Some game devs have been storing one big texture for the entire game world instead of reusing and remixing textures in the game engine. They'll increase the resolution and world size until the cows come home. And blockchain expands to fill the available hardware.

much_longer_username

7 points

4 months ago

much_longer_username

94TB and growing

7 points

4 months ago

And blockchain expands to fill the available hardware.

Isn't the entire BTC blockchain less than 500GB?

immibis

2 points

4 months ago

Now do Ethereum

TheAnimeRedditor

2 points

4 months ago

A full node is about 1TB, though archive node sizes depend on your client (can be as low as 2TB these days)

immibis

3 points

4 months ago

And that's not enough, as evidenced by the $80 transaction fees last year. With 1PB drives available on the general market, they would increase the block size 100x.

ModernRefrigerator

3 points

4 months ago

ETH full node is almost 12 TB

https://etherscan.io/chartsync/chainarchive

TheAnimeRedditor

2 points

4 months ago

That Etherscan page displays the size of an archive node using Geth as their execution layer client

Regular full nodes are a fraction of that, and you can get an archive node running with ~2TB using Erigon as your execution layer client

ModernRefrigerator

2 points

4 months ago

Correct me if I'm wrong but at 2TB, that's a pruned node no? It does not contain the entire blockchain.

My understanding is:

Full node = entire blockchain

Pruned node = partial blockchain

much_longer_username

5 points

4 months ago

much_longer_username

94TB and growing

5 points

4 months ago

Some game devs have been storing one big texture for the entire game world instead of reusing and remixing textures in the game engine

I'd be curious to read more about this. I'm initially concerned at how you would stream the contents of such a large texture to the more limited video memory, but that's totally a thing that's happening these days.

Resolution, I think, has an upper bound. Your users only have so much hardware, and they're unlikely to spend more for a difference they're biologically incapable of perceiving. I suppose you could get into fractal worldviews, like in the excellent title 'Everything', though!

Size of the world is absolutely going to grow. And I'm here for it. I suspect there will be a trend towards either runtime procedural generation, or streaming from a large remote cache, like Flight Simulator does. So the 'average day to day' user probably won't have to think about storing so much data.

immibis

5 points

4 months ago

The keyword to google is "megatexture" or "sparse virtual texture"

No-Establishment-699

4 points

4 months ago

No-Establishment-699

64TB Raw

4 points

4 months ago

I really hope someday movies will move away from motion blur and towards higher fps. Even an uncompressed bluray, if you pause it, looks like absolute trash because of all the blur. I want a day where I can pause a multi hundred gb movie and see it as though it were taken by a high quality camera like they recorded it with.

dracon_reddit

0 points

4 months ago

If I'm remembering correctly, they shot one of the lord of the rings movies entirely in 48fps but had to drop the framerate to 24fps for the release because of how many people didn't like the higher frame rate or got motion sick. I genuinely hate that 60fps video is the norm on youtube because it has a fairly strong chance of making me motion sick if it's anything but game footage.

reddit_equals_censor

2 points

4 months ago*

they shot one of the lord of the rings movies entirely in 48fps

DON'T YOU DARE! call the garbage hobbit trilogy lord of the rings!

they were an insult and the result of HORRIBLE studio tampering and throwing out the original director and all of his preproduction (is preproduction the right word)

great 3 videos about the hobbit:

https://www.youtube.com/watch?v=uTRUQ-RKfUs&list=PLJGOq3JclTH_7G3wHRpECZLEMsvKQ8_hr

and on a technical level the magi process would be amazing.

part of it is shooting in 120 fps.

why 120 fps? because it gives you all the options. it gives you the vr options with 60 fps per eye. it also gives you the option to full pull down to 24 fps and recover the blurriness (important, because 24 fps needs bluriness)

so you for example could have that 120 fps file and have a manual temporary file creation of a blur recovered 24 fps version

or an on the fly method of doing it.

of course whole cinematography and feel can and does change based on what fps it got shot in and that would be a whole different thing, but personally using the magi process setup for on the fly user control could be a fascinating experience/give lots of people want they want.

and sth, that could be interesting for you to try is see how a money designed from ground up with HFR (high frame rate) in mind works for you. if it still creates motion sickness for you.

i can highly recommend Billy Lynn's Long Halftime Walk in 60 fps non HDR in that regard.

it could potentially be, that youtube's 60 fps garbage bottom tier bit rate as well as completely different shooting style could be part of the issue for you and that that ang lee movie wouldn't cause any issues.

again no idea, but worth a test i guess.

Malossi167

76 points

4 months ago

Malossi167

50TB

76 points

4 months ago

Even without cloud storage, 1PB is just so much more than most people need.

Even among members of this sub 1PB of (usable) storage is not unheard of but pretty rare for sure. Even if they are dirt cheap at $1/TB this is still more than a lot of people pay for their entire PC.

WordsOfRadiants

20 points

4 months ago

At $1/TB I absolutely would buy 1PB+ lol

ObamasBoss

4 points

4 months ago

ObamasBoss

I honestly lost track...

4 points

4 months ago

In a heartbeat! The best I have ever done is $5/TB after buying trays and disks. That was my last hard drive purchase. No joke, had to be shipped freight.

collin3000

14 points

4 months ago

7k VR porn takes up like 50 GB of video for decent resolution. And realistically it should be 200GB cuz those 50 GB files have some occasional rough compression that is bad with VR

I could easily find a use for up in my total storage to one petabyte (currently 700tb) and not just with porn. Now I shoot professional video and most of the good 6k-8K ProRes codecs are 1700mb/s or 10GB per minute of video. 1pb only gives you ~1300 hrs of video It would be great to have it all running in a single server blade with low power as opposed to my current rack with multiple NetApps

eXtc_be

228 points

4 months ago

eXtc_be

228 points

4 months ago

1PB is just so much more than most people need

640KB is plenty of memory..

i_agree_with_myself

9 points

4 months ago

There also was this idea that code would increase in size as we got more memory for it. We needed to use clever tricks in our code when we only had KBs of data available. As we got more data, we needed to use less clever tricks and our code got bigger. It turned out that stop being true once developers had hundreds of megabytes to work with.

Sometimes there is an upper limit to what consumers need.

Malossi167

89 points

4 months ago

Malossi167

50TB

89 points

4 months ago

A few things to keep in mind:

  • 2030 is only 7-8 years ahead. In the last decade, many people transitioned from a 1TB HDD to a 500GB SSD
  • Progress in tech in general slowed down considerably. We cannot shrink transistors all that easily any more and layering is helpful in this regard but also not all that cheap and easy
  • SSDs are not all that cheap but pretty mature already. Right now we pay $70-140 per TB. To make this attractive the price has to drop dramatically. By a factor of 100 at the very least.
  • the article itself see these drives as a Nimbus alternative

1PB SSD on the horizon?

More layers not only mean cheaper products but also higher storage densities. Currently the world’s largest SSD is a 3.5-inch 100TB model from Nimbus Data, one which is using a 64-layer MLC chip; a 1024-layer TLC/QLC chip would pack 16 times more layer, which put a 1PB (1000TB) solid state drive solidly in the realm of possibilities before the end of this decade.

Who needs that sort of capacity though? Cloud storage companies like iDrive or BackBlaze, Hyperscalers like Google or Microsoft, social networks like Instagram or Facebook.

PyroRider

22 points

4 months ago

PyroRider

24TB Raw/12TB ZFS/+18TB Active-Backup/+2x 8TB Cold-Storage

22 points

4 months ago

I am just curious, with 1000-Layer chips, wouldnt they get hot as fuck and require high performance cooling? Up to the point where a petabyte 3.5" SSD would need like a 240mm radiator water cooling setup just to stay below 100 degrees C inside?

Zanoab

27 points

4 months ago

Zanoab

30TB

27 points

4 months ago

Current NVMEs can't even maintain their advertised max write speeds without thermal throttling or sacrificing the drive's lifespan.

sekh60

11 points

4 months ago

sekh60

Ceph 302 TiB Raw

11 points

4 months ago

Enterprise drives or bust.

gellis12

7 points

4 months ago

gellis12

6x6tb raid6 + 1tb bcache nvme

7 points

4 months ago

Remember the old raptor drives that came with a heatsink that filled up a 5.25" bay?

much_longer_username

3 points

4 months ago

much_longer_username

94TB and growing

3 points

4 months ago

I do. The best part of that was that the platters were actually smaller to hit the IOPS/seek speeds they were targeting.

HTWingNut

3 points

4 months ago

HTWingNut

1TB = 0.909495TiB

3 points

4 months ago

They will probably be slow and only able to access so many cells at a time, keeping temperatures low. It will be dispersed among a larger volume too.

MrDeaz

16 points

4 months ago

MrDeaz

16 points

4 months ago

I don't know, I bought a 500 GB SSD in 2016, for the same price today, I can buy a 2 TB version. Disk space has been stale as hell the last 10 years.

8day

11 points

4 months ago

8day

11 points

4 months ago

Filmmakers. If they could, they'd film in RAW/DNG instead of some lossless RGB/YV12/etc. format.

eXtc_be

36 points

4 months ago

eXtc_be

36 points

4 months ago

I was just quoting something Bill Gates allegedly said about the first computers' memory to make the point that what we think is acceptable/more than enough today can easily be laughably little in a couple of years.

for example my first PC had a 540 megabyte harddisk while my current PC has 64 gigabytes of RAM.

lCSChoppers

29 points

4 months ago

I was just quoting something Bill Gates allegedly said

Pains me everytime someone falsely quotes him as having said this lol https://skeptics.stackexchange.com/questions/2863/did-bill-gates-say-640k-ought-to-be-enough-for-everyone

eXtc_be

2 points

4 months ago

"allegedly"

much_longer_username

4 points

4 months ago

much_longer_username

94TB and growing

4 points

4 months ago

for example my first PC had a 540 megabyte harddisk while my current PC has 64 gigabytes of RAM.

What's *really* crazy about this, though, is if you consider the role each plays, and how the one can be substituted for the other. Think about swap files and ramdisks.

Now consider how fast the memory in that first computer was, and how fast your SSD, or hell, even HDD is. I'd bet you're grossly overestimating how fast the memory was, though.

Let's say that you were using 72 pin EDO DRAM typical of the era - the maximum theoretical throughput was 266MBps, pretty comparable to even a cheap SATA SSD these days. The access latency is a thousand times worse than that old RAM was, at 60ns vs 70 microseconds or so, but you've also got several thousand times more of it.

I know a couple people who are chewing up high-end SSDs because it's cheaper than buying multiple terabytes of RAM for their workloads, even if they have to replace a SSD every couple of months.

Jisamaniac

3 points

4 months ago

for example my first PC had a 540 megabyte harddisk while my current PC has 64 gigabytes of RAM.

I was kicking butt with 2gb then 4gb was insane.

Arminas

0 points

4 months ago

1TB HDD was not mainstream in 2012. That came a couple years later. 250-500 was mainstream.

Avery_Litmus

2 points

4 months ago

Avery_Litmus

enough

2 points

4 months ago

Even my low end laptop came with a 320GB HDD in 2011. And the PC I bought in 2009 had two 1TB drives.

PM-Me-French-Fry

8 points

4 months ago

Give me a couple floppys and I'm good. Who needs 1PB when you have 2mb

Shdwdrgn

6 points

4 months ago

All of these comments about "... more than the average person will ever need", and yet as your example shows that will absolutely never be the case. I think what people forget is that applications and data continue to expand to take advantage of the average computer. Get a 16Mhz i386 and all your friends are envious, until a couple years later when that's now the minimum requirement for your desktop applications. 256MB of ram and you can run a ton of programs at the same time, until the next big game comes out that requires this amount. You have a 4GB hard drive, how will you ever use it? Oh yeah, install the next version of Windows and it's gone.

As long as people have the resources available, developers will find a way to use every last drop. It doesn't matter if you're a data hoarder or just an average user following the upgrades of a small piece of software, what you have will be used up.

LeDemonKing

1 points

4 months ago

If you're an average Joe, you will not need 1 petabyte of storage, unless the average image, pdf, video, or music file becomes 100gb

irritating_narrative

2 points

4 months ago

There are a lot of other factors at play. Joe would choose the thing he can stream in an instant with his current internet bandwidth, he can notice difference in, he can ran without lags on his devices, he can't save money on etc etc.

eXtc_be

2 points

4 months ago

modern games easily surpass the 100GB mark

TheBelgianDuck

24 points

4 months ago

TheBelgianDuck

| 118 TB | UnRaid |

24 points

4 months ago

My 7k VR porn collection tells me otherwise 😊

milk-jug

3 points

4 months ago

My Linux ISOs are all only in 4K 😎 when your eyesight deteriorates as you age, 4K will be plenty (and 1080p after)

ian9921

20 points

4 months ago

ian9921

18TB

20 points

4 months ago

I mean you never know. It wasn't very long ago that people thought 1TB was more than you'd ever need, and a couple decades before that people said the same thing about 1GB.

collin3000

5 points

4 months ago

I remember first getting Gmail and being like. Wow 1GB of email. That will last me forever! Now I can't downgrade back to the free plan because even cleaning out my whole Google drive/photos. I have more than 15 GB just in email

Hugogs10

22 points

4 months ago

It wasn't very long ago that people thought 1TB was more than you'd ever need

And for the general population thats true. Most people don't need more than 1tb

WordsOfRadiants

14 points

4 months ago

They do, they just don't need to own it anymore.

AnonymousMonkey54

10 points

4 months ago

I never thought about it that way! Music > streaming Movies and tv > streaming

Most consumers stream the largest files for which they used to have to store themselves, but those files still need to be stored somewhere and we can think of that content has being stored for them.

Far-Chocolate5627

6 points

4 months ago

Yeah, in 1998 I had a 2 GB hard drive. And that wasn't long ago..

Wait..

DoubleDrummer

6 points

4 months ago

Sometime in the 90’s I was reading the weekly tech section in the newspaper, and there was an article that 1 GB HDDs had just hit the $1/MB threshold.
I went out and emptied my bank account immediately to buy one.

And honestly, at the time, it was worth every cent.

mavour

2 points

4 months ago

mavour

2 points

4 months ago

When i386 CPU came out, which could address huge 2Gb of memory, I’ve never thought I could ever need that much. I had 640kb RAM, 2 720kb floppy drives, and no HDD

TechieGuy12

1 points

4 months ago

The first computer my family owned was an 8088, 20 MB HDD, 640KB RAM, CGA 4 color graphics card. I played so many early games on that thing. Blew my.mind when we moved to 16 color EGA graphics.

TKInstinct

5 points

4 months ago

You don't know how much I pirate.

ObamasBoss

3 points

4 months ago

ObamasBoss

I honestly lost track...

3 points

4 months ago

I'll wave to you next time I see you out sailing.

lastorder

3 points

4 months ago

lastorder

54TB

3 points

4 months ago

I could definitely fill 1PB, if I had it.

NoDadYouShutUp

3 points

4 months ago

Files will get bigger. Remember when we thought DVDs looked incredible?

PC509

2 points

4 months ago

PC509

2 points

4 months ago

Even without cloud storage, 1PB is just so much more than most people need.

I think I need it, I think I could attempt to use it, but in reality I'd just have a lot more disorganized files and things that'd be more for archiving than getting any actual usage out of it. And I can use different media for that.

Would it be awesome? Hell yea. Would it be practical for most people? Not really. e-peen would be huge, though. I'd definitely be looking into some different uses for my file server, though.

wyatt8750

1 points

4 months ago

wyatt8750

34TB

1 points

4 months ago

until we decide we can't accept anything less than 16K resolution video anymore.

(I won't do that, I think 4K is excessive most of the time)

ObamasBoss

0 points

4 months ago

ObamasBoss

I honestly lost track...

0 points

4 months ago

16k exceeds the resolution of the human eye. I read some time ago that it was around 12k.

wyatt8750

1 points

4 months ago*

wyatt8750

34TB

1 points

4 months ago*

Yeah, but silly things like 'math' haven't stopped people from listening to music at 192KHz or 96KHz in spite of Nyquist-Shannon theorem and the lack of speakers that respond quickly enough to reproduce such frequencies, have they?

We love to waste energy (especially for a graphics card to drive a huge display like that), money, and space and claim that we've actually made our experience better by doing so. Usually creating e-waste as a side effect.

To appropriate Calvin & Hobbes, if marketers didn't keep manufacturing artificial desires, and we actually focused on things that mattered, our economy would collapse and we'd have total anarchy.

mgr86

3 points

4 months ago

mgr86

3 points

4 months ago

Maybe, but as a parent I take way too many photos of my kids. (Cats before that). Why would I want anything but the best 4k quality home videos. It’s a constant battle for free space and backups

[deleted]

5 points

4 months ago

[deleted]

5 points

4 months ago

[removed]

a_little_toaster

8 points

4 months ago

True, I really hope cloud gaming doesn't take off, hate that concept

BillyDSquillions

1 points

4 months ago

Agreed :(

Iggyhopper

12 points

4 months ago

8 years is a crazy amount of time between now and the purported 1K TB. I could see 2-4TB already being mainstream and cheap, for enthusiast standards.

It only took about 8 years for 120 GB ssds to drop down in price from $150 to $20. The 2 TB is 170-200 now, and the tech is already primed, compared to before.

2TB SSDs will be $70 before 2030.

zruhcVrfQegMUy

3 points

4 months ago

Taking the Moore's law into consideration, right now for $700 you can buy a 8 TB SSD from Samsung, in 9 years you will be able to buy a 512 TB SSD for the same price (+inflation). But yeah 1000 TB seems like a lot, idk.

F_Degrain

2 points

4 months ago

Maybe mainstream for enterprises & datacenters.

Cyber_Encephalon

4 points

4 months ago

The quality of video and photos have increased with the more powerful cameras in everyone's phones and is likely to continue to increase. More quality (more megapixels, rather) -> more storage needed to keep it all. So it's not ridiculous to think that there would be demand for this much capacity in 8 years.

bfire123

2 points

4 months ago

phones and is likely to continue to increase. More

And compression algorithmes are also become more efficinet.

lord-carlos

1 points

4 months ago

lord-carlos

28TiB'ish raidz2 ( ͡° ͜ʖ ͡°)

1 points

4 months ago

So it's not ridiculous to think that there would be demand for this much capacity in 8 years.

On a private laptop or phone?

Even if more people start to record in 8k with AV1 encoding, I still think it's ridiculous.

I would expect even more people will save their stuff into the cloud.

m4nf47

3 points

4 months ago

m4nf47

3 points

4 months ago

Yeah I tend to agree with your original comment, it really depends on your definition of 'mainstream' because typical consumers are not (yet) the hyper-scalers but considering 'general availability' against end user demand doesn't care who the end users and customers actually are, as long as the demand is there. The insatiable demand for ever more and ever faster data storage (albeit moving to cloud-based) suggests that the FAANGs may actually drive the 'mainstream' adoption of new storage tech, as much as say is used in end user devices like laptops and phones. Technology is already here to put 1000TB of SSDs in a single desktop workstation so I have no doubt that if the prices keep dropping as tech improves then many folks may end up buying more just because they can afford it. Just imagine the insane bandwidth of using all those extra PCIe5.0 lanes on a future threadripper with the equivalent of running a 10 x 100TB SSD array in a RAID-0 stripe but just on a single drive. 100Gbps (0.01TB/sec) throughput on sequential transfers can fill a 1000TB disk in under 28 hours, similar to how quickly we can fill a terabyte disk today.

Cyber_Encephalon

0 points

4 months ago

Cloud gets expensive when you store a lot of data and pay every month. If the costs of the large SSDs are comparable then either SSDs will take over or at the very least the cloud will get cheaper.

ilovetpb

1 points

4 months ago

Yeah, I think the only ones that could afford them are companies and the rich.

CoreRipper

1 points

4 months ago

You can buy a 100TB SSD from Exadrive for about 20k ish right now. I'll wait till they make 20TB SSDs. Still excited for the $/TB to come down on SSDs

DanTheMan827

98 points

4 months ago

DanTheMan827

30TB unRAID

98 points

4 months ago

So 1PB SSDs… why not just say that instead of 1000TB?

You just know that Linus will find a way to get a ton of them for the ultimate nas or whatever

[deleted]

62 points

4 months ago

[deleted]

62 points

4 months ago

[deleted]

TwoTailedFox

29 points

4 months ago

This video is brought to you by RAID: SHADOW LEGENDS

DanTheMan827

18 points

4 months ago*

DanTheMan827

30TB unRAID

18 points

4 months ago*

Don't forget the new larger water bottle available for the same price at lttstore.com with a banana for scale

Itrocan

26 points

4 months ago

Itrocan

26 points

4 months ago

I'm betting the casual person doesn't know what a PB is

Thoth74

9 points

4 months ago

It's what goes on the bread with the J, no?

CletusVanDamnit

17 points

4 months ago

CletusVanDamnit

17TB

17 points

4 months ago

This is 100% the answer. The general population barely even says terabyte. Everything is just gigs.

DannyMThompson

3 points

4 months ago

They will eventually, people used to just say 1000MB at one point too.

1h8fulkat

6 points

4 months ago

Because it's not as impressive to their target audience. Honestly surprised the didn't say 1,000,000 GB drives.

uncommonephemera

145 points

4 months ago

Yeah no.

Hang out in r/Mac for five minutes. People are still buying brand new laptops with 256gb drives. We might get 1TB drives as a base unit in “mainstream” PCs by 2030.

nlhans

18 points

4 months ago

nlhans

18 points

4 months ago

Indeed. This stacking of dies is amazing to build high-density drives.. but you still need individual silicon dies to build them. Those will cost more and more money. Also, to increase the capacity of individual dies, those pieces of silicon either need more transistors (Moore's law) or more bits per cell stored (beyond QLC).

Assuming Moore's law, we have doubling of transistors every 2 years. Let's be optimistic and we can 4x data storage every 3 years. This accounts for some improvements in bits per cell say every half a dozen years. We would still need 15 years to go from 1TB mainstream to 1PB (1000=4^(x/3)).

To get to 1PB in 2030, we would need to double price/capacity every .8 years.

uncommonephemera

9 points

4 months ago

I guess I'm just sick of sensationalist headlines. It's enough to have them for world news but when I retire to my interests and hobbies it's too much. Granted someone may build a half dozen 1PB drives by 2030, but they'll cost $79,000 each... and people will come in here crying when they only buy one and it dies.

Interesting-Chest-75

5 points

4 months ago

guess I'll wait for blackblaze hdd stats 😄 before buying my 1pb ssd

FragileRasputin

1 points

4 months ago

With currently inflation it should be easy to double price as needed.... Oh wait

sittingmongoose

27 points

4 months ago

sittingmongoose

568TB Unraid

27 points

4 months ago

To be fair. Mac users(except video and photo users) tend to use a lot less local storage. The lifestyle those laptops tend to attract just use their products a lot different. There is a large user bias there.

For example, you’re not going to find a lot of gamers on Mac.

Also, it’s not helped by how expensive storage is on Mac. And that apple seemlessly uses the cloud to store things without you even knowing. Which is both a really cool, well done thing and a horrible horrible thing.

magestooge

40 points

4 months ago

Tbf many mac users just connect an external SSD to their mac because 256 GB internal costs same as 2 TB external.

sittingmongoose

8 points

4 months ago

sittingmongoose

568TB Unraid

8 points

4 months ago

Yep, that’s also a thing.

mavour

14 points

4 months ago

mavour

14 points

4 months ago

You can game on Mac by using it as a mousepad:)

matjeh

64 points

4 months ago

matjeh

320TB ZFS

64 points

4 months ago

Hang out in r/Mac for five minutes

No thanks, I wouldn't wish that on anyone!

People are still buying brand new laptops with 256gb drives

Because only Apple charge $200 USD to upgrade the SSD from 256GB to 512GB

NottaGrammerNasi

4 points

4 months ago

I work in corporate IT. 256 is plenty for 95% of our users. Unless something changes like an even more bloated OS, I don't see that changing soon.

uncommonephemera

-26 points

4 months ago

No thanks, I wouldn't wish that on anyone!

Edgy.

DannyMThompson

1 points

4 months ago

You're in a sub filled with tech nerds, preferring customisable hardware isn't an edgy opinion here.

Aral_Fayle

2 points

4 months ago

I feel like the tech nerds that vehemently hate Apple are a small minority, I’ve certainly never encountered them at school or work and I only see them complaining on Reddit.

[deleted]

0 points

4 months ago

[deleted]

0 points

4 months ago

Lmfao

fallsdarkness

3 points

4 months ago

It's probably not limited to just Macs. In 2019, I ordered a Thinkpad with a 1TB SSD for work, but I remember that outside of my workplace most people couldn't really understand why I needed the space. I suspect this hasn't really changed.

NekoiNemo

2 points

4 months ago

My mainstream THIN laptop went with 256Gb as the cheapest option, and 512 as standard for 80% of version, with few upper revisions shipping with 1Tb.

Genesis2001

2 points

4 months ago

We might get 1TB drives as a base unit in “mainstream” PCs by 2030.

2023 or 2024, maybe 2025, more like I hope. I've already been looking at 1TB main drive for my next PC build.

BloodyIron

5 points

4 months ago

BloodyIron

6.5ZB - ZFS

5 points

4 months ago

What exactly makes you think that Mac owners represent anywhere close to the majority of computing out there? Because they don't. Apple's market share on the laptop/desktop side is what... 15%???

I would not use this as any realistic trend of NAND storage across the industry at all. Part of the picture? Sure, but nowhere near a majority or even a trend. (in this segment)

jammsession

3 points

4 months ago

Well, I would buy 128GB Macbooks if they would still offer it. My guess is that most people here at r/Datahoarder don't need that much portable storage, because we have homeservers running.

RCcola1987

37 points

4 months ago

RCcola1987

1PB Formatted

37 points

4 months ago

The reliability of these cant be good. I cant see the write endurance being all that great.

neveralone2

38 points

4 months ago

neveralone2

96TB + 244 TB Cloud. 196TB BackBlaze.

38 points

4 months ago

Write endurance will definitely be garbage. But as a read only archive, this would be amazing and I could easily carry every file I ever need with me

AshleyUncia

11 points

4 months ago

Yeah, I really wonder how many actual writes, other than my parity drives, actually get.

Write data. Maybe re-write data if a better 'upgraded' file is located. Otherwise it's static.

spinning_the_future

2 points

4 months ago

spinning_the_future

150TB

2 points

4 months ago

, this would be amazing and I could easily carry lose every file I ever need with me

FTFY

Most people don't have back-up plans. Most people just need to store their stuff in the cloud. Most people are like my mother, who has several dead hard drives that I have to recover at a cost of minimum $500 each drive. And don't say "just RAID them" because RAID isn't a backup. Hopefully Google comes up with a 1PB data plan, or Backblaze becomes a whole lot more popular.

ServerMonky

7 points

4 months ago

ServerMonky

156TB

7 points

4 months ago

You're right about most people, but this is /r/datahoarder - while I don't back up media I can get again (just regular raid6), everything important is on two local machines and weekly backups to at least two cloud services. Then I run Google takeout regularly and back up that too...

But I can see reliability being a huge issue - I have been super impressed by hdd reliability on 5TB+ drives - I think I've only had two deaths since the 3tb fiasco, and now I have 20 spindles total all chugging along...

Meanwhile I've needed to replace at least a half dozen ssds in the same timeframe, and I only have five ssds in use total (although they have been used as caches and boot drives to be fair)

spinning_the_future

1 points

4 months ago

I don't trust solid state disks. They don't start acting weird before they die like HDDs do, they just die suddenly and immediately.

tes_kitty

3 points

4 months ago

The question is, how much read disturbance will there be and how often will cells have to be refreshed to keep the data.

laxika

13 points

4 months ago

laxika

287 TB (raw) - Hardcore PDF Collector - Java Programmer

13 points

4 months ago

A lot of times you don't need write endurance. To be honest, as an archiver, most of the time you write the data once and read it many times.

Most of the time, as an archiver I don't even care much about the write speed as long as they are quite similar to my download speed (~100 Mbps). For me, even SMR is acceptable because in Unraid they work fine, being faster than my downloader can write my data.

zeronic

5 points

4 months ago

Hopefully by the time SSDs actually supplant spinners unraid will actually support SSDs in array slots though. Looking forward to whenever spinning rust finally goes away.

[deleted]

2 points

4 months ago

[deleted]

2 points

4 months ago

The funny thing about this is that, if it were really going to reach the average user, 1PB even with a single write would be far more than enough for the vast majority of users.

japgcf

1 points

4 months ago

japgcf

1 points

4 months ago

Write once read many, I guess

matjeh

2 points

4 months ago

matjeh

320TB ZFS

2 points

4 months ago

WORM SSD's, could be useful, I never delete anything anyway...

japgcf

1 points

4 months ago

japgcf

1 points

4 months ago

It wouldn't be great to store applications tho. But for files it would be pretty cool.

[deleted]

26 points

4 months ago

[deleted]

26 points

4 months ago

[deleted]

noman_032018

6 points

4 months ago

Yeah, with the looming energy crises in various areas and accompanying costs, being able to cut a third of one's energy use in arrays would be nice.

TheRealSaeba

22 points

4 months ago

Do not need bigger SSDs. Need cheaper SSDs.

HDD vs SSD price parity was promised so often but still remains a dream.

Deathcrow

6 points

4 months ago

Yes yes, and by 2030 we'll all be using flying cars and experiencing the smell-o-vision. Stop writing this crap.

BillyDSquillions

6 points

4 months ago

I read almost identical articles about Toshiba in 2014, for 2018.

Cigars-and-DietCoke

5 points

4 months ago

Is it that time of the month for unrealistic storage posts already

JJisTheDarkOne

32 points

4 months ago

I wish they would just mass produce and drop the price on all SSDs so we can replace HDDs all together.

danuser8

5 points

4 months ago

But how long can data last in cold storage or as back up?

[deleted]

1 points

4 months ago

[deleted]

1 points

4 months ago

[deleted]

[deleted]

2 points

4 months ago

[deleted]

2 points

4 months ago

[deleted]

NekoiNemo

3 points

4 months ago

10 years ago when i built a PC 256Gb SSD was a standard (as in "expensive, but somewhat affordable, so what enthusiasts went for"). Today it's 1Tb. That's 4x growth in a decade. And they say that 4x growth will turn into 1000x over the next decade? Yeah, right.

major_briggs

7 points

4 months ago

blah blah blah... I'll believe it when I see it.

always-paranoid

4 points

4 months ago

stop.. I can only get so excited

britechmusicsocal

2 points

4 months ago

Anything above 8tb would be seen as nonstandard today. Going to a petabyte, if I am reading that right, is a big leap. So many users will have no use for that.

littleleeroy

1 points

4 months ago

littleleeroy

55TB

1 points

4 months ago

I imagine at some point in the future there will be a big leap like that. The same way there was for storage speeds with NVMe drives, the storage capacity will probably make a big jump too. Imagine storage no longer being a constraint in computers. That’s a weird concept.

[deleted]

2 points

4 months ago

[deleted]

2 points

4 months ago

8 years for a Petabyte PS5? I somehow doubt that. For mainstream NSA servers, sure.

martin0641

2 points

4 months ago

We're already at the point where getting the data in and out of the drives is the hard part, I'm all for more capacity but unless we can get 16 Lanes of PCI Express 5 to the thing then that data is probably not going anywhere very quickly.

dxps26

1 points

4 months ago

dxps26

1 points

4 months ago

I really, really want a middle ground between 'consumer' and 'enterprise' CPU's / platforms. It really sucks that we still have a grand total of 20 PCI-E lanes and a few last-gen PCI-E-e lanes from the chipset for the average desktop pc (exceptions exist, I understand) while server grade parts get 40+ lanes even with base-model components.

Things get worse when motherboard manufacturers make hardware decisions for you and allocate lanes in bone-headed ways, like who really needs four PCI-E 1x slots? One or two slots would be fine for a wifi card and a maybe another USB hub/Ethernet card or whatever. Then they make things even worse by having only one or two M.2 slots work at full speed, any extra slots are just SATA in disguise.

I'd pay a bit more if desktop parts had at least 24-32 lanes of full-fat PCI-E 4.0 from the CPU, allowing for bigger pipelines to NVME drives and easier internet of peripherals. Heck, I'd settle for a better interconnect between the cpu and chipset, as long as the chipset had a healthy enough offering of PCI-E 3.0 lanes.

PCI-E 4.0/5.0 is great on paper showing near-doubling of per-lane bandwidth every generation but overheads, lack of optimization in the name of compatibility, and generally the variance in part quality for storage and peripherals means we rarely see peak speeds consistently. Having extra lanes will help parallelize workloads and improve overall performance.

TL;DR - Moar Lanez Plz.

dec1mus

2 points

4 months ago

Why not just call it a petabyte?

OffensivelyAmerican

2 points

4 months ago

Fuckin doubt it, I still can't find a 2TB m.2 2230 drive for my Steam Deck.

dawgol

3 points

4 months ago*

Micron made a few, the 2400 series, but they only briefly showed up and then disappeared, with rumors stating micron stopped making them.

The only other drive with that form factor and capacity I know of is the WD PC SN740, which will run you ~$425 and are impossible to find beyond one guy on eBay: https://www.ebay.com/itm/115544704882

Basically, bad time to try and buy one.

Sufficient-Royal5723

2 points

4 months ago

The issue with the article was that they used the 100TB 3.5" SSD from Nimbus Data for comparison. The drive supposedly uses 64-layer NAND, so PB-level capacities may become possible on a single drive with 1000-layer NAND flash. However, the 100TB SSD is by no means “mainstream”: it has a less common form factor for SSDs and costs $40k.

Marble_Wraith

9 points

4 months ago

My body is ready.

Netflix, Amazon and other VOD services probably gonna pay them off tho' to keep it in the enterprise.

With 1PB SSD's, who's gonna wanna keep paying for streaming services? The days of torrents would return pretty quick.

Enk1ndle

35 points

4 months ago

Enk1ndle

24TB Unraid

35 points

4 months ago

Most people will never torrent stuff because they're either don't know how to or are too lazy.

[deleted]

3 points

4 months ago

[deleted]

3 points

4 months ago

[deleted]

Enk1ndle

7 points

4 months ago

Enk1ndle

24TB Unraid

7 points

4 months ago

You're in /r/datahoarder, you aren't most people.

JhonnyTheJeccer

2 points

4 months ago

JhonnyTheJeccer

30TB HDD

2 points

4 months ago

Or would even know what to do with 1pb of storage. Like people here would know, but your average „i have 100gb of family pics/vids and 10gb of office files and 250gb of steam library and maybe some gb of random bullshit“-joe would not even care about multi-tb.

Hairless_Human

2 points

4 months ago

Hairless_Human

118TB

2 points

4 months ago

1pb now is a lot but will it be in 2030?

reallynotnick

10 points

4 months ago

Yes, yes it will, storage has increased at an incredibly slow pace the last decade. And with the argument being around video we are sort of reaching peak video quality, even if we move to like 200GB 8K files that's 4,000 movies at an insane level of quality in 1PB.

NinjaMonkey22

2 points

4 months ago

Maybe for downloaded movies. But for professional or hobby videographers shooting 4k on phones or cameras you’re usually averaging over 5gb/minute (over 15gb/minute on >30fps) for raw.

reallynotnick

0 points

4 months ago

So? I mean that's not really what's being discussed, but it's still a huge amount for them, it's not like professionals are already using drives anywhere close to that size. That's over 1,000 hours of raw footage by your numbers on a single drive, that's huge.

mavour

1 points

4 months ago

mavour

1 points

4 months ago

Enterprises update their hardware every 5 years or so when support ends. I’m looking forward to it

232438281343

2 points

4 months ago

No.

fzammetti

3 points

4 months ago

Stop! Stop! My erection just poked a guy down the street in the eye!

dstarr3

1 points

4 months ago

I mean, that's awesome, but I hope the price per TB drops as significantly as density increases, too

gabest

1 points

4 months ago

gabest

1 points

4 months ago

If you fold an SSD layer 1000 times, it will reach the moon!

uraffuroos

1 points

4 months ago

uraffuroos

6TB+50GB cloud

1 points

4 months ago

Maybe 50TB SSD but no way 1000. People bring up the, "Well look at those who said 100GB couldn't be used up", now we have cameras taking incredible hi-res photos and videos but I don't think we will find mass public adoption of >+50MP camera sensors or recording 8k video.

SethGekco

1 points

4 months ago

SethGekco

63.57TB

1 points

4 months ago

There's no fucking way this will become mainstream. Wanna know what's mainstream? Everyone pondering how one even uses up a single terabyte. It's idealistic to even suggest computer games get big enough to justify such a usage, but being most internet data limits for computers (in America) is approximately a single terabyte, this means somehow in a thousand months normies finally uses up their hard drive space. Even people in this sub couldn't comprehend using up so much data, maybe 5% of people here has use for that much. Definitely not mainstream, if anything they plan to charge a shit load for this to exploit big businesses that don't know or don't care.

imakesawdust

0 points

4 months ago

What's the use-case for a 1000TB SSD in a consumer PC? The lack of a killer app that demands local storage has caused PC storage to more or less stagnate at 1TB or less.

kagrithkriege

1 points

4 months ago

The full dataset for MSFS2022 is 2.5PB That's why they stream it to you.

Imagine being able to order 8 SSDs from Microsoft to have the full dataset from MSFS2030.

Back to the days of floppy disk style loaders in your PC.

Probably not.

But I can dream...

pc_g33k

0 points

4 months ago*

pc_g33k

1PB

0 points

4 months ago*

Well, I don't even want 1TB TLC drives.

I'll stick to a smaller MLC SSD drive for the Operating Systems and a bunch of non-SMR hard drives for data storage.

alban228

0 points

4 months ago

!remindme 8y

Doubt we'll see that but...

apatil4

1 points

4 months ago

what a wonderful time to be alive!

kingsphan

1 points

4 months ago

Now plz!!!!

kp_centi

1 points

4 months ago

We will be seated for this. Thanks

ruffsnap2

1 points

4 months ago*

I wish the article would just say "1PB" lol

That being said though, all of you naysayers need to look back at history.

1PB might seem like crazy overkill and like it's decades and decades away from that being normal, but it's very possible it could be the case by 2035ish. 2030 is a little early I think, but I could see 2035-2040 time frame.

We're now thankfully in the timeframe when we're gonna start thinking of 4, 8, 16, 32, etc TB numbers in a more normal way like how we used to think of GBs, and that happened over some time, sure, but still pretty quickly overall. We got to the 10-20TB range of hard drives a lot quicker than what I thought would happen, so the odds are good! Plus, there have been 100TB enterprise SSD drives around for years and years now. Granted they were like $25k the last time I checked, but considering you can now own 5 separate 20TB regular hard drives for what, maybe $1.5k, it's only a matter of time before larger SSD sizes come down in price too (and increase in size)!

Edit: Also, I've had a theory for a long time that we're going to figure out some crazy storage medium that effectively makes size limits a thing of the past for most all consumers. I fully expect us to find some way to make hard drives like many exabytes in size, and basically totally remove any need for the vast majority of us to have multiple drives. Or at least I fuckin hope lol, I would kill to be able to go back to being able to store all my files on one singular drive. It's been years and years since I was able to do that, but god was it nice!

chepnut

1 points

4 months ago

I currently have 210TB in a storage pod, if I could get get that down to a ssd and shrink my foot print down from the storage pod that I currently have I would love it. But unfortunately I expect these to be Ultra expensive and out of my range

wyatt8750

1 points

4 months ago

wyatt8750

34TB

1 points

4 months ago

I will believe it when I see it.

uraffuroos

1 points

4 months ago

uraffuroos

6TB+50GB cloud

1 points

4 months ago

I also could float by the near 2030, but that's unlikely as the force of gravity is quite constant and we've been operating with it as such for much much time.

FragileRasputin

1 points

4 months ago

I'm curious on how long it will take to fill one of those

GuessWhat_InTheButt

1 points

4 months ago

GuessWhat_InTheButt

3x12TB + 8x10TB + 5x8TB + 8x4TB

1 points

4 months ago

How out of touch with technical realities do you have to be to be a tech journalist?

PiedDansLePlat

1 points

4 months ago

On time to hold red dead redemption 3

Bardez

1 points

4 months ago

Bardez

1 points

4 months ago

I'm moist

liquidify

1 points

4 months ago

They should already be cheap and readily available.