Singapore Expats Forum

Any Thoughts About Network Attached Storage (NAS)?

Discuss about computers & Internet. Including mobile phones, home appliances & other gadgets. Read about Windows security risks or virus updates.
User avatar
zzm9980
Governor
Governor
Posts: 6841
Joined: Wed, 06 Jul 2011
Location: Once more unto the breach

Postby zzm9980 » Thu, 07 Aug 2014 9:05 am

What model is it Brah?

Raid0 means you're using two drives (striped) as one logical drive.

Raid1 means your two drives are mirrored copies of each other. If one drive is bad in a Raid1 it is normal that reading off of it is slower; most Raid1s will read from both drives at the same time to increase read speed.

On most devices, the easiest way to tell which drive in a Raid1 mirror is bad is to pull them out one at a time while it is running. If you pull the bad drive out, nothing should change from your current situation. If you pull the good drive out, the whole thing will go offline. You can "double confirm" then by shutting it off, putting both drives in, powering it up, and pulling the other drive (should be the bad one) out.

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Thu, 07 Aug 2014 9:38 am

zzm9980 wrote:On most devices, the easiest way to tell which drive in a Raid1 mirror is bad is to pull them out one at a time while it is running.

If unlucky he can lose data this way and fall into more serious trouble - what if he removes a good one while something is writing to it to the file table? Don't pull any drives out when the thing is running. Stop it first, switch it off, remove the drive, switch it on and see what happens, but before knowing what hardware is that I would not advise doing it. Copy first what possible and only then play with it.

User avatar
Brah
Manager
Manager
Posts: 1962
Joined: Sat, 18 Dec 2010

Postby Brah » Thu, 07 Aug 2014 9:18 pm

x9200 wrote:Failing performance probably indicates bad blocks appearing on one or two hdds.

this is what I suspect, hopefully just one

Any diagnostic software that came with the drive?
Yes:
WD Drive Manager, a monitoring tool when it runs which shows utilization and RAID state (0 or 1)

WD DataLifeguard, which can do a scan on selected discs; first a quick test test, which it passes, (but while doing so Drive Manager shows it toggle between RAID 1 and RAID 0)

Test Option: QUICK TEST
Model Number: WD My Book (RAID 1)
Firmware Number: 1012
Capacity: 1000.20 GB
SMART Status: PASS
Test Result: PASS

I'm running the extended test now, says it will take 15 hours; it failed this test when I ran it a few times when the problem first occurred, but I forget the error message.


What is exactly the drive? Brand, model?
WD MyBook WD20000H2U-00, 2 1tb Seagate drives,

Fail-safe approach: try to copy the data patiently over night, few days whatever reasonable time it takes, to an external drive. Start with the most important data and keep the drive in a well ventilated space to prevent overheating.

Good idea on the copy, but with 77% of the RAIDed 1tb utilized (I configured it RAID 1 from the first time I used it), not sure this will work. I use TeraCopy, which is great, but I think it hung a few times, when I tried, or was just so slow I gave up. While it's 2tb RAIDed = 1tb effective storage, with it at RAID 1, that could be more than 1tb's worth of scans against both discs.

User avatar
zzm9980
Governor
Governor
Posts: 6841
Joined: Wed, 06 Jul 2011
Location: Once more unto the breach

Postby zzm9980 » Fri, 08 Aug 2014 1:20 am

x9200 wrote:
zzm9980 wrote:On most devices, the easiest way to tell which drive in a Raid1 mirror is bad is to pull them out one at a time while it is running.

If unlucky he can lose data this way and fall into more serious trouble - what if he removes a good one while something is writing to it to the file table? Don't pull any drives out when the thing is running. Stop it first, switch it off, remove the drive, switch it on and see what happens, but before knowing what hardware is that I would not advise doing it. Copy first what possible and only then play with it.


He could, if he's 1 in a million unlucky or using a filesystem from two decades ago. He could also get struck by lightning or hit by a car tomorrow too. In the real world the chance of actually losing data by pulling the good drive from a hot-swap raid is extremely low. I would assume he wouldn't test this in the middle of a large write operation. Background activity to a network attached volume will be quite minimal. I think x9200's expectation of trouble is just quite high because he is a Linux user. It's up to you Brah.

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Fri, 08 Aug 2014 9:01 am

Brah wrote:
x9200 wrote:Failing performance probably indicates bad blocks appearing on one or two hdds.

this is what I suspect, hopefully just one

Any diagnostic software that came with the drive?
Yes:
WD Drive Manager, a monitoring tool when it runs which shows utilization and RAID state (0 or 1)

WD DataLifeguard, which can do a scan on selected discs; first a quick test test, which it passes, (but while doing so Drive Manager shows it toggle between RAID 1 and RAID 0)

Test Option: QUICK TEST
Model Number: WD My Book (RAID 1)
Firmware Number: 1012
Capacity: 1000.20 GB
SMART Status: PASS
Test Result: PASS

I'm running the extended test now, says it will take 15 hours; it failed this test when I ran it a few times when the problem first occurred, but I forget the error message.


What is exactly the drive? Brand, model?
WD MyBook WD20000H2U-00, 2 1tb Seagate drives,

Fail-safe approach: try to copy the data patiently over night, few days whatever reasonable time it takes, to an external drive. Start with the most important data and keep the drive in a well ventilated space to prevent overheating.

Good idea on the copy, but with 77% of the RAIDed 1tb utilized (I configured it RAID 1 from the first time I used it), not sure this will work. I use TeraCopy, which is great, but I think it hung a few times, when I tried, or was just so slow I gave up. While it's 2tb RAIDed = 1tb effective storage, with it at RAID 1, that could be more than 1tb's worth of scans against both discs.


You may do the copying in small blocks so, for example, copy first one directory, after it's done, remove it from the original drive and do the same for the next directory. This way, if it freezes/stops at one point you don't need to start over again.

The SMART PASS status normally indicates there should not be any physical bad blocks. If the drive "thinks" it has it (physical reading/writing errors), it logs info about it internally and this info should stay there forever.

When the drive starts to misbehave you may try to take a look at the events log. It is available under Administrative Tools / Event viewer (Windows Logs, System). It may give some further clues.

PS. Seagate drives in a WD enclosure?

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Fri, 08 Aug 2014 9:17 am

zzm9980 wrote:
x9200 wrote:
zzm9980 wrote:On most devices, the easiest way to tell which drive in a Raid1 mirror is bad is to pull them out one at a time while it is running.

If unlucky he can lose data this way and fall into more serious trouble - what if he removes a good one while something is writing to it to the file table? Don't pull any drives out when the thing is running. Stop it first, switch it off, remove the drive, switch it on and see what happens, but before knowing what hardware is that I would not advise doing it. Copy first what possible and only then play with it.


He could, if he's 1 in a million unlucky or using a filesystem from two decades ago. He could also get struck by lightning or hit by a car tomorrow too. In the real world the chance of actually losing data by pulling the good drive from a hot-swap raid is extremely low. I would assume he wouldn't test this in the middle of a large write operation. Background activity to a network attached volume will be quite minimal. I think x9200's expectation of trouble is just quite high because he is a Linux user. It's up to you Brah.


There is likely a faulty raid drive here, it may not be mirroring anything so will behave just like an ordinary drive and the likelihood of data lost with pulling out an ordinary drive hooked up to any modern OS is high, very high - there may be some files corrupted already within a few attempts. Various processes/applications access the drives frequently. For a critical data corruption making the drive not accessible the likelihood is still sufficiently high not to advise it to someone who already has problems with the drive, especially that the right way to do it will just take 30s longer.

User avatar
zzm9980
Governor
Governor
Posts: 6841
Joined: Wed, 06 Jul 2011
Location: Once more unto the breach

Postby zzm9980 » Fri, 08 Aug 2014 11:11 am

x9200 wrote: and the likelihood of data lost with pulling out an ordinary drive hooked up to any modern OS is high, very high


I've been on and managed forensics teams with a lots of work (just look at how many lawsuits my company is involved in!) and that is not our experience, at all. If this was a drive with an OS you were booting off of or applications running at the time the drive was pulled, sure you might lose files you were actively writing to disk. You're not going to lose random files or corrupt your filesystem just by pulling a drive with data files on it.

Various processes/applications access the drives frequently.


Again, It's a NAS used for storage. I highly doubt Brah is running applications off of it, using it for swap, or running applications off of it.

He's more likely to zap something with static electricity and fry it while handling it than losing data the way you're describing.

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Fri, 08 Aug 2014 12:22 pm

Well, it's enough to google. You are the first person I know who does not recommend the safe, soft way of removing this type of hardware before physically unplugging it (BTW, the Brah's is not a NAS). OSX is screaming loudly if you unplug any usb rw storage device. Windows has a special button to safely remove such hardware. Are they all so wrong? There are or may be processes that scan drives and write to them some informations. They are transparent, not necessary showing up any way caching (for example) multimedia information or indexing various data. We don't know what Brah has installed on his PC. Corrupted data on the hard drive will not necessary show up immediately in a catastrophic failure.

Sorry, I don't get the link with the lawsuits and forensics. You mean people do not come over with this sort of problems? Why should they?

By data loss I mean any data loss/corruption, NOT the total data loss of all the data on the drive. Something writes to the drive, you unplug it and the system does not update the file table. This is still data loss. I don't think people hire forensic experts and sue anybody for such reason.

I will buy you a beer if you take an usb hd/thumb drive (RW mode), plug it in and out 20x (still far away from the strike of lightning likelihood) the way you suggested and there will be no inconsistencies within the file system.
You will buy me as many bottles of beer as the No of errors found within the file system after this experiment. Deal?

User avatar
zzm9980
Governor
Governor
Posts: 6841
Joined: Wed, 06 Jul 2011
Location: Once more unto the breach

Postby zzm9980 » Fri, 08 Aug 2014 1:24 pm

The forensic reference was in regards to expertise in filesystems on the drives, and what happens when drives are improperly unmounted, that's all. They're constantly working on and imaging drives that were not properly shut down and dismounted. Data has never been lost, even on drives using full-disk encryption, unless the drive itself already had bad sectors.

I think with this long circular argument, we're mostly agreeing now. No, I know there will not be catastrophic data loss. Yes, you may lose what is writing at the time. I just assumed Brah would know not to write anything intentionally to the disk while doing what I suggested. Sure, some background processes may be indexing something on the disk and writing background metadata, but its inconsequential. If *that* is corrupted in some way it will be fixed and rebuilt the next time the indexing process restarts. While yes your way is safer, so is jaywalking while wearing a helmet or double bagging it when you go to OT. IMO it's just not needed.

I just assumed Brah had a NAS since that is what this thread is about and he said "when he connects the drive". I assume it isn't always in use. Not sure what kind of RAID or device it is then.

I'll gladly take you up on the "USB Beer challenge". Sounds fun actually. :D
I'll even let you pick the OS, as long as it is a filesystem created in the past 15 years. (No FAT32 or ext2)


x9200 wrote:Well, it's enough to google. You are the first person I know who does not recommend the safe, soft way of removing this type of hardware before physically unplugging it (BTW, the Brah's is not a NAS). OSX is screaming loudly if you unplug any usb rw storage device. Windows has a special button to safely remove such hardware. Are they all so wrong? There are or may be processes that scan drives and write to them some informations. They are transparent, not necessary showing up any way caching (for example) multimedia information or indexing various data. We don't know what Brah has installed on his PC. Corrupted data on the hard drive will not necessary show up immediately in a catastrophic failure.

Sorry, I don't get the link with the lawsuits and forensics. You mean people do not come over with this sort of problems? Why should they?

By data loss I mean any data loss/corruption, NOT the total data loss of all the data on the drive. Something writes to the drive, you unplug it and the system does not update the file table. This is still data loss. I don't think people hire forensic experts and sue anybody for such reason.

I will buy you a beer if you take an usb hd/thumb drive (RW mode), plug it in and out 20x (still far away from the strike of lightning likelihood) the way you suggested and there will be no inconsistencies within the file system.
You will buy me as many bottles of beer as the No of errors found within the file system after this experiment. Deal?

bgd
Editor
Editor
Posts: 1358
Joined: Wed, 25 Jul 2007

Postby bgd » Fri, 08 Aug 2014 2:06 pm

Oh no, we've been overrun by geeks :o

Goodluck with the challenge - publish the consumption. :)

User avatar
the lynx
Governor
Governor
Posts: 5272
Joined: Thu, 09 Dec 2010
Location: Location: Location: Location: Location: Location: Location: Location: Location: Location: Location:

Postby the lynx » Fri, 08 Aug 2014 2:15 pm

zzm9980 wrote:The forensic reference was in regards to expertise in filesystems on the drives, and what happens when drives are improperly unmounted, that's all. They're constantly working on and imaging drives that were not properly shut down and dismounted. Data has never been lost, even on drives using full-disk encryption, unless the drive itself already had bad sectors.

I think with this long circular argument, we're mostly agreeing now. No, I know there will not be catastrophic data loss. Yes, you may lose what is writing at the time. I just assumed Brah would know not to write anything intentionally to the disk while doing what I suggested. Sure, some background processes may be indexing something on the disk and writing background metadata, but its inconsequential. If *that* is corrupted in some way it will be fixed and rebuilt the next time the indexing process restarts. While yes your way is safer, so is jaywalking while wearing a helmet or double bagging it when you go to OT. IMO it's just not needed.

I just assumed Brah had a NAS since that is what this thread is about and he said "when he connects the drive". I assume it isn't always in use. Not sure what kind of RAID or device it is then.

I'll gladly take you up on the "USB Beer challenge". Sounds fun actually. :D
I'll even let you pick the OS, as long as it is a filesystem created in the past 15 years. (No FAT32 or ext2)


x9200 wrote:Well, it's enough to google. You are the first person I know who does not recommend the safe, soft way of removing this type of hardware before physically unplugging it (BTW, the Brah's is not a NAS). OSX is screaming loudly if you unplug any usb rw storage device. Windows has a special button to safely remove such hardware. Are they all so wrong? There are or may be processes that scan drives and write to them some informations. They are transparent, not necessary showing up any way caching (for example) multimedia information or indexing various data. We don't know what Brah has installed on his PC. Corrupted data on the hard drive will not necessary show up immediately in a catastrophic failure.

Sorry, I don't get the link with the lawsuits and forensics. You mean people do not come over with this sort of problems? Why should they?

By data loss I mean any data loss/corruption, NOT the total data loss of all the data on the drive. Something writes to the drive, you unplug it and the system does not update the file table. This is still data loss. I don't think people hire forensic experts and sue anybody for such reason.

I will buy you a beer if you take an usb hd/thumb drive (RW mode), plug it in and out 20x (still far away from the strike of lightning likelihood) the way you suggested and there will be no inconsistencies within the file system.
You will buy me as many bottles of beer as the No of errors found within the file system after this experiment. Deal?


Is this challenge open to audience? I will bring buffalo wings along to watch.

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Fri, 08 Aug 2014 2:26 pm

Good, we will have to wait for opportunity to arrive and with your nicely relaxed criteria I may even consider writing my own file system to maximize the outcome.

BTW, It looks you meant a permanent loss and I meant any. Most of the time it is recoverable, but it is highly circumstantial as it depends also on the software, but still, why to take the unnecessary risk potentially wasting time or money?

zzm9980 wrote:The forensic reference was in regards to expertise in filesystems on the drives, and what happens when drives are improperly unmounted, that's all. They're constantly working on and imaging drives that were not properly shut down and dismounted. Data has never been lost, even on drives using full-disk encryption, unless the drive itself already had bad sectors.

I think with this long circular argument, we're mostly agreeing now. No, I know there will not be catastrophic data loss. Yes, you may lose what is writing at the time. I just assumed Brah would know not to write anything intentionally to the disk while doing what I suggested. Sure, some background processes may be indexing something on the disk and writing background metadata, but its inconsequential. If *that* is corrupted in some way it will be fixed and rebuilt the next time the indexing process restarts. While yes your way is safer, so is jaywalking while wearing a helmet or double bagging it when you go to OT. IMO it's just not needed.

I just assumed Brah had a NAS since that is what this thread is about and he said "when he connects the drive". I assume it isn't always in use. Not sure what kind of RAID or device it is then.

I'll gladly take you up on the "USB Beer challenge". Sounds fun actually. :D
I'll even let you pick the OS, as long as it is a filesystem created in the past 15 years. (No FAT32 or ext2)


x9200 wrote:Well, it's enough to google. You are the first person I know who does not recommend the safe, soft way of removing this type of hardware before physically unplugging it (BTW, the Brah's is not a NAS). OSX is screaming loudly if you unplug any usb rw storage device. Windows has a special button to safely remove such hardware. Are they all so wrong? There are or may be processes that scan drives and write to them some informations. They are transparent, not necessary showing up any way caching (for example) multimedia information or indexing various data. We don't know what Brah has installed on his PC. Corrupted data on the hard drive will not necessary show up immediately in a catastrophic failure.

Sorry, I don't get the link with the lawsuits and forensics. You mean people do not come over with this sort of problems? Why should they?

By data loss I mean any data loss/corruption, NOT the total data loss of all the data on the drive. Something writes to the drive, you unplug it and the system does not update the file table. This is still data loss. I don't think people hire forensic experts and sue anybody for such reason.

I will buy you a beer if you take an usb hd/thumb drive (RW mode), plug it in and out 20x (still far away from the strike of lightning likelihood) the way you suggested and there will be no inconsistencies within the file system.
You will buy me as many bottles of beer as the No of errors found within the file system after this experiment. Deal?

User avatar
x9200
Moderator
Moderator
Posts: 9313
Joined: Mon, 07 Sep 2009
Location: Singapore

Postby x9200 » Fri, 08 Aug 2014 2:37 pm

the lynx wrote:Is this challenge open to audience? I will bring buffalo wings along to watch.


Yes, buffalo wings is the right sort of food for this particular challenge.

User avatar
the lynx
Governor
Governor
Posts: 5272
Joined: Thu, 09 Dec 2010
Location: Location: Location: Location: Location: Location: Location: Location: Location: Location: Location:

Postby the lynx » Fri, 08 Aug 2014 2:47 pm

x9200 wrote:
the lynx wrote:Is this challenge open to audience? I will bring buffalo wings along to watch.


Yes, buffalo wings is the right sort of food for this particular challenge.


Hence the choice :wink:

User avatar
nakatago
Moderator
Moderator
Posts: 8333
Joined: Tue, 01 Sep 2009
Location: Sister Margaret’s School for Wayward Children
Contact:

Postby nakatago » Fri, 08 Aug 2014 2:47 pm

zzm9980 wrote:The forensic reference was in regards to expertise in filesystems on the drives, and what happens when drives are improperly unmounted, that's all. They're constantly working on and imaging drives that were not properly shut down and dismounted. Data has never been lost, even on drives using full-disk encryption, unless the drive itself already had bad sectors.


I've had my fair share of computer forensics.

"Here's dead memory. Rebuild the filesystem and see why it died."
"Uh, ok."


  • Similar Topics
    Replies
    Views
    Last post

Return to “Computer, Internet, Phone & Electronics”

Who is online

Users browsing this forum: No registered users and 1 guest