RAW file system problem

It may happen when you take out the storage device (memory card) not using "safely remove" feature, the next time you connect the storage device you can receive a message "You need to format the disk in drive X: before you can use it".

Typically this means that the filesystem on the drive became RAW. Obviously, initial file system was corrupted because the data stored in the buffer during disconnection of the drive was lost due to inappropriate disconnection.

To get rid of the RAW file system issue you need just to format the drive, however, be aware that your data would be gone forever according to the format type you apply. If the data is important you need to recover data first and after that do the format procedure. To recover data you can just use some data recovery program.

Booting from a SW RAID

In the previous post I said that you cannot boot from the software RAID array. In fact, this is not true. All the widespread bootloaders, from Windows NT, Linux LILO, and GRUB, would successfully load OS from a RAID 1 (mirror volume).
It should be noted that some further actions must be undertaken for RAID 1 boot volume when you plan to build a RAID.

Since the mirroring technique does not extend to the Master Boot Record (MBR), you must copy the MBR manually between drives in a software mirror. Otherwise, once the primary (boot) drive fails, you will end up with an unbootable system.

Surely, there will be no irreversible data loss - all you need is to just mount the disk into some other machine and read it, but you get no automatic recovery in case the disk fails in operation, requiring a reboot.

Make a software RAID bootable?

Is it possible to create a bootable software RAID0, RAID 5, or spanned volume containing a Windows installation?
Unfortunately, no. It is not possible to boot from a software RAID0, RAID5, or spanned volume. A hardware RAID controller is required to do this.

You cannot start an OS from the software RAID because the RAID is not accessible until the OS is fully started up, and the operating system itself is on the array.

Why it is critical to read data off both disks in RAID1?

When using RAID 1 it is needed to access data stored on the second member disk as well. Otherwise it may lead to that a bad sector appears on the second member disk. In this situation the RAID 1 doesn't have redundancy any longer and on top of that you do not know it. Absolutely the same is true for hot spare drives.

Typically one of the causes of this issue is the array being idle. To avoid this situation you can inspect both copies of data while reading. It should be noted that heavily loaded RAIDs do not have such issue.

In Windows, at least up to Windows 2003, RAID1 software implementation usually reads from the first drive, rendering RAID 1 unreliable.

NAS and backup power

Just cutting off the power is not sufficient to test an UPS. Generally, Network Attached Storage receives the messages from its UPS unit constantly and reacts at once. It should be noted that during the start up the notifications about power problems are not received. Additionally, one should take into account that the entire cycle (start up then shut down) may require several minutes.
 Consider the following situation:

1. power failure happens
2. NAS takes power from the UPS battery for some time and then shuts down
3. the grid power turns on and the Network Attached Storage powers up and begins to start up
4. here power is off again.

Now NAS cannot process a report about power failure event until it properly starts up. Once NAS loads, it finds out that the power failure has happened and commands to shut down. Further it may happen that UPS is not able to provide power long enough (to shut down NAS correctly) for the battery is already exhausted during the first cycle.
 One can avoid this by setting the NAS up so that it shuts down as soon as it gets a message that power failure happens.

 All this allows saving battery charge to do one more cycle.

 Alternatively you can try to set Network Attached Storage up in such a way that it does not power up automatically, but per operator's command after the power failure has happened.

Benchmark software I tested

I wanted to test my data storage devices in terms of performance and so used three benchmark utilities:

• HD Tune Pro
• Crystal Disk Mark
• BenchMe

With HD Tune Pro I managed to get quite a number of diverse parameters, however, I'm sure that only linear read speed and average disk access time are of interest. Additionally, it was difficult to find such a basic performance parameter as I/O operations per second.

As for CrystalDiskMark, it cannot benchmark a physical storage, but is limited to partitions only. In my opinion this might be a drawback. It gives read/write speed for different number and size of queries. It is also possible to get benchmark both for linear and random read/write speed. This benchmark tool is designed so that user is able to change amount of data which is required to read during tests.

As for another benchmark tool BenchMe, it is a simple to use tool to benchmark storage devices. Unlike CrystalDiskMark, BenchMe handles physical devices, but not volumes. Using this tool one can obtain the following performance characteristics: linear read speed, distribution of disk access time, and a list of device capabilities. IOPS are measured for queue depth 1 and 32.

BenchMe and CrystalDiskMark are free programs, but when it comes to interface and graphical representation of obtained benchmark data, BenchMe has the best of both paid HD Tune Pro and free CrystalDiskMark.

How to use hot spare disks properly?

Nowadays, many storage systems utilize a hot spare disk which is dedicated to save your ass if a disk fails. But a few people really understand how to handle hotspare disks correctly - these disks required to be checked periodically.
Let's consider such a situation - you create your own RAID and put N drives (along with the hotspare disk) to the array. In this case there exists 1/N probability that exactly hotspare disk will be the first to fail. So now you no longer have a hot spare.
 When one of the RAID member disks fails you are surprised to learn that the hot spare disk already failed.
To avoid this you need to check a hotspare periodically or stick to RAID6E or RAID 5E/EE layout.

Wiping the hard disk at home

The drive which you are going to sell or dispose of may still contain either some personal data. This data is to be removed off the disk before getting rid of it.

As far as software is concerned, there are several programs both free and paid that can overwrite the  information by either 0s or random noise. Once overwritten, the data is irreversibly deleted.

To obtain the same result yourself as with the secure erase software, just format the hard disk and load it fully with some stuff you don't care about (similar to multiple copies of House MD videos). Once the big file fits no more, continue adding smaller files. You can also do this with an USB box. On modern full encryption disks, just changing a password gets you the same result as after a secure erase.

Before building an array

When building some really big data storage, you should keep in mind the following concerns:

• array size
• fault-tolerance
• speed
• amount of money you are going to spend

Taking into account the above then it is needed to decide if you try to build your own storage unit or just order off-the-shelf data storage. If you choose the latter option then consider at least this one of RAID tips.
If you decide to build your own RAID then check all of the www.raidtips.com.

Does a RAID decrease access time?

Storage device performance is described by:

• access time
• throughput is sustained average speed of data transfer.

We know that RAID0 can increase throughput, as stated e.g. here.

Access time is more tricky. Access time in a regular hard disk includes the time needed to move a read head above the target track (seek time) and the time which is needed a hard disk to bring a sector under a read head (rotational latency).

Regardless of the number of member disks in RAID 0 it may occur that the cache doesn't have a requested sector which in turn is the furthest from the read head. If this sector is needed the access time is the same (not better) as in case of a single drive. If you are willing to improve random access time you should get rid of rotational hard drives and get an Solid State Drive instead.

The data file recovery

There exists a variety of tools doing the repairs or data extraction from a given file type, like DOC/DOCx, ZIP and so on. The file recovery is actually one more recovery type, similar to photo recovery which was mentioned earlier.

The file recovery software may be useful if the file got damaged because the file data was not fully written to a drive, say, because of the power failure during the write. Or because the program working with the file crashed before completing the write. Should this happen, the actual file content (or a worthwhile part of it) might still be written to the file.

Practically, it is pointless to apply such software to "fix" files which data recovery software found on the damaged drive. The data recovery process, such as the one used to recover RAW file system, typically either gets OK files, or the files are fully damaged, not containing any trace of original data. If the files do not open in their respective programs after a data recovery, this means most certainly no subsequent fixing applied to these files can restore them because the useful data is just not there.

Why are only the thumbnails are sometimes recovered?

More often than not while recovering digital photo files, typically off a memory card, the small image previews, called thumbnails, are recovered successfully, while high-resolution images themselves are damaged.

This behavior is explained by "file fragmentation". The fragmentation is said to occur when the file is stored on the disk in more than one non-contiguous blocks. The sketch explaining the process can be found at the Photo Recovery Limitation page.

The FAT or FAT32 filesystem is most likely to be used on a memory stick or a thumb drive.

Once the file gets erased on the FAT filesystem, the only information that is still available is how to locate the 1st fragment. Any subsequent blocks cannot be recovered.

Thumbnail is not large and placed at the beginning of the file, therefore the thumbnail is usually extracted OK, and still the high resolution image is garbled. Of all the data recovery types, this limitation affects mostly digital photo recovery and unformat on FAT.