Another LaCie 5big Network 2 problem

[tenhouse]

Hello hello

I got pointed to this forum from this thread (http://forum.nas-portal.org/showthread. ... -Laufwerke) by Mijzelf.

Since I don't expect you all speak german I will gladly give you a short summary of what happend:

- lacie support is the best (*haha*)
- I have a Lacie 5big Network 2, with 15 TB, and RAID 5, age about 1 year
Suddenly overnight one disk showed as "missing" in the dashboard, I just unplugged the disk and put it back in and the raid array was rebuilt. After that I had access to my data for about 10 minutes. Then access was gone again - but RAID array still showed as "healthy". My whole dashboard interface was (and is) very slow and showed weired stuff (for example no IP address or 0.0.0.0 as IP and so on). Suddenly 2 disks where shown as missing, I just rebooted the NAS and tadaaa RAID Array ways healthy again. Then as a next step, according to the lacie support ,I reseted the device (without data loss) and 1 disks (its always the "middle" one by the way) was missing again, after another reboot 2 disks were missing, after another reboot 3 disks were missing and then again only the one in the middle.
Seems legit to me.

Anyway what I now did is:
I downloaded the fvdw-sl-console and loaded the standalone kernel.
Then I tried some things from here (http://www.plugout.net/viewtopic.php?f=4&t=1412) but didn't want to go any further since I don't really know what I'm doing :)

Some outputs:

Code: Select all

GPT fdisk (gdisk) version 0.8.5

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Command (? for help): p
Disk /dev/sda: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 4F7CB08D-65F3-42C5-9F53-0909ECC9C4BE
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 31410 sectors (15.3 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1         4036608         4038655   1024.0 KiB  0700  primary
   2         4048896      5860530447   2.7 TiB     0700  primary
   3         4040704         4042571   934.0 KiB   0700  primary
   4         4044800         4046848   1.0 MiB     0700  primary
   5            2048          514047   250.0 MiB   0700  primary
   6          530432          546497   7.8 MiB     0700  primary
   7          546816          578945   15.7 MiB    0700  primary
   8          579584         2266409   823.6 MiB   0700  primary
   9         2267136         4018221   855.0 MiB   0700  primary
  10         4020224         4036289   7.8 MiB     0700  primary



Code: Select all

root@(none):/ # fdisk -l

Disk /dev/mtdblock1: 16 MB, 16777216 bytes
255 heads, 63 sectors/track, 2 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/mtdblock1 doesn't contain a valid partition table

Disk /dev/mtdblock2: 250 MB, 250609664 bytes
255 heads, 63 sectors/track, 30 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/mtdblock2 doesn't contain a valid partition table
fdisk: device has more than 2^32 sectors, can't use all of them

Disk /dev/sdc: 2199.0 GB, 2199023255040 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdc1               1      267350  2147483647+ ee EFI GPT
fdisk: device has more than 2^32 sectors, can't use all of them

Disk /dev/sdb: 2199.0 GB, 2199023255040 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1               1      267350  2147483647+ ee EFI GPT
fdisk: device has more than 2^32 sectors, can't use all of them

Disk /dev/sda: 2199.0 GB, 2199023255040 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1               1      267350  2147483647+ ee EFI GPT


Code: Select all

some demesg stuff:

[  103.066122] WARNING: at drivers/ata/libata-sff.c:1322 ata_sff_queue_pio_task+0x30/0x54()
[  103.074177] Modules linked in:
[  103.077246] [<c0231a10>] (unwind_backtrace+0x0/0xe4) from [<c023e464>] (warn_slowpath_common+0x4c/0x64)
[  103.086615] [<c023e464>] (warn_slowpath_common+0x4c/0x64) from [<c023e494>] (warn_slowpath_null+0x18/0x1c)
[  103.096241] [<c023e494>] (warn_slowpath_null+0x18/0x1c) from [<c0525084>] (ata_sff_queue_pio_task+0x30/0x54)
[  103.106036] [<c0525084>] (ata_sff_queue_pio_task+0x30/0x54) from [<c0526c1c>] (ata_sff_qc_issue+0x1a8/0x1f4)
[  103.115834] [<c0526c1c>] (ata_sff_qc_issue+0x1a8/0x1f4) from [<c052f1c8>] (mv_qc_issue+0x458/0x4a0)
[  103.124856] [<c052f1c8>] (mv_qc_issue+0x458/0x4a0) from [<c0514c04>] (ata_qc_issue+0x284/0x2dc)
[  103.133531] [<c0514c04>] (ata_qc_issue+0x284/0x2dc) from [<c0514f24>] (ata_exec_internal_sg+0x2c8/0x5c0)
[  103.142981] [<c0514f24>] (ata_exec_internal_sg+0x2c8/0x5c0) from [<c0515298>] (ata_exec_internal+0x7c/0x8c)
[  103.152693] [<c0515298>] (ata_exec_internal+0x7c/0x8c) from [<c05270fc>] (sata_pmp_read+0x88/0xc0)
[  103.161625] [<c05270fc>] (sata_pmp_read+0x88/0xc0) from [<c0527160>] (sata_pmp_read_gscr+0x2c/0x80)
[  103.170647] [<c0527160>] (sata_pmp_read_gscr+0x2c/0x80) from [<c05277dc>] (sata_pmp_error_handler+0x2d4/0xab0)
[  103.180616] [<c05277dc>] (sata_pmp_error_handler+0x2d4/0xab0) from [<c0522900>] (ata_scsi_port_error_handler+0x160/0x44c)
[  103.191537] [<c0522900>] (ata_scsi_port_error_handler+0x160/0x44c) from [<c0522c9c>] (ata_scsi_error+0xb0/0xe4)
[  103.201599] [<c0522c9c>] (ata_scsi_error+0xb0/0xe4) from [<c0505198>] (scsi_error_handler+0x80/0x3d0)
[  103.210789] [<c0505198>] (scsi_error_handler+0x80/0x3d0) from [<c0257158>] (kthread+0x84/0x8c)
[  103.219379] [<c0257158>] (kthread+0x84/0x8c) from [<c022cff4>] (kernel_thread_exit+0x0/0x8)
[  103.227696] ---[ end trace aa6825215f6dd33c ]---
[  103.925405] ata2.00: hard resetting link
[  104.275819] ata2: EH complete
[  104.295192] Freeing init memory: 2152K
[  654.396459] UDF-fs: No partition found (1)
[  669.071870] uncorrectable error :
[  669.075214] end_request: I/O error, dev mtdblock0, sector 0
[  669.080944] Buffer I/O error on device mtdblock0, logical block 0
[  669.087213] uncorrectable error :
[  669.090442] end_request: I/O error, dev mtdblock0, sector 8
[  669.096175] Buffer I/O error on device mtdblock0, logical block 1
[  669.102407] end_request: I/O error, dev mtdblock0, sector 16
[  669.108048] Buffer I/O error on device mtdblock0, logical block 2
[  669.114278] uncorrectable error :
[  669.117519] end_request: I/O error, dev mtdblock0, sector 24
[  669.123340] Buffer I/O error on device mtdblock0, logical block 3
[  669.129659] uncorrectable error :
[  669.132890] end_request: I/O error, dev mtdblock0, sector 0
[  669.138623] Buffer I/O error on device mtdblock0, logical block 0
[ 1588.725453] uncorrectable error :
[ 1588.728700] end_request: I/O error, dev mtdblock0, sector 0
[ 1588.734431] Buffer I/O error on device mtdblock0, logical block 0
[ 1588.740701] uncorrectable error :
[ 1588.743929] end_request: I/O error, dev mtdblock0, sector 8
[ 1588.749662] Buffer I/O error on device mtdblock0, logical block 1
[ 1588.755905] end_request: I/O error, dev mtdblock0, sector 16
[ 1588.761543] Buffer I/O error on device mtdblock0, logical block 2
[ 1588.767773] uncorrectable error :
[ 1588.771003] end_request: I/O error, dev mtdblock0, sector 24
[ 1588.776817] Buffer I/O error on device mtdblock0, logical block 3
[ 1588.783160] uncorrectable error :
[ 1588.786405] end_request: I/O error, dev mtdblock0, sector 0
[ 1588.792137] Buffer I/O error on device mtdblock0, logical block 0


And I already uploaded mdadm with tftp - so I would be ready to use it if needed.

Any help would be appreciated :) You can also just point me to the other thread if I have do to the exactly same things.

[Mijzelf]

Well, first do a 'cat /proc/partitions' to see if all partition tables are still alive. (It can also be done by examining all partition table, but this is faster).
Then, for all found sd[abcde]2, do a 'mdadm --examine /dev/sd[abcde]2', to see if the raid headers are still alive. Do you also have a 'smartctl'? In that case you could execute a 'smartctl -a /dev/sd[abcde]' for each disk, to see if it thinks it's healthy.

[tenhouse]

Code: Select all

root@(none):/ # cat /proc/partitions
major minor  #blocks  name

  31        0       1024 mtdblock0
  31        1      16384 mtdblock1
  31        2     244736 mtdblock2
   8        0 2930266584 sdc
   8        1       1024 sdc1
   8        2 2928240776 sdc2
   8        3        934 sdc3
   8        4       1024 sdc4
   8        5     256000 sdc5
   8        6       8033 sdc6
   8        7      16065 sdc7
   8        8     843413 sdc8
   8        9     875543 sdc9
   8       10       8033 sdc10
   8       16 2930266584 sdb
   8       17       1024 sdb1
   8       18 2928240776 sdb2
   8       19        934 sdb3
   8       20       1024 sdb4
   8       21     256000 sdb5
   8       22       8033 sdb6
   8       23      16065 sdb7
   8       24     843413 sdb8
   8       25     875543 sdb9
   8       26       8033 sdb10
   8       32 2930266584 sda
   8       33       1024 sda1
   8       34 2928240776 sda2
   8       35        934 sda3
   8       36       1024 sda4
   8       37     256000 sda5
   8       38       8033 sda6
   8       39      16065 sda7
   8       40     843413 sda8
   8       41     875543 sda9
   8       42       8033 sda10


Code: Select all

root@(none):/ # mdadm --examine /dev/sdc2
/dev/sdc2:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x0
     Array UUID : 61170a64:3186e2d5:980d05ef:30413472
           Name : (none):4  (local to host (none))
  Creation Time : Tue Jul 10 15:36:48 2012
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 5856481280 (2792.59 GiB 2998.52 GB)
     Array Size : 11712962560 (11170.35 GiB 11994.07 GB)
   Super Offset : 5856481536 sectors
          State : clean
    Device UUID : ac938392:1dba4fe3:ef6ea7b1:300f9d68

    Update Time : Wed Sep  4 16:20:12 2013
       Checksum : 307c5e60 - correct
         Events : 892723

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AA... ('A' == active, '.' == missing)


Code: Select all

root@(none):/ # mdadm --examine /dev/sdb2
/dev/sdb2:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x0
     Array UUID : 61170a64:3186e2d5:980d05ef:30413472
           Name : (none):4  (local to host (none))
  Creation Time : Tue Jul 10 15:36:48 2012
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 5856481280 (2792.59 GiB 2998.52 GB)
     Array Size : 11712962560 (11170.35 GiB 11994.07 GB)
   Super Offset : 5856481536 sectors
          State : clean
    Device UUID : 0a0dc206:3aefe30e:7640232a:b3b8d642

    Update Time : Wed Sep  4 16:20:12 2013
       Checksum : 230487e4 - correct
         Events : 892723

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AA... ('A' == active, '.' == missing)



Code: Select all

root@(none):/ # mdadm --examine /dev/sda2
/dev/sda2:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x0
     Array UUID : 61170a64:3186e2d5:980d05ef:30413472
           Name : (none):4  (local to host (none))
  Creation Time : Tue Jul 10 15:36:48 2012
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 5856481280 (2792.59 GiB 2998.52 GB)
     Array Size : 11712962560 (11170.35 GiB 11994.07 GB)
   Super Offset : 5856481536 sectors
          State : clean
    Device UUID : 26a04adc:785082a5:0a0d4501:d6c7286d

    Update Time : Wed Sep  4 16:13:49 2013
       Checksum : 909d567b - correct
         Events : 892716

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : AAA.A ('A' == active, '.' == missing)



Hmm do I see this right: right now I only have 3 out of the 5 hds here? Maybe I should reboot a few times...maybe one of the others will come back...

"smartctl" is not found.

[Mijzelf]

The SoC actually has only 2 sata interfaces, it uses sata multipliers to get 5 intern plus 2 (3?) extern sata connections. According to this table, the first 3 disks are connected to one channel, and the other 2 to the other channel. So it might be possible that either a sata interface or a multiplier is damaged.
That dmesg log you posted also seems not to be happy about something sata/ata related.

Which disks does the firmware say that are dropped? If fvdw meanwhile solved all device name mangling problems related to the 5big2, it seems disks 4 and 5 are out.

Your raid looks healthy, and when you've actually got a hardware problem, chances are that your data is fine.

BTW, according to the mtdblocks you've got a 'diskless' sample.

[tenhouse]

Okay I booted in the normal lacie-mode and saw that at first only disk 4 shows as missing and after a minute or something disk 5 is also showing as missing. Then I rebooted again and only disk 4 is missing.. and its online for about 5 minutes.
Also note that before it was mostly the middle disk (3) that was shown as missing.
edit: and now disk 3, 4 and 5 are showing as missing / empty.

So the more I see of this the more I also think it's a hardware problem.

Problem is: I don't have another lacie device at hand where I could put the disks in... sending it in is also no option, since they support-guy "promised" me that they will erase everything if I do that, he also told me he won't send another device in advance and I send the broken device back after.... only thing I cuold try is to convince him to send the empty device to them and they would send me a new device without disks back....
Or as already someone in my other thread mentioned I could try to take the disks out and put them to another SATA controller and restore the RAID on this way :S but I really fear I will break something if I do this.

edit2:
rebooted back with the tool and now this:

Code: Select all

root@(none):/ # cat /proc/partitions
major minor  #blocks  name

  31        0       1024 mtdblock0
  31        1      16384 mtdblock1
  31        2     244736 mtdblock2
   8       16 2930266584 sdb
   8       17       1024 sdb1
   8       18 2928240776 sdb2
   8       19        934 sdb3
   8       20       1024 sdb4
   8       21     256000 sdb5
   8       22       8033 sdb6
   8       23      16065 sdb7
   8       24     843413 sdb8
   8       25     875543 sdb9
   8       26       8033 sdb10
   8       32 2930266584 sda
   8       33       1024 sda1
   8       34 2928240776 sda2
   8       35        934 sda3
   8       36       1024 sda4
   8       37     256000 sda5
   8       38       8033 sda6
   8       39      16065 sda7
   8       40     843413 sda8
   8       41     875543 sda9
   8       42       8033 sda10


[Mijzelf]

Now only 2 disks left? :shock:

I wonder if your power supply is failing. Do you have a possibility to check that?

Or as already someone in my other thread mentioned I could try to take the disks out and put them to another SATA controller and restore the RAID on this way :S but I really fear I will break something if I do this.

Assembling the array using any other Linux box is not really different than assembling it inside the NAS, using fvdw's tools. But yes, you'll have to fiddle around with disks.

Do you have much data on that disks?

[tenhouse]

Had exactly the same thought as you with the power problem. So what I did is just I took the whole device to another place (where not so many devices are on the same "power supply") - but it didn't help. (And I don't think the power-supply of the NAS is broken, can't say for sure though).

I rebooted and now 3 disks back again :> I really don't think this says anything anymore. Also its wierd, when I watch the LEDS at the back of the device 4 out of 5 are blue and only one is not burning at all. But the dashbaoard says "drive 4 emtpy" and drive 5 "missing" 3 TB.
edit: ok after some minutes 3 disks are gone again, and 2 LEDs are off)

I have about ~ 3-4 TB of data on it. Most of it is a huge movie / music collection (which hurts to lose, but ok, shit happens). What pisses me off much more that me (the *** idiot) put all his data (which came togtehter over all my "pc" years) on this disk and recently erased a disk where I had a backup on it because I needed it and was like "oh hey no problem have RAID 5, it won't burn down or something" ... yeah good job douche..).

[Mijzelf]

I meant the power supply between the mains and the Lacie. Or does the Lacie have a 230V input?

I have about ~ 3-4 TB of data on it

Great. So if you manage to assemble the array, you still have a problem where to put it?

[tenhouse]

I meant the power supply between the mains and the Lacie. Or does the Lacie have a 230V input?

I have about ~ 3-4 TB of data on it

Great. So if you manage to assemble the array, you still have a problem where to put it?

I have actually 2 external 2 TB HDs and one 1 TB HD - so I would have enough space to save the stuff.

No no it has a power supply in between. It's this one http://www.fspgroupusa.com/fsp150ahan1/p/717.html . I don't think its broken .

[fvdw]

I see you are in good hands, if the the output you posted is from dmesg command when running the standalone kernel then definetily something is wrong with detecting the hardware. Or you are using not the latest standalone kernel
But which standalone kernel did you use ?