mdadm raid problem

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

mdadm raid problem

Istvan Gabor-3
Hello:

I have this problem in openSUSE 13.2. My mdadm version is:

mdadm --version
mdadm - v3.3.1 - 5th June 2014

I have some degraded RAID1 arrays. Before adding back the failed
devices
to the arrays I would like to mount the devices independently and
compare
their contents.

For example:

cat /proc/mdstat
Personalities : [raid1]
md11 : active raid1 sdb11[3]
       62918468 blocks super 1.0 [2/1] [U_]


md11 array runs with one device: /dev/sdb11, and it (/dev/md11) is
mounted.
It's failed counterpart is /dev/sdc11.

I would like to assemble /dev/sdc11 to a new array and mount the new
array.
Previously (in openSUSE 12.2, eg.) I could simply do this:

mdadm -A /dev/md101 /dev/sdc11

and it worked without problem, assembled md101 that I could mount.

But in openSUSE 13.2 (mdadm - v3.3.1) it gives error:

mdadm -A /dev/md101 /dev/sdc11
mdadm: Found some drive for an array that is already active: /dev/md/11
mdadm: giving up.

I know it means that running md11 array has the same uuid as the "would
be
created" new md101 array and therefore mdadm prevents its assembly.

But how can I then assemble /dev/sdc11 to an independent array?

I tried to assemble the new array with updating the uuid:

# mdadm -A -U uuid /dev/md101 /dev/sdc11
mdadm: Found some drive for an array that is already active: /dev/md/11
mdadm: giving up.

or:

mdadm -A -U uuid -u 4ff5944f:b14c863f:9fddd970:c1ea6abc /dev/md101
/dev/sdc11
mdadm: Found some drive for an array that is already active: /dev/md/11
mdadm: giving up.

(I changed the last 3 digits of the original uuid.)


mdadm man says:

-U, --update=
               Update the superblock on each device while assembling the
array.  lhe argument given to this flag can be one of sparc2.2,
summaries,  uuid,
               name, homehost, resync, byteorder, devicesize, no-bitmap,
bbl, no-, metadata, or super-minor.

               The uuid option will change the uuid of the array.  If a
UUID is given with the --uuid option that UUID will be used as a new
UUID and will
               NOT be used to help identify the devices in the array.  
If no --uuid is given, a random UUID is chosen.

Even if I could assemble /dev/sdc11 with a new uuid, how can I then
add it back to md11 later? The new uuid will prevent it, I guess.
I will have to change back the uuid to the original. It is tedious
and error-prone.

What would be the best solution?

Thanks,

Istvan



--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

David C. Rankin
On 06/07/2017 04:29 PM, Istvan Gabor wrote:

> Hello:
>
> I have this problem in openSUSE 13.2. My mdadm version is:
>
> mdadm --version
> mdadm - v3.3.1 - 5th June 2014
>
> I have some degraded RAID1 arrays. Before adding back the failed devices
> to the arrays I would like to mount the devices independently and compare
> their contents.
>

What do the following show:

  # mdadm -D /dev/md101

  # mdadm -E /dev/sd[bc]11

and look at /dev/disk/by-id

It sounds like you are attempting to add a drive back to the array that the
array considers failed. The mdadm commands should show what the array thinks
its drives are and what the drives think their mdadm uuids are. Hard to tell.

--
David C. Rankin, J.D.,P.E.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Andrei Borzenkov
In reply to this post by Istvan Gabor-3
08.06.2017 00:29, Istvan Gabor пишет:

> Hello:
>
> I have this problem in openSUSE 13.2. My mdadm version is:
>
> mdadm --version
> mdadm - v3.3.1 - 5th June 2014
>
> I have some degraded RAID1 arrays. Before adding back the failed devices
> to the arrays I would like to mount the devices independently and compare
> their contents.
>
> For example:
>
> cat /proc/mdstat
> Personalities : [raid1]
> md11 : active raid1 sdb11[3]
>       62918468 blocks super 1.0 [2/1] [U_]
>
>
> md11 array runs with one device: /dev/sdb11, and it (/dev/md11) is mounted.
> It's failed counterpart is /dev/sdc11.
>
> I would like to assemble /dev/sdc11 to a new array and mount the new array.
> Previously (in openSUSE 12.2, eg.) I could simply do this:
>
> mdadm -A /dev/md101 /dev/sdc11
>
> and it worked without problem, assembled md101 that I could mount.
>
> But in openSUSE 13.2 (mdadm - v3.3.1) it gives error:
>
> mdadm -A /dev/md101 /dev/sdc11
> mdadm: Found some drive for an array that is already active: /dev/md/11
> mdadm: giving up.
>
> I know it means that running md11 array has the same uuid as the "would be
> created" new md101 array and therefore mdadm prevents its assembly.
>
> But how can I then assemble /dev/sdc11 to an independent array?
>
> I tried to assemble the new array with updating the uuid:
>
> # mdadm -A -U uuid /dev/md101 /dev/sdc11
> mdadm: Found some drive for an array that is already active: /dev/md/11
> mdadm: giving up.
>
> or:
>
> mdadm -A -U uuid -u 4ff5944f:b14c863f:9fddd970:c1ea6abc /dev/md101
> /dev/sdc11
> mdadm: Found some drive for an array that is already active: /dev/md/11
> mdadm: giving up.
>
> (I changed the last 3 digits of the original uuid.)
>

I do not think it is possible. You can change UUID only doing assembly
and you cannot do assembly for duplicated UUID. You need to stop
original array first.

> Even if I could assemble /dev/sdc11 with a new uuid, how can I then
> add it back to md11 later?

Just wipe off metadata and add as new empty disk.


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

David C. Rankin
In reply to this post by David C. Rankin
On 06/07/2017 10:32 PM, David C. Rankin wrote:

> On 06/07/2017 04:29 PM, Istvan Gabor wrote:
>> Hello:
>>
>> I have this problem in openSUSE 13.2. My mdadm version is:
>>
>> mdadm --version
>> mdadm - v3.3.1 - 5th June 2014
>>
>> I have some degraded RAID1 arrays. Before adding back the failed devices
>> to the arrays I would like to mount the devices independently and compare
>> their contents.
>>
>
> What do the following show:
>
>   # mdadm -D /dev/md101
>
>   # mdadm -E /dev/sd[bc]11
>
> and look at /dev/disk/by-id
>
> It sounds like you are attempting to add a drive back to the array that the
> array considers failed. The mdadm commands should show what the array thinks
> its drives are and what the drives think their mdadm uuids are. Hard to tell.
>

Sorry to reply to a reply, but before you can add back your failed drive from
an array (I guess /dev/sdc11 is the failed drive and md101 is the array in
this case) you must remove it from the array (e.g. --fail then --remove it),
then you can add it back. For example:

# mdadm /dev/md101 --fail /dev/sdc11 --remove /dev/sdc11

(if it was already failed, it will just report 'set device faulty failed
/dev/sdc11: No such device)

Then you can add it back with:

# mdadm /dev/md101 --add /dev/sdc11

(it should report: 'mdadm: re-added /dev/sdc11')

What is considered the most up to date drive in the array for re-syncing
purposes will be the drive with the largest event count if all other things
are equal.

You can fail, remove and re-add disks to the array without any problem. (you
can do it for fun or for testing) mdadm is quite robust in that regard.

--
David C. Rankin, J.D.,P.E.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Istvan Gabor-3
In reply to this post by Andrei Borzenkov
On Thu, 8 Jun 2017 06:57:58 +0300, Andrei Borzenkov wrote:

>
> I do not think it is possible. You can change UUID only doing
> assembly
> and you cannot do assembly for duplicated UUID. You need to stop
> original array first.

OK. I stopped running md11 which had /dev/sdb11 device.
Then I could assemble a new array that has /dev/sdc11, using the update
uuid option. But the array is always assembled as /dev/md127 and not
to the name I specify:

mdadm -A -U uuid /dev/md111 /dev/sdc11
mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it (use
--run to insist).

mdadm -D /dev/md111
mdadm: cannot open /dev/md111: No such file or directory

cat /proc/mdstat
Personalities : [raid1]
md127 : inactive sdc11[2](S)
       62918468 blocks super 1.0

Is this a bug?
How can I assemble the array as md111 or whatever I specify?


>> Even if I could assemble /dev/sdc11 with a new uuid, how can I then
>> add it back to md11 later?
>
> Just wipe off metadata and add as new empty disk.

OK, I'll do that or just set the original uuid back.

Thanks,

Istvan

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Andrei Borzenkov
08.06.2017 13:20, Istvan Gabor пишет:

> On Thu, 8 Jun 2017 06:57:58 +0300, Andrei Borzenkov wrote:
>
>>
>> I do not think it is possible. You can change UUID only doing assembly
>> and you cannot do assembly for duplicated UUID. You need to stop
>> original array first.
>
> OK. I stopped running md11 which had /dev/sdb11 device.
> Then I could assemble a new array that has /dev/sdc11, using the update
> uuid option. But the array is always assembled as /dev/md127 and not
> to the name I specify:
>
> mdadm -A -U uuid /dev/md111 /dev/sdc11
> mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it (use
> --run to insist).
>
> mdadm -D /dev/md111
> mdadm: cannot open /dev/md111: No such file or directory
>

Your array is not yet running. What happens if you start it first as
previous message tells you?

> cat /proc/mdstat
> Personalities : [raid1]
> md127 : inactive sdc11[2](S)
>       62918468 blocks super 1.0
>
> Is this a bug> How can I assemble the array as md111 or whatever I specify?
>
>
>>> Even if I could assemble /dev/sdc11 with a new uuid, how can I then
>>> add it back to md11 later?
>>
>> Just wipe off metadata and add as new empty disk.
>
> OK, I'll do that or just set the original uuid back.
>
> Thanks,
>
> Istvan
>


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

David C. Rankin
On 06/09/2017 12:04 AM, Andrei Borzenkov wrote:

>> mdadm -A -U uuid /dev/md111 /dev/sdc11
>> mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it (use
>> --run to insist).
>>
>> mdadm -D /dev/md111
>> mdadm: cannot open /dev/md111: No such file or directory
>>
> Your array is not yet running. What happens if you start it first as
> previous message tells you?
>

You can also provide an express 'missing' argument telling mdadm you know the
drive is missing, e.g.

 # mdadm -A -U uuid /dev/md111 /dev/sdc11 missing

--
David C. Rankin, J.D.,P.E.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Wols Lists
In reply to this post by Istvan Gabor-3
On 08/06/17 11:20, Istvan Gabor wrote:

> On Thu, 8 Jun 2017 06:57:58 +0300, Andrei Borzenkov wrote:
>
>>
>> I do not think it is possible. You can change UUID only doing assembly
>> and you cannot do assembly for duplicated UUID. You need to stop
>> original array first.
>
> OK. I stopped running md11 which had /dev/sdb11 device.
> Then I could assemble a new array that has /dev/sdc11, using the update
> uuid option. But the array is always assembled as /dev/md127 and not
> to the name I specify:
>
> mdadm -A -U uuid /dev/md111 /dev/sdc11
> mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it (use
> --run to insist).
>
> mdadm -D /dev/md111
> mdadm: cannot open /dev/md111: No such file or directory
>
> cat /proc/mdstat
> Personalities : [raid1]
> md127 : inactive sdc11[2](S)
>       62918468 blocks super 1.0
>
> Is this a bug?
> How can I assemble the array as md111 or whatever I specify?

You can't.

That's why the recommended way now is to use array names, not numbers,
eg /dev/md/root, /dev/md/home.

It's like /dev/sdX and UUIDs, array numbers are now (normally) assigned
sequentially from 127 in reverse order. If you want a unique
non-changing id of some sort, you have to either use the md-UUID, or a
name of your choice.

Oh - and if you want to look at the second (kicked off) drive, I think
you did it the right way. You have to either stop the original array, or
move the drive you want to look at to another machine.

Note that if the array has bitmaps enabled (the default nowadays), when
you re-add the drive, it knows what it needs to sync so it won't sync
the entire disk - a rather slow option on today's multi-terabyte
drives... :-)

Cheers,
Wol

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Istvan Gabor-3
In reply to this post by Andrei Borzenkov
On Fri, 9 Jun 2017 08:04:39 +0300, Andrei Borzenkov wrote:

> 08.06.2017 13:20, Istvan Gabor пишет:
>> On Thu, 8 Jun 2017 06:57:58 +0300, Andrei Borzenkov wrote:
>>
>>>
>>> I do not think it is possible. You can change UUID only doing
>>> assembly
>>> and you cannot do assembly for duplicated UUID. You need to stop
>>> original array first.
>>
>> OK. I stopped running md11 which had /dev/sdb11 device.
>> Then I could assemble a new array that has /dev/sdc11, using the
>> update
>> uuid option. But the array is always assembled as /dev/md127 and not
>> to the name I specify:
>>
>> mdadm -A -U uuid /dev/md111 /dev/sdc11
>> mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it
>> (use
>> --run to insist).
>>
>> mdadm -D /dev/md111
>> mdadm: cannot open /dev/md111: No such file or directory
>>
>
> Your array is not yet running. What happens if you start it first as
> previous message tells you?

Thanks, Andrei:

If I assemble and start the array in separate commands, the array is
assembled as md127 and then is started as md127.

If I assemble and run the array in the same command, it is started
with the name the command specifies. E.g.

mdadm -A -R /dev/md111 /dev/sdc11

assembles and runs the array as md111, not md127.

I consider this a buggy behavior.

Thanks again,

Istvan



--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Istvan Gabor-3
In reply to this post by Wols Lists
On Fri, 9 Jun 2017 22:04:23 +0100, Wols Lists wrote:

> On 08/06/17 11:20, Istvan Gabor wrote:
>> But the array is always assembled as /dev/md127 and not
>> to the name I specify:
>>
>> mdadm -A -U uuid /dev/md111 /dev/sdc11
>> mdadm: /dev/md111 assembled from 1 drive - need all 2 to start it
>> (use
>> --run to insist).
>>
>> mdadm -D /dev/md111
>> mdadm: cannot open /dev/md111: No such file or directory
>>
>> cat /proc/mdstat
>> Personalities : [raid1]
>> md127 : inactive sdc11[2](S)
>>       62918468 blocks super 1.0
>>
>> Is this a bug?
>> How can I assemble the array as md111 or whatever I specify?
>
> You can't.

Thanks.
I can if I assemble and run the array in one command, that is
I use -A -R.

> That's why the recommended way now is to use array names, not
> numbers,
> eg /dev/md/root, /dev/md/home.
>
> It's like /dev/sdX and UUIDs, array numbers are now (normally)
> assigned
> sequentially from 127 in reverse order. If you want a unique
> non-changing id of some sort, you have to either use the md-UUID, or
> a
> name of your choice.


Thanks, I did not know this. I'll try to remember it.

> Oh - and if you want to look at the second (kicked off) drive, I
> think
> you did it the right way. You have to either stop the original array,
> or
> move the drive you want to look at to another machine.
>
> Note that if the array has bitmaps enabled (the default nowadays),
> when
> you re-add the drive, it knows what it needs to sync so it won't sync
> the entire disk - a rather slow option on today's multi-terabyte
> drives... :-)

I'll look into this too.

Thanks again,

Istvan


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: mdadm raid problem

Wols Lists
On 13/06/17 13:44, Istvan Gabor wrote:
>>> Is this a bug?
>>> How can I assemble the array as md111 or whatever I specify?
>>
>> You can't.
>
> Thanks.
> I can if I assemble and run the array in one command, that is
> I use -A -R.

Yup. I should have thought of that. But what's happening is you're
giving it a name in the assembly command, so it uses the name you give
it. If you run it in a separate command, it now assigns it a "random"
number, which is 127, or 126, or whatever ...

Cheers,
Wol

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Loading...