FreeNAS 0.7.3953, RAID 1, growfs... oh my!

Fooling around in VMWare, preparing to build my two NAS boxen, I started playing with disk upgrade scenarios in a two-drive RAID 1 configuration. I have the embedded FreeNAS on /dev/ad0, and the two mirror disks are /dev/ad2 and /dev/ad3. First, I had them both at 512MB each, then bumped one to 1GB, fixed the mirror, then bumped the other to 1GB and fixed the mirror again. At this point, I had two 1GB disks with a 512MB (give or take) mirror since it was sized from the initial installation with two 512MB disks. Time to figure out growfs...

I tinkered around with growfs, finally got it to work, and ended up with a mirror to match the size (more or less) of the actual disks. Now, let's do it again to make sure it wasn't a fluke! I bumped one disk to 2GB, rebuilt the mirror, then bumped the other disk to 2GB, rebuilt mirror, thus leaving me with a pair of 2GB disks and a 1GB mirror. Then I followed the growfs process again and took some notes this time, here for your reading pleasure.

NOTE -- This worked for me and I make no guarantees, express or implied, that it will work for you. If you follow my instructions, you may come through this just fine, or you may not. Always back up your data before starting on something like this, and understand the commands you are using and what the impact is. If you can, play in a VM sandbox for a while until you are comfortable.


Notes

---

At this point, I shutdown the FreeNAS VM and replaced /dev/ad3, which was 1GB, with a 2GB disk to match /dev/ad2.

1. At the web gui, edit the /dev/ad3 disk and just click on 'Save' and the new size will be picked up automatically.

2. The raid is still degraded, as expected, so do the 'replace disk' dance...

• edit the raid1 array
• click on 'tools', then choose RAID1 (volume name) and command 'forget' and click on 'Send Command!'
• click on 'tools', then choose RAID1 (volume name), disk ad3, command 'insert', and click on 'Send Command!'
• go back to the raid1 management tab and the status should be COMPLETE once the mirror is rebuilt

3. The mount point should be in 'Error - Retry' state right now, so click on Retry to ensure that it mounts ok.

[root@freenas ~]# [root@freenas ~]# [root@freenas ~]# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0 80430 74634 5796 93% / devfs 1 1 0 100% /dev /dev/md1 31470 12 28942 0% /var /dev/ad0a 39406 34216 5190 87% /cf /dev/mirror/RAID1p1 888526 69300 748144 8% /mnt/NAS1 [root@freenas ~]# [root@freenas ~]#

4. At this point, the mount is back online with the old 1GB mirror size, so let's get into the meat of it.

5. Go back to the mount point configuration and delete the mount point

6. Go to the raid management and delete the raid1 array

7. Add a new raid 1 array, which I will call RAID1, and choose 'create and initialize RAID' option, then click on 'Add'

8. Next, head to the beloved (for me, anyway) CLI via SSH and do the hardcore stuff...

[root@freenas ~]# [root@freenas ~]# [root@freenas ~]# mount /dev/md0 on / (ufs, local) devfs on /dev (devfs, local) /dev/md1 on /var (ufs, local, union) /dev/ad0a on /cf (ufs, local, read-only) [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0 80430 74612 5818 93% / devfs 1 1 0 100% /dev /dev/md1 31470 12 28942 0% /var /dev/ad0a 39406 34216 5190 87% /cf [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# [root@freenas ~]# cd /dev/mirror [root@freenas /dev/mirror]# alias ls='ls -al' [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# ls total 1 dr-xr-xr-x 2 root wheel 512 Dec 16 16:40 . dr-xr-xr-x 5 root wheel 512 Dec 16 16:40 .. crw-r----- 1 root operator 0, 81 Dec 16 16:40 RAID1 crw-r----- 1 root operator 0, 85 Dec 16 16:40 RAID1p1 [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt remove -i 1 /dev/mirror/RAID1 gpt remove: /dev/mirror/RAID1: error: no secondary GPT header; run recover [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt recover /dev/mirror/RAID1 gpt recover: /dev/mirror/RAID1: recovered secondary GPT table from primary gpt recover: /dev/mirror/RAID1: recovered secondary GPT header from primary [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 2097084 1 GPT part - FreeBSD UFS/UFS2 2097118 2097152 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt remove -i 1 /dev/mirror/RAID1 /dev/mirror/RAID1p1 removed [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4194236 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt add -i 1 -t ufs /dev/mirror/RAID1 [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4194236 1 GPT part - FreeBSD UFS/UFS2 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show -l /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4194236 1 GPT part - "RAID1" 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt label -i 1 -l RAID1 /dev/mirror/RAID1 /dev/mirror/RAID1p1 labeled [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4194236 1 GPT part - FreeBSD UFS/UFS2 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# gpt show -l /dev/mirror/RAID1 start size index contents 0 1 PMBR 1 1 Pri GPT header 2 32 Pri GPT table 34 4194236 1 GPT part - "RAID1" 4194270 32 Sec GPT table 4194302 1 Sec GPT header [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# growfs /dev/mirror/RAID1p1 We strongly recommend you to make a backup before growing the Filesystem Did you backup your data (Yes/No) ? Yes new file systemsize is: 1048559 frags Warning: 262076 sector(s) cannot be allocated. growfs: 1920.0MB (3932160 sectors) block size 16384, fragment size 2048 using 15 cylinder groups of 128.00MB, 8192 blks, 16384 inodes. with soft updates super-block backups (for fsck -b #) at: 1835168, 2097312, 2359456, 2621600, 2883744, 3145888, 3408032, 3670176 [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# mount /dev/md0 on / (ufs, local) devfs on /dev (devfs, local) /dev/md1 on /var (ufs, local, union) /dev/ad0a on /cf (ufs, local, read-only) [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0 80430 74612 5818 93% / devfs 1 1 0 100% /dev /dev/md1 31470 12 28942 0% /var /dev/ad0a 39406 34216 5190 87% /cf [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]#

9. Ok, go back to the webui and create the mount point again, then check the size either via the webui or via 'df' from your ssh session and you should be at full capacity.

[root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0 80430 74614 5816 93% / devfs 1 1 0 100% /dev /dev/md1 31470 12 28942 0% /var /dev/ad0a 39406 34216 5190 87% /cf /dev/mirror/RAID1p1 1904078 69300 1682452 4% /mnt/NAS1 [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]# [root@freenas /dev/mirror]#