Creating large (>2TB) Linux Partitions

Now that the era of 1 TB drives is upon us, we're rapidly closing in on a limitation imposed by the legacy x86 MBR which contains the OS loader and partition tables. Due to the 32-bit nature of the MBR, the maximum partition size is 2 TB; a limit that must have seemed impossibly large 20 years ago, when hard disks were measured in MB and only the largest hard drives were more than one hundred megabytes in size!

To circumvent this problem, a new, more modern system has been created to replace the MBR: GPT, or GUID Partition Table. GPT is an extension to the legacy MBR, keeping the MBR intact in the first sector of the hard drive. More importantly to us in this article, is that GPT uses 64 bits to address LBAs, extending partition size limits to about 16 exabytes!

Now, because GPT is still fairly unused, there are may be some caveats with getting programs like grub to properly install and recognize GPT partitions. The best way to avoid difficulties is to simply keep your OS separate from your data; that is, leave your OS partitions on a separate disk or array using a legacy MBR for booting, and write a GPT to the disk/array that is larger than 2TB.

On many server systems which provide BIOS level virtual disks, such as the Dell PowerEdge 2950 with it's PERC 5/i controller, this is cake - simply create a RAID5 array using (all) your disks, create a small, virtual disk for your OS (maybe 20-120 GB, depending upon your needs), and then create another virtual disk for the remaining data (if you're using 6x1TB disks, you should have something around 5 TB remaining).

Once you've installed the operating system and booted into it, Linux will happily see the two devices as /dev/sda and /dev/sdb - with your OS on /dev/sda and your unpartitioned /dev/sdb ready for GPT. From here, getting your system up and running is only a few commands away. Let's launch parted and take a look at the disk first:

[root@pdbfs4 ~]# parted /dev/sdb
GNU Parted 1.8.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p

Model: DELL PERC 5/i (scsi)
Disk /dev/sdb: 4945GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start  End  Size  Type  File system  Flags

Very empty. Note the 4945GB of available space (yum!). We'll need to use this number later when creating the partition. First however, we need to create a GPT label on the disk (as opposed to the legacy MBR, called msdos in parted):

(parted) mklabel gpt
Warning: The existing disk label on /dev/sdb will be destroyed and all data on
this disk will be lost. Do you want to continue?
Yes/No? Yes
New disk label type[msdos]? gpt
(parted) p

Model: DELL PERC 5/i (scsi)
Disk /dev/sdb: 4945GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags

Ah, much better. parted now recognizes it as a GPT partition table, but no partitions have been created yet so we'll need to go back in and create one with the mkpart command (or two, or three, or however you want to break it up...):

(parted) mkpart
Partition name?  []? DATA
File system type[ext2]? xfs
Start? 0
End? 4945GB
(parted) p

Model: DELL PERC 5/i (scsi)
Disk /dev/sdb: 4945GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name  Flags
 1      17.4kB  4945GB  4945GB               DATA

The Partition name is arbitrary, so call it whatever you want. Don't worry about what you put for the filesystem, as you can just mkfs over whatever is created after the program is complete; however if you know what you want before hand (and more importantly, if parted will let you) parted will mkfs for you. Now that we've created our partition, we can quit and should discover /dev/sdb? partitions (if you don't, try rebooting to force the computer to rescan the device's partition table).

You can now mkfs your filesystem of choice (I prefer XFS). Note that due to numbering, df will only see something like 4.5T free (don't panic!).

As you can see from this article, creating partitions larger than 2TB is quite easy using the mklabel and mkpart parted commands. If you have problems, make sure that GUID partition tables are supported by your kernel. Sometimes a reboot is required to force the kernel to rescan the partition table, so don't freak out if /dev/sdb? doesn't exist. As always, please post comments and questions below!

reply

I could create on 3TB volume following guide above. But when rebooting, the pratition tabled is wrong.
true religion

replay

Typically with Linux and large disks, I prefer just to use LVM directly on the disk, no partitioning at all. I find this to be very stable and manageable. business voip

Need HELP!!!

hey i have been trying to partition up a perc 5/i array in RAID 5 and create a virtual disk up to 8 TB, i can create the partition, but when i try to send information to it, i sometimes (sometimes) get input/output errors. but only with users that aren't root.

if i try and restart the server, the booting fails stating that the filesystem couldn't read the super blocks.

any help you could give would be great!

Can't use parted, gives device geometry error

I hv 2 HP MSA60s connected to both channels on a P800 Array controller, both have 12x750G drives configured as RAID5 with one drive as hot spare.

OS is RHEL AS4 Update4- 32 bit installed on a DL365 server.

When I try to run parted on either /dev/cciss/c0d0 and/or /dev/cciss/c0d1 it gives me this error.

"Unable to determine geometry of file/device /dev/cciss/c0d0. You should not use parted unless you REALLY know what you are doing"

and then gives me the options ignore/cancel

What am I doing wrong here. Any pointers will be highly appreciated

Thanks

Virtual Disks

It's a little tricky, and the PERC controller's BIOS will limit your options depending on how you've configured your array. I'll assume you've put all your disks into a RAID5 array from your description, so it should be fairly straightforward. I'm doing this from memory, so bear with me...

You can't carve up a disk you've already created, so if you have already created a virtual disk you'll need to delete it (sorry!). Now you'll be left with zero virtual disks, so create one. Select RAID5, and check all the drives in the list. Now on the top of the right side of the menu you should see an option that says something like "Size" - just fill in the value you want (something like 20480 will give you a 20 GB disk). Now select any advanced options and hit OK.

You should now see one virtual disk listed. You can create another one by scrolling down to the free space pool and repeating the process. This time, you wont be able to change the array configuration (you're stuck with RAID5) but you can change the disk size if you wish. The default is the remaining disk space, so if you're fine with that just hit OK again and reboot.

Let me know if you need additional help and I'll put up some screenshots this weekend.

-Vlad

Similiar configuration

I have the similiar configuration,

I have a DELL 2950 with 6 750GB hard disks. Just different than yours. I have the similiar RAID Controller as well.

My question is how did you partition the Virtual Disks ?

Quote:
simply create a RAID5 array using (all) your disks, create a small, virtual disk for your OS (maybe 20-120 GB, depending upon your needs)

In the line " create a small virtual disk" , how did you manage to do so?

I can only create a virtual disk the size of 3.5TB and it appears to be an entire volume. I wish to split this up.

You just put an end to days of woe

Thanks so much..
Sadly I had pulled all my hair out before I find this.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Copy the characters (respecting upper/lower case) from the image.