How to , and other stuff about linux, photo, php … A linux, photography blog. To remember some linux situation, and fix them quickly.

March 19, 2012

Creating a Partition Size Larger Than 2TB, 3TB

Filed under: Linux — Tags: , , , , , , , , — admin @ 4:04 pm

Hello
Well recently I have to do a partition biger then 2TB, because I wanted to use all drive for backup.
Because fdisk won’t work, well I figure it out that I have to use parted
so here we go.
parted /dev/sdb
the output will be something like
GNU Parted 2.1
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted)
after tis we create a label
(parted) mklabel gpt
Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? Yes
After this we specify the unit that we want to use
(parted) unit TB
We tell parted to use all space
(parted) mkpart primary 0 -0
And the result
(parted) print
Model: ATA ST3000DM001-9YN1 (scsi)
Disk /dev/sdb: 3.00TB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number Start End Size File system Name Flags
1 0.00TB 3.00TB 3.00TB linux-swap(v1) primary

(parted) quit

Now we format the partition
mkfs.ext4 /dev/sdb1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=1 blocks, Stripe width=0 blocks
183148544 inodes, 732566272 blocks
36628313 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
22357 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

The result
mkdir /backup
mount /dev/sdb1 /backup/
df -h
/dev/sdb1 2.7T 201M 2.6T 1% /backup

January 13, 2012

RAID10 ephemeral storage on AWS EC2

Filed under: Linux — Tags: , , , , , , , , — admin @ 3:54 pm

If you’re thinking of doing Radi10  on the ephemeral storage disks attached to an Ec2 instance, this post is for you. Well first of all you have to chose a instance with 4 drives.

You may chose m1.xlarge or c1.xlarge  or cc2.8xlarge . This are only instances with 4 drives. On other instances you may chose to make Raid0, witch in some case is good also.

Well first of all you have to boot your instance, after this please check if you have mounted one of them ( in some cases only one is mounted as ephemeral0 )

You have to umount that drive.

After that fdisk on oll of them and make one drive with full size ( in this manner we will make a raid10 ) with all space.

To see your drivers just run

ls -1 /dev/sd*

and you will see something like :
/dev/sda1
/dev/sdb
/dev/sdc
/dev/sdd
/dev/sde

What I want to do is to make sdb, sdc, sdd, sde one raid10

I’ll just create a single partition on each one. Using fdisk, I choose the fd (Linux raid auto) partition type and create partitions using the entire disk on each one. When I’m done, each drive looks like this:

fdisk -l /dev/sdb

Disk /dev/sdb: 450.9 GB, 450934865920 bytes
255 heads, 63 sectors/track, 54823 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc696d4f6

Device Boot Start End Blocks Id System
/dev/sdb1 1 54823 440365716 fd Linux raid autodetect

Now I create the raid

mdadm  -v --create /dev/md0 --level=raid10 --raid-devices=4 /dev/xvdb1 /dev/xvdc1 /dev/xvdd1 /dev/xvde1

This is taking some time, so to verify if the construction of raid is ready run

 watch cat /proc/mdstat

When is ready you have to see something like this

Personalities : [raid10]
md127 : active raid10 xvdc1[1] xvdb1[0] xvdd1[2] xvde1[3]
880729088 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]

unused devices: <none>

Ok now we have to create a new partition. Just fdisk /dev/md0  create your partition.

fdisk /dev/md0 

mkfs -t ext4 /dev/md0p1

mkdir /mnt/raidd

mount  /dev/md0p1  /mnt/raidd

After you done this, reboot your server . After the server is up and running, on amazon it appear that they will be rename . So md0p1 will be someting lik md127

You may run

grep md /var/log/dmesg

and you will see something like this
[ 0.436792] md: bind<xvde1>
[ 0.444720] md: bind<xvdd1>
[ 0.526356] md: bind<xvdb1>
[ 0.543458] md: bind<xvdc1>
[ 0.547763] md: raid10 personality registered for level 10
[ 0.548234] md/raid10:md127: active with 4 out of 4 devices
[ 0.548311] md127: detected capacity change from 0 to 901866586112

After this you may add to fstab bellow line:

/dev/md127p1 /mnt/raidd auto defaults,comment=cloudconfig 0 2

Now if you run

mount /mnt/raidd 

you shuld have raid mounted

df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 9.9G 2.4G 7.0G 26% /
tmpfs 7.4G 0 7.4G 0% /dev/shm
/dev/md127p1 827G 6.7G 779G 1% /mnt/raidd

 

Powered by WordPress