Tagged as amazon

s3fs and centos 6.2

Published on Friday, April 20th, 2012 in Linux.

Well today I was trying to install s3fs on a centos 6.2 server . So first of all I try this

yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap fuse fuse-devel
wget http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz
tar xvzf s3fs-1.61.tar.gz
cd s3fs-1.61/
./configure --prefix=/usr
make
make install

However the configure wasn’t working because of fuse version. I have installed 2.8.3 and I need 2.8.4

So the solution was to recompile the fuse also

  yum remove fuse fuse* fuse-devel
  yum install gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap
  cd /usr/local/src
  wget "https://downloads.sourceforge.net/project/fuse/fuse-2.X/2.8.4/fuse-2.8.4.tar.gz?r=&ts=1299709935&use_mirror=cdnetworks-us-1"
  tar -xzvf fuse-2.8.4.tar.gz
  rm fuse-2.8.4.tar.gz
  mv fuse-2.8.4 fuse
  cd fuse/
  ./configure --prefix=/usr
  make
  make install
  export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
  ldconfig
  modprobe fuse
  pkg-config --modversion fuse (confirm that 2.8.4 is the version displayed)
  cd ../
  cd s3fs-1.61/
 ./configure --prefix=/usr
  make
 make install

Automatically assign an Elastic IP at Launch time

Published on Thursday, February 23rd, 2012 in Linux.

Today I have face a problem . We want to have all time same ip on our instances because of some rules on a firewall. So how to add automatically elastic ip on a ec2 instance when start automaticaly by load balancer ?
Well this is not possible to be done automaticaly using amazon, so we have to wrote a bash script to be run on start up.

Here is the script:

#!/bin/bash
date
export EC2_HOME=/opt/aws/apitools/ec2-1.4.4.2
export JAVA_HOME=/usr/lib/jvm/jre
export CLASSPATH=${EC2_HOME}/lib
export EC2_PRIVATE_KEY=/root/pk.pem
export EC2_CERT=/root/cert.pem
export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin:/root/bin

ip=`/opt/aws/bin/ec2-describe-addresses -K /root/pk.pem -C /root/cert.pem |grep -v i-|head -1|awk {'print $2'}`
host=`hostname`
instance=`/opt/aws/bin/ec2-describe-instances -K /root/pk.pem -C /root/cert.pem  | grep $host |awk {'print $2'}`
runmore=`/opt/aws/bin/ec2-describe-addresses -K /root/pk.pem -C /root/cert.pem|grep $instance`

if [[ $runmore == '' ]];then
    if [[ $ip != '' ]];then
        /opt/aws/bin/ec2-associate-address -K /root/pk.pem -C /root/cert.pem -i $instance $ip
    fi
fi

Now all you have to do is to add this script on /etc/rc.d/rc.local to be run on startup . If your instance have a elastic ip asigned will not do anything, else will assing first free elastic ip.

EBS hangs in “attaching” mode

Published on Thursday, February 9th, 2012 in Linux.

Well with the script bellow automatic script for backup up redis on a amazon ebs
I saw that was a problem.
From time to time the ebs hang with “attaching”
And the mount command die in D state. And the load on that server grow.
Well, we have modify the script to wait for the volume to be actually “attached” .

This is a improvement version of the script.

#!/bin/bash
date
instance=i-xxxxx
volume=vol-xxxx
iplocalhost=10.00.00.111

export EC2_HOME=/path/ec2-1.4.4.2
export JAVA_HOME=/usr/lib/jvm/jre
export CLASSPATH=${EC2_HOME}/lib
export EC2_PRIVATE_KEY=/root/path/pk.pem
export EC2_CERT=/root/path/cert.pem
export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin:/root/bin

if [ ! -d /mnt/backup ]; then
mkdir /mnt/backup
fi

if [ ! -f /tmp/redis1 ]; then
echo "Cron Started"
touch /tmp/redis1
else
echo "Cron already running"
exit 0
fi
atached=1
while [ $atached -eq 1 ];do
echo "">/root/status.txt
/opt/aws/bin/ec2-describe-volumes -K /root/path/pk.pem -C /root/path/cert.pem > /root/status.txt
if grep -q "available" "/root/status.txt" ; then
echo "Volume is available."
echo "Attaching Volume"
/opt/aws/bin/ec2-attach-volume -K /root/path/pk.pem -C /root/path/cert.pem $volume -i $instance -d /dev/sdh
atached=0
else
echo "Volume is already attached on other instance. Sleeping for 60 seconds"
sleep 60
fi
done
avaiable=1
COUNTER=1
while [ $avaiable -eq 1 ];do
/opt/aws/bin/ec2-describe-volumes -K /root/path/pk.pem -C /root/path/cert.pem $volume > /root/status.txt
if grep -q "attached" "/root/status.txt" ; then
echo "Volume is attached."
avaiable=0
echo "Mounting Volume"
mount /dev/sdh /mnt/backup
sleep 10
if [ -f /mnt/backup/montat ]; then
rsync -vrplogDtH /var/lib/redis/ /mnt/backup/redis
fi
else
echo "Volume is attaching. Sleeping for 60 seconds"
sleep 60
fi
if [ $COUNTER -eq 5 ]; then
echo "Quiting ater 5 minutes"
break
fi
let COUNTER+=1
done
umount -lf /mnt/backup
sleep 5
/opt/aws/bin/ec2-detach-volume -f -K /root/path/pk.pem -C /root/path/cert.pem $volume -i $instance -d /dev/sdh
sleep 5
rm -rf /tmp/redis1
exit;

service php-fpm restart with error after upgrade from 5.3.8 to 5.3.10

Published on Tuesday, February 7th, 2012 in Linux.

Well
I have to upgrade a server from php 5.3.8 to 5.3.10 on a centos server.
After this the service php-fpm restart didn’t work.

Stopping php-fpm:                                          [FAILED]

So after a short look to the problem, what I discover is that problem is on this file

/etc/init.d/php-fpm

What I have was

pidfile=${PIDFILE-/var/run/php-fpm/php-fpm.pid}

and this line should be

pidfile=${PIDFILE-/var/run/php-fpm.pid}

autoscaling with amazon ec2 and elb

Published on Thursday, January 26th, 2012 in Linux.

Creating an auto scaled system using an Amazon load balancer is an interesting task that I did recently.

Here are the list of commands that I used to setup from the command line :

as-create-launch-config ec2elbconfig --image-id ami-xxxxx --instance-type m1.large --key key_name

Also if you don’t have already setup credentials you may append

-I amazonid -S secretkey

Do not forget to add –key because after the instance is up you won’t be able to log on it.

Next command is:

as-create-auto-scaling-group MyAutoScalingGroup --launch-configuration ec2elbconfig --availability-zones us-east-1c --min-size 2 --max-size 6 --load-balancers MyLoadBalancer

Where : MyLoadBalancer – is the name of your ELB ( loading balancer ) from amazon

Nex we create two policy rules, one for adding server and one for remover:

[root@server]# as-put-scaling-policy HighCpuPolicy --auto-scaling-group MyAutoScalingGroup --adjustment=1 --type ChangeInCapacity --cooldown 300
arn:aws:autoscaling:us-east-1:xxx:scalingPolicy:xxx-xx-xx-xx-xxxxx:autoScalingGroupName/MyAutoScalingGroup:policyName/HighCpuPolicy
[root@server]# as-put-scaling-policy LowCpuPolicy --auto-scaling-group MyAutoScalingGroup --adjustment=-1 --type ChangeInCapacity --cooldown 300
arn:aws:autoscaling:us-east-1:xxx:scalingPolicy:xxx-xx-xx-xx-xxxxx:autoScalingGroupName/MyAutoScalingGroup:policyName/LowCpuPolicy

Here is important to remember the output.
After this we must create two monitor rules that will scale our balancer, so will need two more rules:

mon-put-metric-alarm HighCpuAlarm --comparison-operator GreaterThanThreshold --evaluation-periods 4 --metric-name CPUUtilization --namespace "AWS/EC2" --period 60 --statistic Average --threshold 30 --alarm-actions arn:aws:autoscaling:us-east-1:xxx:scalingPolicy:xxx-xx-xx-xx-xxxxx:autoScalingGroupName/MyAutoScalingGroup:policyName/HighCpuPolicy --dimensions "AutoScalingGroupName=MyAutoScalingGroup"

and

mon-put-metric-alarm LowCpuAlarm --comparison-operator LessThanThreshold --evaluation-periods 4 --metric-name CPUUtilization --namespace "AWS/EC2" --period 60 --statistic Average --threshold 20 --alarm-actions arn:aws:autoscaling:us-east-1:xxx:scalingPolicy:xxx-xx-xx-xx-xxxxx:autoScalingGroupName/MyAutoScalingGroup:policyName/LowCpuPolicy --dimensions "AutoScalingGroupName=MyAutoScalingGroup"

After this you will have allways 2 instance up, and if the load go up then 30% more then 4 minutes the system will go up with one more server, if the load go bellow 20% more then 4 minutes will remove one server from load balancer.

If you need to remove this you must delete rules from bottom to top using bellow commands with name of your rules.

mon-delete-alarms
as-delete-policy
as-delete-auto-scaling-group
as-delete-launch-config
← Older Entries