Free Souls (Don't LiVe to GeEk; GeEk to LiVe)

MCollective using a vhost other than / with RabbitMQ

2011-12-23T12:02:00.000+05:30

We'd recently began using MCollective along with Puppet in our setup. MCollective is a framework to build server orchestration or parallel job execution systems. We had MCollective running with ActiveMQ as MOM for management of our EC2 infrastructure, however for our setup at Softlayer we decided to go ahead and make use of RabbitMQ as our MOM.

Thanks to R.I. Pienaar we were quickly able to get solution to some of our issues and very recently one of our developers suggested that we should be making use of a separate vhost for use of mcollective. Seems like a great idea, we all thought ! However after discussions on mcollective user group we found that stomp connector provided with mcollective, by default doesn't allow for passing of host header information to RabbitMQ.

R.I. Pienaar once again came as a saviour and suggested some changes to our stomp connector which now allow us to make use of /mcollective as a vhost while communicating with RabbitMQ.

So here is a change we made to /usr/libexec/mcollective/mcollective/connector/stomp.rb (This location can change with distribution).

@connection = connector.new(user, password, host, port, true, 5, {"accept-version" => '1.0,1.1', "host" => "/mcollective\"})

User-Agent based redirect in haproxy

2011-03-18T22:33:00.000+05:30

We've been using haproxy as our loadbalancer for quite sometime now and I'd say it's been more than wonderful. We've been using it for serving around 2 million page-views per day and it's been pretty good at handling that kind of HTTP traffic. Recently we decided to launch our mobile site and the idea was to allow users to visit our website through normal URL that we use for our website, however upon detection of user-agent we wanted to rewrite the URL to something else. We knew that haproxy was able to parse and identify HTTP Headers and hence we were pretty positive about the idea of using haproxy to do the redirection, however to our surprise, haproxy 1.3 doesn't allow for conditional rewrites and the worst aspect of it is that it doesn't even complain about implementing conditional rewrites, instead it silently ignores the condition and implements the rewrite to every request.

So we finally decided to migrate to haproxy 1.4 and we use following configuration to setup conditional redirect:

1) Create new ACL to identifiy user-agent:

acl mobile_user_agent hdr_sub(User-Agent) -i iphone

2) We didn't want to redirect any requests for .css, .js, .bmp, .jpg, .png, .jpeg, .gif and .ico so created a new ACL

acl is_static_file url_reg .*\.(css|js)\?[0-9.]+

acl is_static_img url_reg .*\.(png|bmp|jpg|jpeg|gif|ico)\?[0-9a-z.]+

acl is_static url_reg .*\.(css|js|png|bmp|jpg|jpeg|gif|ico)

3) If request was for /mobile/* we didn't want to redirect so yet another ACL :)

acl is_mobile url_reg ^\/mobile.*

4) Do a URL rewrite as follows:

reqrep ^([^\ ]*)\ /(.*)   \1\ /mobile/\2 if iphone !is_static_file !is_static_img !is_static !is_mobile

So now any request coming from a mobile device for any thing other than .css, .js, .png, .gif, .bmp, .jpg, .jpeg, ico will be redirected to /mobile/.

So a request like www.abc.com/pqr will become www.abc.com/mobile/pqr !

Access mongodb collection through shell script

2011-03-14T19:38:00.000+05:30

We've been using mongodb for quite some time for various purpose, like newsfeed, analytics etc. Recently a need was found to setup a monitoring for some queries to mongodb and unlike MySQL I was not able to find any means to query the database collection. However a quick look at help of mongo yielded a nice solution. We can put series of statements that we execute at mongo shell inside a .js file and finally execute :

mongo .js

Regex handling is indeed black magic

2011-03-13T17:31:00.001+05:30

Mathematicians Vs. Computer Scientists

2011-03-13T07:48:00.002+05:30

I always had this confusion in my mind about what Computer Scientists do. We've mathematicians who are usually associated with design of algorithms and than we have electronics wizards who are designing computers of today and tommorrow. This thinking always confused about relevance of Computer Science as a subject. However while reading an article about Parsing, I came across this note by Authors (Dick Grune and Ceriel J.H. Jacobs).

"There is a considerable difference between a mathematician's view of world and a computer scientist's. To a mathematician all structures are static: they have always been and always be; the only time dependence is that we just haven't discovered all of them yet. The computer scientist is concerned with (and fascinated by) the continuous creation, combination, separation and destruction of structures: time is of essence. In the hands of a mathematician, the Peano axioms create the integers without reference to time, but if a computer scientist uses them to implement integer addition, he finds that they describe a slow process, which is why he'll be looking for a efficient approach. In this respect the computer scientist has more in common with physicist and chemist; like these, he cannot do without a solid basis in several branches of applied mathematics, but, like these, he is willing (and often virtually obliged) to take on faith certain theorems handed over to him by mathematicians. Without rigor of mathematics all science would collapse, but not all inhabitants of a building need to know all the spars and griders that keep it upright. Factoring off certain knowledge to specialists reduces intellectual complexity of a task, which is one of the things computer science is about."

Rules about Roman Numerals

2011-03-03T01:33:00.002+05:30

While reading about regular expressions, found few interesting things about Roman Numerals today.

In Roman Numerals there are 7 characters which are repeated and combined in various different ways to represent numbers.

I = 1
V = 5
X = 10
L = 50
C = 100
D = 500
M = 1000

Following are general rules for constructing Roman numerals:

Characters are additive. I is 1, II is 2 and III is 3. VI is 6, VII is 7 and VIII is 8.
Tens characters (I, X, C and M) can be repeated upto 3 times.
At 9, we need to subtract from the next highest tens character.
The five characters can not be repeated.
Roman numerals are always written highest to lowest, and read left to right, so the order of characters matters very much.

Puppetizing Amazon EC2

2011-02-22T10:27:00.002+05:30

For those who aren't there yet ?

Amazon EC2 is a cloud computing service from Amazon. Using various mechanisms like API, Web Console, Command Line tools, one can reap benefits of IAAS facility provided by Amazon.

Puppet is systems configuration management software developed by Puppet Labs (Formally Reductive Labs). Puppet provides a framework to simplify the majority of technical tasks that System Administrators perform. It provides a declarative language, which can be used to express system configuration.

Why bother about Puppet? Didn't we have ghost do same job for us in real server scenario. And since I learnt about EC2, so there is concept of AMIs as well !

Yeah, AMIs are great way to package a group of software and configuration files on top of an Operating System, but that's all that's there about them. With development in today's circumstances going Agile, we see a lot of changes in configurations and software installed on a system. Re creation of AMIs at regular intervals is definitely not the coolest thing one would like to do. Imagine running 10 webservers on EC2, which were spawned using a private custom AMI with httpd package pre-installed and configuration built into the AMI. During the course of day, developers made changes to the code base and also added some features, which required 10 changes to the configuration of httpd. This situation is nothing less than a nightmare for system administrators when deployments go out in Agile fashion. Our imaginary circumstance would require 10 changes to AMIs and we'd be required to propagate these changes to either 10 existing servers, 10 times or re spawn 10 servers every time a deployment went out to production.

Puppet comes to our rescue !

With puppet these changes are done using it's declarative language in a file called recipe. So in our setup, we make required changes to the recipe and push the code change to puppetmaster server and puppet client on the webservers connect to puppetmaster after a specified interval (default=30 minutes) and pull the new configuration. Puppet allows designing a setup where in machines are classified under various roles and we can write down configuration for these roles using the declarative language of Puppet.

Cloud and Puppet

EC2 instances can be spawned using command line tools (provided by ec2-api-tools) and we've created a set of shell scripts wrappers around these commands to start of new instances on EC2 and assign them some role through which the instances will get configured using appropriate puppet recipe.

How did we do it ?

Slideshare has been using EC2 for building up it’s conversion stack and as most sysadmins will agree, performance of machines tend to degrade after they’ve been operational for a certain amount of time. We’ve tried to kill this problem by designing conversion stack where new machines on EC2 are spawned at regular interval and old instances are killed.

We've setup a puppet server on one EC2 instance and used Apache proxy to proxy through multiple instances of puppetmasterd running on different ports. We've also reconfigured init script for puppetmasterd to allow for stop and start of puppetmasterd service on different ports using well known /etc/init.d/ start/stop syntax.

Puppet client by default tries to connect to a host named puppet on port 8140 and hence we've setup firewall settings for this host to allow connection to be established on 8140 port and setup apache to listen on 8140 port. Apache than proxies the request to puppetmasterd server running on different ports in backend. This allows for easy scaling of the setup.

Puppet server is configured to automatically self sign the certificate request by the client and this allows new clients to connect directly to puppetmaster and get required configuration.

We’ve also created a separate set of private AMIs which we use to fire up adhoc instances in times of high load on our regular instances. These instances are meant to be running for a small time duration and have all the software and configuration baked into the AMI itself.

We’ve created our own home grown scripts which spawn instances on EC2 at regular intervals and code on these machines take care of killing the machine after a fixed time.

Following is excerpt of the script which we use to spawn different instances. Role of the instance is passed on as argument to the script:

#!/bin/bash

...

#Place request for spot instance

ec2-request-spot-instances $AMI -n $NUM_INSTANCES -p $PRICE -t $TYPE -k $KEY_PAIR --group $SECURITY_GROUP > /tmp/ec2_spot_instance_request

SIR_REQUEST=`cat /tmp/ec2_spot_instance_request | cut -f 2`

rm -f /tmp/ec2_spot_instance_request

#Capture status of request. Initially request has STATUS=open and we need it to be active in order to continue

STATUS=`ec2-describe-spot-instance-requests | grep $SIR_REQUEST | cut -f 6`

#We won't want to wait till infinity for instance to spawn up. Our threshold is 5 minutes.

COUNT=1

#This variable checks if instance is spot or regular

IS_SPOT=1

#Wait for spot instance request to succeed.

REQUIRED_STATUS="active"

while [ $STATUS != $REQUIRED_STATUS ]

sleep 60

STATUS=`ec2-describe-spot-instance-requests | grep $SIR_REQUEST | cut -f 6`

if [ $COUNT -gt 5 ]

then

IS_SPOT=0

break

COUNT=`expr $COUNT + 1`

done

if [ $IS_SPOT -eq 1 ]

then

INSTANCE_ID=`ec2-describe-spot-instance-requests | grep $SIR_REQUEST | cut -f 12`

else

#Kill spot instance request we made earlier

ec2-cancel-spot-instance-requests $SIR_REQUEST

#Spawn up a regular instance

ec2-run-instances $AMI -n $NUM_INSTANCES -t $TYPE -k $KEY_PAIR --group $SECURITY_GROUP > /tmp/ec2_instance_request

INSTANCE_ID=`cat /tmp/ec2_instance_request | tail -1 | cut -f2`

STATUS=`cat /tmp/ec2_instance_request | tail -1 | cut -f6`

rm -f /tmp/ec2_instance_request

REQUIRED_STATUS="running"

while [ $STATUS != $REQUIRED_STATUS ]

sleep 60

STATUS=`ec2-describe-instances $INSTANCE_ID | tail -1 | cut -f6`

done

sleep 120

#Instance is now active. Capture data associated with instance like instance-id, external and internal dns.

INSTANCE_EXTERNAL_DNS=`ec2-describe-instances $INSTANCE_ID | tail -1 | cut -f 4`

INSTANCE_INTERNAL_HOSTNAME=`ec2-describe-instances $INSTANCE_ID | tail -1 | cut -f 5 | cut -f 1 -d'.' `

#We need to do the record keeping

echo "`date`: $INSTANCE_ID: $INSTANCE_EXTERNAL_DNS: $INSTANCE_INTERNAL_HOSTNAME: PDF2SWF" >> $DB_FILE

#If we are not able to get internal hostname within next 1 minutes for some reason then quit

while [ -z $INSTANCE_INTERNAL_HOSTNAME ]

COUNT=`expr $COUNT + 1 `

sleep 10

INSTANCE_INTERNAL_HOSTNAME=`ec2-describe-instances $INSTANCE_ID | tail -1 | cut -f 5 | cut -f 1 -d'.' `

INSTANCE_EXTERNAL_DNS=`ec2-describe-instances $INSTANCE_ID | tail -1 | cut -f 4`

if [ $COUNT -ge 6 ]

then

exit 1

done

#Configure puppetmaster to associate relevant class with node

$SCP root@$PUPPET_MASTER:/etc/puppet/manifests/nodes.pp /tmp/nodes.pp

grep $INSTANCE_INTERNAL_HOSTNAME /tmp/nodes.pp

if [ $? -eq 0 ]

then

sed "/$INSTANCE_INTERNAL_HOSTNAME/d" /tmp/nodes.pp > /tmp/newnodes.pp

mv /tmp/newnodes.pp /tmp/nodes.pp

echo "node $INSTANCE_INTERNAL_HOSTNAME { include $1 }" >> /tmp/nodes.pp

$SCP /tmp/nodes.pp root@$PUPPET_MASTER:/tmp

$SSH root@$PUPPET_MASTER "mv /tmp/nodes.pp /etc/puppet/manifests/nodes.pp"

rm -f /tmp/nodes.pp

$SSH root@$INSTANCE_EXTERNAL_DNS "cat /tmp/puppet_host >> /etc/hosts"

...

#Install puppet on newly built ec2 host

$SSH root@$INSTANCE_EXTERNAL_DNS "apt-get -y install puppet"

This script tries to spawn spot instances on EC2 (for more details about spot instances, refer to this wonderful article by Jonathan Boutelle, CTO, Slideshare). Since availability of spot instances is not certain and we wanted our script to spawn up a instance in stipulated time, so we decided to set a time limit of 5 minutes, to wait for ascertaining availability of spot instance. If we don't get a spot instance in 5 minutes, we spawn up a regular instance instead.

Once an instance has been spawned we do some record keeping tasks and add an entry for puppet server in /etc/hosts file of the node. Puppet client by default tries to connect to a host named puppet and hence our entry in /etc/hosts looks like follows:

XXX.XXX.XXX.XXX puppet

Finally we make an entry for this newly spawned node in nodes.pp on puppet master. This is a tricky part as DNS on EC2 reassigns previously assigned hostname to a new node and this can cause duplicate entries in nodes.pp also many a times it so happens that we are able to spawn off an instance however the script is unable to get hostname using ec2-api-tools and this can cause an invalid entry to appear in nodes.pp. We've tried to circumvent these errors by implementing few error checks.

There are several improvements that can be made to existing setup. We are relying on DNS service of EC2 and this causes lots of issues while identifying instances in cases of troubleshooting errors. This can be resolved by setting up a home grown DNS server on EC2.

We’ve tried to setup a nagios monitoring system on EC2 as well to monitor our services running on cloud. This can be very useful as most of the monitoring facilities provided by Amazon are pretty basic and are more oriented towards system health checks rather than service health checks.

Python source code editing using vim

2011-01-23T17:55:00.004+05:30

Recently found myself handling a bit of python code and not being able to use a GUI environment, it wasn't very easy to code using standard vim editor. Googled a bit and found few nice tips for enabling auto indent, syntax highlighting, and best of the best autocompletion of module details.

For autocomplete feature you'll need to get pydiction installed. The .vim files and supporting files are available at http://www.vim.org/scripts/script.php?script_id=850.

Finally my ~/.vimrc file looks like following:

filetype plugin on
set nocompatible syntax on
set background=light
set showmatch
set ignorecase
set showmode
set ts=4
set sw=4
set autoindent
set smartindent

let g:pydiction_location = '/usr/local/share/pydiction/complete-dict'

Monitor Amazon SQS Queue length

2010-08-03T22:58:00.002+05:30

We make quite a bit of use of Amazon SQS for some of critical applications at our setup and as per our monitoring requirements I wanted to make sure that we get SQS queue lengths being monitored through Nagios. I had earlier made use of boto library during one of our unsuccessful attempts at using Hadoop for some analytics project. However this time around I was able to make good use of boto libraries to get the count of messages in a SQS queue with just few lines of codes. Following is code snippet used to get count of number of messages in a SQS queue:

import sys
from boto.sqs.connection import SQSConnection
conn=SQSConnection('AWS_ACCESS_KEY_ID','AWS_SECRET_KEY')
q=conn.get_queue(sys.argv[1])
attrib = q.get_attributes("ApproximateNumberOfMessages")
print int(attrib['ApproximateNumberOfMessages'])

In order to use the above code snippet, just follow following steps:

1. Install boto. To quickly setup boto, get latest boto source code from http://code.google.com/p/boto/. Extact the compressed file and do a python setup.py install as root.
2. Copy paste the above code to a temporary file say /tmp/test.py. Replace 'AWS_ACCESS_KEY_ID' and 'AWS_SECRET_KEY' with relevant details.
3. Execute the code snippet using : python test.py [SQS_QUEUE_NAME]

Transfer instance from one EC2 account to other

2010-06-21T00:23:00.003+05:30

Recently we decided to migrate our instances to a newly created EC2 account. I was assigned the task of migration and initially my impression of work was that it would only involve fiddling with aws credentials and I'd be done. However I couldn't have been any wrong. Migration of instances from one EC2 account to new one was a job in itself for me. Following are the steps which I had to follow to migrate the instances:

1. Create AWS key pair for new account.
2. Create X.509 certificate for new account.

There are separate procedures which I too to migrate Linux and Windows instances.

Linux instance migration:

1. Copy X.509 certificate to some directory on instance which we want to migrate. I copied private key and certificate to /opt/ec2/keys directory.

2. Login to instance.

3. Download and extract ec2-ami-tools to /opt/ec2.

4. Set a variable EC2_HOME: export EC2_HOME=/opt/ec2

5. Set PATH: export PATH=$PATH:$EC2_HOME/bin

6. Bundle the instance:

ec2-bundle-vol -d /mnt -k /opt/ec2/keys/pk-XXX.pem -c /opt/ec2/keys/cert-XXX.pem -u XXXXXXXXX -e /opt/ec2 -s 10000

For details of using ec2-bundle-vol, use --help

7. Upload bundle: before uploading bundle I create a S3 bucket called ami_from_other_account on my newly created account.

ec2-upload-bundle -b ami_from_other_account -m /mnt/image.manifest.xml -a XXXXX -s XXXXXX

8. Register the AMI: in order to register the AMI we need ec2-api-tools and they need to be setup properly. One needs JRE and JAVA_HOME variable appropriately set. I had setup ec2-api-tools on my machine. I had also copied private key and certificate files to /opt/ec2/keys on my machine
The private key and Certificate are one's for newly created account.

ec2-register ami_from_other_account/image.manifest.xml -K /opt/ec2/keys/pk-XXX.pem -C /opt/ec2/keys/cert-XXX.pem -n AMI-Blah-Blah-Blah.

Windows instance migration:

1. Download and extract ec2-ami-tools and ec2-api-tools to /opt/ec2 directory.

2. Download private key and certificate file to /opt/ec2/keys
3. Share AMI of existing Windows instance with other user.

4. Run the instance thru new user account.

5. I created a S3 bucket called win_fac to hold my new AMI files

6. Bundle running instance

ec2-bundle-instance i-XXXX -b win_fac -o AWS_ACCESS_KEY -w AWS_SECRET_KEY

7. Register new AMI

ec2-register win_fac/image.manifest.xml -K /opt/ec2/keys/pk-XXX.pem -C /opt/ec2/keys/cert-XXX.pem -n WIN_AMI.

Penalty of living a digital life

2010-04-25T20:29:00.002+05:30

Got my gmail accnt compromised yesterday. WTF ! even Mac isn't safe .... The account remained in compromised state for 8 minutes and damage was done in that period. Everyone in my contact list was sent some sort of SPAM message. Worst was that my boss in one of my earlier company received a mail from me asking him "Would you love to try out Viagra !". Though I sent an apology but still it made me realize what kind of world am I living in and yeah I'm 10 x more cautious now.
Even sent a mail to Mr. Jobs about this :)

Regular Expression builder.

2010-04-16T14:58:00.001+05:30

Found a real nice tool in form of http://rubular.com/ It served as a life saver while I was struggling with regular expression.

Configuring puppet with mongrel and apache load balancer

2010-04-13T12:26:00.001+05:30

We've been using EC2 quite a bit now for running some services which needs machines to be spawned up on daily basis. This kind of setup is a real pain if it was all to be handled manually. Our backend engineers were smart to design an automated setup using custom AMI which can configure themself using shell scripts. However this meant that upgrading to new versions would be a PITA. Idea of using puppet seem really useful in such situations and I decided to go for installation of puppetmaster on EC2.

Following are the steps I followed to configure puppetmaster with mongrel running behind apache load balancer (I was using plain vanilla CentOS 5.4 AMI):

Install puppet-server by following steps mentioned in http://ifireball.wordpress.com/docs/howto-install-puppet-on-centos-50/

Edit /etc/sysconfig/puppet and add following statement to end of file:
PUPPETMASTER_PORTS=( 18140 18141 18142 18143 )

Install mongrels : yum install rubygem-mongrel

Install apache and mod_ssl : yum install httpd && yum install mod_ssl

Create a file /etc/httpd/conf.d/puppet.conf with following content:

Listen 8140

<proxy>
BalancerMember http://127.0.0.1:18140
BalancerMember http://127.0.0.1:18141
BalancerMember http://127.0.0.1:18142
BalancerMember http://127.0.0.1:18143
</proxy>

<virtualhost>
SSLEngine On
SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA
SSLCertificateFile /var/lib/puppet/ssl/certs/puppet.compute-1.internal.pem
SSLCertificateKeyFile /var/lib/puppet/ssl/private_keys/puppet.compute-1.internal.pem
SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
SSLCACertificateFile /var/lib/puppet/ssl/ca/ca_crt.pem
SSLVerifyClient optional
SSLVerifyDepth 1
SSLOptions +StdEnvVars

RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

<location>
SetHandler balancer-manager
Order allow,deny
Allow from all
</location>

ProxyPass / balancer://puppetmaster/
ProxyPassReverse / balancer://puppetmaster/
ProxyPreserveHost On

ErrorLog /var/log/httpd/balancer_error_log
CustomLog /var/log/httpd/balancer_access_log combined
CustomLog /var/log/httpd/balancer_ssl_requests "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</virtualhost>

Edit /etc/init.d/puppetmaster :


#!/bin/bash
# puppetmaster  This shell script enables the puppetmaster server.
#
# Author:       Duane Griffin 
#
# chkconfig: - 65 45
#
# description: Server for the puppet system management tool.
# processname: puppetmaster

PATH=/usr/bin:/sbin:/bin:/usr/sbin
export PATH

lockfile=/var/lock/subsys/puppetmaster

# Source function library.
. /etc/rc.d/init.d/functions

if [ -f /etc/sysconfig/puppetmaster ]; then
 . /etc/sysconfig/puppetmaster
fi

PUPPETMASTER_OPTS=""
[ -n "$PUPPETMASTER_MANIFEST" ] && PUPPETMASTER_OPTS="--manifest=${PUPPETMASTER_MANIFEST}"
if [ -n "$PUPPETMASTER_PORTS" ]; then
 PUPPETMASTER_OPTS="$PUPPETMASTER_OPTS --servertype=mongrel"
elif [ -n "$PUPPETMASTER_PORTS" ]; then
    PUPPETMASTER_OPTS="${PUPPETMASTER_OPTS} --masterport=${PUPPETMASTER_PORTS[0]}"
fi

[ -n "$PUPPETMASTER_LOG" ] && PUPPETMASTER_OPTS="${PUPPETMASTER_OPTS} --logdest=${PUPPETMASTER_LOG}"
PUPPETMASTER_OPTS="${PUPPETMASTER_OPTS} \
 ${PUPPETMASTER_EXTRA_OPTS}"

RETVAL=0

prog=puppetmasterd
PUPPETMASTER=/usr/sbin/$prog

start() {
 echo -n $"Starting puppetmaster: "

 # Confirm the manifest exists
 if [ -r $PUPPETMASTER_MANIFEST ]; then
      if [ -n "$PUPPETMASTER_PORTS" ]; then
          for ((i=0; i<4; masterport="${PUPPETMASTER_PORTS[$i]}" pidfile="/var/run/puppet/puppetmaster.${PUPPETMASTER_PORTS[$i]}.pid" ret="$?;" retval="$ret" retval="$?" i="0;" ret="$?;" retval="$ret" retval="$?" i="0;" ret="$?;" retval="$ret" retval="$?">

Create AMI from existing EC2 instance.

2010-04-06T19:08:00.003+05:30

There are times when we've configured an EC2 instance as per our requirements and we find need to create an AMI from the very instance. Recently I found this as a requirement while automating hadoop setup on EC2. We are using clouder-hadoop AMI for booting up hadoop cluster. However these machines need some additional softwares and we wanted to manage the setup using puppet.

So our requirement was to have an AMI with puppet installed and /etc/hosts containing entry corresponding to puppet master. We also wanted puppet to startup during bootup so that every software needed for our purpose was installed before machine was put to production use.

We followed following steps to get an AMI for our purpose:

1. Launch an EC2 instance using cloudera-hadoop AMI. Install puppet and configure /etc/hosts as per our requirement. We'll presume our instance id to be i-6666

2. Log off from the EC2 instance we just created.

3. On our machine we download ec2-ami-tools.zip and ec2-api-tools.zip

4. Create a directory ~/ec2

5. Extract ec2-ami-tools.zip and ec2-api-tools.zip to ~/ec2.

6. Copy EC2's PEM encoded X.509 certificate named something like cert-xxxxxxx.pem and PEM encoded RSA private key named something like pk-xxxxxxx.pem to ~/ec2. We can get these by logging to Amazon Web Services account at http://aws.amazon.com.

7. Setup following environment variables (I'm on Mac OS X, you might need to modify your bash_profile accordingly)

# Setup Amazon EC2 Command-Line Tools
export EC2_HOME=~/.ec2
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=`ls $EC2_HOME/pk-*.pem`
export EC2_CERT=`ls $EC2_HOME/cert-*.pem`
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/

8. Source the bash_profile. source ~/.bash_profile

9. Execute the command ec-bundle-instance. This command helps create an AMI from existing EC2 instance, bundle it and deploy it on S3. Following is an e.g. of the command in action:

ec2-bundle-instance i-6666 -b mybucket -p hadoopami -o -w

mybucket is name of S3 bucket and aws_access_key and aws_secret_key are Access Key ID and AWS Secret Access Key for the owner of the Amazon S3 bucket.

10. The above process takes some time and once done the EC2 instance will reboot. We can track the progress using

ec2-describe-bundle-tasks

11. Once image has been uploaded to S3 we need to register the image. We do so using ec2-register. Following is an e.g. of command in action:

ec2-register mybucket/hadoop-ami

Email Spam Facts (By Rackspace Hosting)

2010-03-29T22:38:00.002+05:30

Hosted Exchange

Green Buildings the future of homes and workplaces

2010-02-25T15:47:00.001+05:30

You might have recently heard about grren buildings or sustainable buildings. I thought let me just put some light on the same. According to Wikipedia Green or sustainable building, is the practice of creating structures and using processes that are environmentally responsible and resource-efficient throughout a building's life-cycle: from siting to design, construction, operation, maintenance, renovation, and deconstruction. This practice expands and complements the classical building design concerns of economy, utility, durability, and comfort.

New technology is constantly being developed for the same though the main goals of green buildings are reduce the overall impact of the built environment on human health and the natural environment by

efficiently use of energy, water, and other resources
Protecting occupant health and improving employee productivity
Reducing waste, pollution and saving enviroment

How ever the green buildings are subject to certain standards in every country. In India they are called Griha and have been drawn up by TERI and LEED together forming Indian Green building council.

Some of the certified green buildings in india are Mahindra Splendour in Mumbai, ITC Green Centre in Gurgaon, CII-Sohrabji Godrej Green Business Centre in Hyderabad, Olympia Tech Park in Chennai, Technopolis in Kolkata.

Recently the Suzlon One Earth Global Headquarters at Hadapsar, designed by Christopher Benninger, was chosen as the best corporate building by the Architects, Engineers and Surveyors’ Association (AESA). The building was termed as one of the greenest and most energy efficient building world over.

With developments like this India surely will emerge as the one of coolest destinations for green buildings world over.

Mantra for Startups !

2010-02-20T16:55:00.002+05:30

Recently found this wonderful article by Paul Graham about startups (http://www.paulgraham.com/start.html).

Definitely learned a thing or two from it and hope it will guide me to a better decision making process in years to come :)

If you are using SQUID then MySAR is a must have

2010-02-07T20:43:00.002+05:30

Since a long time I've been thinking of using a tool which can help me analyze usage of my proxy server i.e. Squid. I've a network of around 500 hosts and the entire traffic comes thru my proxy and I was just inquisitive to know what my users been doing :)

Earlier I was using webalizer to generate analysis of the Squid log, however the problem with webalizer is that it's able to generate an efficient analysis after a day. So the latest record that I can see is of yesterday. However my requirement was to view the present log and this is where I stumbled upon mysar (MySQL SQUID Access Report). This tool parses the access_log file generated by Squid and stores the content of the file in a mysql database. It allows for log rotation without worrying about loosing older data. I've set the record life for 60 days and hence I can view the reports for last 60 days, though my log rotation happens every week.

The web interface of mysar is pretty simple and powerful. One can customize the display to his/her preference. I like to view the report with most byte consuming host first :)

Since it's usage I've been able to identify patterns of web usage in my setup and have been able to formulate a more efficient policy which has allowed me to manage my setup more efficiently.

Climate Change: think more about it

2009-12-08T14:46:00.001+05:30

I am sure by now all of you must have heard about Cop15. If you have not, then here is a brief backgrounder courtesy Wikipedia. “The United Nations Framework Convention on Climate Change (UNFCCC or FCCC) is an international environmental treaty produced at the United Nations Conference on Environment and Development (UNCED), informally known as the Earth Summit, held in Rio de Janeiro from 3 to 14 June 1992. The objective of the treaty is to stabilize greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic interference with the climate system.”

The Rio summit has been followed up by several similar summits known as COP (Conferences of the Parties) as the treaty itself sets no mandatory limits on greenhouse gas emissions for individual countries and contains no enforcement mechanisms. These summits decided updates (called "protocols") that would set mandatory emission limits. The principal update is the Kyoto Protocol on Climate Change (1997), or COP 3 which has become much better known than the UNFCCC itself.

COP 15 is currently taking place in Copenhagen, Denmark from 7 December to 18 December 2009.

The overall goal for the COP 15/MOP 5 United Nations Climate Change Conference in Denmark was to establish an ambitious global climate agreement for the period from 2012 when the first commitment period under the Kyoto Protocol expires.

Now I know all of us people interested in technology do know about climate change. We have seen it around us. The weather is getting unpredictable. Himalayas, specifically Uttaranchal where I have spent my teenage is seeing Glaciers shrink at an alarming rate. I think we all should think , analyze and then share our views about this serious matter. If we talk about India in particular, we have some good thoughts about it. Like we want to have 15 per cent of all our energy consumption from renewable energy by 2020. But it is a hard target and we need to consider all our alternatives to meet this target. For a more detailed discussion on the same , check this link Green Global.

What's with shooting a photograph ?

2009-12-07T11:52:00.002+05:30

Ever since I was a kid, I'd an interest in photography. Earlier we'd a B&W camera from which we upgraded ourself to a Color camera and now to Panasonic FZ28. However I always used to think what makes a good photograph ? Is it the quality of camera which is solely responsible for it or does it also depend upon the knowledge of concerned person who is taking the photograph ? Recently got a chance to read online lessons on photography by Jodie Coston and life has never been so colorful for me ever since. It's a wonderful series of lectures which allow beginners to learn nitty gritty details like shutter speeds, exposure, aperture, lens, focus etc.

Presentations on Ted

2009-09-05T17:06:00.003+05:30

Watched a wonderful presentations by Ken Robinson and Elizabeth Gilbert on Ted today. Both are must watch for every human being.

Broadband over Power Lines

2009-09-05T17:01:00.002+05:30

I was just wondering if we can manage data transmission over a power cable and landed up at a page from CarbonMetrics. If what is mentioned in the page can be implemented then time is not far away when we can see every corner of this planet connected by high speed communication network.

What's with infinity ?

2009-09-01T18:32:00.002+05:30

As has always been the case I got stuck up at problem of defining countable and uncountable sets. This time though I had wikipedia and other resources on Internet to help me out, yet I was not able to comprehend the Cantor's diagonalization argument clearly, until I read an article Infinity: You can't get there from here at www.mathacademy.com. One of the few good articles for layman to understand about the concepts of infinity.

Cardinality of Power Set

2009-09-01T15:19:00.002+05:30

Took some time off to give lectures on Logic and Proof Theory to students enrolled at FPGDST course at C-DAC, Mumbai. It was a wonderful experience and I must say I learnt a lot through this experience. During a session on Set Theory we were discussing PowerSets. Now we all know that a Power Set is a set of all subsets of a set. Or if stated in language of sets:

Given a set S = {x1,x2,...,xn}, the power set P(S) is set of all subsets of S.

Now just after the definition of Power Set was given I found my slide to contain a seemingly easy statement about cardinality of a power set. And it said that if S has n elements then P(S) will have 2^n elements. That's pretty obvious isn't it ? But I don't know why, I decided to proove the statement and boy it wasn't that easy a job and it has been a week and I am still struggling to find a decent proof for the same. Will revert back with a proof as soon as I get one which satisfies me. Till then have fun with mathematics :)

Saving money for a playstation 3

2009-08-10T15:21:00.002+05:30

My friend Ruchir is pursuing his masters in hotel management from AIHS, Canberra . He seems to be calling me quite often from his cell phone unlike several of my other friends abroad who just chat with me on net using gtalk or yahoo. When I asked him his secret, he told me that his parents had bought him a matrix Pre Activated mobile connection when he went to Australia last year for his masters. And recently he just renewed his subscription for a second year under a plan called cap 29 plan which gives him a talk time value of around 180 Australian $ on payment of just 29$ which also is incidentally paid by his parents in India. This allows him to make international calls of around 5 minutes everyday for free. And I thought he must be spending a bomb making all these calls. So guys think smart and find money saving deals like this and spend your cash on other more essential necessities like the latest PS 3.

Firefox Memory Management

2009-07-29T14:18:00.002+05:30

As a user and as a systems administrator, I've seen many people fuming over the fact that their Firefox installation seems to be running very slow and is even making machines go for a toss. A wonderful article on how to reduce memory usage in Firefox can be found at Mozillazine's Knowledge Base.

Some facts about RHEL License.

2009-07-20T23:58:00.004+05:30

Here is what I learnt from one of my friend about RHEL License and this shows why we can't download iso images and copy the media to distribute RHEL.

" Except for a few components provided by third parties (for example, Java) all the code in Red Hat products is open source and licensed under the GPL (or a similar license, such as the LGPL, BSD, MIT). So one always has free access to the source code. However, Red Hat does not provide free access to the binaries of Red Hat Enterprise Linux, and these, combined with an annual subscription to Red Hat Network, access to upgrades, and a selected support services, are the components that Red Hat bundles into each Red Hat Enterprise Linux solution. Since every Red Hat Enterprise Linux product includes support for the system on which it is installed, Red Hat supplies the products with a per-system usage/support subscription. This simple model ensures that systems which use Red Hat Enterprise Linux are able to access the maintenance, services and product upgrades to which they are entitled. Of course, as mentioned before, this has no impact on one's access to the Red Hat Enterprise Linux source code."

So in summary we can conclude that RHEL = Linux Kernel + GPL/LGPL/MIT etc. licensed software + Support + RHEL art work and logos

The last component of RHEL is registered product of Redhat and this is included in the iso images and thus we can't copy RHEL disks and also can't download 'em from Internet.

Fiddling with colors :)

2009-07-19T13:27:00.002+05:30

Disable CPU cores in a multicore system.

2009-07-19T13:19:00.002+05:30

Got my hands on new Dell R710 server today and this is an awesome machine, with huge power underneath. It's powered by two Intel Xeon Quad Core CPU's and has 8 GB of DDR3 RAM inside it. However as has been the story in past, with Intel focusing more on Quad Core CPU's, Dual cores are slowly moving out of the market and this has raised a peculiar situation. As most of the software vendors tend to sell their products based upon licenses which are socket based or CPU based, so it is upto discretion of the vendor, if they decide to consider one Quad Core CPU to be a single CPU or 4 CPU's. With cost of software licenses running into lakhs of rupees, this is a real serious matter. However there is a provision for users to disable some of the CPU core, if they want to. With Dell R710, the provision is available in BIOS setup itself and one can disable upto 2 cores on a Quad core machine. However with IBM, the case isn't so, as they don't allow for this feature. Windows 2003/2008/XP/Vista/7, however allow users to disable some of the cores on multicore CPU. To use this capability, one has to run 'msconfig' and then goto Boot section. In advanced section, one can choose to decide how many CPUs do they wish to enable. But considering the monopoly, some of the vendors have in software market, for instance Oracle in Enterprise Database, it would be interesting to see how they react to such situations.

Amazing Hand Art !

2009-07-12T15:43:00.002+05:30

While searching for some techniques for origami I landed up at a blog which had a collection of photographs of hand art by Guido Daniele. Visit Freshpics to get a view of the amazing work done by this great artist.

How to swap two variables without using third variable ?

2009-07-10T15:22:00.006+05:30

Swap routine is perhaps one of the most common routine we find in many programming books. Almost every book related to high level programming language carries a page about how to swap contents of two variables. It's generally explained as follows:

a=5, b=7 //two variables which are to be swapped
temp = a //temp is our temporary variable
a = b //this makes variable a contain value of variable b
b = temp //b contains value of temp which is containing original value of a

However there is a method by which we can easily swap the values of these variables without making use of third variable and the answer lies in logic theory. Exclusive-OR operation can be used to accomplish this task as follows:

A = A x B
B = A x B
A = A x B

In above equations, x is denoting XOR operation.

For e.g. if A = 5 and B = 3 then we can write them in binary as A = 0000 0101, B = 0000 0011
=> A = A x B = 0000 0110
=> B = A x B = 0000 0110 x 0000 0011 = 0000 0101 = 5
and A = A x B = 0000 0110 x 0000 0101 = 0000 0011 = 3

DIY - LCD System

2009-07-08T11:01:00.002+05:30

Few days back, had an idea to build a custom LCD display panel for my newly built computer. I wanted to use this panel to display all sorts of things like my current system statistics, my playlists etc. and I was lucky to find a DIY guide for this purpose at Overclockers.

Connecting to remote FTP site through IPtables

2009-07-07T15:00:00.004+05:30

We'd recently installed a Linux based Proxy/Gateway server and we'd used iptables to manage the firewall aspect of the server. However even though we'd port 21 open on this firewall, we were not able to initiate a data session to remote FTP site. Although connect and authentication sessions were happily working, but when a data session command like ls was issued, the client just couldn't connect to remote host. Solution to the problem was understood, after we read the RFC for FTP. The problem rested on design of FTP protocol. FTP can work in two modes, namely active and passive. Following figure illustrates the difference between the two:

So as illustrated in figure above the connection request to FTP server is made at port 21 and hence we were able to connect to remote host. However the data connection is established by server in an active connect mode. In this mode the server tries to connect to a port > 1024 on client side. Moreover server originates this request from port 20. This means that iptables or any other firewall would need to allow connect from port 20 of remote host to port > 1023. This can be a security risk at times and hence need for passive mode is felt.

In passive mode, client tries to connect to a port > 1023 on server. The client generates this request from port > 1023. Now if we enable ip_conntrack_ftp module via modprobe, then we can easily connect to remote FTP site and also initiate the data transfer.

Hardware report in Linux

2009-07-07T10:11:00.003+05:30

Few days back one of my friend came up to me and asked if I can tell him a command to view details of hardware on his Linux machine. Given my knowledge all I could recall was, using /proc/cpuinfo to get information about CPU, using /proc/meminfo to get information about Memory and using commands like lspci and lsusb to get information about PCI and USB devices. However yesterday just out of curiosity I looked up at my system and found a command lshw. This command helps in generating entire system report and provides good enough details about various components of the system. It was even able to find out vendor id for my optical drive ! One thing which I couldn't decipher though was the fact that the command always complained about running under root credentials. Why would such a command have to be executed under root credentials ?

Understanding fstab

2009-07-02T13:42:00.004+05:30

/etc/fstab contains information about all partitions and storage devices on the computer. It contains information about where the partitions and storage devices should be mounted and how. Following is sample from fstab file:

/dev/hda1 / ext3 defaults 1 1 /dev/hdb1 /music ext3 defaults 1 2 /dev/fd0 /media/floppy auto ro,noauto,user,exec 0 0

The fstab file is divided into 6 columns where first is device name, second is mount point, third is filesystem, fourth is mount options, fifth is dump option, and sixth is filesystem check option.

The option auto in filesystem type simply means that filesystem type is automatically detected.

Mount options:

auto and noauto: with auto option the device is mounted automatically. with noauto option, device can be mounted only explicitly.

user and nouser: user option allows normal user to mount the device, whereas nouser lets only root mount the device. nouser is default.

exec and noexec: exec lets one execute binaries residing on that partition, while noexec blocks that. exec is the default option

ro: mount readonly

rw: mount filesystem read-write.
sync and async: sync means that when a command is issued to say copy a file to filesystem, then changes are physically written to device at same time. However with async changes can be made to physical device at some later stage. async is default option. With removable devices like floppy disks, one should make use of sync option, as generally users try to remove the disk without unmounting the device, which can result in data corruption.

default: uses the default options, that are rw, suid, dev, exec, auto, nouser and async.

Fifth column in fstab is dump option. dump checks it and uses the number to decide if a filesystem should be backed up. If set to 0, the filesystem is ignored.

Sixth column in fstab is fsck option. fsck looks at sixth column to determine in which order the filesystems should be checked. If it's 0, the filesystem won't be checked.

Decibel's Naagin - The Lady Cobra !

2009-06-30T19:02:00.006+05:30

Heard Nagin by Decibel last night and ever since have been glued to the song. Sexcellent work on 6 strings by Ripple. The history of band and the song is available at Decibel's Blog.

Following is a rocking performance by the band @ IIT-D Rock Fest.

And if I've heard correct then there is part 2 of the song as well, titled "Naagin - The Lady with No Bra !"

Trek to Bheemashankar (Maharashtra)

2009-06-19T20:43:00.003+05:30

My new machine

2009-06-19T18:48:00.003+05:30

Thanks to Mother India Part III, a.k.a. Gayatri ma'am, I got myself a new machine (Mother India Part I is my mum and Part II is my aunt !) . The specs are wonderful at Celeron 2.3 GHz., 512 MB RAM and a 40 GB hard disk . Got to get Linux From Scratch installed on this machine tonight along with some softwares from BLFS.

Is free reporting really Free Memory ?

2009-06-10T20:00:00.002+05:30

Today while looking at a system we found that free command was reporting too much memory usage while ps -aux was showing something different all together. So where was this additional memory been reported by free as used up was being used. Well the answer to this question is fairly simple if we are to blame software developers, cauz' all the memory that wasn't been displayed as free but was not been used by processes either was memory which wasn't freed up by processes which had died earlier. Now this is pretty bad and we needed to find a way to get over this issue and following trick mentioned in Mige Mengenai's blog helped us out.

1. dd if=/dev/zero of=junk bs=10MB count=200
2. rm -f junk

Well yessssssssss! All that needed to be done was this and we were able to free around 300 MB of memory ! Well what exactly happened was that command #1 tried to create a 2000MB file and as we were having very less memory left, it tried to get all available memory and there was lot of swapping in and out done as well. Finally when file was created we were left with a clean free memory space. Atleast that's what I understood by the entire process. Nyways thanks to Mige for a wonderful trick :)

Price Lists of Computer Peripherals

2009-06-09T14:04:00.002+05:30

Thanks to Vaidy, I came to know of following two sites from where I can get price lists of various Computer peripherals:

1. http://www.theitwares.com
2. http://www.deltapage.com

ODT vs. DOC

2009-06-09T13:44:00.002+05:30

With Open Document Format now almost a standard, it won't be too long when Microsoft will start providing support for these formats. However until that day comes, Sun has a solution for people who can't wait for Microsoft to take a stance on Open Document Format. Sun has provided a plugin called ODF Plugin for Microsoft Office, and this plugin allows users of Microsoft Office software suite to open and edit documents in Open Document Format.

For a quick comparison of ODF/ODT against DOC, visit 3 Monkeys.

TFTP on a Linux NAT box.

2009-06-04T21:53:00.003+05:30

Apart from following instructions in article about running M$ RIS/WDS through Linux dhcpd server we also need to configure iptables for allowing tftp server running on Linux server to serve the clients. And mind you, running tftp server on a Linux box with multiple interfaces and iptables handling routing, is a real pain. However netfilter has two modules ip_conntrac_tftp and ip_nat_tftp which are just what doctor would've advised for this scenario. Just a modprobe ip_conntrac_tftp and modprobe ip_nat_tftp does the job for us and now we can happily run tftp server on a natted linux machine.

And here is another one !

2009-06-02T22:19:00.001+05:30

A new T Design !

2009-06-02T13:36:00.003+05:30

With a pack of Iron On Transfer Papers this is what I came up with on my newly bought T Shirt:

Is Windows XP secure without a password ?

2009-05-31T17:14:00.004+05:30

Found a new target machine today with IP address: 79.113.131.224 . Now this machine is running Windows XP and has local administrator password set to blank. I always thought that this was a big security risk. But today I was proven wrong as remote access to a machine with blank password is not possible and you are always presented with a screen saying "You are unable to logon because of account restrictions". This is the way M$ is trying to protect it's poor customers who've forgotten to set their administrator passwords but haven't forgotten o hook up their machine to Internet ! Anyone interested in working on getting into the remote machine can connect to the above mentioned IP through Remote Desktop session.

n00b vs. newb

2009-05-31T15:59:00.001+05:30

Here is a nice article explaining the difference :)

BING !!!

2009-05-31T12:23:00.002+05:30

Despite several attempts it seems that Microsoft has not been able to take on Google as far as Web Services go and Bing is latest attempt by M$ to take on Google. Bing is a shift by M$ from it's traditional Live Search. The technology behind Bing is provided by a company called Powerset which was recently acquired by the software giant. The service is going to be provided for Beta testing from 1st June 2009 and will be available at www.bing.com . Will Microsoft be able to take on Google in search arena is something which only time will tell. But the promotional video does promise a much better search experience. By the way on a lighter note, Bing stands for "But it's not Google !"

What's with Load average ?

2009-05-26T19:24:00.002+05:30

Today while setting up the gateway/proxy server a question raised as to how to analyze the performance of this new setup ? There is a nice small article available at KnowledgeBase which explains this question. However upon reading the article we came to a place where the author had mentioned about sysstat. Now I've used sysstat for sometime now and uptime is a command which most *nix people are familiar with. But the big question this time was "How does the command actually gives the Load average ?". A bit of googling and we found a nice explanation at Mike Malon's blog.

M$ RIS through Linux DHCPD

2009-05-20T01:34:00.002+05:30

Since the change in our network setup we'd to think about how RIS server could be intergrated with non MS DHCP server. We've migrated from MS DHCP server to dhcpd server running on RHEL 5. As we'd thought earlier, the process was not as easy as adding next-server and filename entry to /etc/dhcpd.conf. To make RIS work with non MS DHCP server we've to understand how RIS actually works.

The behavior of the RIS client is different depending on whether or not the RIS server and the DHCP server are the same machine. This distinction is significant, so we'll look at each case in theory, and then describe our situation.

Let's pretend for a minute that we're in an all-Microsoft environment - MS DHCP, MS RIS server.

Case 1: The (MS) RIS server is the same box as the (MS) DHCP server.

 The client boots off the floppy or PXE BIOS, and sends out a DHCPDISCOVER request, via broadcast, asking for an
IP address, netmask,gateway, maybe some other stuff (i.e., DNS, WINS, time servers, etc), and some information
about the RIS server.
The DHCP/RIS server then responds with a DHCPOFFER reply, offering a lease for an IP, netmask, gateway, maybe
some other stuff, and also the hostname and IP address of the RIS server (which in this case is the same as the
hostname/IP of the DHCP server) as well as the initial TFTP path to the boot file (which by default is \oschooser
\i386\startrom.com).
The client looks at the DHCPOFFER and notices that it contains both basic lease information (the IP, netmask,
and so on) as well as the RIS-specific stuff.  Because the RIS stuff AND the basic stuff is in there, it 'knows'
that the DHCP server also offers RIS service.
The client then completes the DHCP handshake (sends a DHCPREQUEST asking to use that DHCPOFFERed lease, and gets
a DHCPACK from the DHCP/RIS server confirming).  It then downloads the startrom.com file from the DHCP/RIS
server at the path specified in the lease, and initiates the RIS process (so you start getting blue OSCML
screens on your console).
Off you go.

Case #2: The (MS) RIS server is on a different machine than the (MS) DHCP server.

Again, the client boots off the floppy or PXE BIOS, and sends out a DHCPDISCOVER request, via broadcast, asking
for an IP address, netmask, gateway, maybe some other stuff, and some information about the RIS server.
The DHCP server then responds with a DHCPOFFER reply, with a lease for an IP, netmask, gateway, and maybe some
other stuff, but no information about the RIS server.
The RIS server *also* responds with a separate DHCPOFFER reply, which contains the hostname and IP address of
the RIS server, and the initial TFTP path to the boot file. But, this DHCPOFFER does *not* contain any
basic information such as IP address, netmask, etc.
The client looks at both offers and ignores the one from the RIS server, since it doesn't contain an IP,
netmask, etc. It completes the handshake for the lease offered by the DHCP server, and gets itself on the air.
The client then sends out a *second* DHCPDISCOVER broadcast. The DHCP server responds as before. The RIS server
responds as before. The client again looks at both offers. This time it ignores the offer from the DHCP server
(since it already has that information) and pulls the information it needs (the RIS-specific stuff) out of the
lease offered by the RIS server - i.e., the hostname and IP of the RIS server and the path to the startrom.com
file. (I don't know if it bothers to complete the second handshake - I doubt it.)
The client then uses that information to contact the RIS server, download the startrom.com file, and start the
ball rolling.

Now, alter the environment slightly:

Case #3: The RIS server and the DHCP server are on separate machines.

 The client is on a different subnet than the RIS server or the DHCP server.(The DHCP and RIS servers can be on
the same or different subnets from each other - doesn't matter.)  DHCPDISCOVER broadcast traffic is forwarded
from the subnet where the client is, directly to the DHCP server (i.e, the RIS server does not hear any
DHCPDISCOVER traffic from the client).

The DHCP server may or may not be running MS DHCP (in our case, it's running the dhcpd server on Linux).

 Again, the client boots off the floppy or PXE BIOS, and sends out a DHCPDISCOVER request, via broadcast, asking
for an IP address, netmask, gateway, maybe some other stuff, and the information about the RIS server.
Since the DHCPDISCOVER traffic is getting forwarded directly to the DHCP server, and the RIS server never hears
it, the RIS server will never send the client a DHCPOFFER with information about itself.  So, the DHCP server
must be configured to provide the RIS specific information in it's DHCPOFFERed lease, even though it's not
running RIS.
The DHCP server responds to the client with a DHCPOFFER reply, offering a lease for an IP, netmask, gateway,
maybe some other stuff, as well as the hostname and IP address of the RIS server (which in this case is a
*different* hostname/IP than the DHCP server) and also the initial TFTP path to the boot file.

BUG:

 Since the DHCPOFFERED lease contains both basic and the RIS-specific information (i.e., it looks like a lease
offered by a DHCP server that's also running RIS), the client attempts to reconnect to the DHCP server to
download the initial startrom.com file at the path specified in the lease.  It does this even though the lease
tells it that the RIS server is on another box.

FIX:

 Run TFTP on the DHCP server, and place a copy of startrom.com at the appropriate path. Once the client downloads
the initial startrom.com file, it becomes smart enough to look at the information in the lease, and goes and
talks to the real RIS server for the ntldr file and everything else.

NOTE:

 The client expects to find startrom.com and ntldr at the same relative paths. That is, if the lease specifies
that the path to startrom.com is \foo\bar\startrom.com, then in this situation the client downloads startrom.com
from \\dhcp-server\tfptproot\foo\bar\startrom.com, and looks for ntldr at \\ris-server\reminst\foo\ba\ntldr (and
NOT \\ris-server\reminst\OSChooser\i386\ntldr).

This is an important point as our DHCP server is not able to interpret the backslashes in the \OSChooser\i386 path correctly. We ended up putting startrom.com in the root TFTP directory on the DHCP server, which meant that we also had to put a copy of ntldr at \\ris-server\reminst\ntldr.

Following are the configuration settings that had to be done for allowing existing MS RIS to work with Linux DHCP:

1. The global section of dhcpd.conf has following more entries:

 server-name "ris-server.domain";
next-server ;
filename "startrom.com";

2. The files ntldr and startrom.n12 have also been copied from \\ris-server\reminst\OSChooser\i386 to \\ris-server\reminst.

3. TFTP server has been installed on gateway server and root of the tftp server is /tftpboot. This directory contains startrom.com file, which the DHCP server will provide to the client. The tftp service is being run through xinted and the configuration file for xinetd is /etc/xinetd.conf, while the configuration file for tftp is /etc/xinetd.d/tftp

4. /etc/hosts.allow had to be modified to allow clients from 192.161.0.0, 192.161.1.0 and 192.161.2.0 to access tftp services.

Thanks to Greg and Alex for sharing information about internal working of M$ RIS.

Wanna be a DJ ?

2009-05-17T18:40:00.002+05:30

While googling for Digital Turntables I stumbled upon a small video presentation about Technics SL-DZ1200 Direct-Drive Digital Turntable.

Technics SL-DZ1200 Digital Turntable
by PeteRock

Having seen the presentation an idea came across to me, as to why not have a software which allows me to use my Touchscreen panel as a digital turntable and upon googling it I found that ATTIGO TT, a futuristic DJ Turntable (TT) created by Scott Hobbs, a student at Dundee University, Scotland.

Final Product // ATTIGO TT from Scott Hobbs on Vimeo.

As for aspiring DJs who don't have money to shelve out, you can try out Mixxx. Mixxx is an open source digital music mixing software with oodles of features to satisfy your apetite.

Wolfram|Alpha is online !

2009-05-17T13:07:00.002+05:30

After Mathematica and A New Kind of Science Stephen Wolfram is back with a very ambitious project called Wolfram|Alpha. It's a distributed computing project which is going to give answers to various kind of queries a user puts through it's web based interface. As Stephen wrote in his Wolfram|Alpha blog, Alpha will run on 5 distributed colocation facilities.

What computing power have they gathered in these facilities for launch day? Two supercomputers, just about 10,000 processor cores, hundreds of terabytes of disks, a heck of a lot of bandwidth, and what seems like enough air conditioning for the Sahara to host a ski resort.

One of their launch partners, R Systems, created the world’s 44th largest supercomputer (per the June 2008 TOP500 list). They call it the R Smarr. It will be running Wolfram|Alpha on launch day!

Dell is another of the launch partners with a data center full of quad-board, dual-processor, quad-core Harpertown servers. What does it all add up to? The ability to handle 175 million queries (yielding maybe a billion) per day—over 5 billion queries (encompassing around 30 billion calculations) per month.

Wolfram|Alpha is going to be one of the most computationally intensive websites on the Internet.

Sysadmin Woes

2009-05-15T12:26:00.002+05:30

A nice and funny document which tries to see through the pain of a Sysadmin.

Inside RAM

2009-05-11T12:24:00.002+05:30

Few days back during a discussion at office I heard about two different kinds of RAMs (yes yes we all know about SDRAMs and DDRs), one of 'em was ECC Registered and another was FBD. Now many people use technical jargons every now and then without actually understanding the details. So as usual google was there to help us morons, and after few clicks here and there I landed up at this wonderful article about different types of RAMs. The article explains in easy to understand language about the features and difference between Unbuffered, Registered, ECC, ECC Registered and FBD memory modules.

Our Trip to Matheran

2009-05-10T21:23:00.001+05:30

Our Trip To Pindari Glacier

2009-05-10T21:22:00.003+05:30

Our Trip To Alibaug

2009-05-10T21:16:00.000+05:30

My New T Logo

2009-05-10T15:59:00.002+05:30

"lshw" one of the must have commands.

2009-05-06T15:59:00.003+05:30

Today got a chance to setup a brand new server which will be used as a web-caching proxy and content filtering system at our setup. I wanted to pump in as much RAM as possible into the machine and asked my boss for memory upgrade and as always he came up with 2 DIMMS of 2 GB RAM modules. As the IBM eServer (this was the server where we were supposed to host our application) was having 1 GB of RAM already and there were 2 free slots, so we decided to plug in both the modules. One interesting thing about this setup was that we can't plugin memory modules here are there, just as we are used to in case of Desktop. It has a specific sequencing pattern which has to be followed. It requires one to plugin lower capacity modules first and higher capacity modules later (later here refers to being closer to CPU).

Anyways after fitting in the RAM modules we were anxious to see our machine with 5 GB of RAM and I was really anxious to get OS up and running. The BIOS also showed us 5 GB of RAM installed and we were all happy. Finally the OS was also installed and I don't know why but we decided to check our memory utilitization. However as soon as we entered 'cat /proc/meminfo', we got a big shock as the OS was reporting to have 4 GB of RAM installed. It was at this moment did we realized that BIOS was making a fool of us all, as the maximum memory limit of a 32 bit machine is 4 GB and OS was correct at detecting it.

So this brought us to next big question about determining if our machine was a 32 bit machine or a 64 bit one ? If manuals aren't there then how can one determine this ? And the answer to this question is lshw. It's a small nifty utility which reports hardware information and also determines if the machine is 64 bit or 32 bit. For more information about lshw, visit it's website.

Argument list too long

2009-05-03T16:51:00.002+05:30

When working on Linux, it seems that most of the utilities provided with coreutil package have some kind of limitation imposed upon 'em. Today I was trying to delete some files from one of our servers and as I issued the command rm -f heldmsg-* I was surprised to see bash throwing me an error saying -bash: /bin/rm: Argument list too long

It initially seemed to me that it's a bug on part of coreutils package but upon googling I found that this wasn't the case and it's neither a bug of coreutils nor of bash. Actually it's an architecture limitation of Unix like operating systems. Further details about the issue are available at GNU Core Utilities FAQs.

Interestingly Linux kernel 2.6.23 doesn't suffer from this problem any more, as it has removed the classic ARG_MAX limitation. It will eventually propagate into the release software distributions which include it. However glibc's getconf ARG_MAX hasn't caught up yet and still reports a fixed value.

Anyhow as for the solution to the problem of removal of the files, one solution is to use find command as follows:

find . -name 'heldmsg-*' | xargs -0 rm

and another method is through a simple shell script:

for x in *; do rm $x; done

Mobility Modder

2009-05-03T15:14:00.003+05:30

It has happened millions of times that I was looking for a driver update and though the chipset manufacturer reports of having a new version, my laptop vendor doesn't show it available on it's website. The biggest question here is, "What to do next ?". As most of the users are aware, a corrupt driver can be worst nightmare a user has to face. This is where Mobility Modder comes to rescue. It's a small nifty tool designed just for that purpose. For more details visit Mobility Modder's webpage.

Bumptop is live

2009-05-03T13:58:00.003+05:30

Much awaited Bumptop 3D Desktop is live and available for download now. One can download the Free and Commercial version from Bumptop's Official Website. There is also a nice video explaining the features of this wonderful piece of software. Happy bumping !

National Knowledge Network

2009-05-03T12:04:00.002+05:30

Few days back got information from our Executive Director about commissioning of National Knowledge Network in India. All together it seems a very ambitious initiative by Government of India to interconnect educational and research institutions in India.

This opens up all new possibilities for collaborative research and sharing of resources. However I am not too sure if it's going to be a success considering the kind of attitude I've seen among people who are into research. They are highly secretive about the research they are doing and moreover with availability of Internet sharing of resources (if it means sharing of thesis papers) was already possible. Yes this network does provide a means for Grid computing projects to happen more effectively but doesn't Garuda project already serve this purpose ?

The only scope I feel this project has is in Virtual classrooms and Virtual libraries. However these projects are also going to be useful only if required applications are deployed on the network asap. Virtual classrooms aren't a big necessity for major colleges like IITs, IISc or NITs, instead they are more required for colleges in remote areas where there is lack of adequate faculty. If this network has to be successful then it has to reach those remote places in first place, as people from those places are going to be biggest consumers of the applications running on this network. Also virtual libraries are only going to be successful if we digitize the information contained in books. However this is going to raise several legal issues related to licensing and I'm not sure if management in most of the institutions would be interested in dealing with these issues.

But if in case virtual classrooms do begin to run, then we've a great oppurtunity at our hands to design and develop communication technologies which provide high definition audio/video content, which can be used to deliver the media on the network. I guess Narendra is already feeling great about availability of a platform to deliver his Eduvid project :)

Switchfoot - Lonely Nation

2009-04-29T22:23:00.001+05:30

One hell of a fucking great song. Some thought provoking lyrics after a looooooong time.

The Road to Reality

2009-04-29T22:02:00.003+05:30

Was reading "The Road To Reality" by Roger Penrose today. Don't know why but after reading first few chapters a very disturbing thought came into mind. These thoughts make me feel really uncomfortable and I was confused once again. Am I doing right by trying to understand the details of how things work around me, or is it just that I'm wasting my time. I know it may sound absurd to many but this is a question which I keep on asking myself when I try to do something. Maybe it's fear of failure which makes me think like this, I don't know for sure but something is wrong deep inside somewhere. Got to find a solution for this before it gets too late. By the way the book is an excellent and must read.

Auto Complete in Vim

2009-03-30T22:04:00.002+05:30

Thanks to Rahul Sir, I remembered a long forgotten trick which is very helpful while coding using Vim. Autocomplete feature of Vim is a wonderful feature which allows for autocompletion of text which is currently being inserted inside Vim editor.

To use autocomplete one must be in insert mode. Then press Ctrl and x to go to autocomplete mode. The next ctrl + ’some key’ sequence determines the output of autocomplete. Eg. ctrl + ‘p’ will search backwards in the file for the first matching text and insert that, while ctrl + ‘n’ searches forward in the file for the same. You can use ctrl + ‘f’ to insert files names present in current directory (useful when inserting header files).

This feature becomes all more effective and efficient when dictionary is used along with Vim.

Ctrl+l Complete whole lines of text
Ctrl+n Complete words from current buffe
Ctrl+k Complete words from dictionaries
Ctrl+t Complete words from thesaurus
Ctrl+i Words from current and included files
Ctrl+s Spelling suggestions (Vim 7.0 and newer only)

I Forgot My Windows XP Password !

2009-03-29T09:27:00.002+05:30

Many a times it happens that user forgets his/her password and things become really messed up when there is a single user i.e. administrator and user forgets the password for administrator account. However there is a way to reset the password and that too without using any particular software. All that's required is a bootable Windows XP CD and one just needs to follow the steps outlined in an article by Vic Ferri.

Google's Power Supply Unit

2009-03-26T21:19:00.002+05:30

Watch http://www.youtube.com/watch?v=M5wfv7RE_J4 to get a glimpse of what's happening at Google as far as efficient data center setup is concerned.

Good discussion

2009-03-25T21:00:00.004+05:30

Had a good discussion about a project idea related to eLearning solution today with my colleagues. I really appreciate critical reviews of an idea and that's what I got from my friends today. Thanks a zillion to 'em for a great discussion and if that wasn't enough the day ended on a very informative article I read on Rajesh Jain's Weblog on Emerging Technologies, Enterprises and Markets. Thanks a lot to Rajesh for providing the content. So what's next, is the question I got to ask myself tonight when I hit bed tonight :)

Apache Proxy For SSL enabled sites

2009-03-24T19:10:00.002+05:30

Somedays back I'd an opportunity to do proxy setup for Apache webserver and as if problems are my good company, I landed up with a problem today when I had to bring a SSL enabled site running in my setup, under the umberalla of unified URL namespace.

However as with every problem associated with task of systems administration, this too had a pretty easy solution and this solution involved usage of SSLProxy which is a directive provided by mod_ssl module of Apache webserver.

So all that's needed to be done is to enable the SSLEngine and specify proxy path of the urls as illustrated below:

#Enable SSL Proxy
SSLEngine On

#Proxy paths
ProxyPass /
ProxyPassReverse /

So now any one visiting / is taken to .

However I'm not sure why, but I'm unable to do a URL Rewrite as it was happening automatically with normal Apache Proxy. Will see to it and revert back in days to come.

Network Analysis using Overlook fing

2009-03-23T11:04:00.002+05:30

While searching for a utility which allows for scanning of entire local area network I found a small but very useful tool called Overlook-fing. The utility can be used for network and service discovery. Taking advantage of a brand new cross-platform network engine, it reaches an impressive discovery sharpness and speed: you’re going to discover your whole network in few seconds and generate customized real-time reports.

Fing's smart discovery automatically detects the network type and uses the best technique to take the picture of it. The best results are surely achieved on Ethernet networks (Wireless ones included), where Fing is able to detect all network hosts, fire walled ones included, reaching 100% hit. The service discovery feature quickly detects active TCP services on a target host or network.

The software is available for Windows, Linux and Mac. For more details visit www.lookatlan.com

How to test your Antivirus/AntiSpyware Solution ?

2009-03-22T15:36:00.002+05:30

Few days back I was asked by my boss to evaluate some antivirus and anti-spyware solutions for our setup. As usual first step was to ask vendors to come up with their data sheets and as usual all of 'em hurried up with their emails and we were all very much excited to learn about all cool features they had to offer.

However no matter how much features these solutions offer, like device control, application control, even NAC, at the end of day the key task they ought to perform is efficient detection of virus/worms/trojans/malwares and also removal of them. We also wanted to have a demo of the software solutions but this raised a question as to how to test these solutions to their limit. This lead me to Spycar and EICAR which provide test files to test the functionality of the antivirus/anti-spyware solutions. I don't know for sure as to how good these test files are, but atleast we've a starting point and I'll post how they fare in days to come.

Handling fonts in Linux

2009-03-22T12:22:00.002+05:30

This has been a cause of trouble for me ever since I started reading News portals. Some portals use specific fonts and this makes it difficult for browsers on Linux to render the page successfully. The problem rests on the fact that browsers make use of fonts available on the system and as these portals require installation of particular fonts not available on Linux so it becomes impossible for rendering. So the key issue before me was to understand how to install fonts on Linux.

Linux makes use of fontconfig library for font substitution when either entire fonts or individual characters are not present. If system is not having a particular font, it tries to display the content by making use of some other font. Applications in Linux looks in directories specified in /etc/fonts/fonts.conf for location to search for a font.

In order to install a new font on Linux box, you can do one of the following:

 # Copy font to one of the directories mentioned in /etc/fonts/fonts.conf.
# Issue command fc-cache 
 # Restart the application and the font should be available to that application for use.

 # Copy font to ~/.fonts
# Issue command fc-cache ~/.fonts
# Restart the application.

In second case the font changes are visible only to user in whose home directory the fonts have been installed, whereas in first case the changes are visible to all users on the machine.

Gmail and PGP Keys !

2009-03-22T10:13:00.002+05:30

Thanks to Ashish Shukla whose mails on ILUGD were always PGP signed, I finally decided to have my own PGP key setup and integrate it with my personal email account on Gmail. The setup is fairly simple, all softwares I needed were there on my machine including GnuPG and Seahorse (In case the softwares aren't available then they can be installed easily via apt or yum). One software missing on my machine was FireGPG. It's a Mozilla Firefox extension which integrates with Gmail and allows for signing and encryption of mails easily. Once softwares are in place all that's required is to create a PGP key using Seahorse and start using FireGPG to sign and encrypt mails. An excellent tutorial on using GnuPG is available at Johnny Chadda's site.

How to enable routing in Linux kernel ?

2009-03-16T19:28:00.002+05:30

The Linux kernel by default doesn't allow for routing and it has to be enabled through sysctl. Edit /etc/sysctl.conf and set net.ipv4.conf.default.accept_source_route = 1

Networking in Fedora 10

2009-03-16T10:36:00.002+05:30

Got Fedora 10 installed on my machine today and Plymouth is great ! However once I'd booted into the environment I just couldn't connect to the network. Fedora 10 has NetworkManager which is used by default for maintaining network connections and also there is another service called network. Now one thing which really baffles me is "Why in the hell two services to do the same job ?". Anyways as I was new to this scheme of things I tried to restart network service quite a few times but all in vein. A bit of googling finally made me realise that this wasn't the way things were supposed to work and I had to first disable the NetworkManager service before I could make use of network service.

Size of Swap Space

2009-03-14T17:07:00.002+05:30

Enough is enough and I had heard too much of this idea of having swap space with size twice that of RAM. Now why in the hell do we have to have such a constraint ? I mean there has to be some reason behind it ( now this is ironical as I think we don't know reasons for most of the problems we tackle in day to day life, with computers ).

A bit of googling and voilla I landed up at this wonderful article by Russel Coker (etbe). Thanks to author for this wonderful post.

Finally we have our own Wiki !

2009-03-12T12:58:00.003+05:30

Thanks to my boss, we finally got permission to host our own Wiki at office ! The installation was smooth and hasslefree. Just downloaded the mediawiki tar file from Mediawiki Site and installed the WAMP software on one of Windows 2003 server. Next step was to configure the httpd.conf file, as I already had IIS running on port 80. After setting the listening port to 8080, the time had come to configure the newly installed mediawiki, so I fired up the browser and pointed it to http://localhost:8080/wiki . Coool so the page says that mediawiki has been installed but hasn't been configured as of yet, so next step is to click on link which says "click here to setup the wiki". The background configuration script does an environment check for the first time to verify if everything is working as it should. Once a green signal is given we can easily go ahead and setup our new wiki. All we now need to do is to supply some information about wiki, like wiki name, administrator information, database information and some tweaking info about caching etc. and that's it, our new wiki is up and ready for edits :)

Next step which I presume almost every wiki admin does for first time is to set the logo of his/her wiki by editing $wgLogo in LocalSettings.php and boy my wiki is ready to be presented to the world. Next thing which I wanted was to remove some of the links from navigation bar and also wanted to edit the disclaimer information. That's quite easy as well, just search for MediaWiki:Sidebar and you can edit the page you get in search result. You may even add a new link.

Now the task ahead is to see how permissions are managed by MediaWiki and if we can integrate our existing Active Directory setup with Mediawiki for user authentication, and then the server will be up and ready for deployment.

How can someone do this ?

2009-03-09T16:42:00.002+05:30

Today missed the bus again in morning, well it had to inevitably happen as last night I hit bed too late at around 4:30 a.m. in morning, thanks to the wonderful book "God Created Integers ..." by Stephen Hawking. As usual I hired an auto, plugged in newly bought Creative EP-630 headphones into the ipod and started enjoying a brand new Monday morning. With Holi festivities nearing the roads seemed a bit empty and within no moments I was near H2O at Bandra. The signal was red and I saw few new born puppies enjoying a warm sunshine beneath a shed near the crossing. Suddenly few guys came and for no reason one of them kicked 2 of the poor little dogs. I just don't understand how can someone do such a thing ? It's not the first time I've seen such an incidence and I am sure something is seriously wrong with such people who try to show their power on helpless beings.

Lockhart's Mathematical Lament

2009-03-09T12:12:00.002+05:30

This post is regarding an article called A Mathematician's Lament by Paul Lockhart. Paul is a mathematics teacher at Saint Ann's School in Brooklyn, New York. It is, quite frankly, one of the best critiques of current education norms being followed at schools for teaching Mathematics. Written by a first-class research mathematician who elected to devote his teaching career to K-12 education.

Paul became interested in mathematics when he was about 14 (outside of the school math class, he points out) and read voraciously, becoming especially interested in analytic number theory. He dropped out of college after one semester to devote himself to math, supporting himself by working as a computer programmer and as an elementary school teacher. Eventually he started working with Ernst Strauss at UCLA, and the two published a few papers together. Strauss introduced him to Paul Erdos, and they somehow arranged it so that he became a graduate student there. He ended up getting a Ph.D. from Columbia in 1990, and went on to be a fellow at MSRI and an assistant professor at Brown. He also taught at UC Santa Cruz. His main research interests were, and are, automorphic forms and Diophantine geometry.

After several years teaching university mathematics, Paul eventually tired of it and decided he wanted to get back to teaching children. He secured a position at Saint Ann's School, where he says "I have happily been subversively teaching mathematics (the real thing) since 2000."

He teaches all grade levels at Saint Ann's (K-12), and says he is especially interested in bringing a mathematician's point of view to very young children. "I want them to understand that there is a playground in their minds and that that is where mathematics happens. So far I have met with tremendous enthusiasm among the parents and kids, less so among the mid-level administrators," he wrote in an email. Now where do you hear such a thing? But enough of my words. Read Paul's dynamite essay. It's a 25-page PDF file.

Lockhart's Lament

Why is 0^0 =1 ?

2009-03-07T16:31:00.002+05:30

Seems like a naive question but I was definitely not sure about the reason behind the truth of equation 0⁰ = 1, until I visited betterexplained.com . It's one of those sites which tries to take mind away from usual ways of looking at things and it surely helped me understand reason behind the truth of the equation. The logic also made me clear about the logic behind finding answers to questions like "What's 2 ^1.5 ?"

Places in India I've Seen

2009-03-07T16:25:00.000+05:30

visited 21 states (60%)
Create your own visited map of India or another interesting project

What's my Ethernet Device ?

2009-03-05T18:16:00.003+05:30

This is one problem which I have been facing since quite a long time. Generally machines come with just one network interface card and Linux happily detects them as device eth0. However with servers things become a bit complicated as they generally have 2 or more NICs and Linux detects these devices as eth0, eth1 and so on. Now if I'm going to a site with a server having 2 NICs and there is some problem in network configuration and thus I've to figure out which of the NICs is eth0 and which one is eth1, then with only a damn gloomy console, figuring out answer to this question is difficult, if not impossible.

Here comes the utility called ethtool. I think every Linux admin should have this utility in his tool pack. This utility can be used to configure ethernet cards. One can even set the Transfer rate and duplex modes of the card. A real nice option of this command is -p.

If we try out command "# ethtool -p eth1 " then if we monitor the network card physically, we can see lights blinking on the NIC. This way we can easily figure out which of the card is eth0 and which one is eth1 !

Gotta Be Somebody by Nickelback

2009-03-04T13:07:00.003+05:30

This song for reasons I ain't sure about has touched me so deeply that I can't express in words. However this also raised a question which is quite philosophical in a way. Why do we like certain kind of music at certain point of time ? I am yet to meet a person who doesn't appreciate good quality music and it's also a known fact that a person's taste for music keeps on changing with time. I still remember the days when I was a die hard MLTR fan but I've also been through times when all that I loved was hard core metal. Recently learnt from an article published in IEEE about some research group working on finding the reasons behind this question. Hope they come up with some reasoning soon.

As for the song by Nickelback, it's simply amazing. The video is available on YouTube!.

Lyrics goes as follows:

This time, I wonder what it feels like
To find the one in this life, the one we all dream of
But dreams just aren't enough
So I'll be waiting for the real thing, I'll know it by the feeling
The moment when we're meeting, will play out like a scene
Straight off the silver screen
So I'll be holding my own breath, right up 'til the end
Until that moment when, I find the one that I'll spend forever with

'Cause nobody wants to be the last one there
'Cause everyone wants to feel like someone cares
Someone to love with my life in their hands
There's gotta be somebody for me like that
'Cause nobody wants to do it on their own
And everyone wants to know they're not alone
There's somebody else that feels the same somewhere
There's gotta be somebody for me out there

Tonight, out on the street, out in the moonlight
And dammit this feels too right, it's just like déjà vu
Me standing here with you
So I'll be holding my own breath, could this be the end
Is it that moment when, I find the one that I'll spend forever with

'Cause nobody wants to be the last one there
'Cause everyone wants to feel like someone cares
Someone to love with my life in their hands
There's gotta be somebody for me like that
'Cause nobody wants to do it on their own
And everyone wants to know they're not alone
There's somebody else that feels the same somewhere
There's gotta be somebody for me out there

You can't give up, looking for a diamond in the rough
You never know, when it shows up, make sure you're holding on
'Cause it could be the one, the one you're waiting on
'Cause nobody wants to be the last one there
And everyone wants to feel like someone cares
Someone to love with my life in their hands
There's gotta be somebody for me, ohhh

Nobody wants to do it on their own
And everyone wants to know they're not alone
There's somebody else that feels the same somewhere
There's gotta be somebody for me out there
Nobody wants to be the last one there
'Cause everyone wants to feel like someone cares
There's somebody else that feels the same somewhere
There's gotta be somebody for me out there

Link to Learn Haskell

2009-02-22T10:13:00.003+05:30

For most of CS students Haskell seems wierd as it's drastically different from many other programming languages that they learn, like C,Java and even Assembly ! For all such people Eric Etheridge has compiled a tutorial Haskell for C Programmers. This is an easy to follow manual and will surely help one understand the tit-bits of an otherwise seemingly difficult language.

Solution to Dynamic Double Dim. Array

2009-02-19T14:59:00.004+05:30

Sometime back I had posted a problem regarding dynamic allocation of double dimensional arrays and finally after a sufficiently large time finally solution to the problem has surfaced :) Thanks to Vinod Ji of C-DAC, Mumbai for his help. I'm not sure if I'd have cracked the solution without his guidance. Anyways following is the solution to the problem:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

main()
{
FILE *fp;
char **word;
char *str;
int c,i=0,j=0;

fp=fopen("input.txt","r");

str=(char *)malloc(sizeof(char));
// word=(char **)malloc(sizeof(long));
word=0; //we couldv'e used above line instead.

while ((c=getc(fp))!=EOF)
{
if (c==' ' || c=='\t' || c=='\n')
{
str[i]='\0';
word=(char **)realloc(word,sizeof(long)*(j+1));
*(word+j)=(char *)malloc((strlen(str)+1)*sizeof(char));
strcpy(word[j],str);
j++;
str=(char *)malloc(sizeof(char));
i=0;
}
else
{
str[i++]=c;
str=(char *)realloc(str,(i+1)*sizeof(char));
}
}

for(i=0;i<j;i++)
printf ("%s\n",word[i]);
getchar();
}

Apache Web Server: Reverse Proxy

2009-02-19T12:31:00.003+05:30

We've a setup where applications are hosted on various servers dispersed across different servers. Yesterday we came to know that our Executive Director wants a unified URL namespace for all the applications. Specifically he wanted all applications to be accessible via http://[domain_name]/[product_name] . I've seen mod_alias and Redirect/Rewrite rules but for applications dispersed across different servers I had no clue how to go about it. This is where mod_proxy came to our rescue. This is a wonderful module which allows Apache to behave like a forward and reverse proxy. We can even implement caching using mod_cache.

Apache can be configured in both forward and reverse proxy (also known as gateway) mode. An ordinary forward proxy sits between client and server. Client makes request for specific server through the proxy and the proxy server fetches content from server and passes it onto the client. In this mode of operation, client needs to be specifically configured. This mode is very much like using a proxy server to access Internet.

A reverse proxy (or a gateway) on other hand is appears to client like an ordinary web server. No special configuration on client is required. The client makes ordinary request for content in the namespace of reverse proxy. The reverse proxy then decides where to send the request, and returns the content as if it's the origin.

A typical usage of a reverse proxy is to provide Internet users access to a server that is behind a firewall. Reverse proxies can also be used to balance load among several back-end servers, or to provide caching for a slower back-end server. In addition, reverse proxies can be used simply to bring several servers into the same URL space.

The forward proxy is activated using the ProxyRequests directive. A reverse proxy is activated using the ProxyPass directive or the [P] flag to the RewriteRule directive. It is not necessary to turn ProxyRequests on in order to configure a reverse proxy.

A basic configuration goes as follows:

Forward Proxy

ProxyRequests On ProxyVia On Order deny,allow Deny from all Allow from internal.example.com

Reverse Proxy

ProxyRequests Off Order deny,allow Allow from all ProxyPass /foo http://foo.example.com/bar ProxyPassReverse /foo http://foo.example.com/bar

This wonderful module has made a seemingly difficult task, a piece of cake for me :) Long live the author of module !

Configure WDS for Live CD deployment

2009-02-17T14:28:00.006+05:30

We have been using Windows Deployment Server (WDS) for installation of Windows and Linux operating systems since quite some time now. However with advent of Ubuntu and distributions based upon it, we had to find some way to allow Live CDs to boot directly from WDS. Today we finally managed to get the job done. Following additional tools are required for this purpose:

1. NFS on Windows. (SFU 3.5 provides both server for NFS and client for NFS).

Following steps are required to get Live CD boot off PXE through Windows Deployment Server:

1. Extract the contents of iso image of Live CD to some location on Windows Deployment Server.
(Let's assume location to be D:\Linux\live. Henceforth in this article, this location shall be referred to as $LINUXLIVE)

2. Create following directories:
a) $RIS_VOL\Setup\English\Images\Ubuntu
b) $RIS_VOL\Setup\English\Images\Ubuntu\i386
c) $RIS_VOL\Setup\English\Images\Ubuntu\i386\templates
d) $RIS_VOL\Setup\English\Images\Ubuntu\i386\templates\pxelinux.cfg

3. Download netboot files for Ubuntu from http://archive.ubuntu.com/ubuntu/dists/gutsy/main/installer-i386/current/images/netboot/netboot.tar.gz
Since my distribution if Gutsy so I'm using gutsy repo. In case of Hardy or Fiesty use appropriate netboot image. As a quick fix just replace gutsy in link above with hardy or fiesty :)

4. Extract the downloaded netboot file to $RIS_VOL\Setup\English\Images\Ubuntu\i386\templates

5. Create a file called default in $RIS_VOL\Setup\English\Images\Ubuntu\i386\templates\pxelinux.cfg folder and add following content:

DISPLAY ubuntu-installer/i386/boot-screens/boot.txt

F1 ubuntu-installer/i386/boot-screens/f1.txt
F2 ubuntu-installer/i386/boot-screens/f2.txt
F3 ubuntu-installer/i386/boot-screens/f3.txt
F4 ubuntu-installer/i386/boot-screens/f4.txt
F5 ubuntu-installer/i386/boot-screens/f5.txt
F6 ubuntu-installer/i386/boot-screens/f6.txt
F7 ubuntu-installer/i386/boot-screens/f7.txt
F8 ubuntu-installer/i386/boot-screens/f8.txt
F9 ubuntu-installer/i386/boot-screens/f9.txt
F0 ubuntu-installer/i386/boot-screens/f10.txt

DEFAULT LiveDesktopCD

LABEL install
kernel ubuntu-installer/i386/linux
append vga=normal initrd=ubuntu-installer/i386/initrd.gz --
LABEL Ubuntu
kernel vmlinuz
append initrd=initrd.gz boot=casper netboot=nfs nfsroot=: --
LABEL linux
kernel ubuntu-installer/i386/linux
append vga=normal initrd=ubuntu-installer/i386/initrd.gz --
LABEL cli
kernel ubuntu-installer/i386/linux
append tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL expert
kernel ubuntu-installer/i386/linux
append priority=low vga=normal initrd=ubuntu-installer/i386/initrd.gz --
LABEL cli-expert
kernel ubuntu-installer/i386/linux
append tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false priority=low vga=normal initrd=ubuntu-installer/i386/initrd.gz --

LABEL rescue
kernel ubuntu-installer/i386/linux
append vga=normal initrd=ubuntu-installer/i386/initrd.gz rescue/enable=true --

PROMPT 1 TIMEOUT 0
6. Ensure that Server for NFS service is started on the machine running WDS.

7. Right click on folder D:\Linux\live and select Sharing & Security.

8. In NFS Sharing tab select Share this folder and also choose "Allow anonymous logon".

9. Ensure that Everyone and Anonymous Logon have Full Control access to the folder which we have just shared using NFS.

10. Copy vmlinuz and initrd.img from $LINUXLIVE\casper to $RIS_VOL\Setup\English\Images\Ubuntu\i386\templates

11. That's all that's required to be done and now we can PXE boot a machine and test the Live CD !

TAB

2009-02-15T10:05:00.003+05:30

I never knew that a simple thing as TAB character would make me wake up for entire night. The problem arose when I was trying to figure out how expandtabs function behave in Python. My brother had asked me to find out how this function behaves and he never asks dumb questions :) So when I ran following code the surprise was infront of my eyes:

Code:

s='abcdef\tabc\ta'
print s
print len(s)
x=s.expandtabs()
print x
print len(x)

In my opinion the output should have been :


abcdef        abc        a
12
abcdef        abc        a
17

But to my surprise this wasn't the case and the output I received was :


abcdef  abc               a
12
abcdef  abc               a
17

Why in the hell is first tab space smaller then the second tab space ? This was the question which was making my mind go nuts. After looking up for the issue for about 2 hrs or so I finally got the answer to the problem and it lies in the way TAB space works.

Tab space has different interpretations and these different interpretations are the reason for all the confusion I had in mind. Traditionally tab character meant "move to the right until the current column is a multiple of 8.'' In many other implementations like Windows and MAC editors the interpretation is same, except that multiples of 4 are used instead of multiples of 8. Yet another interpretation for ASCII character is to mean "indent to next tab stop", where tab stops are set arbitrarily, they might not be equally distanced from each other.

Generally tabs are used for indentation purposes and considering the confusion we've with regards to it's implementation, it's advisable to use White Space for this purpose.

What's with time ?

2009-02-09T20:31:00.002+05:30

Many a times I used to be confused with questions which referred to time. For e.g. a very common question which is asked in many competitive examinations is " If it's 11 o' clock now, then what'll be the time after 16 hours ?" Now this is a fairly simple problem but I don't know why, I always used to get my stomach turn upside down, having seen such problems. However today I found a nice way to tackle such problems. My approach lies with modular arithmetic. We use a clock cycle of 12 hrs so after every 12 hr the clock returns to 1 and so on. So we just add up 11 and 16 and get 27 (11+16=27) and finally we take 27mod(12) which is equal to 3. So after 16 hrs, clock will be indicating 3 o' clock. Another approach couldv'e been to take 16mod(12)=4, add 11 and 4 = 15 and take 15mod(12) = 3.

Breaking Chocolate Bars

2009-02-07T13:54:00.004+05:30

I still remember this problem which was asked by one of the Profs during my interview at IIT Delhi. The problem is very simple to be stated and goes as follows:

Problem:

Assume you have a chocolate bar consisting, as usual, of a number of squares arranged in a rectangular pattern. Your task is to split the bar into small squares (always breaking along the lines between the squares) with a minimum number of breaks. How many will it take?

And I still remember that I said that answer for a 4*3 standard Chocolate bar should be 11. However I was too lazy to prove that the answer is indeed correct. And today when one of my cousin posed the very same question I think I've found the answer to the problem.

Solution:

We can solve the problem using concept of Principle of Mathematical Induction.

1. If we have a a chocolate bar with 2 squares then it will require just 1 break.
2. Let's assume that a chocolate bar with m squares needs m-1 breaks.

3. Let us break the chocolate bar with N squares into two bars each with x and y squares respectively. Now by our previous hypothesis, we'd require x-1 breaks to split the chocolate bar with x squares and for chocolate bar with y squares we'd need y-1 breaks. Thus total number of breaks to split apart chocolate bar with N squares = (x-1) + (y-1) + 1
= (x+y) - 1 = N-1

Thus we'd require N-1 breaks to split apart a chocolate bar with N squares to individual squares.

Resizing ext3 partition using parted

2009-02-03T15:23:00.003+05:30

Got a chance to partition a ext3 partition which was hosting / on it today. As is the case usually I just couldn't find any Live CD which was having qtparted or kparted on it :) So now I was back to command line and parted was the utility which I could use to resize an exisiting partition. Looking from a layman's view it seems very easy to resize an existing ext3 partition but it isn't so when we do a hands on job.

Here are the steps one needs to follow in order to resize an existing ext3 partition using parted:

1. Check for partition features using debugfs

# debugfs -R "feature"

2. Generally available features are: has_journal ext_attr resize_inode dir_index filetype large_file and sparse_super.

3. parted requires us to remove some of the features from the filesystem, especially has_journal, ext_attr, resize_inode, dir_index. We'll use tune2fs and debugfs to remove these features from our filesystem.

4. Following set of commands will remove the listed features from our filesystem:

# tune2fs -O^has_journal
# debugfs -w -R "feature -ext_attr -resize_inode -dir_index"

5. After issuing above commands we'd be required to run e2fsck. Supply -y option to e2fsck to allow e2fsck to finish the job non-interactively.

# e2fsck -y

6. Now comes the resizing part. We can either issue the command directly or use interactive shell provided by parted. Interactive shell is easier to use.

# parted (parted) resize

One can get the start size by issuing print command in parted interactive shell.

And now all we need to do is wait for resizing operation to complete, and as my experience says "Keep burger and coffee handy :) "

7. Once partition resizing is done, we need to re-enable the filesystem features that we'd disabled earlier. Following commands will do the task for us:

# debugfs -w -R "feature ext_attr resize_inode dir_index"
# tune2fs -O has_journal

Search For String in Multiple Files

2009-02-03T13:24:00.002+05:30

Today my boss was stuck with a peculiar problem. He wanted to search for a name in some text files which were arranged in few folders. Now on first look it seems pretty easy and I advised him to use File Search utility which comes with Windows OS. However to my surprise I couldn't utilize "A word or phrase in the file" search criterion and found out this is an issue which has been problematic for quite some time now. It happens because a filter component is not registered for the file type that contains the text that we specified, or the filter component ignored the text that we specified. There is an entire procedure to be followed if we want this functionality to work properly with Windows XP (This was not a problem with earlier versions of Windows). Now having been ditched by search facility available with Windows XP, there was no other way but to resort to good age old method of performing search using some command line utility. There is a wonderful utility available with Windows and it has been available since Windows NT 4.0. The utility is known as findstr and it allows for searching string(s) in file(s). The syntax of command is as follows:

findstr [/b] [/e] [/l] [/c:string] [/r] [/s] [/i] [/x] [/v] [/n] [/m] [/o] [/g:file] [/f:file] strings files

Power of Geometric Series !

2009-01-31T22:30:00.002+05:30

While watching lectures on Geometric Progression I found this wonderful problem which touches upon our day to day life to an extent. The problem goes as follows:

Suppose we manage to win a lottery of say Rs. 1,00,000 but due to exisiting norms we've to pay 1/3 of the amount as tax to Government. Now suppose the lottery company is willing to pay for the tax amount, then how much should it actually pay us, so that we can take back home exactly Rs. 1,00,000, after paying the tax to Government ? The key point here to note is that every amount being paid to us by lottery company is being taxed by Government.

The answer is Rs. 1,50,000.

Why some people can't put 2 and 2 together ?

2009-01-27T20:34:00.002+05:30

Here is a nice article on dyscalculia which is a form of mental disability which doesn't allow numerical ability of person to grow with age and experience.

Trip To Allahabad

2009-01-27T12:49:00.003+05:30

Recently I had an oppurtunity to visit famous city of Prayag (now known as Allahabad). I don't know for sure but I was not aware of some facts which I am mentioning in this post:

If you are visiting Allahabad for first time then do visit Sangam (it's a culmination point of Ganga, Yamuna and Saraswati). Ganga and Yamuna are distinctly visible as the color of their water is very much different, with Ganga being lot more clear then Yamuna. Saraswati however is said to be flowing underneath and hence can't be seen.
Hanuman temple at Sangam is the sole temple where one can see the deity lying. In every other temple the sculpture of Hanuman is in standing position.
The temple of Shiva at Sangam is also a must visit. It's a 3 story high temple and the top floor holds a great sculpture of Shiva Linga.
Anand Bhawan is another place worth visiting while travelling to Allahabad. This place has been converted to a Museum and one can see the house as it used to be in days when Mahatama Gandhi Ji, Pandit Jawaharlal Nehru, Mrs. Indira Gandhi and other people of immense political and historical importance used to live here. Mrs. Indira Gandhi was married to Firoz Gandhi at this very place.
The Airport at Allahabad (Bamrauli Airport) is a small airport and presently only Air India and Jetlite are two passenger carriers to this Airport. However it's quite ironic that Allahabad has distinction of hosting event of first flight in India (Click Here for more details).
The first Air Mail was also delivered from Allahabad.
One must enjoy the wonderful food at Netrams at Katra. Speciality include small size Samosas which can last for about 3-4 months.
El Chico was a good restaurant where one can enjoy some wonderful cuisine.
Bars and Pubs are hard to find at Allahabad. One can enjoy a drink or two at bar in Kanha Hotel at Civil Lines, Allahabad. However as public consumption of alcohol is not looked upon as a dignified act, so it's advisable to refrain from alcohol consumption.

Something Is Missing.

2009-01-16T10:12:00.003+05:30

I don't know why but it seems to me that something's missing inside and I am unable to point out accurately as to what that thing is. I feel like a wagabond element at times who is just wandering in the wilds of various domains of knowledge created by some poor souls of earlier centuries. Things seem to be in grasp but still they are so so far away from being getting materialized into concrete realities. Maybe I am expecting too much out of myself but it can't just be that, because deep inside, somewhere I feel that this is the right path which will finally help me realize my role in this life.

Some numerical facts

2009-01-13T11:48:00.003+05:30

I had never thought before that numbers can be so interesting. I mean for once I was really astonished to see such wonderful and interesting properties of natural numbers. So here are two such problems which have really intrigued me for quite some time now:

1. If you think of any number, say your phone number, your pin code, your car number or any other number then you can find some power of 2 where you can find this number as left most digit of that power.

2. We take any whole number n greater than 0. If n is even, we halve it (n/2), else we do "triple plus one" and get 3n+1. The conjecture is that for all numbers this process converges to 1. Hence it has been called "Half Or Triple Plus One". Actually it ends in a tail comprising of 1,4,2,1,4,2,1..... This is actually an unsolved conjecture and Thwaites (1996) has offered a £1000 reward for resolving the conjecture.

Unsuccessful Attempt To Dhak Bahiri

2009-01-04T13:00:00.014+05:30

I guess nature has it's own way of proving it's supremacy and we all found an e.g. during course of visit to Dhak Bahiri. This was second of my treks in Maharashtra's Shayadri ranges. Prior research did gave indications that it wasn't an easy to do trek, but considering my earlier experiences in Himalayas I was quite confident that we'd make it. So we (me, Nipun, Santosh, Vishal and Vaidyanathan) started of our journey by taking last local from CST to Karjat. It's a comfortable journey of around 3 hrs. By 3:30 A.M. we were at Karjat and it was nice to find chai wala at station. After some enquiry we found that first bus to Sandsi village was to start at 5:30 A.M., so after some rest at station we went off to bus stop at Karjat. There was another group as well at bus station which was going for trekking to Bheemashankar. State Transport buses are perhaps one of best things which I've seen in Maharashtra. I mean, no matter how remote the village is, one can find good frequency of buses to/from these places. Our bus had only 6 guys, our group and one other fellow. We reached Sandsi in 30 min. and it was still dark. There were few villagers who were waking up to a new morning and we found a place where a guy was sleeping in front of fire. It was comfortable to warm ourselves by sitting across fire. We finally started our trek to Dhak Bahiri at 7:00 A.M.

This is our target

It was still dark as sun had not risen past the mountains but as visibility improved to a decent level we started marching towards the mountain. We were accompanied by two dogs whom we even gave names, Julie and Kapish :) We passed through the fields at Sandsi village and it really makes me wonder why in the hell there is no provision of alternative water supply in this region. The fields were dead dry and one can easily see massive cracks in the soil. I really feel for the farmer community living in such hardships and still trying to find means to grow crops for people like us. After we'd crossed the fields we found ourselves at bank of a dry river. A villager earlier had told us to follow the river and not cross it. We tried very hard to follow his advise and started following river up towards the hill. We halted in between at a place where there was some water, to get through with daily chores. It was really nice to be in nature's abode and it really feels strange and nice at times to me to be in such remoteness. As we moved along the jungle became more dense by every step and there was a point beyond which we couldn't find the path ahead. Boy, we are lost !!! And to add to the thrill were sounds coming from hill above us, which were initially reported by some of us as to be being made by some bear but in reality they were some monkeys howling :) The dogs too were looking skeptical and we finally decided to return back. As we were moving back Santosh found another path and dogs too looked much more at ease on that path and we decided to stick to it, and voila within few min. we were on right track which was also marked by some white arrows on rocks. Within few minutes the hike also began and I must admit it was not an easy one and reminded me of hike I made during my treks in Uttaranchal (especially Loharkhet to Dhakuri during my visit to Pindari). After hiking for about 2 hrs we finally landed at a place where there was a plain surface and we decided to have our breakfast there. It was strange to see the dogs not taking any food (God knows why in the hell they accompanied us !). We found few guys from some nearby village wandering there and they told us that they were in search of some bird prey. This reminded me of my childhood days when once I went for hunting in Himalayas with my Grandpa (I really miss those good old days :( and dearly wish them to come back). Those guys told us that it was just 45 min walk from that point to Bahiri caves and this brought in a new wave of enthusiasm in our life. However this enthusiasm was to last for a very short time period as we again lost our way during the trek and landed up at another big field near by the point where we had our breakfast. Some asshole had even placed a railway ticket from Lonawala to Pune along that path as a marker. This was really irritating as we were unable to find our way out. So again with no other option we'd to back trek and Vaidy who was in front was lucky enough to find the way out to right trek. The journey ahead to top from this point onwards was filled with fun and we passed through some dense jungles and a point where we had to hike up on a rock which was inclined at almost 65-70 degree angle. Vishal was dead tired by then and so were we all. Finally we managed to reach the place they call Naali (It's a narrow passage between two hills). Nipun as he always is was over enthusiastic and was running all over the places to get caught on camera. Rest of us were a bit tired and I was really beginning to worry about how will we cross this passage with our luggage on our back. However in God's name we all marched ahead and what the hell it was a dead end after this passage. I mean all I could see was a feet long path down and we'd to move down by almost 2-3 feet from top to that place. Even if we were to come down, I could see that there was no way we could climb back. But as has always been the case with guys, we couldn't accept this fact and decided to move ahead. Nipun was first one to go down. We decided to give our rucksacks to him so that we could move down without any luggage on our backs and suddenly I heard from Santosh that one of the bags has toppled down the hill. What the hell !!!! And to add to agony it was my bag which had moved down. We all were in deep trouble now, as all the food for night was in that bag and I was in much big trouble as my cellphone and wallet were also in that very bag. Anyhow we all finally came down and took our luggage to a place which we considered safest. Nipun decided to go down and have a look (It was really brave of him to do so). Meanwhile Vaidy moved ahead and decided to see if he could find some helping hand. Nipun wasn't able to move much deep down the hill and it was very wise of him too. Suddenly we found few guys coming down from Bahiri caves towards us. These guys were trekkers from Pune and they were also carrying ropes with them. They agreed to help us out but even they could not move down very deep due to geography of the hill. While 2 of the guys from Pune were down, one of other guys from their group asked us, if we had ever come before to this place. And when we told him that this was our first trek ever in such remote areas in Sahayadris, he posed a real surprised look on his face. When we asked for the reason, he told us something, which made all of us look dumb silly and stupid and also brave at same time. He told us that this was one of the toughest treks in Sahayadris and almost 100 people had lost their life during the trek. This brought shiver down spine of some of our group members but real trouble right now was to find the bag. With failure of the trekkers from Pune, now our only help rested on some villager from Sandsi village. And luckly for us, one group from Mumbai had also come up to Bahiri and they were accompanied by a local villager from Sandsi. One guy from Pune agreed to help us out and he went to Bahiri caves and talked to the villager. Now all we had to do was to sit in chilling heat and wait for villager to come down. When he finally met us, he agreed to help us out but now we had a situation as we couldn't complete our trek, cauz we had to show the group which had come with villager the path down to Sandsi. Anyhow we decided to go ahead with it and moved back. Travelling back was even tougher then climbing up as, sand on the hills is really slippry and one has to be really cautious while moving down. Anyhow after a long and tireing journey down we finally managed to meet the local villager and I was glad to find my wallet back. Sadly my cellphone was lost but I belive everything happens for a good cause and we are going out to complete this Mission to reach caves of Dhak Bahiri again in coming holiday.

Our final target

Dangerous Knowledge

2008-12-26T18:24:00.002+05:30

One of few good videos about people like Cantor, Turing, Godel and others can be found at http://video.google.com/videoplay?docid=-5122859998068380459.

Plotting Graphs

2008-12-26T17:17:00.003+05:30

Today after running program for Collatz conjecture for all integers (Visit Wolfram Mathworld for more details) , I thought of generating a graph which could help me understand how the program was behaving with respect to different numbers. This was the first time though when I wanted to plot a curve for some data (so unscientific of me !). So here I was with my data file containing 2 columns. First column contained input ranging from 2-32766 and second column contained steps taken by algorithm to reach 1 for every number. To my surprise I found myself in a helpless situation for once, as I couldn't think of any software which could suite my needs. Googling a few times brought me to GNUPlot and I must admit, it's one of the really excellent softwares as far as graph plotting is concerned. One can give it a formula or even a data set separated by tab space/blanks/or any other delimiter. Here is the graph I got for my data: