comm: another way to diff

I recently had a need to compare two files of two different versions.
This is usually what you use <enter your favorite diff tool here> for.
However, these files had similar entries (lines) in different places in the file, and most diff tools are not smart enough to cope with that. What I wanted to do is to eliminate those and to be able to compare only the relevant changes.

These specific files had some irrelevant lines (read: comments), which (conveniently enough) begins with a '#' character. So first I stripped the comments from both files:

grep -Ev '#' .config > .config_no_comments

Now that we have two files that contain only the changes we're interested in, we can use comm to compare them.
comm can be used on sorted files only, so we need to sort our two files:

sort -o .config_no_comments_sorted .config_no_comments

Now, suppose I have those two files and I want to see only the unique lines that file1 has (that are not present in file2), I would do:

comm -23 file1 file2

If I want to see only the unique lines that file2 has (that are not present in file1), I would do:

comm -13 file1 file2

And this will output the lines that appear in both files:

comm -12 file1 file2

RTFM for more options.

Valgrind for MIPS

Support for the MIPS architecture was recently added to Valgrind.

If you want to build Valgrind for your MIPS target, you should definitely refer to the README.mips file, but I wanted to outline the general steps it took to me to successfully build and run Valgrind:

1. Get the latest source code - support for MIPS was just recently integrated into the main tree, and currently no official release support it:
   svn co svn://svn.valgrind.org/valgrind/trunk valgrind
2. cd into the valgrind directory.
3. ./autogen.sh
4. ./configure --host=mipsel-linux-gnu --with-pagesize=4 CC=my-mips-xccompiler CXX=my-mips-xcppcompiler CFLAGS="-I/path/to/target/kernel -I/path/to/target/kernel/include -D__STRUCT_EXEC_OVERRIDE__ --prefix=/path/to/installation/dir
5. make
6. make install

Some notes on step [4]:

• --host=mipsel-linux-gnu - mipsel is for little endian architecture and mips is for big endian.
• with-pagesize=4 - this should be set to whatever your kernel's PAGE_SIZE is configured.
• -D__STRUCT_EXEC_OVERRIDE__ - not sure why, but my kernel is missing the a.out.h in the arch/mips directory, which is needed by Valgrind. I had to define this so Valgrind will not include a.out.h.
• --prefix - you want to set this up - this will create three directories - /bin, /lib and include, which you want in your target's root filesystem.

Before you can run Valgrind on the target, you'll need to issue the following command:

export VALGRIND_LIB=/lib/valgrind

That's it, your'e all set - type 'valgrind --help' for all available options.

Adding a new service to Fedora 16

Suppose that you have a new service that you want to control with the 'service' command, or that you want to start every time the system boot.
This post explains how to do it in a few simple steps.

Step 1 - Create a service script
Create a file under /etc/init.d, let's call it 'myservice', and add the following to it:

#!/bin/bash
#
# chkconfig: 1000 50 50
#
# description: myservice service
#

start() {
  /usr/share/myservice/start.sh
}

stop() {
  /usr/share/myservice/stop.sh
}

restart() {
  stop
  start
}

case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
*)
echo $"Usage: $0 {start|stop|restart}"
exit 1
esac

exit 0

The 'chkconfig' and 'description' lines are mandatory for the script to be used with the chkconfig tool.

From chkconfig's man page:

For example, random.init has these three two  lines:
       # chkconfig: 2345 20 80
       # description: Saves and restores system...
This  says  that the random script should be started in levels 2, 3, 4, and 5, that its start priority should be 20, and that its stop priority should  be  80.

Don't forget to make the script executable:

sudo chmod +x /etc/init.d/myservice

Step 2 - Using the script

sudo service myservice start
sudo service myservice stop
sudo service myservice restart

Step 3 - Run the script automatically at boot time (optional)

sudo systemctl enable myservice.service

Mississippi John Hurt

I've recently came across the amazing story of Mississippi John Hurt.
Reading this guy's Wikipedia entry, I've literally WTFed every other line; he was quite extraordinary.

To try and sum it all in one paragraph, he was a gifted bluesman and an extraordinary guitar player, who actually invented a new style of guitar-playing back in the 1920s.
After several commercial failures, he abandoned the music career and became a farmer, only to be discovered 30(!) years later in the early 1960s.
After this discovery, he was all the rage for a couple of years, until he died in 1966.

This is one of my favorite songs from this wonderful artist:

Here you can see that he was also a real character:

I think that the simplicity of his voice and the laid back guitar-playing style is a winning combination, making him my current favorite artist*.


* Disclaimer: this may vary daily.

How to make Django play with apache

So, I actually haven't written any blog entry since my decision to do so 4 years ago.
Banging my head on my desk for a couple of hours, trying to understand wtf is going on with django and apache, actually made me decide to write my first entry today.
So here goes.

First thing first - why would you want to use apache HTTP web server with Django?
Two main reasons:

1. Apache is one of the most successful and professional open source community, and their HTTP web server is one of the most powerful and used web servers in the world.
2. Django's integrated web-server is intended to be used only in the development phase.

I was under the impression that hooking apache in to replace Django's web server is going to be easy.
I was wrong.

There are a couple of resources out there on doing this, but none helped me solve this; they all just provided another piece of the puzzle.
I'm sure that you can google and find all the stuff that I have, so I'll just skip the references part, and just write what I did to make it work.

Step 1 - install the needed software
For me, this meant installing apache (obviously) and mod_wsgi.
I'm using Fedora 16 as my workstation, so this was fairly easy:

sudo yum install httpd
sudo yum install mod_wsgi

At one point I was so frustrated from not being able to get this thing going, that I actually tried going with mod_python, only to find out later that it's not supported anymore.
I'm stating this just because I'm not sure if installing mod_python has something to do with the all thing working, but just in case, I also did:

sudo yum groupinstall "Web Server"
sudo yum install mod_ssl mod_python

Again, I don't think that this is a necessary step.

Step 2 - configure apache
In Fedora, apache's configuration file is /etc/httpd/conf/httpd.conf.
Aside from the usual basic configuration that you may want to apply, the most notable change that I had to do is adding mod_wsgi to the list of loaded modules:

LoadModule wsgi_module modules/mod_wsgi.so

You need to restart apache after saving your changes:

sudo service httpd restart

Step 3 - RTFM
Read Django's documentation on how to make apache work with Django.
Of course that this is not enough, or else I wouldn't have to write this post.

After reading some security-related articles, I've decided to put my django.wsgi in a new directory - /usr/local/www/wsgi-scripts/.
I've added this path to httpd.conf, like the documentation indicates:

WSGIScriptAlias / /usr/local/www/wsgi-scripts/django.wsgi

My django.wsgi looks like this:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

path = '/home/apache/myproject'
if path not in sys.path:
sys.path.append(path)

If you've followed Django's documentation, you might notice that I have two things that are different than the ones indicated there.
These two things are what made my setup eventually work.

The first thing is the line that set the DJANGO_SETTINGS_MODULE environment variable.
The documentation states that 'mysite.settings' should be used.
I tried wrapping my head around this mysite.settings thing, but I just couldn't understand what does it mean - is mysite should be replaced by my own site's name? Is it just a constant that should be used as-is?
Let me make things short - I still don't know why 'mysite' is being used in the example, but what worked for me was just 'settings'.
After trying to understand if 'settings' have anything to do with the file 'settings.py' in your site's directory, I've come to a conclusion that this "special" settings file is being generated by Django when it's starting to run, by using the information from the file 'settings.py'.

The other thing that is different, is the path to the application code.
I've giving up trying to put the code under my user's name, since apache kept shouting that it can't find stuff (even though my python path variable was set appropriately). I've came to a conclusion that this is a permission problem, and being the n00b system-admin that I am, I've decided to solve this by creating a home directory for the apache user.
When you install apache (httpd), an 'apache' user and an 'apache' group is created in the system, for all the web-servicing purposes.
I've simply copied my code into apache's home directory and changed the ownership to apache:

sudo cp -r /home/tomer/code/myproject /home/apache/
sudo chown -R apache /home/apache/myproject
sudo chgrp -R apache /home/apache/myproject

Tips and tricks

• You should always have apache's log tailed, so you can see all the error messages it outputs. In Fedora, the log files are by default in /etc/httpd/logs, so you should:
  tail -f /etc/httpd/logs/error_log
• After moving the files to apache's home directory, you might need to remove all the .pyc files and re-run the app, so new settings (if there are any) will take effect.

I think that Django's documentation is top-notch. However, in this case, I felt like the specific subject was left for the more advanced sys-admins.
I hope that this article will help all the other n00bs out there.
Or at least one, cuz I know I'm not the only one... :)

Hello, world

First post ever!

PS - Thanks mom, for reading this.