libmesa conflicts with Gnome 3 on Fedora 18 – run into black screen after libmesa update

Posted on June 4, 2013 by daveti

A recent update on libmesa caused my Fedora 18 running into black screen during displaying the login screen. Note that Fedora 18 is using Gnome 3 as the default login desktop before you could switch to others, like my favorate MATE. Unfortunately, ‘yum history undo’ was unable to downgrade the libmesa (even in init 3 mode) for certain reasons. After a few days struggling, I find a workaround to bypass this issue before libmesa/Gnome could realize and fix it. NOTE again, this workaround  needs the other desktop instead of Gnome 3 (in my case, I have MATE installed).

1. Fedora 18, kernel 3.9.4, libmesa version…
[root@daveti daveti]# uname -a
Linux daveti.me 3.9.4-200.fc18.x86_64 #1 SMP Fri May 24 20:10:49 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@daveti daveti]# yum history info 142
Loaded plugins: auto-update-debuginfo, langpacks, presto, refresh-packagekit
Transaction ID : 142
Begin time     : Sat Jun  1 00:16:12 2013
Begin rpmdb    : 2779:e709b9f553182476eadaa5bdf69ad710af96301b
End time       :            00:17:28 2013 (76 seconds)
End rpmdb      : 2780:d845c48ab457707b0d33a46f18707a810d652f97
User           : root,DES 224,541-953-0921,541-953-0921 <root>
Return-Code    : Success
Command Line   : update
Transaction performed with:
Installed     rpm-4.10.3.1-1.fc18.x86_64                              @updates
Installed     yum-3.4.3-54.fc18.noarch                                @updates
Installed     yum-metadata-parser-1.1.4-7.fc18.x86_64                 @anaconda
Installed     yum-plugin-auto-update-debug-info-1.1.31-10.fc18.noarch @updates
Installed     yum-presto-0.9.0-1.fc18.noarch                          @anaconda
Packages Altered:
Updated     eclipse-changelog-2:2.8.0-4.fc18.noarch                     @updates
Update                        2:2.8.0-8.fc18.noarch                     @updates
Updated     eclipse-linuxtools-1.2.0-1.fc18.noarch                      @updates
Update                         1.2.0-7.fc18.noarch                      @updates
Updated     eclipse-oprofile-1.2.0-1.fc18.noarch                        @updates
Update                       1.2.0-7.fc18.noarch                        @updates
Updated     eclipse-rpm-editor-1.2.0-1.fc18.noarch                      @updates
Update                         1.2.0-7.fc18.noarch                      @updates
Updated     eclipse-rpmstubby-1.2.0-1.fc18.noarch                       @updates
Update                        1.2.0-7.fc18.noarch                       @updates
Updated     eclipse-valgrind-1.2.0-1.fc18.noarch                        @updates
Update                       1.2.0-7.fc18.noarch                        @updates
Updated     ibus-anthy-1.5.0-1.fc18.x86_64                              @updates
Update                 1.5.3-1.fc18.x86_64                              @updates
Updated     khrplatform-devel-9.1-3.fc18.noarch                         @updates
Update                        9.2-0.7.20130528.fc18.noarch              @updates
Updated     libtevent-0.9.17-1.fc18.x86_64                              @anaconda
Update                0.9.18-1.fc18.x86_64                              @updates
Updated     libxml2-2.9.1-1.fc18.i686                                   @updates
Updated     libxml2-2.9.1-1.fc18.x86_64                                 @updates
Update              2.9.1-1.fc18.1.i686                                 @updates
Update              2.9.1-1.fc18.1.x86_64                               @updates
Updated     libxml2-devel-2.9.1-1.fc18.x86_64                           @updates
Update                    2.9.1-1.fc18.1.x86_64                         @updates
Updated     libxml2-python-2.9.1-1.fc18.x86_64                          @updates
Update                     2.9.1-1.fc18.1.x86_64                        @updates
Updated     llvm-libs-3.2-2.fc18.x86_64                                 @updates
Update                3.3-0.4.rc2.fc18.x86_64                           @updates
Updated     mesa-dri-drivers-9.1-3.fc18.x86_64                          @updates
Update                       9.2-0.7.20130528.fc18.x86_64               @updates
Obsoleted   mesa-dri-filesystem-9.1-3.fc18.x86_64                       @updates
Obsoleting  mesa-filesystem-9.2-0.7.20130528.fc18.x86_64                @updates
Updated     mesa-libEGL-9.1-3.fc18.i686                                 @updates
Updated     mesa-libEGL-9.1-3.fc18.x86_64                               @updates
Update                  9.2-0.7.20130528.fc18.i686                      @updates
Update                  9.2-0.7.20130528.fc18.x86_64                    @updates
Updated     mesa-libEGL-devel-9.1-3.fc18.x86_64                         @updates
Update                        9.2-0.7.20130528.fc18.x86_64              @updates
Updated     mesa-libGL-9.1-3.fc18.i686                                  @updates
Updated     mesa-libGL-9.1-3.fc18.x86_64                                @updates
Update                 9.2-0.7.20130528.fc18.i686                       @updates
Update                 9.2-0.7.20130528.fc18.x86_64                     @updates
Updated     mesa-libGL-devel-9.1-3.fc18.x86_64                          @updates
Update                       9.2-0.7.20130528.fc18.x86_64               @updates
Updated     mesa-libgbm-9.1-3.fc18.i686                                 @updates
Updated     mesa-libgbm-9.1-3.fc18.x86_64                               @updates
Update                  9.2-0.7.20130528.fc18.i686                      @updates
Update                  9.2-0.7.20130528.fc18.x86_64                    @updates
Updated     mesa-libglapi-9.1-3.fc18.i686                               @updates
Updated     mesa-libglapi-9.1-3.fc18.x86_64                             @updates
Update                    9.2-0.7.20130528.fc18.i686                    @updates
Update                    9.2-0.7.20130528.fc18.x86_64                  @updates
Updated     mesa-libxatracker-9.1-3.fc18.x86_64                         @updates
Update                        9.2-0.7.20130528.fc18.x86_64              @updates
Updated     opencc-0.4.0-1.fc18.x86_64                                  @updates
Update             0.4.3-1.fc18.x86_64                                  @updates
Updated     selinux-policy-3.11.1-96.fc18.noarch                        @updates
Update                     3.11.1-97.fc18.noarch                        @updates
Updated     selinux-policy-devel-3.11.1-96.fc18.noarch                  @updates
Update                           3.11.1-97.fc18.noarch                  @updates
Updated     selinux-policy-doc-3.11.1-96.fc18.noarch                    @updates
Update                         3.11.1-97.fc18.noarch                    @updates
Updated     selinux-policy-targeted-3.11.1-96.fc18.noarch               @updates
Update                              3.11.1-97.fc18.noarch               @updates
Updated     xorg-x11-drv-ati-7.0.0-0.9.20121015gitbd9e2c064.fc18.x86_64 @updates
Update                       7.1.0-5.20130408git6e74aacc5.fc18.x86_64   @updates
Updated     xorg-x11-drv-intel-2.21.6-1.fc18.x86_64                     @updates
Update                         2.21.8-1.fc18.x86_64                     @updates
Dep-Install xorg-x11-glamor-0.5.0-5.20130401git81aadb8.fc18.x86_64      @updates
history info
[root@daveti daveti]#

2. Workaround – change the default desktop of ‘startx’ (from Gnome 3 to MATE)
a. Ctrl-Alt-F2: force into init3 mode
b. cd /etc/X11/xinit
c. Modify Xclients (with the changes below)
#daveti: make mate as the default
#GSESSION=”$(type -p gnome-session)”
GSESSION=”$(type -p mate-session)”
STARTKDE=”$(type -p startkde)”

# check to see if the user has a preferred desktop
PREFERRED=
if [ -f /etc/sysconfig/desktop ]; then
    . /etc/sysconfig/desktop
    if [ “$DESKTOP” = “GNOME” ]; then
        PREFERRED=”$GSESSION”
    elif [ “$DESKTOP” = “KDE” ]; then
        PREFERRED=”$STARTKDE”
    fi
fi

if [ -n “$PREFERRED” ]; then
    exec “$PREFERRED”
fi

# now if we can reach here, either no desktop file was present,
# or the desktop requested is not installed.

if [ -n “$GSESSION” ]; then
    # by default, we run GNOME.
    exec “$GSESSION”
elif [ -n “$STARTKDE” ]; then
    # if GNOME isn’t installed, try KDE.
    exec “$STARTKDE”
fi
d. startx!

Posted in OS | Tagged , , , , | 2 Comments

excel2csv – Parse Excel file into CSV files / Parse CSV files into CSV data structure

Posted on June 4, 2013 by daveti

excel2csv is a tool used to read an Excel file in and convert it into a bunch of CSV files or just read a CSV file and parse it into the CSV data structure which could be used by upper layer application. The design and implementation of excel2csv is simple and stupid, feel free to make it part of your project!

Design/Implementation
Excel2csv.java – a static main class/function used to read an Excel file and convert it into few CSV files
CsvParser.java – a CSV parser (to parse the CSV file into CSV data structure/class)
Csv.java – Csv data structure
CsvColumn – Csv column data structure

Project Name: excel2csv
Destination: Excel/CSV Parser
Language: Java
IDE: NetBeans 7.2
Library: jExcel/jxl
Project Web: https://github.com/daveti/excel2csv
Git Read Only: https://github.com/daveti/excel2csv.git

Posted in Dave's Tools | Tagged , , , , | Leave a comment

iptables, firewalld and Fedora 18

Posted on May 19, 2013 by daveti

After setting up httpd in my Fedora 18, I was not able to get access to my own web server. I thought it was related with firewall within the local network, like port 80 being blocked. However, thanks to Mingwei (Daniel), we find the root cause is the default firewall within Fedora 18 – iptables. Further more, it is unable to tune the iptables in Fedora 18 (there is even no configuration file for iptables). Yes, Fedora 18 is using a new thing called firewalld to take control of all the things happened in iptables. This post is providing hints about the old iptables and the new firewalld things. May it help.

1. iptables

The reason why port 80 is blocked by default in Fedora 18 is the rules in iptables. Run ‘iptables -L‘, you should be able to see all the rules configured in your system. If you are using non-Fedora 18, like CentOS 6.4, you should be able to find the configuration file ‘/etc/sysconfig/iptables‘, where you could add rules to unblock ports.

# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
# daveti: add nat table
*nat
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp –dport 80 -j REDIRECT –to-port 8080
-A PREROUTING -p udp -m udp –dport 80 -j REDIRECT –to-port 8080
COMMIT
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 22 -j ACCEPT
# Open port for http
-A INPUT -m state –state NEW -m tcp -p tcp –dport 80 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 8080 -j ACCEPT
# Open port for VNC 1
-A INPUT -m state –state NEW -m tcp -p tcp –dport 5801 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 5901 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 6001 -j ACCEPT
# Open port for VNC 2
-A INPUT -m state –state NEW -m tcp -p tcp –dport 5802 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 5902 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 6002 -j ACCEPT
-A INPUT -j REJECT –reject-with icmp-host-prohibited
-A FORWARD -j REJECT –reject-with icmp-host-prohibited
COMMIT

The green parts are used to open port 80 and 8080. Generally, ports 80 and 8080 need to be opened within the *filter rules. Moreover, if http request forwarding (like from 80 to 8080, needed by Tomcat) is needed, *nat rules need to be created. The blues parts are used to opens 2 sets of VNC ports, including java client, vnc-server and vnc client. Before and after the changes, ‘service iptables stop’ and ‘service iptables start’ are needed to make the newly added rules work.

The good thing of iptables is the power to create complicated rules to provide the ability of firewall and security. However, the bad thing, as you can tell, is the awkward way to write down these rules.

2. firewalld, firewall-cmd

In Fedora 18, iptables is replaced or taken control by firewalld, which tends to provide a friendly interface to tune the iptables. firewalld is the demaeon process and firewall-cmd is the cmd interface used to talk with firewalld. Run ‘firewall-cmd –list-all-zones‘, you will find something like ‘iptables -L’ but firewalld provides the idea of ‘zone’. The way to open port 80 and enable VNC in firewalld is like below:

firewall-cmd –permanent –zone=public –add-service=http
firewall-cmd –permanent –zone=public –add-service=vnc-server

Seems not only the famous ports (1-1024) are combined with service name directly but also some other not that ‘famous’ ports are binded with service name. Actually, the definition of service for firewalld are written in xml under ‘/usr/lib/firewalld/services‘

[root@daveti services]# pwd
/usr/lib/firewalld/services
[root@daveti services]# ll
total 116
-rw-r—–. 1 root root 374 Apr  3 03:37 amanda-client.xml
-rw-r—–. 1 root root 320 Apr  3 03:37 bacula-client.xml
-rw-r—–. 1 root root 346 Apr  3 03:37 bacula.xml
-rw-r—–. 1 root root 455 Apr  3 03:37 cluster-suite.xml
-rw-r—–. 1 root root 338 Apr  3 03:37 dhcpv6-client.xml
-rw-r—–. 1 root root 346 Apr  3 03:37 dns.xml
-rw-r—–. 1 root root 374 Apr  3 03:37 ftp.xml
-rw-r—–. 1 root root 448 Apr  3 03:37 https.xml
-rw-r—–. 1 root root 353 Apr  3 03:37 http.xml
-rw-r—–. 1 root root 372 Apr  3 03:37 imaps.xml
-rw-r—–. 1 root root 454 Apr  3 03:37 ipp-client.xml
-rw-r—–. 1 root root 427 Apr  3 03:37 ipp.xml
-rw-r—–. 1 root root 517 Apr  3 03:37 ipsec.xml
-rw-r—–. 1 root root 385 Apr  3 03:37 libvirt-tls.xml
-rw-r—–. 1 root root 389 Apr  3 03:37 libvirt.xml
-rw-r—–. 1 root root 424 Apr  3 03:37 mdns.xml
-rw-r—–. 1 root root 324 Apr  3 03:37 nfs.xml
-rw-r—–. 1 root root 389 Apr  3 03:37 ntp.xml
-rw-r—–. 1 root root 335 Apr  3 03:37 openvpn.xml
-rw-r—–. 1 root root 357 Apr  3 03:37 pop3s.xml
-rw-r—–. 1 root root 446 Apr  3 03:37 radius.xml
-rw-r—–. 1 root root 384 Apr  3 03:37 samba-client.xml
-rw-r—–. 1 root root 461 Apr  3 03:37 samba.xml
-rw-r—–. 1 root root 550 Apr  3 03:37 smtp.xml
-rw-r—–. 1 root root 463 Apr  3 03:37 ssh.xml
-rw-r—–. 1 root root 393 Apr  3 03:37 telnet.xml
-rw-r—–. 1 root root 301 Apr  3 03:37 tftp-client.xml
-rw-r—–. 1 root root 437 Apr  3 03:37 tftp.xml
-rw-r—–. 1 root root 475 Apr  3 03:37 vnc-server.xml
[root@daveti services]# cat http.xml
<?xml version=”1.0″ encoding=”utf-8″?>
<service>
  <short>WWW (HTTP)</short>
  <description>HTTP is the protocol used to serve Web pages. If you plan to make your Web server publicly available, enable this option. This option is not required for viewing pages locally or developing Web pages.</description>
  <port protocol=”tcp” port=”80″/>
</service>
[root@daveti services]# cat vnc-server.xml
<?xml version=”1.0″ encoding=”utf-8″?>
<service>
  <short>Virtual Network Computing Server (VNC)</short>
  <description>A VNC server provides an external accessible X session. Enable this option if you plan to provide a VNC server with direct access. The access will be possible for displays :0 to :3. If you plan to provide access with SSH, do not open this option and use the via option of the VNC viewer.</description>
  <port protocol=”tcp” port=”5900-5903″/>
</service>
[root@daveti services]#

Reference:

http://www.cyberciti.biz/faq/linux-iptables-firewall-open-port-80/
http://serverfault.com/questions/231381/iptables-forward-port-80-to-port-8080
http://fedoraproject.org/wiki/Features/firewalld-default
http://serverfault.com/questions/478148/fedora-linux-18-firewalld-blocking-all-ports-after-firewall-cmd-enable

Posted in Network | Tagged , , , | 2 Comments

Add a new syscall into Linux kernel – for fedora 18 with kernel 3.8.7

Posted on April 17, 2013 by daveti

Adding a new syscall into Linux kernel would be tricky for the beginners (like myself), especially considering the kernel source tree structure changes among different versions. It seems most of the online tutorials focusing on older versions of kernel (2.6.X) or early version of 3.X. Things are little bit different using the latest version of kernel (3.8.7, and I have no idea from which version did all these changes happen…). Anyway, we will try to compare the older way and the latest way to add a new syscall into the kernel. Good luck! (NOTE: this post is for x86 arch!)

A. Older version of kernel (2.6.X) or early version of 3.X
This post (http://arvindsraj.wordpress.com/2012/10/05/adding-hello-world-system-call-to-linux/) actually gives a great and detailed reference on how to do that. Summary is listed down below.

Step 0: Define the new syscall (sys_hello())
Make a new directory called ‘hello’ under the kernel directory and create the source file for syscall sys_hello() as well as its makefile.
file:
hello/hello.c
add:

#include <linux/kernel.h>
asmlinkage long sys_hello(void)
{
    printk("Hello worldn");
    return 0;
}
file:
hello/Makefile
add:
obj-y := hello.o
Step 1: Add the hello directory into kernel Makefile
file:
Makefile
original:
core-y          += kernel/ mm/ fs/ ipc/ security/ crypto/ block/
after:
core-y          += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/

Step 2: Add the new syscall into syscall table (Assembly file actually)
file:
arch/x86/kernel/syscall_table_32.S (for x86-64 arch, the file should be syscall_table_64.S)
add:
.long sys_hello

Step 3: Add the syscall number (macro definition eventually and the number varies)
file:
arch/x86/include/asm/unistd_32.h
add:
#define __NR_hello 349
add:
#define NR_syscalls 350
file:
arch/x86/include/asm/unistd_64.h
add:

#define __NR_hello 312
 __SYSCALL(__NR_hello, sys_hello)

Step 4: Add the new syscall into syscall header file
file:
include/linux/syscalls.h
add:
asmlinkage long sys_hello(void);

Step 5: Compile, update the kernel and run the testing file (using dmesg to check the kernel log)
file:
$HOME/sysHello.c
add:

#include <stdio.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#include <unistd.h>
#define __NR_hello 312 //349 if you are running a 32bit kernel and following my tutorial
long hello_syscall(void)
{
    return syscall(__NR_hello);
}
int main(int argc, char *argv[])
{
    long int a = hello_syscall();
    printf("System call returned %ldn", a);
    return 0;
}
B. Linux kernel 3.8.7

Step 0: Define the new syscall (sys_hello())
Make a new directory called ‘hello’ under the kernel directory and create the source file for syscall sys_hello() as well as its makefile.
file:
hello/hello.c
add:

#include <linux/kernel.h>
asmlinkage long sys_hello(void)
{
    printk("Hello worldn");
    return 0;
}
file:
hello/Makefile
add:
obj-y := hello.o
Step 1: Add the hello directory into kernel Makefile
file:
Makefile
original:
core-y          += kernel/ mm/ fs/ ipc/ security/ crypto/ block/
after:
core-y          += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/

Step 2: Add the new syscall as well as the number into syscall table (real table file and the number varies)
file:
arch/x86/syscalls/syscall_32.tbl
add (at the end of the file):
351    i386    daveti_hello    sys_hello
file:
arch/x86/syscalls/syscall_64.tbl
add (at the end of the file but before x32-specific syscalls):
314    common    daveti_hello    sys_hello

Step 3: Add the new syscall into syscall header file
file:
include/linux/syscalls.h
add:
asmlinkage long sys_hello(void);

Step 4: Compile, update the kernel and run the testing file (using dmesg to check the kernel log)
file:
$HOME/sysHello.c
add:
#include <stdio.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#include <unistd.h>

int main()
{
    long int rtn = syscall(351);
    printf(“System call sys_hello return %ldn”, rtn);
    return 0;
}

Posted in OS | Tagged , , , , | Leave a comment

Linux kernel linked list

Posted on April 3, 2013 by daveti

This post gives a sample file used to play with Linux kernel linked list <linux/list.h>, as well as a makefile used to build this kernel module. lsmod, insmod, rmmod and dmesg are needed. The reference is the book “Operating System Concepts” 9E (chapter 2).

[root@daveti kernel_module_list]# cat birthdayList.c
/*
 * A kernel linked list usage example with kernel module
 * Reference: OSC9E-CH2
 * April 2, 2013
 * daveti
 *
 */
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/slab.h>

#define NUM_OF_PERSON    5

/* Example struct we will use. */
typedef struct _birthday {
    int day;
    int month;
    int year;
    struct list_head list;
}birthday;

/* Declare and init the head of the linked list. */
LIST_HEAD(birthday_list);

/* This function is called when the module is loaded. */
int birthdayList_init(void)
{
           printk(KERN_INFO “Loading Modulen”);

    /* Allocate 5 birthdays from kernel */
    birthday *person;
    int i = 0;
    for(i = 0; i < NUM_OF_PERSON; i++)
    {
        /* Request malloc to the kernel. */
        person = kmalloc(sizeof(*person), GFP_KERNEL);
        /* Assign value to the struct. */
        person->day = i+1;
        person->month = i+1;
        person->year = i+1;
        /* Init the list within the struct. */
        INIT_LIST_HEAD(&person->list);
        /* Add this struct to the tail of the list. */
        list_add_tail(&person->list, &birthday_list);
    }

    printk(KERN_INFO “Display the list:n”);

    /* Go thru the list and print. */
    birthday *ptr;
    list_for_each_entry(ptr, &birthday_list, list)
    {
        printk(KERN_INFO “day: %d, month: %d, year: %dn”,
                ptr->day,
                ptr->month,
                ptr->year);
    }

    printk(KERN_INFO “Display donen”);

           return 0;
}

/* This function is called when the module is removed. */
void birthdayList_exit(void)
{
    printk(KERN_INFO “Removing Modulen”);

    /* Go thru the list and free the memory. */
    birthday *ptr, *next;
    list_for_each_entry_safe(ptr, next, &birthday_list, list)
    {
        printk(KERN_INFO “Removing – day: %d, month: %d, year: %dn”,
                ptr->day,
                ptr->month,
                ptr->year);
        list_del(&ptr->list);
        kfree(ptr);
    }

    printk(KERN_INFO “Memory free donen”);
}

/* Macros for registering module entry and exit points. */
module_init( birthdayList_init );
module_exit( birthdayList_exit );

MODULE_LICENSE(“GPL”);
MODULE_DESCRIPTION(“BirthdayList Module”);
MODULE_AUTHOR(“daveti”);

[root@daveti kernel_module_list]# cat Makefile
obj-m += birthdayList.o
all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
[root@daveti kernel_module_list]#

Posted in OS | Tagged , , , | 2 Comments

Linux kernel compiling, updating and kernel module compiling, loading and removing

Posted on March 28, 2013 by daveti

Brief intro to Linux kernel compiling and updating, as well as Linux kernel module compiling, loading and removing. The Linux kernel we are playing with is 3.8.4.

1. Linux kernel compiling and updating
Ubuntu:
sudo apt-get install wget
sudo apt-get install gcc
sudo apt-get install libncurses5-dev
sudo apt-get update
sudo apt-get upgrade
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.8.4.tar.xz (NOTE: this URL is from http://www.kernel.org)
sudo tar -xvf linux-3.8.4.tar.xz -C /usr/src/
cd /usr/src/linux-3.8.4/
sudo make menuconfig
sudo make oldconfig
sudo make
sudo make modules_install install
reboot

Fedora (using root login):
yum install wget
yum install gcc
yum install ncurses-devel
yum update (NOTE: yum may update the kernel eventually…if it does, skip this cmd!)
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.8.4.tar.xz
tar -xvf linux-3.8.4.tar.xz -C /usr/src/kernels/
cd /usr/src/kernels/linux-3.8.4
make menuconfig
make oldconfig
make
make modules_install install
reboot

Reference:
http://www.howopensource.com/2011/08/how-to-compile-and-install-linux-kernel-3-0-in-ubuntu-11-04-10-10-and-10-04/
http://www.howopensource.com/2011/08/how-to-install-compile-linux-kernel-3-0-in-fedora-15-and-14/

2. Linux kernel module compiling, loading and removing
[root@daveti ch2]# cat simple.c
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>

/* This function is called when the module is loaded. */
int simple_init(void)
{
       printk(KERN_INFO “Loading Modulen”);

       return 0;
}

/* This function is called when the module is removed. */
void simple_exit(void) {
    printk(KERN_INFO “Removing Modulen”);
}

/* Macros for registering module entry and exit points. */
module_init( simple_init );
module_exit( simple_exit );

MODULE_LICENSE(“GPL”);
MODULE_DESCRIPTION(“Simple Module”);
MODULE_AUTHOR(“SGG”);

[root@daveti ch2]# cat Makefile
obj-m += simple.o
all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
[root@daveti ch2]#

make (build the module)
insmod simple.ko (Load the module)
lsmod | grep simple (You will find the module – simple)
dmesg (You will find the log ‘Loading Module’)
rmmod simple (remove the module)
lsmod | grep simple (You will NOT find the module – simple)
dmesg (You will find the log ‘Removing Module’)

Reference:
http://www.os-book.com/

Posted in OS | Tagged , , | Leave a comment

Linux File System / Data Recovery

Posted on March 13, 2013 by daveti

Recently encountered a kernel panic issue caused by ‘fsck -y’ and reboot, which caused the partition table missing. Tried some tools to recovery the file system as well as the data. Generally, we should never run into data recovery as we should always have data backup. However, sometimes it just happens – broken file system and no data backup.

0. Data backup before ‘fsck’
‘fsck’ is helpful and dangerous. Even Read-Only file system happens, we should try to backup all the data before we run ‘fsck’ trying to repair the file system!

1. File System recovery
Either hardware failure, like bad blocks, or software failure, like partition table missing, will cause the file system crash. When it happens, the data within the disk/partition could not be accessed unless the file system is repaired. The tools we could use here are:
fdisk (fdisk -l) – list all visible devices(disks)
lvm – help mount the invisible devices to /dev, which is necessary for fsck
fsck (fsck -y) – repair the bad file system
parted – a GUI based partition edit/recovery tool like PartitionMagic under Windows
gparted – Try its best to guess the partition
testdisk – like gparted but much more powerful and famous!

2. Data recovery
Sometimes, we just get no luck on the file system recovery. Then it is time to try data recovery directly. To avoid the duplicate failure during data recovery on the raw disk, it is a good idea to create an image for the whole disk or partition and do the data recovery on the image. The tools we could use here are:
dd – create the image of the whole disk/partition
ddrescue – do the data recovery based on the image generated by dd
magicrescue – similar with ddrescue
testdisk/photorec – photorec is a subtool from testdisk which is used to recovery certain files
R-StudioEmergencyCD – create the image of the whole disk/partition (Commercial but could run trial)
R-Studio – do the data recovery (based on the image generated by EmergencyCD, commercial but could run trail)

3. General procedure
a. Try the OS installation CD/DVD – get into rescue mode to search for the partition or repair the file system with fsck
b. Try different Linux bootable live CD – get into the live system as well as the tools mentioned above (Recommend: GParted and Ubuntu Rescue Remix)
c. Try to create the image of the whole disk/partition before data recovery using external hard drive
d. Run the data recovery tools based on the image
e. There are 3rd party commercial service focusing data recovery if data privacy is not cared so much

4. R-Studio
Well, I have tried most of the tools to recover the file system but failed. I have tried most of the tools above to recover the data but seems most of them are focusing on specific files like JPEG or ELF. Then R-Studio saved me, though it is not free ($79.99). You may find some issues using the R-Studio Linux version. But after I switched to the Windows version based on the image, R-studio worked like a charm!

5. Reference
Different Linux live CDs
http://www.cgsecurity.org/wiki/TestDisk_Livecd
GParted live CD
http://gparted.sourceforge.net/livecd.php
Ubuntu Rescue Remix
http://ubuntu-rescue-remix.org/
TestDisk step by step
http://www.howtoforge.com/data_recovery_with_testdisk
General Data Recovery intro from Ubuntu
https://help.ubuntu.com/community/DataRecovery
R-Studio
http://www.r-tt.com/
A good tutorial for partition/data recovery
http://www.dedoimedo.com/computers/linux-data-recovery.html

Posted in OS | Tagged , , , , , , , , , | 2 Comments

webcam on Linux (Fedora) – issues with Skype

Posted on March 9, 2013 by daveti

Install a webcam on Linux is kind of tricky. The problem is almost none of the manufacturer would provide the Linux driver when delivering their products. And again, the power of open source community would be the hero again. There are some different kinds of Linux webcam drivers which cover different range of the products. In this post, we will try to give a general guidance on how to choose a webcam, install and make it work. The real example happens here is Logitech WebCam c310 and Fedora 18 with kernel 3.8.1. May it help.

1. How to choose the webcam?
This link (https://wiki.archlinux.org/index.php/Webcam_Setup#Identify_your_webcam) lists different kinds of webcams supported by Linux based on different drivers. We will choose ‘Linux-UVC’ here, which is sort of popular and covers the most common fashion webcams. You could find the detailed products from the UVC web (http://www.ideasonboard.org/uvc/). Then I have bought Logitech WebCam c310.

[root@daveti ~]# lsmod | grep uvc
uvcvideo               80924  0
videobuf2_vmalloc      13163  1 uvcvideo
videobuf2_core         38816  1 uvcvideo
videodev              120947  2 uvcvideo,videobuf2_core
media                  20444  2 uvcvideo,videodev

2. Install the uvc kernel module
Note that uvc kernel module (driver) is not necessarily contained in the Linux kernel. ‘lsmod | grep uvc’ to make sure you have the module enabled. Otherwise, please install it using your pkg manager, like yum in Fedora.

3. Make sure your WebCam is recognized when uvc is enabled
[root@daveti ~]# lsusb
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 004 Device 002: ID 1058:1140 Western Digital Technologies, Inc. My Book Essential USB3.0
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 003: ID 413c:2501 Dell Computer Corp.
Bus 002 Device 004: ID 0461:4d81 Primax Electronics, Ltd Dell N889 Optical Mouse
Bus 002 Device 005: ID 413c:2107 Dell Computer Corp.
Bus 002 Device 057: ID 046d:081b Logitech, Inc. Webcam C310

4. Install the WebCam applications
I have installed ‘Cheese’, ‘guvcview’ and ‘Video4Linux Control Panel’ (will cover the communication software later). Much more applications could be found at link (https://wiki.archlinux.org/index.php/Webcam_Setup#Get_software_to_use_your_webcam)

5. Skype/Pidgin
The interesting thing happened here is ‘Pidgin’ could make video calls after ‘Skype’ works (it seems both of them are based on old libv4l instead of using the uvc driver directly). So we will try to tweak our ‘Skype’ then. ‘Skype’ is different with other WebCam application, which could take advantage of the latest drivers, like uvc. ‘Skype’ is totally based on old Linux webcam driver lib – libv4l. More over, there is no 64-bit ‘Skype’ binary till now. Even you were running ‘Skype’ on 64-bit Linux, you were using the 32-bit ‘Skype’ and you would have to install 32-bit libv4l, like the one for fedora (http://rpm.pbone.net/index.php3/stat/4/idpl/20041050/dir/fedora_18/com/libv4l-0.8.8-3.fc18.i686.rpm.html). After that, we need to write a shell wrapper for calling ‘Skype’ in a way that will trigger the libv4l (only in this way, we could make video calls in ‘Skype’)

[root@daveti bin]# cat myskp
#!/usr/bin/bash

# Wrapper to call skype with v4l driver
# to enable the video ability
# 8 Mar, 2013
# daveti

LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so /usr/bin/skype

There is a good link which gives much more info about this tweaking (http://community.linuxmint.com/tutorial/view/219)

6. No voice?
It would be much more frustrating if video works but voice not. Make sure to do 2 things for voice:
sudo yum reinstall pulseaudio alsa-plugins-pulseaudio pulseaudio-esound-compat pulseaudio-libs pulseaudio-libs-glib2 pulseaudio-module-zeroconf pulseaudio-libs-zeroconf xmms-pulse pulseaudio-module-gconf wine-pulseaudio xine-lib-pulseaudio pulseaudio-utils pulseaudio-module-bluetooth gst-mixer padevchooser paman paprefs pavucontrol pavumeter
System->Preferences->VolumeControl->Device: choose the USB device (ID is the one in ‘lsusb’) and make sure you have not mute the microphone.

Posted in Linux Distro | Tagged , , , , , , | Leave a comment

Nvidia, sucks – issues related with the official GPU driver and CUDA on Linux (Fedora 18)

Posted on March 2, 2013 by daveti

Nvidia, you SUCK! If you are planning to install the latest official GPU driver for your latest Linux, which means most probably you wanna try CUDA, then please BE WARE! I had installed the 310.xx version of Nvidia Linux driver, which would cause the system hang after image loading if the kernel > 3.7.8. The more frustrating thing is the CUDA’s tight dependency on version of kernel, gcc and glibc. Finally, I have removed the Nvidia’s driver and come back to Nouveau, which is an open source Nvidia driver for Linux and supports the latest kernel (3.7.9-205) smoothly! Again, Nvidia, you SUCK!

0. If CUDA is your target, make sure your GPU card is capable for it.
https://developer.nvidia.com/cuda-gpus

1. If you really wanna install Nvidia’s GPU driver…
vi /etc/default/grub
old stuffs: GRUB_CMDLINE_LINUX="....quiet rhgb"
after changes: GRUB_CMDLINE_LINUX="quiet rhgb rdblacklist=nouveau nouveau.modeset=0"
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
init 3
./yourOfficialNvidiaLinuxDriver
reboot

2. Even you have that driver bitch installed, you are NOT gonna succeed in CUDA…
http://developer.download.nvidia.com/compute/cuda/5_0/rel/docs/CUDA_Toolkit_Release_Notes_And_Errata.txt

**  The CUDA development environment relies on tight integration with the host 
development environment, including the host compiler and C runtime libraries, and is 
therefore only supported on distribution versions that have been qualified for this 
CUDA Toolkit release. 

**  Distributions Currently Supported  

    Distribution       32 64  Kernel                 GCC         GLIBC        
    -----------------  -- --  ---------------------  ----------  -------------
    Fedora 16          X  X   3.1.0-7.fc16           4.6.2       2.14.90      
    ICC Compiler 12.1     X                                                   
    OpenSUSE 12.1         X   3.1.0-1.2-desktop      4.6.2       2.14.1       
    Red Hat RHEL 6.x      X   2.6.32-131.0.15.el6    4.4.5       2.12         
    Red Hat RHEL 5.5+     X   2.6.18-238.el5         4.1.2       2.5          
    SUSE SLES 11 SP2      X   3.0.13-0.27-pae        4.3.4       2.11.3       
    SUSE SLES 11.1     X  X   2.6.32.12-0.7-pae      4.3.4       2.11.1       
    Ubuntu 11.10       X  X   3.0.0-19-generic-pae   4.6.1       2.13         
    Ubuntu 10.04       X  X   2.6.35-23-generic      4.4.5       2.12.1       

**  Distributions No Longer Supported 

    Distribution       32 64  Kernel                 GCC         GLIBC        
    -----------------  -- --  ---------------------  ----------  -------------
    Fedora 14          X  X   2.6.35.6-45            4.5.1       2.12.90      
    ICC Compiler 11.1     X                                                   
    OpenSUSE 11.2      X  X   2.6.31.5-0.1           4.4.1       2.10.1       
    Red Hat RHEL 6.x   X      2.6.32-131.0.15.el6    4.4.5       2.12         
    Red Hat RHEL 5.5+  X      2.6.18-238.el5         4.1.2       2.5          
    Ubuntu 11.04       X  X   2.6.38-8-generic       4.5.2       2.13

3. Yep, sweet, time to remove that Nvidia bitch and back to Nouveau!
Recall the changes within the /etc/default/grub and grub2-mkconfig again and then reboot

Posted in Linux Distro | Tagged , , , , , , , | Leave a comment

myssh – an interactive ssh management tool

Posted on February 24, 2013 by daveti

Inspired by the putty manager on Windows system, I have spent the whole afternoon to write ‘myssh’ – an interactive ssh management tool providing the ability to save, display, choose and connect your ssh connection automatically. I am not sure if there is any kind of GUI related stuff on Linux just like putty manager. However, if you are a CLI fan with some ssh connections, please try ‘myssh’. Hope it will save you some time and make you fun:)

myssh

[root@daveti blog]# ./myssh
Welcome to myssh – an interactive ssh management tool
=====================================================
0    user0@server0
1    user0@server1
2    user1@server1
q    quit
=====================================================
Please input the number of your choice: q
[root@daveti blog]#

#!/bin/bash

#
# myssh
# An interactive ssh management tool
# dave.jing.tian@gmail.com
# 23 Feb, 2013
#

#============================================
# Configure the ssh information here ~ change
#============================================

# login list – add your login here
loginList=(“user0” “user1”)

# hostname/IPaddr list – add your remote machine here
remoteList=(“server0” “server1”)

# password list – add your password here
passwdList=(“passwd0” “passwd1” “passwd2”)

# ssh list – add the relation rules to construct the ssh entry
# format: “login-remote-passwd”
# example: “0-1-0” stands for a ssh entry as below
# user0@server1 with password passwd0
sshList=(“0-0-2” “0-1-0” “1-1-1”)

#============================================
# Internal used functions ~ no need to change
#============================================

# Display the usage and then exit
usage()
{
echo “Usage: myssh”
exit 1
}

# Get the “login@remote” from ssh entry
getLoginRemoteFromSshEntry()
{
local sshEntry=”$1″
local loginIndex=`echo $sshEntry | cut -d “-” -f1 | tr -d ” “`
local remoteIndex=`echo $sshEntry | cut -d “-” -f2 | tr -d ” “`
echo “${loginList[loginIndex]}@${remoteList[remoteIndex]}”
}

# Get the “passwd” from ssh entry
getPasswdFromSshEntry()
{
local sshEntry=”$1″
local passwdIndex=`echo $sshEntry | cut -d “-” -f3 | tr -d ” “`
echo “${passwdList[passwdIndex]}”
}

# Display all ssh entries within sshList
displaySshList()
{
echo “Welcome to myssh – an interactive ssh management tool”
echo “=====================================================”
# Go thru the sshList to construct the ssh entry menu
for((i=0; i<${#sshList[@]}; i++))
do
local loginRemote=`getLoginRemoteFromSshEntry ${sshList[i]}`
echo ”  $i    $loginRemote”
done
echo ”  q    quit”
echo “=====================================================”
}

#===========================================
# Main function of myssh ~ no need to change
#===========================================

# Check ssh and sshpass at first
SSHCMD=`which ssh`
SSHPASSCMD=`which sshpass`
if [ “$SSHCMD” = “” ] || [ “$SSHPASSCMD” = “” ]
then
echo “Please install both ssh and sshpass before using myssh”
exit 1
fi

# Display the ssh list
displaySshList

# Ask for the input
read -p “Please input the number of your choice: ” index

# Process the input
if [ “$index” = “q” ]
then
exit 1
elif [[ $index =~ ^-?[0-9]+$ ]] && [[ $index -ge 0 ]] && [[ $index -le ${#sshList[@]} ]]
then
# Re-parse this ssh entry within sshList
loginRemote=`getLoginRemoteFromSshEntry ${sshList[$index]}`
passwd=`getPasswdFromSshEntry ${sshList[$index]}`
# Debug
# echo “$loginRemote $passwd”

# Just do it!
$SSHPASSCMD -p “$passwd” $SSHCMD “$loginRemote”

else
echo “Invalid input: $index”
usage
fi

Posted in Dave's Tools, Programming | Tagged , , , | 3 Comments