Adding a new syscall into Linux kernel would be tricky for the beginners (like myself), especially considering the kernel source tree structure changes among different versions. It seems most of the online tutorials focusing on older versions of kernel (2.6.X) or early version of 3.X. Things are little bit different using the latest version of kernel (3.8.7, and I have no idea from which version did all these changes happen…). Anyway, we will try to compare the older way and the latest way to add a new syscall into the kernel. Good luck! (NOTE: this post is for x86 arch!)
A. Older version of kernel (2.6.X) or early version of 3.X
This post (http://arvindsraj.wordpress.com/2012/10/05/adding-hello-world-system-call-to-linux/) actually gives a great and detailed reference on how to do that. Summary is listed down below.
Step 0: Define the new syscall (sys_hello())
Make a new directory called ‘hello’ under the kernel directory and create the source file for syscall sys_hello() as well as its makefile.
file:
hello/hello.c
add:
#include <linux/kernel.h>asmlinkage long sys_hello(void){ printk("Hello worldn"); return 0;}obj-y := hello.ocore-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/Step 2: Add the new syscall into syscall table (Assembly file actually)
file:
arch/x86/kernel/syscall_table_32.S (for x86-64 arch, the file should be syscall_table_64.S)
add:
.long sys_hello
Step 3: Add the syscall number (macro definition eventually and the number varies)
file:
arch/x86/include/asm/unistd_32.h
add:
#define __NR_hello 349
add:
#define NR_syscalls 350
file:
arch/x86/include/asm/unistd_64.h
add:
#define __NR_hello 312 __SYSCALL(__NR_hello, sys_hello)Step 4: Add the new syscall into syscall header file
file:
include/linux/syscalls.h
add:
asmlinkage long sys_hello(void);
Step 5: Compile, update the kernel and run the testing file (using dmesg to check the kernel log)
file:
$HOME/sysHello.c
add:
#include <stdio.h>#include <linux/kernel.h>#include <sys/syscall.h>#include <unistd.h>#define __NR_hello 312 //349 if you are running a 32bit kernel and following my tutoriallong hello_syscall(void){ return syscall(__NR_hello);}int main(int argc, char *argv[]){ long int a = hello_syscall(); printf("System call returned %ldn", a); return 0;}Step 0: Define the new syscall (sys_hello())
Make a new directory called ‘hello’ under the kernel directory and create the source file for syscall sys_hello() as well as its makefile.
file:
hello/hello.c
add:
#include <linux/kernel.h>asmlinkage long sys_hello(void){ printk("Hello worldn"); return 0;}obj-y := hello.ocore-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ hello/Step 2: Add the new syscall as well as the number into syscall table (real table file and the number varies)
file:
arch/x86/syscalls/syscall_32.tbl
add (at the end of the file):
351 i386 daveti_hello sys_hello
file:
arch/x86/syscalls/syscall_64.tbl
add (at the end of the file but before x32-specific syscalls):
314 common daveti_hello sys_hello
Step 3: Add the new syscall into syscall header file
file:
include/linux/syscalls.h
add:
asmlinkage long sys_hello(void);
Step 4: Compile, update the kernel and run the testing file (using dmesg to check the kernel log)
file:
$HOME/sysHello.c
add:
#include <stdio.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#include <unistd.h>
int main()
{
long int rtn = syscall(351);
printf(“System call sys_hello return %ldn”, rtn);
return 0;
}
davejingtian.org 