davejingtian.org

USB mass storage on Gumstix – a tricky issue using Yocto Daisy

Posted on July 1, 2014 by daveti

A recent upgrade of Yocto, from Dora to Daisy, broke the things worked before. Actually, CONFIG_USB_MASS_STORAGE, enabled in the defconfig of the Overo, was not passed to the .config during the kernel build. This weird missing config costed me 24-hour debugging. And it turns out nobody was doing wrong but bitbake could do better indeed. This post talks about building Gumstix image using Yocto Daisy, the missing kernel config, and debug and the take away in the summary. Happy debug!

0. Build the Gumstix Overo image using Yocto Daisy

The Gumstix-Yocto-Repo now uses Yocto Daisy as the default/master branch for its recipes. Before following the gumstix-yocto building procedure, we have to update the Yocto into Daisy:
0.0. Quick Start
http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html
0.1. Download the SDK
https://www.yoctoproject.org/download/yocto-project-16-buildtools-sdk
0.2. Download the toolchain (for 1.6)
http://downloads.yoctoproject.org/releases/yocto/yocto-1.6/toolchain/
0.3. Gumstix-Yocto
https://github.com/gumstix/Gumstix-YoctoProject-Repo
0.4. Check the Yocto version before build (make sure Daisy is invoked instead of others, like Dora)
cat poky/meta-yocto/conf/distro/poky.conf

1. The missing config – CONFIG_USB_MASS_STORAGE

Before “bitbake gumstix-consle-image” is submitted, the new config CONFIG_USB_MASS_STORAGE is enabled as a module in the defconfig of the Overo kernel recipe. Then weird thing happened. The USB mass storage kernel module was not contained in the modules.tgz. Actually, it had never been built.

2. Debug

While the kernel source code locates within the git directory under build/tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.5.7-r0, the building directory is linux-overo-standard-build under the same directory. Checking the .config for the previous build, I found no config for the USB mass storage was enabled. A direct doubt was something wrong with the bb file, as “bitbake -f -c virtual/kernel” triggers the parsing of “poky/meta-gumstix/recipes-kernel/linux/linux-gumstix_3.5.7.bb“. And the parse was good because the defconfig was located under /tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.5.7-r0, as well as bunch of patch files. After hours and hours debugging, “bitbake -v -D -f -c virtual/kernel” showed the clue:

GEN /home/daveti/gumstix-daisy/yocto/build/tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.5.7-r0/linux-overo-standard-build/Makefile
scripts/kconfig/conf –silentoldconfig Kconfig
.config:2751:warning: override: USB_CDC_COMPOSITE changes choice state
#
# configuration written to .config
#
GEN /home/daveti/gumstix-daisy/yocto/build/tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.5.7-r0/linux-overo-standard-build/Makefile

The kernel config checking tool ‘conf’ reported an warning about a config conflict and a config rewriting because of this conflict. After disabling CONFIG_USB_CDC_COMPOSITE in the defconfig, we now have the USB mass storage kernel module.

3. Summary

CONFIG_USB_CDC_COMPOSITE was disabled in the Dora recipes but is enabled in the Daisy recipes for certain reasons. The bug could be found easily by normal “make menuconfig” as it is reported as a warning. However, bitbake silences the warning and makes everything look well even if the .config has been rewritten because of some config conflict.

Posted in Embedded System, Linux Distro | Tagged bitbake, gumstix, kernel, Linux, overo, USB, yocto | 2 Comments

ARP – Linux kernel ARP behavior with multiple NIC

Posted on June 27, 2014 by daveti

This post is the summary based on a recent discussion in the kernelnewbies mail list. Thanks to the collective intelligence, the Linux kernel ARP behavior with multiple NIC is now pretty clear.

0. Original Question

“Assume there are two interfaces eth0 and eth1 each configured with IPs belonging to different subnets. Say IP1 is assigned to eth0 and IP2 to eth1. Now if an ARP request comes on eth0 for IP2, what is the behavior on Linux?”

1. Summary

Without code checking, the answer comes from “Documentation/networking/ip-sysctl.txt” by looking at “/proc/sys/net/ipv4/conf/“

If IP1 and IP2 are in different subnets, as the question, arp_filter is the controller:

“0 – (default) The kernel can respond to arp requests with addresses from other interfaces. This may seem wrong but it usually makes sense, because it increases the chance of successful communication. IP addresses are owned by the complete host on Linux, not by particular interfaces. Only for more complex setups like load-balancing, does this behavior cause problems.“

If IP1 and IP2 are in the same subnets, arp_announce and arp_ignore are the controller.

Posted in Network, OS | Tagged arp, kernel, kernelnewbies, Linux, sysctl | 8 Comments

crypto – use Linux kernel crypto API

Posted on June 18, 2014 by daveti

Crypto is an important part of the Linux kernel source code. One can tell from the standalone crypto directory under the kernel source tree. Device drivers, file system and security all need crypto. This post does not help understand the implementation of Linux kernel crypto API. Instead, a working kernel module is provided to show how to use the kernel crypto API.

0. R.t.D.C.

/*
 * sha1_mod.c
 * A kernel module using SHA1 and Linux kernel crypto API
 * Reference: http://stackoverflow.com/questions/3869028/how-to-use-cryptoapi-in-the-linux-kernel-2-6
 * Jun 9, 2014
 * root@davejingtian.org
 * http://davejingtian.org
 */
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/crypto.h>
#include <linux/err.h>
#include <linux/scatterlist.h>

#define SHA1_LENGTH     20

static int __init sha1_init(void)
{
    struct scatterlist sg;
    struct crypto_hash *tfm;
    struct hash_desc desc;
    unsigned char output[SHA1_LENGTH];
    unsigned char buf[10];
    int i, j;

    printk(KERN_INFO "sha1: %s\n", __FUNCTION__);

    memset(buf, 'A', 10);
    memset(output, 0x00, SHA1_LENGTH);

    tfm = crypto_alloc_hash("sha1", 0, CRYPTO_ALG_ASYNC);
    if (IS_ERR(tfm)) {
	printk(KERN_ERR "daveti: tfm allocation failed\n");
	return 0;
    }

    desc.tfm = tfm;
    desc.flags = 0;

    //crypto_hash_init(&desc);
    //daveti: NOTE, crypto_hash_init is needed
    //for every new hasing!

    for (j = 0; j < 3; j++) {
	crypto_hash_init(&desc);
	sg_init_one(&sg, buf, 10);
	crypto_hash_update(&desc, &sg, 10);
	crypto_hash_final(&desc, output);

	for (i = 0; i < 20; i++) {
		printk(KERN_ERR "%02x", output[i]);
	}
	printk(KERN_INFO "\n---------------\n");
	memset(output, 0x00, SHA1_LENGTH);
    }

    crypto_free_hash(tfm);

    return 0;
}

static void __exit sha1_exit(void)
{
    printk(KERN_INFO "sha1: %s\n", __FUNCTION__);
}

module_init(sha1_init);
module_exit(sha1_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("daveti");

1. Pitfalls

As one might have known already, when we are trying to call crypto API for SHA1, we are querying the crypto manager (cryptomgr) eventually – a kernel thread maintains different algorithm instances and handles crypto operation requests all over the kernel. The truth is if crypto API is called, like crypto_alloc_hash(), before the cryptomgr is initialized during the kernel booting process, then NULL would be return. Make sure the crypto API is called after cryptomgr. Otherwise, you have to provide your own implementation.

If you are working in an IRQ context, be aware that some older crypto APIs may cause trouble. Based on my experience working on CentOS 6 with kernel 2.6.32, crypto_hash_digest() does not work. Instead, crypto_hash_final() works well.

Posted in OS, Programming, Security | Tagged CentOS, crypto, cryptomgr, kernel, Linux, SHA1 | 3 Comments

vmrun – run your VMware VM without GUI

Posted on June 5, 2014 by daveti

Virsh is my favorite cmdline for KVM management. You may wonder if there is sth similar for VMware or want to run VMware without GUI. Here is goes – vmrun.

0. VIX API

VIX API allows users to automate the management of VMs, such as APIs used to start/stop VMs. It supports most products from VMware, including Player, Sever, Workstation, Fusion and vSphere (though VIX API is deprecated as there is a vSphere API).

1. vmrun

After VIX API is installed, you will have a powerful command – vmrun, which leverages the powerful VIX API to provide a similar command line interface for VMware-based VMs, like virsh for KVM.

Take VMware Player and its VMs for example:
# Start the VM without GUI
vmrun -T player start “path/to/yourVM.vmx” nogui
# Stop the VM
vmrun -T player stop “path/to/yourVM.vmx” hard
# List all possible commands supported by vmrun
vmrun -h

2. Issues

There is a command in vmrun – getGuestIPAddress, which is used to get the IP address of the guest VM. First, the VM needs to be started. Second, VMware tools are needed to make the functionality work.
Once you login the VM, you may find top does not work. The same case happens if you try to compile the kernel (make menuconfig). Adding the fix into .profile helps.
root@dave-desktop:~/lsms# top
‘xterm-256color’: unknown terminal type.
Fix: export TERM=xterm

Posted in Programming | Tagged vix, vmrun, VMware | 8 Comments

EXT4-fs error – use Ubuntu to fix Fedora

Posted on May 31, 2014 by daveti

The most terrible thing a Linux could have may be some errors reported by the file system. It is usually caused by power outages or bad blocks/sectors within the hard drive. Eventually, as a Linux user, you may at least have suffered from file system errors once, where you have to reinstall the OS and for sure to lose all the existing data. This post is the summary of the recent file system errors happened on my Fedora 20. May give some hints.

0. File system errors seem to be inevitable

There was sth wrong with my Fedora 20 after a recent kernel update to 3.14.4. After megasas kernel module (a kernel module for LSI RAID cards) init’d, the whole system fell into emergency mode (single user mode). However, a manually CTRL-D would force the system going into the normal mode. I hunted for the changes of megasas which has been stable for years. Then I was looking forward to the next kernel release which may fix this weird issue. Interestingly, things were getting more weird.

After a ‘yum clean all‘ (trying to fix the broken dependency but not working anyway), I was not able to run ‘yum update’ anymore, always with the same error from Python: OSError [Errono 5]: Input/Output error. Tried to reboot the system and rebuild the yum cache manually but still got the same error. While I was wondering what happened within yum, there was sth going wrong within my file system.

1. Use tools to figure out what happens

1.1 dmesg

Please do use this powerful tool to check kernel’s complain and I found sth like below:
May 23 08:28:29 daveti kernel:EXT4-fs (dm-0): warning: mounting fs with errors, running e2fsck is recommended
May 23 08:28:29 daveti kernel:EXT4-fs (dm-0): re-mounted. Opts: (null)
May 23 08:28:29 daveti systemd: Started Remount Root and Kernel File Systems.
May 23 08:28:29 daveti systemd: Starting Configure read-only root support…
May 23 08:28:29 daveti systemd: Starting Import network configuration from initramfs…
May 23 08:28:29 daveti systemd: Starting Load/Save Random Seed…
May 23 08:28:29 daveti systemd: Starting Local File Systems (Pre).
May 23 08:28:29 daveti systemd: Reached target Local File Systems (Pre).
May 23 08:28:29 daveti systemd: Started Load/Save Random Seed.
May 23 08:28:29 daveti systemd: Started Configure read-only root support.
May 23 08:28:29 daveti systemd: Started Import network configuration from initramfs.
May 23 08:28:29 daveti systemd: Starting Emergency Shell…
May 23 08:28:29 daveti systemd: Started Emergency Shell.
May 23 08:28:29 daveti systemd: Starting Emergency Mode.
May 23 08:28:29 daveti systemd: Reached target Emergency Mode.
May 23 08:28:29 daveti systemd-fsck: /dev/sda1: clean, 410/128016 files, 244603/512000 blocks
May 23 08:28:29 daveti systemd: Started File System Check on /dev/mapper/fedora_dyn–184–171–61–7-home.
May 23 08:28:29 daveti systemd: Started File System Check on /dev/disk/by-uuid/c96cf320-40a7-42d5-9406-bf80c866faa6.
May 23 08:28:29 daveti systemd: Mounting /boot…
May 23 08:28:29 daveti systemd: Mounting /home…
May 23 08:33:08 daveti kernel:[ 301.924281] EXT4-fs (dm-0): error count: 45
May 23 08:33:08 daveti kernel:[ 301.924299] EXT4-fs (dm-0): initial error at 1400170854: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:[ 301.924307] EXT4-fs (dm-0): last error at 1400799080: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): error count: 45
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): initial error at 1400170854: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): last error at 1400799080: __ext4_get_inode_loc:3919: inode 2238830: block 8913590

What does this tell us? First, it tells my file system (EXT4) screws. Second, it tells the reason why the system falling into emergency mode (because of file system errors). Last, it tells some screwed inode operations.

1.2 smartctl

If your hard drive support SMART, try ‘smartctl -a /dev/yourDevice‘. Mine’s not. Let me know if this tool helps.

1.3 badblocks

This tool is used to see if there is any bad blocks within the drive – ‘badblocks /dev/yourDevice‘. NOTE, first it does NOT fix these bad blocks and second nothing from badblocks does not mean the file system in a good state. Again, dmesg!

2. Rescue mode?

We probably all know the right solution for this case – umount the bad disk and run fsck! The point is how to do that if you are not able to umount the disk even in single user mode when the root file system is screwed. If you have a second machine, you can unplug the issued hard drive and plug it into the other machine and run fsck there. If not, what else shall we do?

It is said that most Linux distribution installation ROMs have the so-called rescue mode where you can try to fix issues on the current system. I tried Fedora 20 live DVD and booted into ‘Try’ mode where a live image is loaded into the memory as the active OS. Unfortunately, I am still not able to umount the root file system for certain reasons.

3. Workaround

Fortunately, I have got a Ubuntu 14.04 LTS live DVD ROM. Booting from this system, I am able to umount all the partitions and run ‘fsck -yvf /dev/yourPartition‘ to fix all the bad blocks. After a reboot, my Fedora 20 runs smoothly into the Gnome login.

4. At last

File system errors may be inevitable for Linux (and eventually for all OSes). Once it happens, try to fix it ASAP. While I was trying to figure out how to umount the root file system, the number of errors (bad blocks) reported by EXT4 was increasing indeed! And please UMOUNT your disk/partition before fsck, otherwise you will hate Linux…

Posted in Linux Distro, OS | Tagged badblocks, dmesg, e2fsck, fedora, fsck, megasas, smartctl, Ubuntu, yum | 1 Comment

Gumstix Overo – USB gadget mass storage

Posted on April 1, 2014 by daveti

This post introduces the general procedure to make gumstix overo work as an USB gadget mass storage using the Yocto Linux. Thanks Adam Lee (Gumstix) for the gadget configuration consulting and debugging. Thanks Tyler Nichols (OSIRIS Lab) for the hardware debugging. A lot of reference online also make this post possible.

0. Yocto Linux

Yocto Project is an embedded Linux building environment supporting different kinds of embedded devices and Gumstix is using Yocto eventually right now. Before we start, please make sure you have Yocto Project installed in your machine. For details, please check the reference [a]. One key point of choosing tool chain for cross compile is that the architecture of Gumstix Overo is ARMv7.

1. Overo as USB gadget mass storage

Following the procedure [b], can we setup up the building system for Gumstix Overo. To enable the USB gadget mass storage, all we need to do is to add “CONFIG_USB_MASS_STORAGE=m” into the defconfig for Overo under the Yocto building environment, like “/home/daveti/gumstix/yocto/poky/meta-gumstix/recipes-kernel/linux/linux-gumstix-3.5/overo”

(Optional) Kernel Hacking and Rebuild

An initial build is needed before whatever changes you want, which may take hours to generate MLO, u-boot.img, uImage, console-image and etc. Once done, hack into the kernel source code directory, like “/home/daveti/gumstix/yocto/build/tmp/work/overo-poky-linux-gnueabi/linux-gumstix/3.5.7-r0/git“, and do the changes. After that, force the rebuild under the build directory [d]:

bitbake -f -c compile virtual/kernel
bitbake -f -c deploy virtual/kernel

2. Running

Before we actually could get the Overo running like an USB gadget mass storage, some prerequisite has to be done at first.

2.1 Burn the SD card

Reference [f, g] shows the details on how to make a bootable SD card. But reference [b] provides us a smarter way, using the script provided by the gumstix-repo, like “/home/daveti/gumstix/yocto/poky/meta-gumstix-extras/scripts“.

./mk2partsd <block device>

2.2 Load the images

Follow the procedure [b] to copy all the necessary images from deploy to the SD card.

cd build/tmp/deploy/images/{overo|duovero|pepper}
cp MLO u-boot.img uImage /media/boot
tar xaf gumstix-console-image-overo.tar.bz2 -C /media/rootfs --strip-components=1
sync

2.3 Prepare the USB gadget file system

In this section, we will create a FAT file system, which will be used by the USB gadget mass storage. Note that this is done after your login the Overo system.

dd if=/dev/zero of=/home/root/daveti.img bs=1M count=64
losetup /dev/loop0 /home/root/daveti.img
mkfs.vfat /dev/loop0
mount -t vfat /dev/loop0 /mnt (optional)

2.4 Load the kernel module

In your Overo system,

depmod
modprobe g_mass_storage file=/home/root/daveti.img
lsmod/modinfo/rmmod/dmesg...

3. Pitfalls

Note that for my Overo+Palo35, I have to connect the USB ports both to the gumstix and the PC before I can load the g_mass_storage kernel module. Otherwise, the gumstix could not get recognized by the PC [e]. Adam also mentioned a counter example, which has to load the kernel module before USB port is connected. Adam also mentioned that some front USB ports do not work and it would be wise to use the USB ports from the motherboard directly. Well, in summary for this, please try every possible solutions here to make your device work. The Gumstix USB gadget is picky and funky indeed.

Also note that all the kernel hacks/changes are NOT included in the console-image, which provides only the kernel modules of the initial build triggered by the defconfig. Eventually, there are 2 images for a incremental Linux kernel build. One is uImage, which is already included in the previous procedure. The other one is “modules-overo.tgz”, under like “/home/daveti/gumstix/yocto/build/tmp/deploy/images/overo”. This modules-overo.tgz essentially contains the latest build of all the kernel modules enabled. If the kernel hacking is for certain kernel modules, make sure to overwrite the rootfs (unzipped from “gumstix-console-image-overo.tar.bz2”) using this “modules-overo.tgz”

rm -rf $ROOTFS/lib/modules/3.5.7-yocto-standard/kernel
rm -rf $ROOTFS/lib/modules/3.5.7-yocto-standard/modules.builtin
rm -rf $ROOTFS/lib/modules/3.5.7-yocto-standard/modules.order
cp -rf $MODULES/lib/modules/3.5.7-yocto-standard/* $ROOTFS/lib/modules/3.5.7-yocto-standard/

4. Reference

a. http://www.yoctoproject.org/docs/latest/yocto-project-qs/yocto-project-qs.html
b. https://github.com/gumstix/Gumstix-YoctoProject-Repo
c. http://41j.com/blog/2011/11/gumstix-as-usb-storage-device/
d. https://github.com/gumstix/Gumstix-YoctoProject-Repo/wiki
e. http://gumstix.8.x6.nabble.com/defconfig-for-overo-as-usb-gadget-mass-storage-td4968995.html
f. http://41j.com/blog/2011/11/setting-up-a-gumstix-sd-card/
g. http://gumstix.org/create-a-bootable-microsd-card.html

Posted in Embedded System, Linux Distro, OS | Tagged ARM, bitbake, gumstix, Linux, overo, palo35, usb gadget, yocto | 121 Comments

nested KVM – just for fun

Posted on March 30, 2014 by daveti

KVM is based on virtualization instruction set – either Intel vmx or AMD svm, which provides the ability to run the VM directly without emulation or translation. In other words, KVM could only be enabled if it sees vmx/svm in /proc/cpuinfo. By default, there is not way to run a nested KVM within a KVM guest, as the CPU of KVM guest does not have vmx/svm. This post introduces a general way to have nested KVM on Fedora 20. Have fun!

0. Host (Fedora 20, x86_64)

Make sure your CPU has vmx/svm for KVM – ‘cat /proc/cpuinfo | grep vmx‘. If lucky, move on installing KVM – ‘yum install kvm‘. Then we have to enable the nested KVM feature, which is disabled by default (Run ‘cat /sys/module/kvm_intel/parameters/nested‘ – ‘N’). To turn it on (ref a):
echo “options kvm-intel nested=1” | sudo tee /etc/modprobe.d/kvm-intel.conf
(Run ‘cat /sys/module/kvm_intel/parameters/nested’ – ‘Y’)

1. KVM Guest (fedora20kvm, x86_64)

Within the host, use ‘virt-install’ (CLI) or ‘virtual machine manager’ (GUI) to install a Fedora 20 VM named fedora20kvm. It does not matter what is the hard configuration. Just be aware that we will create a nested KVM guest within it. So make sure we will have enough memory and disk space. Once created, we can login our KVM Guest and ‘cat /proc/cpuinfo | grep vmx’ and won’t find it. Installing KVM on the guest currently is not possible. Now it is time to expose the vmx instruction set to the KVM Guest. BTW, I am using the Intel CPU and will talk about vmx in the following. If AMD is the target, replace it with svm.

Run ‘virsh edit fedora20kvm’ to edit the XML configuration for our KVM guest fedora20kvm. Find the CPU tag and add the line in green, which will expose the vmx to the KVM guest (ref a). Note that this XML is usually under /etc/libvirt/qemu/. Manually editing also works.

<cpu mode=’custom’ match=’exact’>
<model fallback=’allow’>SandyBridge</model>
<feature policy=’require’ name=’vmx’/>
</cpu>

After fedora20 is booted, ‘cat /proc/cpuinfo | grep vmx’ will find it – Yes!

2. nested KVM Guest (fedora20kvmkvm, x86_64)

Within the KVM guest (fedora20kvm), we can install KVM now. Once KVM is there, we can go ahead to create a nested KVM guest named fedora20kvmkvm, like what we have done for the KVM guest. However, we may find that we are not able to create/start this fedora20kvmkvm and KVM complains “Network is already in use by interface by eth0“. You may also find the error in /var/log/libvirt/libvirtd.log if you are using Ubuntu. Before we fix this, we needs few words to make clear.

The networking among KVM guests, the host and the Internet is implemented via a bridge called ‘virbr0‘ (or ‘br0’ in older releases). This ‘virbr0’ is a bridge usually to a ‘real’ interface, like ‘eth0’, providing the NAT functionality between the host IP and KVM guest IP. The IP of ‘virbr0’ by default is 192.168.122.1 and all the KVM guests created based on ‘virbr0’ has the IP format ‘192.168.122.X’, where X is in range [2, 254].

Once KVM is installed, libvirtd would try to create this bridge/interface after the system is booted. The bridge would NOT be created if it collides with its host routing (ref b). Running ‘ifconfig’, indeed, we are not able to find ‘virbr0’ within the KVM guest. The reason is simple, within the KVM guest, ‘eth0’ IP is eventually based on ‘virbr0’ of the host, which is something like ‘192.168.122.Y’. Within the route table of the KVM guest (‘cat /proc/net/route‘), there is a routing entry with destination IP ‘192.168.122.0’ and netmask ‘255.255.255.0’ for ‘eth0’. Unfortunately, libvirtd of the KVM guest was trying to create a ‘virbr0’, which would have the same routing entry but with different interface name! It failed to do that.

Now we will fix that. Let us find the default configuration file for ‘virbr0’ used by libvirtd – /etc/libvirt/qemu/networks/default.xml. By default, it looks like below:

<network>
<name>default</name>
<uuid>dc0ed217-7e39-42c4-b1e0-f2112b9bae58</uuid>
<bridge name=”virbr0″ />
<forward/>
<ip address=”192.168.122.1” netmask=”255.255.255.0″>
<dhcp>
<range start=”192.168.122.2” end=”192.168.122.254” />
</dhcp>
</ip>
</network>

Now let’s change it using some other IP block, like 10.0.0.0:

<network>
<name>default</name>
<uuid>dc0ed217-7e39-42c4-b1e0-f2112b9bae58</uuid>
<bridge name=”virbr0″/>
<forward/>
<ip address=”10.0.0.1” netmask=”255.255.255.0″>
<dhcp>
<range start=”10.0.0.2” end=”10.0.0.254“/>
</dhcp>
</ip>
</network>

Reboot your KVM guest to make sure the changes work. After rebooting, you will find the missing ‘virbr0’ and you should be able to create/start a nested KVM – fedora20kvmkvm – Yes!

3. ifconfig

3.1 Host (Fedora 20)

[root@daveti daveti]# ifconfig
em1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 184.171.61.83 netmask 255.255.254.0 broadcast 184.171.61.255
inet6 fe80::1a03:73ff:fe25:3571 prefixlen 64 scopeid 0x20<link>
inet6 2001:468:d01:c6:1a03:73ff:fe25:3571 prefixlen 64 scopeid 0x0<global>
ether 18:03:73:25:35:71 txqueuelen 1000 (Ethernet)
RX packets 1636026 bytes 2213024469 (2.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 696327 bytes 52861586 (50.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 20 memory 0xf3500000-f3520000

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 731874 bytes 2290638009 (2.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 731874 bytes 2290638009 (2.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:a7:cd:75 txqueuelen 0 (Ethernet)
RX packets 459232 bytes 32469028 (30.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 634892 bytes 2123001894 (1.9 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vmnet1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.39.1 netmask 255.255.255.0 broadcast 172.16.39.255
inet6 fe80::250:56ff:fec0:1 prefixlen 64 scopeid 0x20<link>
ether 00:50:56:c0:00:01 txqueuelen 1000 (Ethernet)
RX packets 115 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6059 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vmnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.232.1 netmask 255.255.255.0 broadcast 192.168.232.255
inet6 fe80::250:56ff:fec0:8 prefixlen 64 scopeid 0x20<link>
ether 00:50:56:c0:00:08 txqueuelen 1000 (Ethernet)
RX packets 115 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6054 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fc54:ff:fed9:177a prefixlen 64 scopeid 0x20<link>
ether fe:54:00:d9:17:7a txqueuelen 500 (Ethernet)
RX packets 229738 bytes 16326182 (15.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 364690 bytes 1533862479 (1.4 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@daveti daveti]#

3.2 KVM Guest (fedora20kvm)

[root@localhost networks]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.172 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::5054:ff:fed9:177a prefixlen 64 scopeid 0x20<link>
ether 52:54:00:d9:17:7a txqueuelen 1000 (Ethernet)
RX packets 151421 bytes 322443953 (307.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 104436 bytes 7136384 (6.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 87268 bytes 289788554 (276.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 87268 bytes 289788554 (276.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.0.1 netmask 255.255.255.0 broadcast 10.0.0.255
ether fe:54:00:ff:01:0c txqueuelen 0 (Ethernet)
RX packets 104648 bytes 5653017 (5.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 140706 bytes 321786351 (306.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fc54:ff:feff:10c prefixlen 64 scopeid 0x20<link>
ether fe:54:00:ff:01:0c txqueuelen 500 (Ethernet)
RX packets 104393 bytes 7094414 (6.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149982 bytes 322261530 (307.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@localhost networks]#

3.3 nested KVM Guest (fedora20kvmkvm)

[daveti@localhost ~]$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.0.34 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::5054:ff:feff:10c prefixlen 64 scopeid 0x20<link>
ether 52:54:00:ff:01:0c txqueuelen 1000 (Ethernet)
RX packets 150045 bytes 322268521 (307.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 104432 bytes 7099768 (6.7 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 4 bytes 340 (340.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4 bytes 340 (340.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[daveti@localhost ~]$

4. Something about Ubuntu…

If you are using Ubuntu, be aware. I was trying to set up a nested KVM following the similar procedure using Ubuntu 12.04.4 x86_64 but failed finally. The reason is changing default.xml for ‘virbr0’ did not work. I assume Ubuntu was using some buggy libvirt.

5. Reference

a. http://www.rdoxenham.com/?p=275
b. http://www.redhat.com/archives/libvir-list/2010-May/msg01088.html
c. http://kashyapc.com/2012/01/14/nested-virtualization-with-kvm-intel/
d. http://lxr.linux.no/linux+v3.13.5/Documentation/virtual/kvm/nested-vmx.txt

Posted in Linux Distro, Network, OS | Tagged fedora, intel, kvm, libvirt, libvirtd, SVM, Ubuntu, virbr0, virsh, vmx | 2 Comments

top or glances – trust on /proc/meminfo

Posted on March 25, 2014 by daveti

You may notice that the output of top and glances differs on memory usage (if you are using top and glances the same time). This post will disclose some details about memory info collection by glances and top. And the answer may be known to everyone – /proc/meminfo.

1. top

top is part of procps pkg (http://procps.sourceforge.net/). Going deep into the source code, we will find function meminfo() (procps-3.2.8/proc/sysinfo.c) is the one taking care of memory info collection by reading the /proc/meminfo file.

2. glances

Checking the source of glances (https://github.com/nicolargo/glances/blob/master/glances/glances.py), we will find the glances is based on psutil (https://code.google.com/p/psutil/). However, the memory information processing is a little weird. Instead of totally relying on psutil, glances uses its own computation for certain fields. That is partially because psutil was not mature in the past and glances has to come up with its own hacks. meminfo.py is a code example based on psutil to simulate the current behavior of glances and the latest psutil. Going deep into the psutil (), we could also find that function virtual_memory() is based on reading /proc/meminfo file.

2.1 virtual_memory() (psutil/psutil/_pslinux.py)

def virtual_memory(): total, free, buffers, shared, _, _ = cext.linux_sysinfo() cached = active = inactive = None f = open('/proc/meminfo', 'rb') CACHED, ACTIVE, INACTIVE = b("Cached:"), b("Active:"), b("Inactive:") try: for line in f:

2.2 meminfo.py

#!/usr/bin/env python

# Added Glances output simulation
# Mar 11, 2014
# daveti@cs.uoregon.edu
# http://davejingtian.org

“””
Print system memory information.

$ python examples/meminfo.py
MEMORY
——
Total : 9.7G
Available : 4.9G
Percent : 49.0
Used : 8.2G
Free : 1.4G
Active : 5.6G
Inactive : 2.1G
Buffers : 341.2M
Cached : 3.2G

SWAP
—-
Total : 0B
Used : 0B
Free : 0B
Percent : 0.0
Sin : 0B
Sout : 0B
“””

import psutil
from psutil._compat import print_

def bytes2human(n):
# http://code.activestate.com/recipes/578019
# >>> bytes2human(10000)
# ‘9.8K’
# >>> bytes2human(100001221)
# ‘95.4M’
symbols = (‘K’, ‘M’, ‘G’, ‘T’, ‘P’, ‘E’, ‘Z’, ‘Y’)
prefix = {}
for i, s in enumerate(symbols):
prefix[s] = 1 << (i + 1) * 10 for s in reversed(symbols): if n >= prefix[s]:
value = float(n) / prefix[s]
return ‘%.1f%s’ % (value, s)
return “%sB” % n

def simulate_glances(phymem):
”’
Code below are copied from glances.py source code (V 1.7.5)
”’
output = []
# buffers and cached (Linux, BSD)
buffers = getattr(phymem, ‘buffers’, 0)
cached = getattr(phymem, ‘cached’, 0)
# active and inactive not available on Windows
active = getattr(phymem, ‘active’, 0)
inactive = getattr(phymem, ‘inactive’, 0)
# phymem free and usage
total = phymem.total
free = phymem.available # phymem.free + buffers + cached
used = total – free
percent = phymem.percent
# Format the output
output.append((‘Total’, total))
output.append((‘Available’, -1))
output.append((‘Percent’, percent))
output.append((‘Used’, used))
output.append((‘Free’, free))
output.append((‘Active’, active))
output.append((‘Inactive’, inactive))
output.append((‘Buffers’, buffers))
output.append((‘Cached’, cached))
for o in output:
value = o[1]
if o[0] != ‘Percent’:
value = bytes2human(value)
print_(‘%-10s : %7s’ % (o[0], value))

def pprint_ntuple(nt):
for name in nt._fields:
value = getattr(nt, name)
if name != ‘percent’:
value = bytes2human(value)
print_(‘%-10s : %7s’ % (name.capitalize(), value))

def main():
print_(‘MEMORY\n——‘)
#daveti: Hack for glances simulation
#pprint_ntuple(psutil.virtual_memory())
psvm = psutil.virtual_memory()
pprint_ntuple(psvm)
print_(‘\nGlances\n——‘)
simulate_glances(psvm)
print_(‘\nSWAP\n—-‘)
pprint_ntuple(psutil.swap_memory())

if __name__ == ‘__main__’:
main()

3. summary

top is stable and has not been changed for years; glances is new and based on psutil. According to Nicolas (dev of glances), latest glances is fully leveraging psutil and the current glances will be patched. Meanwhile, cat /proc/meminfo is always there.

Posted in OS | Tagged /proc/meminfo, glances, Linux, psutil, Python, top | Leave a comment

Hack – make VMware Player 6.0.1 start on Linux kernel 3.13.X

Posted on March 4, 2014 by daveti

If you were using VMware Player 6.0.1 on the latest Linux kernel 3.13.X and not able to get it started. Please read thru this post.

0. Error – failure to build vmnet

Inspect the VMware starting log /tmp/vmware-root/modconfig-XXXX.log, we may find the failure like this:

2014-03-04T20:55:19.711-08:00| vthread-3| I120: /lib/modules/3.13.5-200.fc20.x86_64/build/include/linux/version.h not found, looking for generated/uapi/linux/version.h instead.
2014-03-04T20:55:19.711-08:00| vthread-3| I120: using /usr/lib64/ccache/gcc for preprocess check
2014-03-04T20:55:19.720-08:00| vthread-3| I120: Preprocessed UTS_RELEASE, got value “3.13.5-200.fc20.x86_64”.
2014-03-04T20:55:19.720-08:00| vthread-3| I120: The header path “/lib/modules/3.13.5-200.fc20.x86_64/build/include” for the kernel “3.13.5-200.fc20.x86_64” is valid. Whoohoo!
2014-03-04T20:55:19.855-08:00| vthread-3| I120: Invoking modinfo on “vmnet”.
2014-03-04T20:55:19.859-08:00| vthread-3| I120: “/sbin/modinfo” exited with status 256.
2014-03-04T20:55:19.909-08:00| vthread-3| I120: Setting destination path for vmnet to “/lib/modules/3.13.5-200.fc20.x86_64/misc/vmnet.ko”.
2014-03-04T20:55:19.909-08:00| vthread-3| I120: Extracting the vmnet source from “/usr/lib/vmware/modules/source/vmnet.tar“.
2014-03-04T20:55:19.915-08:00| vthread-3| I120: Successfully extracted the vmnet source.
2014-03-04T20:55:19.916-08:00| vthread-3| I120: Building module with command “/usr/bin/make -j4 -C /tmp/modconfig-pSj2y5/vmnet-only auto-build HEADER_DIR=/lib/modules/3.13.5-200.fc20.x86_64/build/include CC=/usr/lib64/ccache/gcc IS_GCC_3=no”
2014-03-04T20:55:21.786-08:00| vthread-3| W110: Failed to build vmnet. Failed to execute the build command.

So the root cause for VMware Player to start up is the failure to build a kernel module called vmnet. Fortunately, the vmnet source are open and available for edit. The log above has already told you where to find vmnet.tar source files.

1. Verify – manually make vmnet

Let us go into the vmnet.tar directory /usr/lib/vmware/modules/source/, untar the source file and do the make directly.

[root@daveti source]# ll
total 5004
-rw-r–r–. 1 root root 757760 Mar 4 20:38 vmblock.tar
-rw-r–r–. 1 root root 1249280 Mar 4 20:38 vmci.tar
-rw-r–r–. 1 root root 1208320 Mar 4 20:38 vmmon.tar
drwxr-xr-x. 2 root root 4096 Oct 18 12:11 vmnet-only
-rw-r–r–. 1 root root 880640 Mar 4 20:38 vmnet.tar
-rw-r–r–. 1 root root 1024000 Mar 4 20:38 vsock.tar
[root@daveti source]# cd vmnet-only
[root@daveti vmnet-only]# ll
total 916
-rw-r–r–. 1 root root 50253 Oct 18 12:11 bridge.c
-rw-r–r–. 1 root root 2018 Oct 18 12:11 community_source.h
-rw-r–r–. 1 root root 1441 Oct 18 12:11 compat_autoconf.h
-rw-r–r–. 1 root root 2298 Oct 18 12:11 compat_module.h
-rw-r–r–. 1 root root 10212 Oct 18 12:11 compat_netdevice.h
-rw-r–r–. 1 root root 7037 Oct 18 12:11 compat_skbuff.h
-rw-r–r–. 1 root root 3074 Oct 18 12:11 compat_sock.h
-rw-r–r–. 1 root root 3827 Oct 18 12:11 compat_version.h
-rw-r–r–. 1 root root 17987 Oct 18 12:11 COPYING
-rw-r–r–. 1 root root 49129 Oct 18 12:11 driver.c
-rw-r–r–. 1 root root 2252 Oct 18 12:11 driver-config.h
-rw-r–r–. 1 root root 52703 Oct 18 12:11 filter.c
-rw-r–r–. 1 root root 1233 Oct 18 12:11 geninclude.c
-rw-r–r–. 1 root root 17919 Oct 18 12:11 hub.c
-rw-r–r–. 1 root root 850 Oct 18 12:11 includeCheck.h
-rw-r–r–. 1 root root 4447 Oct 18 12:11 Makefile
-rw-r–r–. 1 root root 1695 Oct 18 12:11 Makefile.kernel
-rw-r–r–. 1 root root 2642 Oct 18 12:11 Makefile.normal
-rw-r–r–. 1 root root 4542 Oct 18 12:11 monitorAction_exported.h
-rw-r–r–. 1 root root 1296 Oct 18 12:11 netdev_has_dev_net.c
-rw-r–r–. 1 root root 1774 Oct 18 12:11 netdev_has_net.c
-rw-r–r–. 1 root root 5805 Oct 18 12:11 net.h
-rw-r–r–. 1 root root 14142 Oct 18 12:11 netif.c
-rw-r–r–. 1 root root 1592 Oct 18 12:11 nfhook_uses_skb.c
-rw-r–r–. 1 root root 8939 Oct 18 12:11 procfs.c
-rw-r–r–. 1 root root 1402 Oct 18 12:11 skblin.c
-rw-r–r–. 1 root root 161736 Oct 18 12:11 smac.c
-rw-r–r–. 1 root root 9610 Oct 18 12:11 smac_compat.c
-rw-r–r–. 1 root root 2055 Oct 18 12:11 smac_compat.h
-rw-r–r–. 1 root root 3689 Oct 18 12:11 smac.h
-rw-r–r–. 1 root root 27555 Oct 18 12:11 userif.c
-rw-r–r–. 1 root root 11655 Oct 18 12:11 vm_assert.h
-rw-r–r–. 1 root root 72320 Oct 18 12:11 vm_atomic.h
-rw-r–r–. 1 root root 33932 Oct 18 12:11 vm_basic_asm.h
-rw-r–r–. 1 root root 15323 Oct 18 12:11 vm_basic_asm_x86_64.h
-rw-r–r–. 1 root root 18641 Oct 18 12:11 vm_basic_asm_x86.h
-rw-r–r–. 1 root root 16634 Oct 18 12:11 vm_basic_defs.h
-rw-r–r–. 1 root root 26189 Oct 18 12:11 vm_basic_types.h
-rw-r–r–. 1 root root 10876 Oct 18 12:11 vm_device_version.h
-rw-r–r–. 1 root root 3268 Oct 18 12:11 vmnetInt.h
-rw-r–r–. 1 root root 7416 Oct 18 12:11 vm_oui.h
-rw-r–r–. 1 root root 1316 Oct 18 12:11 vmware_pack_begin.h
-rw-r–r–. 1 root root 1336 Oct 18 12:11 vmware_pack_end.h
-rw-r–r–. 1 root root 1956 Oct 18 12:11 vmware_pack_init.h
-rw-r–r–. 1 root root 14627 Oct 18 12:11 vnetEvent.c
-rw-r–r–. 1 root root 1839 Oct 18 12:11 vnetEvent.h
-rw-r–r–. 1 root root 7633 Oct 18 12:11 vnetFilter.h
-rw-r–r–. 1 root root 3280 Oct 18 12:11 vnetFilterInt.h
-rw-r–r–. 1 root root 12538 Oct 18 12:11 vnet.h
-rw-r–r–. 1 root root 8929 Oct 18 12:11 vnetInt.h
-rw-r–r–. 1 root root 2064 Oct 18 12:11 vnetKernel.h
-rw-r–r–. 1 root root 9971 Oct 18 12:11 vnetUserListener.c
-rw-r–r–. 1 root root 73397 Oct 18 12:11 x86cpuid.h
[root@daveti vmnet-only]# make
Using 2.6.x kernel build system.
make -C /lib/modules/3.13.5-200.fc20.x86_64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/kernels/3.13.5-200.fc20.x86_64′
CC [M] /usr/lib/vmware/modules/source/vmnet-only/driver.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/hub.o
/usr/lib/vmware/modules/source/vmnet-only/hub.c: In function ‘VNetHubFindHubByID’:
/usr/lib/vmware/modules/source/vmnet-only/hub.c:132:49: warning: argument to ‘sizeof’ in ‘memcmp’ call is the same expression as the first source; did you mean to provide an explicit length? [-Wsizeof-pointer-memaccess]
memcmp(idNum, currHub->id.pvnID, sizeof idNum))) {
^
/usr/lib/vmware/modules/source/vmnet-only/hub.c: In function ‘VNetHubAlloc’:
/usr/lib/vmware/modules/source/vmnet-only/hub.c:315:36: warning: argument to ‘sizeof’ in ‘memcpy’ call is the same pointer type ‘uint8 *’ as the destination; expected ‘uint8’ or an explicit length [-Wsizeof-pointer-memaccess]
memcpy(hub->id.pvnID, id, sizeof id);
^
CC [M] /usr/lib/vmware/modules/source/vmnet-only/userif.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/netif.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/bridge.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/filter.o
/usr/lib/vmware/modules/source/vmnet-only/filter.c:206:1: error: conflicting types for ‘VNetFilterHookFn’
VNetFilterHookFn(unsigned int hooknum, // IN:
^
/usr/lib/vmware/modules/source/vmnet-only/filter.c:64:18: note: previous declaration of ‘VNetFilterHookFn’ was here
static nf_hookfn VNetFilterHookFn;
^
/usr/lib/vmware/modules/source/vmnet-only/filter.c:64:18: warning: ‘VNetFilterHookFn’ used but never defined [enabled by default]
/usr/lib/vmware/modules/source/vmnet-only/filter.c:206:1: warning: ‘VNetFilterHookFn’ defined but not used [-Wunused-function]
VNetFilterHookFn(unsigned int hooknum, // IN:
^
make[2]: *** [/usr/lib/vmware/modules/source/vmnet-only/filter.o] Error 1
make[1]: *** [_module_/usr/lib/vmware/modules/source/vmnet-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.13.5-200.fc20.x86_64′
make: *** [vmnet.ko] Error 2
[root@daveti vmnet-only]#

Then, we confirmed that the root cause is vmnet.ko. More specifically, VNetFilterHookFn in filter.c causes the building error. And so, what happened there?

2. RCA – root cause analysis

In the latest kernel, e.g, 3.13.5, the signature of netfilter hook function (nf_hookfn) has been changed as the result of code refactoring. However, VNetFilterHookFn is still using the old definition of nf_hookfn. That is why we are not able to compile vmnet.ko. For details, please check http://lxr.linux.no/linux+v3.13.5/include/linux/netfilter.h#L46

3. Fix – diff

[root@daveti vmnet-only]# diff filter.c.orig filter.c
206c206,207
< VNetFilterHookFn(unsigned int hooknum, // IN:
—
> //VNetFilterHookFn(unsigned int hooknum, // IN:
> VNetFilterHookFn(const struct nf_hook_ops *ops, //daveti: fix
255c256,258
< transmit = (hooknum == VMW_NF_INET_POST_ROUTING);
—
> //transmit = (hooknum == VMW_NF_INET_POST_ROUTING);
> //daveti: fix
> transmit = (ops->hooknum == VMW_NF_INET_POST_ROUTING);
[root@daveti vmnet-only]#

4. Verify fix – make again

After the patch for the filter.c, we should be able to make the vmnet kernel module now. Let us make sure the fix work.

[root@daveti vmnet-only]# make
Using 2.6.x kernel build system.
make -C /lib/modules/3.13.5-200.fc20.x86_64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/kernels/3.13.5-200.fc20.x86_64′
CC [M] /usr/lib/vmware/modules/source/vmnet-only/filter.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/procfs.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/smac_compat.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/smac.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/vnetEvent.o
CC [M] /usr/lib/vmware/modules/source/vmnet-only/vnetUserListener.o
LD [M] /usr/lib/vmware/modules/source/vmnet-only/vmnet.o
Building modules, stage 2.
MODPOST 1 modules
CC /usr/lib/vmware/modules/source/vmnet-only/vmnet.mod.o
LD [M] /usr/lib/vmware/modules/source/vmnet-only/vmnet.ko
make[1]: Leaving directory `/usr/src/kernels/3.13.5-200.fc20.x86_64′
make -C $PWD SRCROOT=$PWD/. \
MODULEBUILDDIR= postbuild
make[1]: Entering directory `/usr/lib/vmware/modules/source/vmnet-only’
make[1]: `postbuild’ is up to date
make[1]: Leaving directory `/usr/lib/vmware/modules/source/vmnet-only’
cp -f vmnet.ko ./../vmnet.o
[root@daveti vmnet-only]#

5. Tar and Start

Now it is time to tar the patched vmnet-only directory into vmnet.tar again. Start the VMware Player and enjoy~

Posted in OS | Tagged Linux, vmnet, VMware Player | 14 Comments

Something about float point math – function call vs. operator

Posted on March 2, 2014 by daveti

This post is triggered by a question of Python – “Is there any difference between pow() and ** operator?”. Similarly, this question applies to C and Java – “Is there any difference between pow(1, 2) and 1*1?”. The first thing came into my mind is the function call cost comparing with the operator. One could use the operator rather than the function call to improve the general performance. However, then why would function call exists? The answer is for better precision.

0. why doesn’t GCC optimize a*a*a*a*a*a to (a*a*a*)*(a*a*a)?

This StackOverflow post (http://stackoverflow.com/questions/6430448/why-doesnt-gcc-optimize-aaaaaa-to-aaaaaa) talks about why the compiler would not optimize such expression to an apparently better one (with less assembly instructions). The take-away behind this is Float Point Math is NOT Associative.

1. function call vs operator

We will run the simple code below to see what will happen in C, Python and Java. Note that the testings are run on my x86-64 MacBook Pro. The compilers are GCC(LLVM 5.0), Python 2.7.5 and JDK 1.7.0_40.

/*
* float_number.c
* To verify that Float Point Math is NOT Associative
* Mar 2, 2014
* root@davejingtian.org
* http://davejingtian.org
*/

#include <stdio.h>
#include <math.h>

int main(void)
{
float f;
for (f = 1; f < 2; f += 0.1)
{
printf(“——————————\n”);
printf(“pow(%f, 6)=%f\n”, f, pow(f, 6));
printf(“%f**6=%f\n”, f, f*f*f*f*f*f);
printf(“(%f**3)*(%f**3)=%f\n”, f, f, (f*f*f)*(f*f*f));
}
return 0;
}

Part of output:
——————————
pow(1.900000, 6)=47.045913
1.900000**6=47.045914
(1.900000**3)*(1.900000**3)=47.045918

Don’t be surprise if the results are not the same as above. pow() function gives the best precision comparing with the raw operator. If we change the ‘float f’ to ‘double f’, given the length of the decimal, the output are all the same. However, this does not mean they are the same indeed. Double type just provides a better precision during the computation and shows the same results after rounding.

Python:

# float_math.py
# An example file to show that
# Float Point Math is NOT Associative
# Mar 2, 2014
# root@davejingtian.org
# http://davejingtian.org

import math

float_num = [1., 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.]

for f in float_num:
print(“————————————“)
print(“pow(%f, 6)=%f” %(f, pow(f, 6)))
print(“%f**6=%f” %(f, f**6))
print(“(%f**3)*(%f**3)=%f” %(f, f, (f**3)*(f**3)))

Part of output:
————————————
pow(1.900000, 6)=47.045881
1.900000**6=47.045881
(1.900000**3)*(1.900000**3)=47.045881

Interesting, the output are all the same. What happens here? Well, as I am using default C python as the PVM. The float type in Python is mapped into double in C. It has the same output in C with double type. Other PVM may have different output.

Java:

/*
* float_number.java
* An example to verify Float Point Math is NOT Associative
* Mar 2, 2014
* root@davejingtian.org
* http://davejingtian.org
*/

public class float_number
{
public static void main(String args[])
{
float f;
for (f = 1; f < 2; f += 0.1)
{
System.out.println(“————————“);
System.out.println(“pow(“+f+”, 6)=”+Math.pow(f, 6));
System.out.println(f+”*”+f+”*”+f+”*”+f+”*”+f+”*”+f+”=”+(f*f*f*f*f*f));
System.out.println(“(“+f+”*”+f+”*”+f+”)*(“+f+”*”+f+”*”+f+”)=”+((f*f*f)*(f*f*f)));
}}}

Part of output:
————————
pow(1.9000002, 6)=47.04591287880129
1.9000002*1.9000002*1.9000002*1.9000002*1.9000002*1.9000002=47.045914
(1.9000002*1.9000002*1.9000002)*(1.9000002*1.9000002*1.9000002)=47.045918

Again, similar output like float in C. If double is used, it looks like below. Again, the key point here is double is better but not exactly the same!

————————
pow(1.7000000000000006, 6)=24.137569000000052
1.7000000000000006*1.7000000000000006*1.7000000000000006*1.7000000000000006*1.7000000000000006*1.7000000000000006=24.13756900000005
(1.7000000000000006*1.7000000000000006*1.7000000000000006)*(1.7000000000000006*1.7000000000000006*1.7000000000000006)=24.137569000000045
————————
pow(1.8000000000000007, 6)=34.01222400000008
1.8000000000000007*1.8000000000000007*1.8000000000000007*1.8000000000000007*1.8000000000000007*1.8000000000000007=34.01222400000008
(1.8000000000000007*1.8000000000000007*1.8000000000000007)*(1.8000000000000007*1.8000000000000007*1.8000000000000007)=34.01222400000008
————————
pow(1.9000000000000008, 6)=47.04588100000012
1.9000000000000008*1.9000000000000008*1.9000000000000008*1.9000000000000008*1.9000000000000008*1.9000000000000008=47.04588100000012
(1.9000000000000008*1.9000000000000008*1.9000000000000008)*(1.9000000000000008*1.9000000000000008*1.9000000000000008)=47.04588100000012

2. conclusion

For precision, functions calls (provided by the corresponding Math lib) usually are better than the raw operators. And this is the reason why there are such function calls. Also, double type is better than the float indeed. For performance, operators should be better as without function cost. For optimization, because of Float Point Math in computers is not always equal to the one in reality, compilers usually stick with the user’s input rather than ‘finding a better way’. Like mentioned in the StackOverflow post, if (a*a*a)*(a*a*a) is what you want, write it in that way.

Posted in Programming, Stuff about Compiler | 1 Comment