This post is the summary based on a recent discussion in the kernelnewbies mail list. Thanks to the collective intelligence, the Linux kernel ARP behavior with multiple NIC is now pretty clear.
0. Original Question
“Assume there are two interfaces eth0 and eth1 each configured with IPs belonging to different subnets. Say IP1 is assigned to eth0 and IP2 to eth1. Now if an ARP request comes on eth0 for IP2, what is the behavior on Linux?”
Without code checking, the answer comes from “Documentation/networking/ip-sysctl.txt” by looking at “/proc/sys/net/ipv4/conf/“
If IP1 and IP2 are in different subnets, as the question, arp_filter is the controller:
“0 – (default) The kernel can respond to arp requests with addresses from other interfaces. This may seem wrong but it usually makes sense, because it increases the chance of successful communication. IP addresses are owned by the complete host on Linux, not by particular interfaces. Only for more complex setups like load-balancing, does this behavior cause problems.“
If IP1 and IP2 are in the same subnets, arp_announce and arp_ignore are the controller.