Networking Using Qemu/KVM
I have played with two different systems for providing virtual machines, Qemu/KVM and Xen. Xen and the necessary utilities to manipulate it was provided with CentOS 5, Qemu/KVM is provided with CentOS 6. Of the two, I found Xen easier to get to work, but Qemu/KVM to be far more flexible and generally friendlier in terms of resource usage. Unfortunately, CentOS 6.2 package for Qemu/KVM only includes that portion of Qemu/KVM that provides KVM acceleration which means you can't use it to emulate different hardware. To do that you have to compile Qemu yourself. The first major challenge I faced was networking.
With Xen, networking was easy. I never really became familiar with the internals because I didn't have to. I created virtual machines, configured their interfaces with the network manager and everything just worked.
Not so with Qemu/KVM! It is necessary to setup a bridge for Qemu/KVM networking to work. Don't bother to do it with Network Manager because it doesn't know about bridges. I found advice for how to work around this on the Internet that was wrong which is why I'm documenting what did work here.
All the magic happens in /etc/sysconfig/network-scripts directory and /etc/resolv.conf. Why DNS configuration would be included in the network scripts is a bit of a mystery to me anyway since it's really a service that sits on top of the network, not part of it. I expect it has to do with DHCP. Since DNS servers can be assigned via DHCP along with IP address, it was conveinient to include DNS information in the network scripts.
Normally, Network Manager would read ifcfg-eth0 and use the lines in blue
DNS1=22.214.171.124 DNS2=126.96.36.199 DOMAIN="eskimo.com"to generate /etc/resolv.conf as follows:
# Generated by NetworkManager search eskimo.com nameserver 188.8.131.52 nameserver 184.108.40.206
However, when you create a br0 bridge device, Network Manager will not pick up the DNS server and domain information from ifcfg-br0. Worse, once you create the ifcfg-br0 file, even if you still have the DNS information in ifcfg-eth0, Network Manager won't pick it up there either.
But wait! It gets even better. Creating a resolv.conf file is no big deal to do by hand, but if you do not disable Network Manager in services, it will overwrite resolv.conf with useless crap each time you reboot leaving your machine without the ability to resolve addresses.
Here is where I was mislead by some of what I read on the net. I read that if you put NM_CONTROLLED="no" in the network-scripts, Network Manager will leave them alone. But what I learned is that even if I put this in both ifcfg-eth0 and ifcfg-br0, Network Manager STILL overwrote my resolv.conf file.
So, the very first thing to do is go into System->Administration->Services on the host machine and disable and stop Network Manager. If you do not do this, Network Manager will overwrite your /etc/resolve.conf and leave your machine without working DNS everytime you reboot once you have the bridge device in place.
After that you need to create an ifcfg-br0 file that looks something like this:
DEVICE="br0" NM_CONTROLLED="no" IPV6INIT=no TYPE=Bridge BOOTPROTO=none ONBOOT=yes IPADDR=220.127.116.11 NETMASK=255.255.255.0 GATEWAY=18.104.22.168 DEFROUTE=yes
Adjust the IPADDR, NETMASK, and GATEWAY as appropriate for your network. If you do not have IPv6 addresses, be sure to include IPV6INIT=no. Otherwise it will take a long time for the interface to comeup and things like NTP, NIS, NFS, that get started as the machine boots up, will fail. Some have also suggested putting MTU=9000 on the bridge, but I haven't done so. My thoughts are that unless you have a physical interface and switch capable of jumbo packets, setting the MTU up on the bridge is just going to result in a lot of unnecessary packet fragmentation and reassembly.
You will also need to modify your ifcfg-eth0 file to look something like this:
DEVICE="eth0" NM_CONTROLLED="no" ONBOOT=yes HWADDR=C8:60:00:32:FE:DA IPV6INIT=no BRIDGE=br0
Lastly, you need to manually create an /etc/resolv.conf file. It is quite simple having only the domain, or a list of search domains, and name servers. Something like this:
domain eskimo.com nameserver 22.214.171.124 nameserver 126.96.36.199
Now after you've done this either reboot the machine or type:
service network restart
I'd really suggest a reboot so that you can make sure that Network Manager is really disabled and isn't breaking your resolv.conf file.
This is the network configuration necessary on the host. On the guest machines you can use network manager and configure eth0 just as you would a real physical ethernet. Nothing special is required on the guest side.