KVM and OpenStack are fundamentally different technologies. KVM is a hypervisor, a virtualization scheduler running within the physical machine kernel. It's responsible for fragmenting, isolating, and redistributing low-level resources—such as the CPU's VT-x instruction set, memory's EPT page table mapping, and disk's QEMU block device I/O queues—to virtual machines. OpenStack, on the other hand, is a loosely coupled management framework pieced together from over a dozen components like Nova, Neutron, Cinder, and Keystone. It doesn't virtualize anything itself; its job is to invoke KVM on compute nodes, manage Ceph for distributed storage, control VXLAN tunnels on switches, and maintain tenant quotas and authentication. Comparing the architectures of KVM and OpenStack is like comparing the principles of an internal combustion engine and a vehicle manufacturing platform—the former is the power core, the latter is the production line scheduling system—the levels are completely different.
From a microscopic perspective of the I/O path, the data packet routing in KVM's direct output and OpenStack's overlay network is drastically different. A pure KVM running on a single physical machine uses virtual machine network cards (NICs) directly bridged to the physical NIC via macvtap or SR-IOV. Data packets originate from the virtual machine's kernel protocol stack, pass through the host machine's bridge or OVS kernel module, and are forwarded almost losslessly to the uplink port of the rack switch. This path involves no additional encapsulation/decapsulation overhead, reducing latency to microseconds. Japanese data centers typically connect to local backbone networks like NTT, IIJ, and KDDI, which inherently offer low latency advantages. Combined with KVM's pass-through capability, the RTT from Tokyo to Osaka can be consistently below 3 milliseconds.
However, the introduction of OpenStack changes everything. Neutron's default VXLAN mode requires each virtual machine's Ethernet frame to be tagged with a 24-bit VNI identifier before being encapsulated into a UDP packet and sent through the host machine's physical NIC. This means that the data packet is stripped of an extra layer of data before even leaving the rack. The receiving end also needs to remove the VXLAN header to reconstruct the original frame. While the hardware offloading process partially alleviates the burden of encapsulation and decapsulation, the CPU interrupt response and context switching overhead are still real. In markets like Japan, where bandwidth billing is extremely precise, each additional header byte means a decrease in the effective payload ratio, resulting in less effective data being transmitted for the same amount of traffic. A more practical problem is that when the OpenStack control node experiences a Neutron-server restart or the database connection pool becomes full, all virtual machine network update operations will queue and become blocked, causing port binding timeouts for new VMs and leading to instance creation failures—something that simply doesn't occur in a KVM bare metal pipe environment.
Now let's look at the storage stack differences. When KVM is used with a locally connected NVMe SSD, each IO instruction travels directly from the PCIe bus to the NAND flash, with latency in the tens of microseconds range. fio measurements show 4K random write IOPS reaching hundreds of thousands. However, OpenStack's Cinder volume storage inevitably goes through iSCSI or RBD network protocols. Even with an all-flash backend, the addition of a TCP/IP stack and a RADOS object mapping layer directly doubles the IO latency. Many Japanese data centers use OpenStack environments with shared storage pools. Cross-rack Ceph data synchronization also consumes internal network bandwidth. When the cluster undergoes scrubbing or rebalancing, the performance of the database virtual machines running on it will significantly decrease. You can clearly feel that, with the same 2-core, 4GB configuration, the MySQL QPS in a KVM direct storage environment is 30-40% higher than in an OpenStack + Ceph environment. This is the inherent overhead of the architecture itself, unrelated to hardware configuration.
From a fault domain perspective, the smallest unit of failure in KVM single-host mode is the physical machine itself. If one machine goes down, the virtual machines running on it will also fail, making the impact clear and predictable. Troubleshooting only requires checking ILO logs, viewing kernel panic information, and replacing faulty memory modules. In contrast, the fault domain of OpenStack is a complex topology spanning the control plane, network plane, and storage plane. A common scenario you might encounter is that Nova-scheduler is still scheduling normally, but the backlog in the RabbitMQ message queue causes instance creation requests to time out; or the Glance image service is working normally, but the Swift object storage authentication token has expired, causing newly created VMs to be unable to find the image file. Japanese operations teams emphasize "locating the cause of the problem and providing a temporary workaround within 30 minutes," but in an OpenStack system where components are interdependent, root cause diagnosis often requires consultation with the network group, storage group, and virtualization group simultaneously. The cost of this kind of wrangling is disastrous during peak sales periods.
Regarding resource overselling, KVM's semantics are much clearer. It allows you to set memory overuse coefficients at the host level and compress duplicate memory pages through KSM kernel page merging. This overselling is deterministic; you know the upper limit and the threshold for triggering swap penalties. OpenStack's Flavor specification quota management introduces multiple layers of abstraction. Users see a fixed number of vCPUs and memory values in the control panel, but the underlying resource pool may be composed of a mix of physical machines from different generations. This results in VMs with the same flavor having performance entirely dependent on which older host machine they are scheduled to run on. In the Japanese cloud market, many small local service providers claiming to offer OpenStack services have long neglected to upgrade their underlying QEMU versions to maintain API compatibility. This leads to incompatibility between old and new instruction sets, causing VMs to crash immediately after hot migration to a new physical machine.
The statistical distribution of network latency also illustrates this point. Using real-world test data from Tokyo, for example, in data centers connected to the BBIX exchange, the standard deviation of ICMP latency jitter in a pure KVM environment remains within 0.3 milliseconds, while in an OpenStack environment, affected by VXLAN tunnels and Netfilter rules of virtual routers, the standard deviation is typically above 2 milliseconds. This difference is amplified in real-time audio and video services. Japanese vendors providing WebRTC relay services have largely bypassed OpenStack, preferring to write their own scheduling scripts using the libvirt API rather than bear the unpredictable jumps introduced by Neutron.
In terms of management boundaries, KVM gives you complete control. You can directly modify the host machine's sysctl parameters to optimize the TCP BBR congestion algorithm, manually bind CPU cores to isolate dedicated interrupt handling cores, and customize the huge page size to improve the TLB hit rate for large memory pages. This level of freedom is highly valued in Japan's highly customized enterprise application scenarios because their business code often has specific kernel parameter dependencies. To maintain multi-tenant isolation and API consistency, OpenStack forcibly blocks most low-level tuning interfaces. Want to adjust the vCPU pinning policy of a virtual machine? Yes, but you need to modify the `extra_specs` property of `flavor` in `nova.conf` and then restart the `nova-compute` service, affecting all tenants on the same host machine. This cost is unacceptable to many Japanese CTOs.
The perspectives on dual-site backup and disaster recovery also need to be clear. KVM paired with native LVM snapshots or QEMU incremental backups can achieve block-level precise consistency recovery points. Combined with Japan's high-speed intranet dedicated lines, asynchronous replication latency across availability zones can be reduced to the second level. OpenStack, on the other hand, forcibly relies on Ceilometer monitoring and Aodh alerts to trigger backup policies. This data collection and analysis chain is too long; often, before the backup task even starts, new data has already been written to the source volume. Ultimately, the restored virtual machine state is fragmented, and the business logs and database binlog don't match.
The conclusion is quite clear: don't be fooled by the halo effect of OpenStack. The so-called "OpenStack-based cloud servers" you buy in Japan are essentially still KVM doing the work; OpenStack is just a management puppet attached to the front. The difference lies in how efficient this puppet is and whether it will trip you up at critical moments. If you prioritize deterministic IO performance, transparent underlying operations, and rapid fault recovery, choose KVM virtualization directly and reject those extra management middleware layers. If you need deep resource isolation for multi-tenancy, a self-service quota application process, and a standardized API integration system, then OpenStack is an unavoidable option, but be prepared to invest additional maintenance budget.
EN
CN