=== Full CPUID Control for bhyve

Contact: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>

==== Project Overview

Ongoing work on this project aims to integrate the existing proof-of-concept work into FreeBSD, and to add the following usability features:

- a user-friendly configuration method to override individual bits, parts or even whole CPUID functions as needed, while keeping the rest of the host CPUID information or a pre-defined CPUID configuration
- a user-friendly configuration method for the hypervisor signature reported by bhyve
- a set of predefined CPUID configurations based on common x86 architecture levels, perhaps also including a set of CPUID data for a few real CPU models, and a user-friendly configuration method to choose one for a VM

==== Changes during the last quarter

===== Extensions to the bhyve configuration syntax

The existing bhyve configuration file syntax was limited to assignments to configuration nodes, which parsed into tree-structured set of nvlists internally.

In order to allow a more flexible configuration using pre-defined configuration file snippets, the bhyve configuration file syntax has been augmented with two new features:

- A configuration node can be appended to with the `+=` operator.
  If a value has already been assigned to the same configuration node, the new value is appended to the old value, separated by a comma.
- Another configuration file can be included with an `include` directive.
  Absolute and relative paths are supported, but relative paths are understood as relative to the path of the configuration file the include directive appears in.

===== Yet more flexible CPUID configuration

The CPUID configuration mechanism has been extended to support symbolic feature names, where each name corresponds to a particular bit of a register for a CPUID function.
This is particularly useful for individual instruction set extensions or other optional features that can be turned off or on at will.

So instead of configuring `cpuid.0x00000001+=edx|=0x00040000`, you can now simply write `cpuid.enable+=psn`.
There is also a corresponding option `cpuid.disable` for symbolic features.

Using bit manipulation on CPUID register is of course still supported as not all CPUID features can be expressed in terms of individual feature bits.


===== Using x86 architecture levels

The x86 architecture levels are now implemented as a set of pre-defined configuration file snippets, each higher level building on top of the lower level.

To illustrate this, here's what x86-64-v3.cfg looks now:

-------------------
include x86-64-v2.cfg
cpuid.enable+=fma,movbe,osxsave,avx,f16c
cpuid.enable+=bmi1,avx2,bmi2
cpuid.enable+=abm
-------------------

The CPU or architecture model can be selected either by including the corresponding configuration file from `/usr/share/bhyve/cpu` in the bhyve configuration file, or it can be specified on the command line as part of the vCPU configuration: `bhyve [...] -c 16,model=x86-64-v1 [...]`

===== CPUID constraints

In order to prevent accidentally enabling CPUID features for a guest CPU which the host CPU does not support, the CPUID configuration parser has been extended with a list of CPUID constraints, which are consulted when a particular CPUID register value is set or modified.

For any register in a CPUID function, a constraint can specify which bits are immutable and which may be cleared only.

The constraints are checked after the CPUID configuration has been applied.


==== Plans for next quarter

* Implement overrides for CPUID constraints to ignore any constraints or to warn about violation of a constraint instead of failing VM setup.
* Provide more pre-defined CPU configurations based on real-world CPUs.
* Similar to setting a x86 architecture level, a mechanism will be implemented to override the hypervisor identification without requiring manually changing CPUID bits.
* Get the whole wad reviewed and committed into FreeBSD.

Sponsor: The FreeBSD Foundation
