=== Full CPUID Control for bhyve

Contact: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>

==== Project Overview

Ongoing work on this project aims to integrate the existing proof-of-concept work into FreeBSD, and to add the following usability features:

- a user-friendly configuration method to override individual bits, parts or even whole CPUID functions as needed, while keeping the rest of the host CPUID information or a pre-defined CPUID configuration
- a user-friendly configuration method for the hypervisor signature reported by bhyve
- a set of predefined CPUID configurations based on common x86 architecture levels, perhaps also including a set of CPUID data for a few real CPU models, and a user-friendly configuration method to choose one for a VM

==== Changes during the last quarter

===== Extensions to bhyvectl
The CPUID configuration prototype already implemented a new command for bhyvectl, `--get-cpuid-cfg`, to query the CPUID configuration of a vCPU.

Another new command for bhyvectl has now been implemented, `--get-cpuid`, which fetches CPUID values emulated for a vCPU.

There are several modes of operation of `--get-cpuid`:

`bhyvectl --get-cpuid=<leaf>[,<index>]`:: This fetches the emulated CPUID value for an individual CPUID leaf and optionally index.
`bhyvectl --get-cpuid[=all]`:: This fetches the complete set of emulated CPUID values, for all supported leaves and all supported indices.
`bhyvectl --get-cpuid=all,p`:: Same as above, but the output is parsable as a set of CPUID configuration options for bhyve.
`bhyvectl --get-cpuid=all,s`:: Same as above, but the output is "sparse", meaning that only CPUID leaves and indices are printed if their values on the given vCPU (with the `--cpu` switch) differ from the same leaf and index on vCPU 0.

The latter two can be used to construct a baseline CPUID configuration based on the CPUID emulation of bhyve, which can then be modified as desired.


===== More flexible CPUID configuration
The CPUID configuration mechanism implemented as part of the prototype has been reworked to allow a more flexible and fine-grained configuration that does not require the whole set of CPUID information to be specified.
Here is an example configuration:

-------------------
cpuid.0x00000001=edx|=0x00040000
cpuid.0x00000003=ecx=0x01234567,edx=0x89abcdef
vcpu.1.cpuid.0x00000003=ecx|=0x10000000
vcpu.2.cpuid.0x00000003=ecx|=0x20000000
vcpu.3.cpuid.0x00000003=ecx|=0x30000000
vcpu.4.cpuid.0x00000003=ecx|=0x40000000
vcpu.5.cpuid.0x00000003=ecx|=0x50000000
vcpu.6.cpuid.0x00000003=ecx|=0x60000000
vcpu.7.cpuid.0x00000003=ecx|=0x70000000
vcpu.8.cpuid.0x00000003=ecx|=0x80000000
vcpu.9.cpuid.0x00000003=ecx|=0x90000000
vcpu.10.cpuid.0x00000003=ecx|=0xa0000000
vcpu.11.cpuid.0x00000003=ecx|=0xb0000000
vcpu.12.cpuid.0x00000003=ecx|=0xc0000000
vcpu.13.cpuid.0x00000003=ecx|=0xd0000000
vcpu.14.cpuid.0x00000003=ecx|=0xe0000000
vcpu.15.cpuid.0x00000003=ecx|=0xf0000000
cpuid.0x0000000b,0=edx=0xffffffff
cpuid.0x0000000b,1=edx=0xeeeeeeee
cpuid.0x0000000b,2=edx=0xdddddddd
-------------------

The above CPUID configuration enables the Processor Serial Number (0x00000001, EDX bit 18) feature on all vCPUs and sets a default serial of 0x01234567, 0x89abcef.

On each vCPU other than vCPU 0, it overwrites one digit of the serial number
with the vCPU ID so that each vCPU has a unique serial number.

The last three assignments does not make any sense.
Leaf 0x0000000b contains topology information in all indices, and register EDX always contains the x2APIC ID of the vCPU.
Consequently, these last three assignments will be silently ignored.


===== Using x86 architecture levels

It is now possible to specify a x86 architecture level:

-------------------
cpuid.archlevel=v1
-------------------

This will disable all features of x86 architecture level v2 to v3, restricting the CPU featureset to that of x86 architecture level v1.

==== Plans for next quarter

* The x86 architecture level support should be more robust.
While a direct configuration of CPUID information should allow to change almost everything, for x86 architecture levels we should check whether all the features defined in an architecture level are actually supported by the hardware.
Setting an architecture level that is not supported by the CPU should be disallowed, or at least being warned about.

* Similar to setting a x86 architecture level, a mechanism will be implemented to override the hypervisor identification without requiring manually changing CPUID bits.

* Get the whole wad reviewed and committed into FreeBSD.

Sponsor: The FreeBSD Foundation
