bhyve guest illegal instruction

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

bhyve guest illegal instruction

Chuck Tuffli-2
(apologies for the resend, this time with a Subject)

John / Peter

I'm running an application in a Linux guest under bhyve that is core
dumping because of an Illegal instruction. Running a simplified case in gdb
shows:
$ gdb -q rdtscp
Reading symbols from rdtscp...done.
(gdb) run
Starting program: /home/tuffli/rdtscp

Program received signal SIGILL, Illegal instruction.
main (argc=1, argv=0x7fffffffeae8) at rdtscp.c:12
12 time1 = __builtin_ia32_rdtscp(&dummy);
(gdb) l
7       main(int argc, char *argv[])
8       {
9                uint64_t time1, time2;
10              uint32_t dummy;
11
12              time1 = __builtin_ia32_rdtscp(&dummy);
13              usleep(1000);
14              time2 = __builtin_ia32_rdtscp(&dummy);
15
16              printf("time delta %lu\n", time2 - time1);
17
18              return (EXIT_SUCCESS);
19      }

This same program works on the FreeBSD 12-stable machine hosting the VM as
well as another bare-metal Linux host. Poking around in the vmm code, I
found
                        /*
                         * Hide rdtscp/ia32_tsc_aux until we know how
                         * to deal with them.
                         */
                        regs[3] &= ~AMDID_RDTSCP;
                        break;
in sys/amd64/vmm/x86.c which I _think_ is relevant because lscpu doesn't
show the rdtscp flag. If this is the root cause, what would need to be done
to implement this?

--chuck
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

rdtscp support (was Re: bhyve guest illegal instruction)

Peter Grehan
Hi Chuck,

> 12              time1 = __builtin_ia32_rdtscp(&dummy);

  rdtscp shouldn't be used without checking that it's available via
CPUID first, but as you mentioned the feature is available on the host,
just hidden from the guest.

> This same program works on the FreeBSD 12-stable machine hosting the VM
> as well as another bare-metal Linux host. Poking around in the vmm code,
> I found
>                          /*
>                           * Hide rdtscp/ia32_tsc_aux until we know how
>                           * to deal with them.
>                           */
>                          regs[3] &= ~AMDID_RDTSCP;
>                          break;
> in sys/amd64/vmm/x86.c which I _think_ is relevant because lscpu doesn't
> show the rdtscp flag. If this is the root cause, what would need to be
> done to implement this?

  At a quick glance, if the feature is available on the host you'd need to
   - expose it via CPUID
   - save/restore the TSC_AUX MSR, but using the VMCS MSR h/w
save/restore mechanism that will have to be resurrected. (this avoids
any preemption issues,even at NMI level).
   - set the "enable RDTSCP" VM-execution control to one in the VMCS

  That being said, I've heard anecdotally that rdtscp results in
VM-exits on other hypervisors so there may be reason to emulate it
rather than allow a pass-thru. More investigation may be required.

later,

Peter.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"