Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
Hi,

So I have been trying to get Xen to boot on my laptop. It is getting
stuck with the following error messages:

(XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
(XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
9ce13000. iommu reg - ffff82c000203000
(XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
(XEN) ........... done
(XEN) initial low memory virq threshold set at 0x200 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is keeping VGA console
(XEN) Boot video device 00:02.0
(XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 320 kB init memory
FreeBSD PVH running on xen-3.0-x86_64p
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
1994 The regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
    [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
    Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
    Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
FXSR,SSE,SSE2,HTT>
    Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
    AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
    AMD Features2=0x21<<LAHF,ABM>
    Structured Extended Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
Hypervisor: Origin = "XenVMMXenVMM"
real memory  = 9991770112 (9528 MB)
avail memory = 7936724992 (7569 MB)
(XEN) d0v1 Triple fault - invoking HVM shutdown action 0
(XEN) *** Dumping Dom0 vcpu#1 state: ***
(XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    0020:[<ffffffff8102042b>]
(XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
(XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx: 0000000000000001
(XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi: fffffe0003bb6078
(XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8: 00000000ffffffff
(XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4: 0000000000000020
(XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
(XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
(XEN) Guest stack trace from rsp=fffffe00009fffa0:
(XEN)   Fault while accessing guest memory.
(XEN) Hardware Dom0 halted: halting machine

Any idea why this is happening and how can I fix this?

Thanks,
Pratyush Yadav
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Edward Napierala

> On 18 May 2018, at 12:59, Pratyush Yadav <[hidden email]> wrote:
>
> Hi,
>
> So I have been trying to get Xen to boot on my laptop. It is getting
> stuck with the following error messages:
>
> (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
> (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
> 9ce13000. iommu reg - ffff82c000203000
> (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
> (XEN) ........... done
> (XEN) initial low memory virq threshold set at 0x200 pages.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) Xen is keeping VGA console
> (XEN) Boot video device 00:02.0
> (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
> (XEN) Freed 320 kB init memory
> FreeBSD PVH running on xen-3.0-x86_64p
> Copyright (c) 1992-2018 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
> 1994 The regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
>    [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
> WARNING: WITNESS option enabled, expect reduced performance.
> VT(vga): text 80x25
> CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
>    Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
>    Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
> FXSR,SSE,SSE2,HTT>
>    Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
> POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
>    AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
>    AMD Features2=0x21<<LAHF,ABM>
>    Structured Extended Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
> Hypervisor: Origin = "XenVMMXenVMM"
> real memory  = 9991770112 (9528 MB)
> avail memory = 7936724992 (7569 MB)
> (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
> (XEN) *** Dumping Dom0 vcpu#1 state: ***
> (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    0020:[<ffffffff8102042b>]
> (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
> (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx: 0000000000000001
> (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi: fffffe0003bb6078
> (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8: 00000000ffffffff
> (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4: 0000000000000020
> (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
> (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
> (XEN) Guest stack trace from rsp=fffffe00009fffa0:
> (XEN)   Fault while accessing guest memory.
> (XEN) Hardware Dom0 halted: halting machine
>
> Any idea why this is happening and how can I fix this?

First thing I’d try is to checkout an earlier version - eg from three months ago) and see if it persist.  Another thing to do (in parallel) is to file a PR (http://bugs.freebsd.org).

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Akshay Jaggi-2
Looks to me like a configuration error. Worst case, incompatible hardware.
Iirc, you mentioned Xen and Freebsd were on a USB drive. Can you try using
a hard drive partition?
Also, there was an option to enable verbose logging.

Regards,
Akshay

On Fri 18 May, 2018, 2:13 PM Edward Tomasz Napierała, <[hidden email]>
wrote:

>
> > On 18 May 2018, at 12:59, Pratyush Yadav <[hidden email]> wrote:
> >
> > Hi,
> >
> > So I have been trying to get Xen to boot on my laptop. It is getting
> > stuck with the following error messages:
> >
> > (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
> > (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
> > 9ce13000. iommu reg - ffff82c000203000
> > (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
> > (XEN) ........... done
> > (XEN) initial low memory virq threshold set at 0x200 pages.
> > (XEN) Std. Loglevel: All
> > (XEN) Guest Loglevel: All
> > (XEN) Xen is keeping VGA console
> > (XEN) Boot video device 00:02.0
> > (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input
> to Xen)
> > (XEN) Freed 320 kB init memory
> > FreeBSD PVH running on xen-3.0-x86_64p
> > Copyright (c) 1992-2018 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
> > 1994 The regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
> >    [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
> amd64
> > WARNING: WITNESS option enabled, expect reduced performance.
> > VT(vga): text 80x25
> > CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
> >    Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
> >
> Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
> > FXSR,SSE,SSE2,HTT>
> >
> Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
> > POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
> >    AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
> >    AMD Features2=0x21<<LAHF,ABM>
> >    Structured Extended
> Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
> > Hypervisor: Origin = "XenVMMXenVMM"
> > real memory  = 9991770112 (9528 MB)
> > avail memory = 7936724992 (7569 MB)
> > (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
> > (XEN) *** Dumping Dom0 vcpu#1 state: ***
> > (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
> > (XEN) CPU:    1
> > (XEN) RIP:    0020:[<ffffffff8102042b>]
> > (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
> > (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx:
> 0000000000000001
> > (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi:
> fffffe0003bb6078
> > (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8:
> 00000000ffffffff
> > (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4:
> 0000000000000020
> > (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
> > (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
> > (XEN) Guest stack trace from rsp=fffffe00009fffa0:
> > (XEN)   Fault while accessing guest memory.
> > (XEN) Hardware Dom0 halted: halting machine
> >
> > Any idea why this is happening and how can I fix this?
>
> First thing I’d try is to checkout an earlier version - eg from three
> months ago) and see if it persist.  Another thing to do (in parallel) is to
> file a PR (http://bugs.freebsd.org).
>
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Roger Pau Monné
In reply to this post by Pratyush Yadav
On Fri, May 18, 2018 at 05:29:06PM +0530, Pratyush Yadav wrote:

> Hi,
>
> So I have been trying to get Xen to boot on my laptop. It is getting
> stuck with the following error messages:
>
> (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
> (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
> 9ce13000. iommu reg - ffff82c000203000
> (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
> (XEN) ........... done
> (XEN) initial low memory virq threshold set at 0x200 pages.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) Xen is keeping VGA console
> (XEN) Boot video device 00:02.0
> (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
> (XEN) Freed 320 kB init memory
> FreeBSD PVH running on xen-3.0-x86_64p
> Copyright (c) 1992-2018 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
> 1994 The regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
>     [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
> WARNING: WITNESS option enabled, expect reduced performance.
> VT(vga): text 80x25
> CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
>     Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
>     Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
> FXSR,SSE,SSE2,HTT>
>     Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
> POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
>     AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
>     AMD Features2=0x21<<LAHF,ABM>
>     Structured Extended Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
> Hypervisor: Origin = "XenVMMXenVMM"
> real memory  = 9991770112 (9528 MB)
> avail memory = 7936724992 (7569 MB)
> (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
> (XEN) *** Dumping Dom0 vcpu#1 state: ***
> (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    0020:[<ffffffff8102042b>]
> (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
> (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx: 0000000000000001
> (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi: fffffe0003bb6078
> (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8: 00000000ffffffff
> (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4: 0000000000000020
> (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
> (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
> (XEN) Guest stack trace from rsp=fffffe00009fffa0:
> (XEN)   Fault while accessing guest memory.
> (XEN) Hardware Dom0 halted: halting machine
>
> Any idea why this is happening and how can I fix this?

Can you execute:

addr2line -e /usr/lib/debug/boot/kernel/kernel.debug ffffffff8102042b

This should print a file and line number that would help in order to
debug what's going on.

Since it seems like the crash is caused by a triple fault on an AP,
you can try to boot with dom0_max_vcpus=1 on the xen_cmdline, that
might prevent the panic from happening.

We need however to figure out what's going on and fix it.

Thanks, Roger.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
Hi,

The line is

/usr/src/sys/amd64/amd64/mp_machdep.c:307

Also, I tried with dom0_max_vcpus=1 but it keeps rebooting (despite
noreboot in the xen command line)

On Sat 19 May, 2018, 1:40 PM Roger Pau Monné, <[hidden email]> wrote:

> On Fri, May 18, 2018 at 05:29:06PM +0530, Pratyush Yadav wrote:
> > Hi,
> >
> > So I have been trying to get Xen to boot on my laptop. It is getting
> > stuck with the following error messages:
> >
> > (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
> > (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
> > 9ce13000. iommu reg - ffff82c000203000
> > (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
> > (XEN) ........... done
> > (XEN) initial low memory virq threshold set at 0x200 pages.
> > (XEN) Std. Loglevel: All
> > (XEN) Guest Loglevel: All
> > (XEN) Xen is keeping VGA console
> > (XEN) Boot video device 00:02.0
> > (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input
> to Xen)
> > (XEN) Freed 320 kB init memory
> > FreeBSD PVH running on xen-3.0-x86_64p
> > Copyright (c) 1992-2018 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
> > 1994 The regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
> >     [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
> amd64
> > WARNING: WITNESS option enabled, expect reduced performance.
> > VT(vga): text 80x25
> > CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
> >     Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
> >
>  Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
> > FXSR,SSE,SSE2,HTT>
> >
>  Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
> > POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
> >     AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
> >     AMD Features2=0x21<<LAHF,ABM>
> >     Structured Extended
> Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
> > Hypervisor: Origin = "XenVMMXenVMM"
> > real memory  = 9991770112 (9528 MB)
> > avail memory = 7936724992 (7569 MB)
> > (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
> > (XEN) *** Dumping Dom0 vcpu#1 state: ***
> > (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
> > (XEN) CPU:    1
> > (XEN) RIP:    0020:[<ffffffff8102042b>]
> > (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
> > (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx:
> 0000000000000001
> > (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi:
> fffffe0003bb6078
> > (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8:
> 00000000ffffffff
> > (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4:
> 0000000000000020
> > (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
> > (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
> > (XEN) Guest stack trace from rsp=fffffe00009fffa0:
> > (XEN)   Fault while accessing guest memory.
> > (XEN) Hardware Dom0 halted: halting machine
> >
> > Any idea why this is happening and how can I fix this?
>
> Can you execute:
>
> addr2line -e /usr/lib/debug/boot/kernel/kernel.debug ffffffff8102042b
>
> This should print a file and line number that would help in order to
> debug what's going on.
>
> Since it seems like the crash is caused by a triple fault on an AP,
> you can try to boot with dom0_max_vcpus=1 on the xen_cmdline, that
> might prevent the panic from happening.
>
> We need however to figure out what's going on and fix it.
>
> Thanks, Roger.
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
In reply to this post by Akshay Jaggi-2
Hi,

On Fri 18 May, 2018, 6:55 PM Akshay Jaggi, <[hidden email]> wrote:

> Looks to me like a configuration error. Worst case, incompatible hardware.
> Iirc, you mentioned Xen and Freebsd were on a USB drive. Can you try using
> a hard drive partition?
> Also, there was an option to enable verbose logging.
>

I installed on my HDD, but it makes no difference. Also, I have set log
level to all, still the logs are not being saved.

Regards,

> Akshay
>
> On Fri 18 May, 2018, 2:13 PM Edward Tomasz Napierała, <[hidden email]>
> wrote:
>
>>
>> > On 18 May 2018, at 12:59, Pratyush Yadav <[hidden email]> wrote:
>> >
>> > Hi,
>> >
>> > So I have been trying to get Xen to boot on my laptop. It is getting
>> > stuck with the following error messages:
>> >
>> > (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
>> > (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
>> > 9ce13000. iommu reg - ffff82c000203000
>> > (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
>> > (XEN) ........... done
>> > (XEN) initial low memory virq threshold set at 0x200 pages.
>> > (XEN) Std. Loglevel: All
>> > (XEN) Guest Loglevel: All
>> > (XEN) Xen is keeping VGA console
>> > (XEN) Boot video device 00:02.0
>> > (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch
>> input to Xen)
>> > (XEN) Freed 320 kB init memory
>> > FreeBSD PVH running on xen-3.0-x86_64p
>> > Copyright (c) 1992-2018 The FreeBSD Project.
>> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
>> > 1994 The regents of the University of California. All rights reserved.
>> > FreeBSD is a registered trademark of The FreeBSD Foundation.
>> > FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
>> >    [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
>> amd64
>> > WARNING: WITNESS option enabled, expect reduced performance.
>> > VT(vga): text 80x25
>> > CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class
>> CPU)
>> >    Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
>> >
>> Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
>> > FXSR,SSE,SSE2,HTT>
>> >
>> Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
>> > POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
>> >    AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
>> >    AMD Features2=0x21<<LAHF,ABM>
>> >    Structured Extended
>> Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
>> > Hypervisor: Origin = "XenVMMXenVMM"
>> > real memory  = 9991770112 (9528 MB)
>> > avail memory = 7936724992 (7569 MB)
>> > (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
>> > (XEN) *** Dumping Dom0 vcpu#1 state: ***
>> > (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
>> > (XEN) CPU:    1
>> > (XEN) RIP:    0020:[<ffffffff8102042b>]
>> > (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
>> > (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx:
>> 0000000000000001
>> > (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi:
>> fffffe0003bb6078
>> > (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8:
>> 00000000ffffffff
>> > (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4:
>> 0000000000000020
>> > (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
>> > (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
>> > (XEN) Guest stack trace from rsp=fffffe00009fffa0:
>> > (XEN)   Fault while accessing guest memory.
>> > (XEN) Hardware Dom0 halted: halting machine
>> >
>> > Any idea why this is happening and how can I fix this?
>>
>> First thing I’d try is to checkout an earlier version - eg from three
>> months ago) and see if it persist.  Another thing to do (in parallel) is to
>> file a PR (http://bugs.freebsd.org).
>>
>>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
In reply to this post by Pratyush Yadav
On Sat, May 19, 2018 at 3:45 PM, Pratyush Yadav <[hidden email]> wrote:
> Hi,
>
> The line is
>
> /usr/src/sys/amd64/amd64/mp_machdep.c:307
>
> Also, I tried with dom0_max_vcpus=1 but it keeps rebooting (despite noreboot
> in the xen command line)

Correction: It boots with dom0_max_vcpus=1. It did not boot for me
earlier because I was trying to fix it by trying to make Xen from
source so I had actually deinstalled it when I tested with
max_vcpus=1. Now I reinstalled Xen and it works.

Thanks for the temporary workaround! Let me know if you want more info
about the panic.

>
> On Sat 19 May, 2018, 1:40 PM Roger Pau Monné, <[hidden email]> wrote:
>>
>> On Fri, May 18, 2018 at 05:29:06PM +0530, Pratyush Yadav wrote:
>> > Hi,
>> >
>> > So I have been trying to get Xen to boot on my laptop. It is getting
>> > stuck with the following error messages:
>> >
>> > (XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
>> > (XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
>> > 9ce13000. iommu reg - ffff82c000203000
>> > (XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
>> > (XEN) ........... done
>> > (XEN) initial low memory virq threshold set at 0x200 pages.
>> > (XEN) Std. Loglevel: All
>> > (XEN) Guest Loglevel: All
>> > (XEN) Xen is keeping VGA console
>> > (XEN) Boot video device 00:02.0
>> > (XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input
>> > to Xen)
>> > (XEN) Freed 320 kB init memory
>> > FreeBSD PVH running on xen-3.0-x86_64p
>> > Copyright (c) 1992-2018 The FreeBSD Project.
>> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
>> > 1994 The regents of the University of California. All rights reserved.
>> > FreeBSD is a registered trademark of The FreeBSD Foundation.
>> > FreeBSD 12.0-CURRENT #0 r333606: Mon May 14 19:59:08 UTC 2018
>> >
>> > [hidden email]:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
>> > WARNING: WITNESS option enabled, expect reduced performance.
>> > VT(vga): text 80x25
>> > CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class
>> > CPU)
>> >     Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
>> >
>> > Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,MCA,CMOV,PAT,ACPI,MMX
>> > FXSR,SSE,SSE2,HTT>
>> >
>> > Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE
>> > POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
>> >     AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
>> >     AMD Features2=0x21<<LAHF,ABM>
>> >     Structured Extended
>> > Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
>> > Hypervisor: Origin = "XenVMMXenVMM"
>> > real memory  = 9991770112 (9528 MB)
>> > avail memory = 7936724992 (7569 MB)
>> > (XEN) d0v1 Triple fault - invoking HVM shutdown action 0
>> > (XEN) *** Dumping Dom0 vcpu#1 state: ***
>> > (XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
>> > (XEN) CPU:    1
>> > (XEN) RIP:    0020:[<ffffffff8102042b>]
>> > (XEN) RFLAGS:  0000000000010016    CONTEXT: hvm guest (d0v1)
>> > (XEN) rax: 0000000000000000   rbx: ffffffff8201c100   rcx:
>> > 0000000000000001
>> > (XEN) rdx: 0000000000003078   rsi: ffffffff81b992f8   rdi:
>> > fffffe0003bb6078
>> > (XEN) rbp: fffffe00009ffff0   rsp: fffffe00009fffa0   r8:
>> > 00000000ffffffff
>> > (XEN) r15: 0000000000000400   cr0: 0000000000000011   cr4:
>> > 0000000000000020
>> > (XEN) cr3: 00000000035a4000   cr2: 0000000000000ff0
>> > (XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
>> > (XEN) Guest stack trace from rsp=fffffe00009fffa0:
>> > (XEN)   Fault while accessing guest memory.
>> > (XEN) Hardware Dom0 halted: halting machine
>> >
>> > Any idea why this is happening and how can I fix this?
>>
>> Can you execute:
>>
>> addr2line -e /usr/lib/debug/boot/kernel/kernel.debug ffffffff8102042b
>>
>> This should print a file and line number that would help in order to
>> debug what's going on.
>>
>> Since it seems like the crash is caused by a triple fault on an AP,
>> you can try to boot with dom0_max_vcpus=1 on the xen_cmdline, that
>> might prevent the panic from happening.
>>
>> We need however to figure out what's going on and fix it.
>>
>> Thanks, Roger.



--
Regards,
Pratyush Yadav
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Roger Pau Monné
In reply to this post by Pratyush Yadav
Please try to avoid top posting.

On Sat, May 19, 2018 at 03:45:41PM +0530, Pratyush Yadav wrote:
> Hi,
>
> The line is
>
> /usr/src/sys/amd64/amd64/mp_machdep.c:307

Hm, it seems like dbg_stack is not properly allocated. Can you please
try the above debug patch and paste the boot log?

---8<---
diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c
index 301461420874..8854242b4bf5 100644
--- a/sys/amd64/amd64/mp_machdep.c
+++ b/sys/amd64/amd64/mp_machdep.c
@@ -304,6 +304,7 @@ init_secondary(void)
 
  /* Save the per-cpu pointer for use by the DB# handler. */
  np = ((struct nmi_pcpu *) &dbg_stack[PAGE_SIZE]) - 1;
+printf("ID %d dbg_stack: %p per-cpu: %p\n", cpu, dbg_stack, np);
  np->np_pcpu = (register_t) pc;
 
  wrmsr(MSR_FSBASE, 0); /* User value */
@@ -403,6 +404,7 @@ native_start_all_aps(void)
     M_WAITOK | M_ZERO);
  dbg_stack = (char *)kmem_malloc(kernel_arena, PAGE_SIZE,
     M_WAITOK | M_ZERO);
+printf("allocated dbg_stack: %p\n", dbg_stack);
  dpcpu = (void *)kmem_malloc(kernel_arena, DPCPU_SIZE,
     M_WAITOK | M_ZERO);
 
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
On Mon, May 21, 2018 at 2:33 PM, Roger Pau Monné <[hidden email]>
wrote:
> Please try to avoid top posting.

Sorry, I didn't know. I Googled top posting just now, and realized it is
inconvenient for the reader.

> Hm, it seems like dbg_stack is not properly allocated. Can you please
> try the above debug patch and paste the boot log?
>
> ---8<---
> diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c
> index 301461420874..8854242b4bf5 100644
> --- a/sys/amd64/amd64/mp_machdep.c
> +++ b/sys/amd64/amd64/mp_machdep.c
> @@ -304,6 +304,7 @@ init_secondary(void)
>
>         /* Save the per-cpu pointer for use by the DB# handler. */
>         np = ((struct nmi_pcpu *) &dbg_stack[PAGE_SIZE]) - 1;
> +printf("ID %d dbg_stack: %p per-cpu: %p\n", cpu, dbg_stack, np);
>         np->np_pcpu = (register_t) pc;
>
>         wrmsr(MSR_FSBASE, 0);           /* User value */
> @@ -403,6 +404,7 @@ native_start_all_aps(void)
>                     M_WAITOK | M_ZERO);
>                 dbg_stack = (char *)kmem_malloc(kernel_arena, PAGE_SIZE,
>                     M_WAITOK | M_ZERO);
> +printf("allocated dbg_stack: %p\n", dbg_stack);
>                 dpcpu = (void *)kmem_malloc(kernel_arena, DPCPU_SIZE,
>                     M_WAITOK | M_ZERO);
>

I applied your patch. The boot stops at the same spot. Also, I can't see
any of the two print messages added in the patch. I booted from a Live CD
to look at the logs, but they are not stored. I checked dmesg.boot and
messages, and /var/log/xen/ none of them have the logs. They all have logs
of the previous boot.

This is all what I can see from my screen:

XEN) Scrubbing free RAM on 1 nodes using 4 CPUs
(XEN) [VT-D] DMAR: [DMA Read} Request device [0000:00:1a.0] fault addr
9ce13000. iommu reg - ffff82c000203000
(XEN) [VT-D] DMAR: reason 06 - PTE read access is not set
(XEN) ........... done
(XEN) initial low memory virq threshold set at 0x200 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is keeping VGA console
(XEN) Boot video device 00:02.0
(XEN) ***Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
Xen)
(XEN) Freed 320 kB init memory
FreeBSD PVH running on xen-3.0-x86_64p
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
1994 The regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-CURRENT #0 r333974M: Wed May 11 37:05 IST 2018
    root@freebsd-test:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (2494.22-MHz K8-class CPU)
    Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3
    Features=0x17c1cbf5<FPU,DE,TSC,MSR,PAE,MCE.CX8.APIC,SEP,
MCA,CMOV,PAT,ACPI,MMX
FXSR,SSE,SSE2,HTT>
    Features2=0xf6f83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.
1,SSE4.2,x2APIC,MOVBE
POPCNT,AESNI,XSAVE,AVX,F16C,RDRAND,HV>
    AMD Features=0x20101800<SYSCALL,<s12>NX,LM>
    AMD Features2=0x21<<LAHF,ABM>
    Structured Extended Features=0x2329<FSGSBASE,BMI1,AVX2,BMI2,ERMS,NFPUSG>
Hypervisor: Origin = "XenVMMXenVMM"
real memory  = 5982789632 (5705 MB)
avail memory = 4051214336 (3863 MB)
(XEN) d0v1 Triple fault - invoking HVM shutdown action 0
(XEN) *** Dumping Dom0 vcpu#1 state: ***
(XEN) ----[ Xen-4.7.2 x86_64 debug=n Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    0020:[<ffffffff80b52fbe>]
(XEN) RFLAGS:  0000000000010046    CONTEXT: hvm guest (d0v1)
(XEN) rax: 0000000000000000   rbx: fffff801649a1ff8   rcx: 00000000000000c7
(XEN) rdx: ffffffff81b166f0   rsi: 0000000000000000   rdi: fffff801649a1ff8
(XEN) rbp: fffffe00027e8bb0   rsp: fffffe0027e8b80   r8: fefefefefefefeff
(XEN) r15: ffffffff812f497c   cr0: 0000000000000011   cr4: 0000000000000020
(XEN) cr3: 0000000002e36000   cr2: 0000000000000000
(XEN) ds: 0028   es: 0028   fs: 0028   gs: 0028   ss: 0028   cs: 0020
(XEN) Guest stack trace from rsp=fffffe00009fffa0:
(XEN)   Fault while accessing guest memory.
(XEN) Hardware Dom0 halted: halting machine


When I run addr2line on ffffffff80b52fbe, I get:
./machine/pcpu.h:231

--
Regards,
Pratyush Yadav
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Roger Pau Monné
On Wed, May 23, 2018 at 12:57:59PM +0530, Pratyush Yadav wrote:

> On Mon, May 21, 2018 at 2:33 PM, Roger Pau Monné <[hidden email]>
> wrote:
> > Please try to avoid top posting.
>
> Sorry, I didn't know. I Googled top posting just now, and realized it is
> inconvenient for the reader.
>
> > Hm, it seems like dbg_stack is not properly allocated. Can you please
> > try the above debug patch and paste the boot log?
> >
> > ---8<---
> > diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c
> > index 301461420874..8854242b4bf5 100644
> > --- a/sys/amd64/amd64/mp_machdep.c
> > +++ b/sys/amd64/amd64/mp_machdep.c
> > @@ -304,6 +304,7 @@ init_secondary(void)
> >
> >         /* Save the per-cpu pointer for use by the DB# handler. */
> >         np = ((struct nmi_pcpu *) &dbg_stack[PAGE_SIZE]) - 1;
> > +printf("ID %d dbg_stack: %p per-cpu: %p\n", cpu, dbg_stack, np);
> >         np->np_pcpu = (register_t) pc;
> >
> >         wrmsr(MSR_FSBASE, 0);           /* User value */
> > @@ -403,6 +404,7 @@ native_start_all_aps(void)
> >                     M_WAITOK | M_ZERO);
> >                 dbg_stack = (char *)kmem_malloc(kernel_arena, PAGE_SIZE,
> >                     M_WAITOK | M_ZERO);
> > +printf("allocated dbg_stack: %p\n", dbg_stack);
> >                 dpcpu = (void *)kmem_malloc(kernel_arena, DPCPU_SIZE,
> >                     M_WAITOK | M_ZERO);
> >
>
> I applied your patch. The boot stops at the same spot. Also, I can't see
> any of the two print messages added in the patch. I booted from a Live CD
> to look at the logs, but they are not stored. I checked dmesg.boot and
> messages, and /var/log/xen/ none of them have the logs. They all have logs
> of the previous boot.

It's too early for the logs to be stored anywhere. The point where you
get the crash is when the APs are started, which is way before FreeBSD
starts proving for disk devices.

Can you please try the patch below?

Thanks, Roger.
---8<---
diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c
index 54184898e9bf..52391e2e7c08 100644
--- a/sys/x86/xen/pv.c
+++ b/sys/x86/xen/pv.c
@@ -113,6 +113,7 @@ static int xen_pv_start_all_aps(void);
 extern char *doublefault_stack;
 extern char *mce_stack;
 extern char *nmi_stack;
+extern char *dbg_stack;
 #endif
 
 /*
@@ -329,6 +330,8 @@ start_xen_ap(int cpu)
     (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
  nmi_stack =
     (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
+ dbg_stack =
+    (void *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
  dpcpu =
     (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, M_WAITOK | M_ZERO);
 
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
Hi,

On Wed, May 23, 2018 at 1:45 PM, Roger Pau Monné <[hidden email]> wrote:

> It's too early for the logs to be stored anywhere. The point where you
> get the crash is when the APs are started, which is way before FreeBSD
> starts proving for disk devices.
>
> Can you please try the patch below?
>
> Thanks, Roger.
> ---8<---
> diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c
> index 54184898e9bf..52391e2e7c08 100644
> --- a/sys/x86/xen/pv.c
> +++ b/sys/x86/xen/pv.c
> @@ -113,6 +113,7 @@ static int xen_pv_start_all_aps(void);
>  extern char *doublefault_stack;
>  extern char *mce_stack;
>  extern char *nmi_stack;
> +extern char *dbg_stack;
>  #endif
>
>  /*
> @@ -329,6 +330,8 @@ start_xen_ap(int cpu)
>             (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
>         nmi_stack =
>             (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
> +       dbg_stack =
> +           (void *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
>         dpcpu =
>             (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, M_WAITOK | M_ZERO);
>

I think we have different pv.c files. For me, line 113 is:
/* Xen init_ops implementation. */

The declarations of doublefault_stach, mce_stack, etc are in line 101.

Similarly, line 329 for me is:
{

in function xen_pv_parse_symtab(void). The declarations your diff
mentions in line 329 are in line 224.

This is in sync with the official repository [0]. Maybe you have
modifications that are not yet upstream?

Anyway, I manually made the changes. It still does not boot (I used
make kernel -DKERNFAST, but I don't think that should make a
difference).

Regards,
Pratyush Yadav

[0] https://github.com/freebsd/freebsd/blob/master/sys/x86/xen/pv.c
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Roger Pau Monné
On Wed, May 23, 2018 at 08:27:29PM +0530, Pratyush Yadav wrote:

> Hi,
>
> On Wed, May 23, 2018 at 1:45 PM, Roger Pau Monné <[hidden email]> wrote:
> > It's too early for the logs to be stored anywhere. The point where you
> > get the crash is when the APs are started, which is way before FreeBSD
> > starts proving for disk devices.
> >
> > Can you please try the patch below?
> >
> > Thanks, Roger.
> > ---8<---
> > diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c
> > index 54184898e9bf..52391e2e7c08 100644
> > --- a/sys/x86/xen/pv.c
> > +++ b/sys/x86/xen/pv.c
> > @@ -113,6 +113,7 @@ static int xen_pv_start_all_aps(void);
> >  extern char *doublefault_stack;
> >  extern char *mce_stack;
> >  extern char *nmi_stack;
> > +extern char *dbg_stack;
> >  #endif
> >
> >  /*
> > @@ -329,6 +330,8 @@ start_xen_ap(int cpu)
> >             (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
> >         nmi_stack =
> >             (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
> > +       dbg_stack =
> > +           (void *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
> >         dpcpu =
> >             (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, M_WAITOK | M_ZERO);
> >
>
> I think we have different pv.c files. For me, line 113 is:
> /* Xen init_ops implementation. */
>
> The declarations of doublefault_stach, mce_stack, etc are in line 101.
>
> Similarly, line 329 for me is:
> {
>
> in function xen_pv_parse_symtab(void). The declarations your diff
> mentions in line 329 are in line 224.
>
> This is in sync with the official repository [0]. Maybe you have
> modifications that are not yet upstream?

Sorry, I did indeed have other changes in pv.c. I'm appending the
patch on top of current HEAD.

> Anyway, I manually made the changes. It still does not boot (I used
> make kernel -DKERNFAST, but I don't think that should make a
> difference).

FWIW, I think the recommended way is KERNFAST=1.

Can you paste the error? I think you should no longer get a triple
page fault in mp_machdep.c:307.

Thanks, Roger.
---8<---
diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c
index c7e97c5b2b63..27e98012302b 100644
--- a/sys/x86/xen/pv.c
+++ b/sys/x86/xen/pv.c
@@ -101,6 +101,7 @@ static int xen_pv_start_all_aps(void);
 extern char *doublefault_stack;
 extern char *mce_stack;
 extern char *nmi_stack;
+extern char *dbg_stack;
 #endif
 
 /*
@@ -224,6 +225,8 @@ start_xen_ap(int cpu)
     (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
  nmi_stack =
     (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
+ dbg_stack =
+    (void *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO);
  dpcpu =
     (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, M_WAITOK | M_ZERO);
 

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit

Pratyush Yadav
On Wed, May 23, 2018 at 8:41 PM, Roger Pau Monné <[hidden email]> wrote:

> On Wed, May 23, 2018 at 08:27:29PM +0530, Pratyush Yadav wrote:
>> Hi,
>>
>> I think we have different pv.c files. For me, line 113 is:
>> /* Xen init_ops implementation. */
>>
>> The declarations of doublefault_stach, mce_stack, etc are in line 101.
>>
>> Similarly, line 329 for me is:
>> {
>>
>> in function xen_pv_parse_symtab(void). The declarations your diff
>> mentions in line 329 are in line 224.
>>
>> This is in sync with the official repository [0]. Maybe you have
>> modifications that are not yet upstream?
>
> Sorry, I did indeed have other changes in pv.c. I'm appending the
> patch on top of current HEAD.
>
>> Anyway, I manually made the changes. It still does not boot (I used
>> make kernel -DKERNFAST, but I don't think that should make a
>> difference).
>
> FWIW, I think the recommended way is KERNFAST=1.

Thanks!

> Can you paste the error? I think you should no longer get a triple
> page fault in mp_machdep.c:307.
>

Yeah, the address does change. RIP now points to ffffffff80b52fbe. I
ran addr2line on it and it points to ./machine/pcpu.h:231

I didn't write the full error because it is a lot of typing which
takes time. Most of it is probably not useful information. If you do
want the full error log, let me know and I'll type it out.

One thing, I simply ran make kernel. Does that update kernel.debug, or
do I have to do something else to update it?

--
Regards,
Pratyush Yadav
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "[hidden email]"