vm_page_t's do not appear initialized

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

vm_page_t's do not appear initialized

Austin Shafer
Hi freebsd-hackers,

While trying to write a DRM buffer page fault handler, I've run into an
issue where vm_page_t structs appear to be uninitialized and contain
random data. Most of These "random" fields in the structures remain the
same after rebooting, which makes me think that I have missed something
and am misinterpreting the fields. After a little over a week reading
through source code and trying different things, I wanted to see
if anyone had any insight that could help me learn whats going on.

Thanks in advance!

(For context, here is the project this is from)
https://github.com/aritger/eglstreams-kms-example/issues/7


I have more verbose examples below, but some quick examples of what I'm
seeing are:
 - vm_page_t->phys_addr does not match expected
 - vm_page_t->flags has values not in the PG_* macros
   - same for busy_lock and other entries
 - kgdb can't read memory at vm_page_t->object
 - etc...

It looks like vm_page_init_page was not called on these vm_page_t's I am
interacting with, but I can't seem to figure out why that would not be
the case? I've also booted into single user mode so I can look at things
before any graphics drivers can mess with them, but the pages still look
the same. While writing the fault handler I've been using the i915kms
driver's handler as a reference, and I've included page structs from an
intel laptop as an example of what I expected to see.

_____________________________________________________________________
Here is an example of a vm_page_t I get for the start of device memory
on an NVIDIA 1070:
_____________________________________________________________________

# uname -a
FreeBSD wolfgang 12.0-RELEASE FreeBSD 12.0-RELEASE #5 r343025M: Sun Feb 10 14:29:01 EST 2019
ashafer@wolfgang:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
----------------
pciconf -lbv:
...
vgapci0@pci0:1:0:0:     class=0x030000 card=0x33011462 chip=0x1b8110de rev=0xa1 hdr=0x00
    vendor     = 'NVIDIA Corporation'
    device     = 'GP104 [GeForce GTX 1070]'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 32, base 0xee000000, size 16777216, enabled
    bar   [14] = type Prefetchable Memory, range 64, base 0xd0000000, size 268435456, enabled
    bar   [1c] = type Prefetchable Memory, range 64, base 0xe0000000, size 33554432, enabled
    bar   [24] = type I/O Port, range 32, base 0xe000, size 128, enabled
...
----------------

/* get the page corresponding device mem */
vm_page_t page = PHYS_TO_VM_PAGE(0xd0000000) <--- page == 0xfffff802266a5980

----------------
(kgdb) p/x *(vm_page_t)0xfffff802266a5980
$1 = {plinks = {q = {tqe_next = 0x2c43b579e62bf7ae, tqe_prev = 0xb47eac47a195af64}, s = {ss = {
        sle_next = 0x2c43b579e62bf7ae}, pv = 0xb47eac47a195af64}, memguard = {
      p = 0x2c43b579e62bf7ae, v = 0xb47eac47a195af64}}, listq = {
    tqe_next = 0x6d4683b13bdc1486, tqe_prev = 0x3666c2179bcddb1d},
  object = 0xff2e12d8cc36710b, pindex = 0xb5bd170e69b840a2, phys_addr = 0xaad51419a796c44d,
  md = {pv_list = {tqh_first = 0x38d208abcf6951ec, tqh_last = 0x850ca846e9f2d9f3},
    pv_gen = 0xf277ff0c, pat_mode = 0x564e1542}, wire_count = 0xc810bbe0,
  busy_lock = 0x4ee4b98f, hold_count = 0xbcda, flags = 0x3f53, aflags = 0x37, oflags = 0x66,
  queue = 0xa, psind = 0xc1, segind = 0x23, order = 0xb7, pool = 0x12, act_count = 0x9c,
  valid = 0x22, dirty = 0x71}
----------------
just in case this is related... I think vm_phys_segs drives which pages get
initialized during system boot?
----------------
(kgdb) p/x vm_phys_segs
$2 = {{start = 0x10000, end = 0x6f000, first_page = 0xfffff80443776000, domain = 0x0,
    free_queues = 0xffffffff82057670}, {start = 0x70000, end = 0x87000,
    first_page = 0xfffff80443778700, domain = 0x0, free_queues = 0xffffffff82057670}, {
    start = 0x100000, end = 0x1000000, first_page = 0xfffff8044377c180, domain = 0x0,
    free_queues = 0xffffffff82057670}, {start = 0x1000000, end = 0x2c6b000,
    first_page = 0xfffff804437dd980, domain = 0x0, free_queues = 0xffffffff82057400}, {
    start = 0x2c73000, end = 0x2cab000, first_page = 0xfffff80443896838, domain = 0x0,
    free_queues = 0xffffffff82057400}, {start = 0x2e00000, end = 0x96c80000,
    first_page = 0xfffff804438a0980, domain = 0x0, free_queues = 0xffffffff82057400}, {
    start = 0x98083000, end = 0x9c68f000, first_page = 0xfffff80447538eb8, domain = 0x0,
    free_queues = 0xffffffff82057400}, {start = 0x9cfff000, end = 0x9d000000,
    first_page = 0xfffff8044773d918, domain = 0x0, free_queues = 0xffffffff82057400}, {
    start = 0x100000000, end = 0x44360f000, first_page = 0xfffff80449f75980, domain = 0x0,
    free_queues = 0xffffffff82057400}, {start = 0x100000000, end = 0x44360f000,
    first_page = 0xfffff80449f75980, domain = 0x0, free_queues = 0xffffffff82057400}, {
    start = 0x0, end = 0x0, first_page = 0x0, domain = 0x0,
    free_queues = 0x0} <repeats 53 times>}

_____________________________________________________________________
EXPECTED: This is what the vm_page_t for device memory on a intel laptop
(Different computer, this looks like what I should be getting)
_____________________________________________________________________
pciconf -lbv:
...
vgapci0@pci0:0:2:0:     class=0x030000 card=0xfa121179 chip=0x04168086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '4th Gen Core Processor Integrated Graphics Controller'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 64, base 0xb0000000, size 4194304, enabled
    bar   [18] = type Prefetchable Memory, range 64, base 0xa0000000, size 268435456, enabled
    bar   [20] = type I/O Port, range 32, base 0x4000, size 64, enabled
...
----------------
vm_page_t page = PHYS_TO_VM_PAGE(0xa0000000) <--- page ==
----------------
(kgdb) p/x *(vm_page_t)0xfffff80447875980
$1 = {plinks = {q = {tqe_next = 0x0, tqe_prev = 0x0}, s = {ss = {sle_next = 0x0}, pv = 0x0},
    memguard = {p = 0x0, v = 0x0}}, listq = {tqe_next = 0x0, tqe_prev = 0x0}, object = 0x0,
  pindex = 0x0, phys_addr = 0xa0000000, md = {pv_list = {tqh_first = 0x0,
      tqh_last = 0xfffff804478759b8}, pv_gen = 0x0, pat_mode = 0x1}, wire_count = 0x1,
  busy_lock = 0x1, hold_count = 0x0, flags = 0x4, aflags = 0x0, oflags = 0x0, queue = 0xff,
  psind = 0x0, segind = 0x0, order = 0x0, pool = 0x0, act_count = 0x0, valid = 0x0,
  dirty = 0x0}

_____________________________________________________________________

Again I'm not really sure what I'm running into. I've had to try to
learn the FreeBSD vm system while working on this, so I'm sorry if I
missed anything obvious to those who have more experience. I'm more than
happy to provide any information or test anything and I'd appreciate any
pointers to what can explain this.

Thank you so much for your time!
      Austin Shafer
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: vm_page_t's do not appear initialized

Konstantin Belousov
On Tue, Feb 19, 2019 at 01:12:52AM -0500, Austin Shafer wrote:

> Hi freebsd-hackers,
>
> While trying to write a DRM buffer page fault handler, I've run into an
> issue where vm_page_t structs appear to be uninitialized and contain
> random data. Most of These "random" fields in the structures remain the
> same after rebooting, which makes me think that I have missed something
> and am misinterpreting the fields. After a little over a week reading
> through source code and trying different things, I wanted to see
> if anyone had any insight that could help me learn whats going on.
>
> Thanks in advance!
>
> (For context, here is the project this is from)
> https://github.com/aritger/eglstreams-kms-example/issues/7
>
>
> I have more verbose examples below, but some quick examples of what I'm
> seeing are:
>  - vm_page_t->phys_addr does not match expected
>  - vm_page_t->flags has values not in the PG_* macros
>    - same for busy_lock and other entries
>  - kgdb can't read memory at vm_page_t->object
>  - etc...
>
> It looks like vm_page_init_page was not called on these vm_page_t's I am
> interacting with, but I can't seem to figure out why that would not be
> the case? I've also booted into single user mode so I can look at things
> before any graphics drivers can mess with them, but the pages still look
> the same. While writing the fault handler I've been using the i915kms
> driver's handler as a reference, and I've included page structs from an
> intel laptop as an example of what I expected to see.
>
> _____________________________________________________________________
> Here is an example of a vm_page_t I get for the start of device memory
> on an NVIDIA 1070:
> _____________________________________________________________________
>
> # uname -a
> FreeBSD wolfgang 12.0-RELEASE FreeBSD 12.0-RELEASE #5 r343025M: Sun Feb 10 14:29:01 EST 2019
> ashafer@wolfgang:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
> ----------------
> pciconf -lbv:
> ...
> vgapci0@pci0:1:0:0:     class=0x030000 card=0x33011462 chip=0x1b8110de rev=0xa1 hdr=0x00
>     vendor     = 'NVIDIA Corporation'
>     device     = 'GP104 [GeForce GTX 1070]'
>     class      = display
>     subclass   = VGA
>     bar   [10] = type Memory, range 32, base 0xee000000, size 16777216, enabled
>     bar   [14] = type Prefetchable Memory, range 64, base 0xd0000000, size 268435456, enabled
>     bar   [1c] = type Prefetchable Memory, range 64, base 0xe0000000, size 33554432, enabled
>     bar   [24] = type I/O Port, range 32, base 0xe000, size 128, enabled
> ...
> ----------------
>
> /* get the page corresponding device mem */
> vm_page_t page = PHYS_TO_VM_PAGE(0xd0000000) <--- page == 0xfffff802266a5980
>
> ----------------
> (kgdb) p/x *(vm_page_t)0xfffff802266a5980
> $1 = {plinks = {q = {tqe_next = 0x2c43b579e62bf7ae, tqe_prev = 0xb47eac47a195af64}, s = {ss = {
>         sle_next = 0x2c43b579e62bf7ae}, pv = 0xb47eac47a195af64}, memguard = {
>       p = 0x2c43b579e62bf7ae, v = 0xb47eac47a195af64}}, listq = {
>     tqe_next = 0x6d4683b13bdc1486, tqe_prev = 0x3666c2179bcddb1d},
>   object = 0xff2e12d8cc36710b, pindex = 0xb5bd170e69b840a2, phys_addr = 0xaad51419a796c44d,
>   md = {pv_list = {tqh_first = 0x38d208abcf6951ec, tqh_last = 0x850ca846e9f2d9f3},
>     pv_gen = 0xf277ff0c, pat_mode = 0x564e1542}, wire_count = 0xc810bbe0,
>   busy_lock = 0x4ee4b98f, hold_count = 0xbcda, flags = 0x3f53, aflags = 0x37, oflags = 0x66,
>   queue = 0xa, psind = 0xc1, segind = 0x23, order = 0xb7, pool = 0x12, act_count = 0x9c,
>   valid = 0x22, dirty = 0x71}
> ----------------
> just in case this is related... I think vm_phys_segs drives which pages get
> initialized during system boot?
> ----------------
> (kgdb) p/x vm_phys_segs
> $2 = {{start = 0x10000, end = 0x6f000, first_page = 0xfffff80443776000, domain = 0x0,
>     free_queues = 0xffffffff82057670}, {start = 0x70000, end = 0x87000,
>     first_page = 0xfffff80443778700, domain = 0x0, free_queues = 0xffffffff82057670}, {
>     start = 0x100000, end = 0x1000000, first_page = 0xfffff8044377c180, domain = 0x0,
>     free_queues = 0xffffffff82057670}, {start = 0x1000000, end = 0x2c6b000,
>     first_page = 0xfffff804437dd980, domain = 0x0, free_queues = 0xffffffff82057400}, {
>     start = 0x2c73000, end = 0x2cab000, first_page = 0xfffff80443896838, domain = 0x0,
>     free_queues = 0xffffffff82057400}, {start = 0x2e00000, end = 0x96c80000,
>     first_page = 0xfffff804438a0980, domain = 0x0, free_queues = 0xffffffff82057400}, {
>     start = 0x98083000, end = 0x9c68f000, first_page = 0xfffff80447538eb8, domain = 0x0,
>     free_queues = 0xffffffff82057400}, {start = 0x9cfff000, end = 0x9d000000,
>     first_page = 0xfffff8044773d918, domain = 0x0, free_queues = 0xffffffff82057400}, {
>     start = 0x100000000, end = 0x44360f000, first_page = 0xfffff80449f75980, domain = 0x0,
>     free_queues = 0xffffffff82057400}, {start = 0x100000000, end = 0x44360f000,
>     first_page = 0xfffff80449f75980, domain = 0x0, free_queues = 0xffffffff82057400}, {
>     start = 0x0, end = 0x0, first_page = 0x0, domain = 0x0,
>     free_queues = 0x0} <repeats 53 times>}
>
> _____________________________________________________________________
> EXPECTED: This is what the vm_page_t for device memory on a intel laptop
> (Different computer, this looks like what I should be getting)
> _____________________________________________________________________
> pciconf -lbv:
> ...
> vgapci0@pci0:0:2:0:     class=0x030000 card=0xfa121179 chip=0x04168086 rev=0x06 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '4th Gen Core Processor Integrated Graphics Controller'
>     class      = display
>     subclass   = VGA
>     bar   [10] = type Memory, range 64, base 0xb0000000, size 4194304, enabled
>     bar   [18] = type Prefetchable Memory, range 64, base 0xa0000000, size 268435456, enabled
>     bar   [20] = type I/O Port, range 32, base 0x4000, size 64, enabled
> ...
> ----------------
> vm_page_t page = PHYS_TO_VM_PAGE(0xa0000000) <--- page ==
> ----------------
> (kgdb) p/x *(vm_page_t)0xfffff80447875980
> $1 = {plinks = {q = {tqe_next = 0x0, tqe_prev = 0x0}, s = {ss = {sle_next = 0x0}, pv = 0x0},
>     memguard = {p = 0x0, v = 0x0}}, listq = {tqe_next = 0x0, tqe_prev = 0x0}, object = 0x0,
>   pindex = 0x0, phys_addr = 0xa0000000, md = {pv_list = {tqh_first = 0x0,
>       tqh_last = 0xfffff804478759b8}, pv_gen = 0x0, pat_mode = 0x1}, wire_count = 0x1,
>   busy_lock = 0x1, hold_count = 0x0, flags = 0x4, aflags = 0x0, oflags = 0x0, queue = 0xff,
>   psind = 0x0, segind = 0x0, order = 0x0, pool = 0x0, act_count = 0x0, valid = 0x0,
>   dirty = 0x0}
>
> _____________________________________________________________________
>
> Again I'm not really sure what I'm running into. I've had to try to
> learn the FreeBSD vm system while working on this, so I'm sorry if I
> missed anything obvious to those who have more experience. I'm more than
> happy to provide any information or test anything and I'd appreciate any
> pointers to what can explain this.
>
> Thank you so much for your time!
>       Austin Shafer
I do not know exactly what are you doing, but problems with
PHYS_TO_VM_PAGE() on the framebuffers pages usually indicate incorrect
(or missed) usage of the vm_phys_fictitious_reg_range() KPI.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: vm_page_t's do not appear initialized

Austin Shafer
Konstantin Belousov <[hidden email]> writes:

> I do not know exactly what are you doing, but problems with
> PHYS_TO_VM_PAGE() on the framebuffers pages usually indicate incorrect
> (or missed) usage of the vm_phys_fictitious_reg_range() KPI.

Thanks! That did the trick, I had no idea I needed to call that first.

There are plenty of descriptions of vm pages and objects online, but are
there any docs for the physical allocator portion of virtual memory?
Specifically about vm_phys.c/h? "The design/implementation of FreeBSD"
and the handbook both focus on vm pages.

Thanks again for your help!
       Austin Shafer
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"