ib_unregister_device - OFED related question

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ib_unregister_device - OFED related question

Somayajulu, David
Hi All,

ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>() is typically called as part of the rdma driver module unload for a given device. As part of execution of this function it loops through the following code:



ofed/drivers/inifiniband/core/device.c



void ib_unregister_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_unregister_device>(struct ib_device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=ib_device> *device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>)

{

                    :

                    list_for_each_entry_reverse<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list_for_each_entry_reverse>(client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>, &client_list, list<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=list>)

                                         if (client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove)

                                                             client<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=client>->remove(device<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=device>);

                    :

}



For IB clients, client->remove(device) calls cma_remove_one() (ofed/drivers/inifiniband/core/cma.c) which ends up calling cma_process_remove().



cma_process_remove() calls wait_for_completion(&cma_dev->comp).



Now wait_for_completion() can potentially cause the thread to sleep().



From kern/kern_module.c:

int

module_unload<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_unload>(module_t<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=module_t> mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>)

{

         int error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>;



         mtx_lock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_lock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error> = MOD_EVENT<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_EVENT>(mod<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mod>, MOD_UNLOAD<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=MOD_UNLOAD>);

         mtx_unlock<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=mtx_unlock>(&Giant<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=Giant>);

         return (error<http://fxr.watson.org/fxr/ident?v=FREEBSD10;i=error>);

}

We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. Isn't this a problem with cma_process_remove() or am I missing something?



Thanks

David S. ([hidden email]<mailto:[hidden email]>)










_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: ib_unregister_device - OFED related question

Hans Petter Selasky-6
On 07/07/18 02:49, Somayajulu, David wrote:
> We see that module_unload() grabs the Gaint Lock prior to invoking UNLOAD. Isn't this a problem with cma_process_remove() or am I missing something?

Hi,

The LinuxKPI should DROP_GIANT and PICKUP_GIANT when sleeping. I haven't
checked FreeBSD 10 recently, but this is the case for FreeBSD 11 and
FreeBSD 12.

--HPS
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
To unsubscribe, send any mail to "[hidden email]"