firmware loading

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

firmware loading

Sam Leffler
Florent Thoumie and I have been working on some generic support for
loading firmware using kld's.  You can find a proof of concept at:

http://www.freebsd.org/~sam/firmware.tgz

It has code to manage firmware images and load them on demand by
requesting a kld through standard facilities.  Firmware is packaged
using a genfw program that's included.  You can package one or more
firmware images in a single kld.  I've packaged the iwi firmware as a
boot image in a single kld + kld's for each operating mode that have two
firmware images.  The tarball also includes modified versions of the iwi
and ipw drivers to use the code.  I tested iwi, Florent is working on ipw.

If you're interested in this stuff feel free to pick it up; I've run out
of time to work on it.  There are some potential issues with holding
locks over the linker calls and the genfw program could use some TLC and
probably a new name (plus the man page needs to be completed).

It appears ispfw can be reworked to use this code.  iwi and ipw
definitely can use it.  Not sure who else can/should use it.

        Sam
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Max Laier
On Wednesday 21 December 2005 06:54, Sam Leffler wrote:

> Florent Thoumie and I have been working on some generic support for
> loading firmware using kld's.  You can find a proof of concept at:
>
> http://www.freebsd.org/~sam/firmware.tgz
>
> It has code to manage firmware images and load them on demand by
> requesting a kld through standard facilities.  Firmware is packaged
> using a genfw program that's included.  You can package one or more
> firmware images in a single kld.  I've packaged the iwi firmware as a
> boot image in a single kld + kld's for each operating mode that have two
> firmware images.  The tarball also includes modified versions of the iwi
> and ipw drivers to use the code.  I tested iwi, Florent is working on ipw.
>
> If you're interested in this stuff feel free to pick it up; I've run out
> of time to work on it.  There are some potential issues with holding
> locks over the linker calls and the genfw program could use some TLC and
> probably a new name (plus the man page needs to be completed).
I am generally interested and will probably look at it between Christmas and
the New Year - unless Damien wants to look at it himself.  The current scheme
of loading the firmware is disfunctional with a WITNESS enabled kernel (too
much "sleep while holding mutex") and needs to be fixed anyway.

Is this work in perforce?  Is Florent still actively working on it?

> It appears ispfw can be reworked to use this code.  iwi and ipw
> definitely can use it.  Not sure who else can/should use it.

--
/"\  Best regards,                      | [hidden email]
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier@EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Florent Thoumie
On Wednesday 21 December 2005 14:52, Max Laier wrote:

> On Wednesday 21 December 2005 06:54, Sam Leffler wrote:
> > Florent Thoumie and I have been working on some generic support for
> > loading firmware using kld's.  You can find a proof of concept at:
> >
> > http://www.freebsd.org/~sam/firmware.tgz
> >
> > It has code to manage firmware images and load them on demand by
> > requesting a kld through standard facilities.  Firmware is packaged
> > using a genfw program that's included.  You can package one or more
> > firmware images in a single kld.  I've packaged the iwi firmware as a
> > boot image in a single kld + kld's for each operating mode that have two
> > firmware images.  The tarball also includes modified versions of the iwi
> > and ipw drivers to use the code.  I tested iwi, Florent is working on
> > ipw.
> >
> > If you're interested in this stuff feel free to pick it up; I've run out
> > of time to work on it.  There are some potential issues with holding
> > locks over the linker calls and the genfw program could use some TLC and
> > probably a new name (plus the man page needs to be completed).
>
> I am generally interested and will probably look at it between Christmas
> and the New Year - unless Damien wants to look at it himself.  The current
> scheme of loading the firmware is disfunctional with a WITNESS enabled
> kernel (too much "sleep while holding mutex") and needs to be fixed anyway.
>
> Is this work in perforce?  Is Florent still actively working on it?
        I'm actively trying to get familiar with src/sys, which takes most of my
        time. I won't be able to fix anything in a while though.

        I've been testing Sam's changes (since he has no ipw(4) hardware) and I've
        create new ipw/iwi-firmware ports that you will find in [1].

        [1] http://people.freebsd.org/~flz/firmware/

--
Florent Thoumie
[hidden email]
FreeBSD Committer

attachment0 (194 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Poul-Henning Kamp
In reply to this post by Sam Leffler
In message <[hidden email]>, Sam Leffler writes:
>Florent Thoumie and I have been working on some generic support for
>loading firmware using kld's.  You can find a proof of concept at:
>
>http://www.freebsd.org/~sam/firmware.tgz
>
>It has code to manage firmware images and load them on demand by
>requesting a kld through standard facilities.

Is the firmware kld unloaded again when no longer in use, or does it
stay in KVM once as long as the driver is active ?

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Scott Long-2
Poul-Henning Kamp wrote:

> In message <[hidden email]>, Sam Leffler writes:
>
>>Florent Thoumie and I have been working on some generic support for
>>loading firmware using kld's.  You can find a proof of concept at:
>>
>>http://www.freebsd.org/~sam/firmware.tgz
>>
>>It has code to manage firmware images and load them on demand by
>>requesting a kld through standard facilities.
>
>
> Is the firmware kld unloaded again when no longer in use, or does it
> stay in KVM once as long as the driver is active ?
>

Right now it stays resident.  My understanding is that the wireless
cards that this was designed for need to reload their firmware after
resume.  We don't have any hooks to tell drivers when the filesystem
is ready, so the only non-hackish solution right now is to just keep
it resident.  I agree that this isn't ideal, and I would think that it
would be pretty trivial to add an event handler hook to solve this.
The other thing that I'm not sure about is whether this being a module
dependency will prevent it from being unloaded.  I'm not sure how
ispfw.ko handles this.

Scott
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Sam Leffler
In reply to this post by Max Laier
Max Laier wrote:

> On Wednesday 21 December 2005 06:54, Sam Leffler wrote:
>
>>Florent Thoumie and I have been working on some generic support for
>>loading firmware using kld's.  You can find a proof of concept at:
>>
>>http://www.freebsd.org/~sam/firmware.tgz
>>
>>It has code to manage firmware images and load them on demand by
>>requesting a kld through standard facilities.  Firmware is packaged
>>using a genfw program that's included.  You can package one or more
>>firmware images in a single kld.  I've packaged the iwi firmware as a
>>boot image in a single kld + kld's for each operating mode that have two
>>firmware images.  The tarball also includes modified versions of the iwi
>>and ipw drivers to use the code.  I tested iwi, Florent is working on ipw.
>>
>>If you're interested in this stuff feel free to pick it up; I've run out
>>of time to work on it.  There are some potential issues with holding
>>locks over the linker calls and the genfw program could use some TLC and
>>probably a new name (plus the man page needs to be completed).
>
>
> I am generally interested and will probably look at it between Christmas and
> the New Year - unless Damien wants to look at it himself.  The current scheme
> of loading the firmware is disfunctional with a WITNESS enabled kernel (too
> much "sleep while holding mutex") and needs to be fixed anyway.

I can't recall if that's the driver's lock being held over linker calls
that are potentially blocking, but if you can resolve this then I can
also re-enable dynamic loading of 802.11 crypto modules.

>
> Is this work in perforce?  Is Florent still actively working on it?

It's not in perforce.  Florent has already responded.
>
>
>>It appears ispfw can be reworked to use this code.  iwi and ipw
>>definitely can use it.  Not sure who else can/should use it.
>
>

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Sam Leffler
In reply to this post by Scott Long-2
Scott Long wrote:

> Poul-Henning Kamp wrote:
>
>> In message <[hidden email]>, Sam Leffler writes:
>>
>>> Florent Thoumie and I have been working on some generic support for
>>> loading firmware using kld's.  You can find a proof of concept at:
>>>
>>> http://www.freebsd.org/~sam/firmware.tgz
>>>
>>> It has code to manage firmware images and load them on demand by
>>> requesting a kld through standard facilities.
>>
>>
>>
>> Is the firmware kld unloaded again when no longer in use, or does it
>> stay in KVM once as long as the driver is active ?
>>
>
> Right now it stays resident.  My understanding is that the wireless
> cards that this was designed for need to reload their firmware after
> resume.  We don't have any hooks to tell drivers when the filesystem
> is ready, so the only non-hackish solution right now is to just keep
> it resident.  I agree that this isn't ideal, and I would think that it
> would be pretty trivial to add an event handler hook to solve this.
> The other thing that I'm not sure about is whether this being a module
> dependency will prevent it from being unloaded.  I'm not sure how
> ispfw.ko handles this.

Actually, it is unloaded on last reference.  This is controlled by the
caller (there's a flag to firmware_put that says it's ok to unload).
Unloading on last ref is the most tricky part of the code as I wanted to
handle multiple firmware images packaged in a single kld and that
requires some care to be robust.  For now consumers need to be somewhat
aware of how multi-image kld's are packaged; they must order their
get+put calls to put last on the "master image" in a kld.

Using kld's to do this adds a level of indirection (over just reading
directly from files) but also has lots of advantages.  You can trivially
link stuff statically into the kernel.  You can pre-load stuff from the
loader.  You can print copyright notices to the console on load.  You
can also do things like compress the firmware in the kld.  The big win
is having a common api that hides the implementation details.  In this
regard the code needs some cleanup so consumers don't have to include
linker.h to get the definition of linker_file_t and so they don't know
about implementation details.

        Sam
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Poul-Henning Kamp
In message <[hidden email]>, Sam Leffler writes:

>Actually, it is unloaded on last reference.

That works for me.

--
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[hidden email]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: firmware loading

Max Laier
In reply to this post by Sam Leffler
On Wednesday 21 December 2005 06:54, Sam Leffler wrote:

> Florent Thoumie and I have been working on some generic support for
> loading firmware using kld's.  You can find a proof of concept at:
>
> http://www.freebsd.org/~sam/firmware.tgz
>
> It has code to manage firmware images and load them on demand by
> requesting a kld through standard facilities.  Firmware is packaged
> using a genfw program that's included.  You can package one or more
> firmware images in a single kld.  I've packaged the iwi firmware as a
> boot image in a single kld + kld's for each operating mode that have two
> firmware images.  The tarball also includes modified versions of the iwi
> and ipw drivers to use the code.  I tested iwi, Florent is working on ipw.
>
> If you're interested in this stuff feel free to pick it up; I've run out
> of time to work on it.  There are some potential issues with holding
> locks over the linker calls and the genfw program could use some TLC and
> probably a new name (plus the man page needs to be completed).
>
> It appears ispfw can be reworked to use this code.  iwi and ipw
> definitely can use it.  Not sure who else can/should use it.
An updated version of this work is here:

http://people.freebsd.org/~mlaier/firmware-20060125.tgz

It includes the following changes:
- Firmware module generation with awk and ld (no special tool required).  The
kmod.mk Makefile has been changed to support this and it's now very easy to
build a firmware module.
- Versioning
- firmware_put() safe from any context
- digi(4) converted.  ATTENTION: this was done blindly - if you have digi(4)
hardware I'd appreciate reports!
- Plenty iwi(4) changes which make it work much better for me - though there
are some rough edges still.
- firmware(9) manpage

To try it, just copy the contents of the tarball over src and apply the two
patches in sys/conf to the respective files.  The firmware support can be
loaded as a module itself, so testing is really easy.

My plan is to import the basic firmware support on the weekend (if no
objections are raised).  Drivers would be converted later after some more
testing.  The aim of this is to avoid more and more handrolled sollutions.  I
didn't yet have time to look at ispfw, but will do that as well.

So, any objections?  Comments?  Feedback?  Thanks!

--
/"\  Best regards,                      | [hidden email]
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier@EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News

attachment0 (194 bytes) Download Attachment