Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Scott Long-2
Damien Bergamini wrote:

> I don't like the idea of keeping the firmware in kernel memory.
> It's a rather big file (~200KB).
> And there is one for each operating mode (BSS, IBSS, monitor).
>
> The second reason why I don't like KLDs is because they require
> user intervention and users must know which KLD to load for the
> mode they want to operate in.  And if you put all firmwares in
> the same KLD, you end up with a big fat 1MB file.
>

The isp driver knows how to unload the ispfw module after it's done
with it, so it doesn't stay resident in memory.

> I won't go back to anything based on iwicontrol.  People simply
> don't know how to use it.  Trust me.  There is not a single day
> where I don't get email from people complaining about it.

I would be nice to have a single interface for loading data into the
kernel from within the kernel, but there are complicating factors.  On
one side you have storage drivers like isp that need firmware in order
to run (yeah, the isp hardware does have a basic firmware load, but it's
pretty much useless), therefore you can't wait until the filesystem is
available.  On the other side you have devices that need to load fairly
arbitrary data files, but only after boot.

Loading files into the kernel manually has all sorts of problems.  VFS
is very hard to work with in its 'raw' API form because just about
every operation has locking side effects that are hard to predict.  A
vput() call might simply lower the refcount on a vnode, or it might
wind up calling VOP_INACTIVE and going into the bufcache.  It's hard to
handle this correctly, and it's it's not something that should be
duplicated all over the kernel.  I know that Windows and Linux have
kernel APIs for reading files, but it's simply not a good idea to do it.
Also, putting hardcoded paths into the driver is very unmaintainable.

I guess my vote would be to have a devd mechanism where the kernel
can ask for a module via a particular key, devd maps the key to a file
via its config file and loads it into kernel space as a KLD, and then
the kernel unloads the KLD when its done with it.  For things like isp,
nothing would change, and for iwi you'd get a reliable way of getting
bits into the kernel that doesn't require messing with the VFS layer
or hard-coding knowledge of the filesystem layout into the driver.

Scott

>
> Damien
>
> | > Or have the firmware be embedded in a KLD, like ispfw.
> |
> | I think I've now been repeated by everyone in the conversation.  :)
> |
> | So maybe it's time to solve this?  Move discussion to arch@?
> |
> | --
> | Nate
>

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
jhb>John Baldwin
jhb> The easiest way to accomplish this is for your driver to send a
jhb> message to devd requesting that the firmware be reloaded on
jhb> resume since devd won't run until the kernel is fully back up.

des>Dag-Erling Smørgrav
des> ...or keep the firmware image in memory after loading it the first
des> time around.

scottl>Scott Long
scottl> Or have the firmware be embedded in a KLD, like ispfw.

nate>Nate Lawson
nate> I think I've now been repeated by everyone in the conversation.  :)

Only those people that think it is a good idea have repeated it.

nate> So maybe it's time to solve this?  Move discussion to arch@?

I don't like the kld option.  People have been talking about doing
this since about 4.2 and nothing has happened to make it generic.

The good things about it are that the driver can request that modules
be loaded and unloaded.  Once loaded, the driver can go directly to a
binary representation of the firmware.  These are both good points.

The bad points about this is that you have to generate the firmware
kld module.  This will require a per-driver program to parse the
firmware and some design to place the data into data structures that
the driver can use to bang the data into the card.  It would mean
creating two copies of the firmware because most people will install
the new firmware, then run this program and install the new kld (maybe
the kld generation program would run each installkernel, since you
never know what it depends on and when that might change).  Also, you
have the potential for version skew between the kld and the firmware.
There's one more level of indirection that you need to worry about
when you go the kld route.

The firmware parsing code tends to be relatively simple and may need
access to multiple files, as well as the actual hardware.  The wi
firmware loader, for example, needs to patch certain values from the
card into the wi images before they will work (the current
hand-tweaked symbol CF card has had these re-applied).

To solve the race with "/", one could easily just queue the driver
loading to a taskqueue.  I don't think that the taskqueues aren't
running until after the resume process is complete.  Even if they are,
and you block waiting for / to come back, it wouldn't be the resume
path that's blocking.  One could also use devd for this, but if the
driver already knows how to loads its firmware, that seems like an
overly complicated solution.

Warner
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Scott Long-2
Warner Losh wrote:

> jhb>John Baldwin
> jhb> The easiest way to accomplish this is for your driver to send a
> jhb> message to devd requesting that the firmware be reloaded on
> jhb> resume since devd won't run until the kernel is fully back up.
>
> des>Dag-Erling Smørgrav
> des> ...or keep the firmware image in memory after loading it the first
> des> time around.
>
> scottl>Scott Long
> scottl> Or have the firmware be embedded in a KLD, like ispfw.
>
> nate>Nate Lawson
> nate> I think I've now been repeated by everyone in the conversation.  :)
>
> Only those people that think it is a good idea have repeated it.
>
> nate> So maybe it's time to solve this?  Move discussion to arch@?
>
> I don't like the kld option.  People have been talking about doing
> this since about 4.2 and nothing has happened to make it generic.
>
> The good things about it are that the driver can request that modules
> be loaded and unloaded.  Once loaded, the driver can go directly to a
> binary representation of the firmware.  These are both good points.
>
> The bad points about this is that you have to generate the firmware
> kld module.  This will require a per-driver program to parse the
> firmware and some design to place the data into data structures that
> the driver can use to bang the data into the card.  It would mean
> creating two copies of the firmware because most people will install
> the new firmware, then run this program and install the new kld (maybe
> the kld generation program would run each installkernel, since you
> never know what it depends on and when that might change).  Also, you
> have the potential for version skew between the kld and the firmware.
> There's one more level of indirection that you need to worry about
> when you go the kld route.
>
> The firmware parsing code tends to be relatively simple and may need
> access to multiple files, as well as the actual hardware.  The wi
> firmware loader, for example, needs to patch certain values from the
> card into the wi images before they will work (the current
> hand-tweaked symbol CF card has had these re-applied).
>
> To solve the race with "/", one could easily just queue the driver
> loading to a taskqueue.  I don't think that the taskqueues aren't
> running until after the resume process is complete.  Even if they are,
> and you block waiting for / to come back, it wouldn't be the resume
> path that's blocking.  One could also use devd for this, but if the
> driver already knows how to loads its firmware, that seems like an
> overly complicated solution.
>
> Warner

Making the driver aware of the filesystem layout is a bad idea.
Encapsulating data into a KLD is not hard to do.

Scott
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
In reply to this post by Scott Long-2
scottl> Scott Long
scottl> I guess my vote would be to have a devd mechanism where the kernel
scottl> can ask for a module via a particular key, devd maps the key to a file
scottl> via its config file and loads it into kernel space as a KLD, and then
scottl> the kernel unloads the KLD when its done with it.  For things like isp,
scottl> nothing would change, and for iwi you'd get a reliable way of getting
scottl> bits into the kernel that doesn't require messing with the VFS layer
scottl> or hard-coding knowledge of the filesystem layout into the driver.

Apart from my other objections to kld, I really don't like getting
devd involved in this.  devd is a passive listener.  It listens for
events and then runs programs.  It is not there to interact with the
kernel in synchronous manner.  Everything is queued and there's
deliberately no meachanism for a reply.

There's also no reason to have a third party involved to load the
module.  None of the other automatic kldload parts of the kernel need
this.  Why is driver firmware any different?  You can easily define
the interface to be load module "$driver-fw-$type" and the driver can
take care of the sprintf to fill in the string.  A carefully chosen
interface here can help eliminate the extra layers of complexity.  How
the heck would devd know how to convert the particular key into
something meaningful?  How would concentrating the quirks of all
possible drivers in devd be better than localizing them in the driver
itself?

Warner
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
In reply to this post by Scott Long-2
> Making the driver aware of the filesystem layout is a bad idea.

Your opinion has been noted.  Repeating it often isn't going to help.
It has its advantages from the user point of view (I put a file here
and it is used, my tool of choice can always be "cp").

> Encapsulating data into a KLD is not hard to do.

Neither is a dozen other things that we do automatically for the
user.  From a user point of view this is a pita.  I put this file
here, and then I have to remember which magic tool to run to convert
the iwi firmware into a kld.  But that's a different tool from the wi
driver and that's different from the iwp driver, etc.

Each approach has its pro's and con's.  Getting dogmatic about 'X is a
bad idea, therefor you must do Y' without looking at why 'Y' might be
a worse alternative to 'X' isn't going to help this discussion.

Warner
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Scott Long-2
In reply to this post by Warner Losh
Warner Losh wrote:

> scottl> Scott Long
> scottl> I guess my vote would be to have a devd mechanism where the kernel
> scottl> can ask for a module via a particular key, devd maps the key to a file
> scottl> via its config file and loads it into kernel space as a KLD, and then
> scottl> the kernel unloads the KLD when its done with it.  For things like isp,
> scottl> nothing would change, and for iwi you'd get a reliable way of getting
> scottl> bits into the kernel that doesn't require messing with the VFS layer
> scottl> or hard-coding knowledge of the filesystem layout into the driver.
>
> Apart from my other objections to kld, I really don't like getting
> devd involved in this.  devd is a passive listener.

yes, it passively listens to requests for firmware.

> It listens for
> events and then runs programs.

Excellent!  Exactly that is needed!

> It is not there to interact with the
> kernel in synchronous manner.  Everything is queued and there's
> deliberately no meachanism for a reply.
>
> There's also no reason to have a third party involved to load the
> module.

Maybe you want a program that can on-the-fly patch the wi firmware?

> None of the other automatic kldload parts of the kernel need
> this.  Why is driver firmware any different?

Why hack the kernel module loader?

> You can easily define
> the interface to be load module "$driver-fw-$type" and the driver can
> take care of the sprintf to fill in the string.  A carefully chosen
> interface here can help eliminate the extra layers of complexity.  How
> the heck would devd know how to convert the particular key into
> something meaningful?

Maybe via some sort of configuration file?  I thought that devd had one
of those.  Guess I'm wrong?

> How would concentrating the quirks of all
> possible drivers in devd be better than localizing them in the driver
> itself?

Why put a lot of complexity into a program that corrupts your data when
it has a bug?

Scott
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Scott Long-2
In reply to this post by Warner Losh
Warner Losh wrote:

>>Making the driver aware of the filesystem layout is a bad idea.
>
>
> Your opinion has been noted.  Repeating it often isn't going to help.
> It has its advantages from the user point of view (I put a file here
> and it is used, my tool of choice can always be "cp").
>
>
>>Encapsulating data into a KLD is not hard to do.
>
>
> Neither is a dozen other things that we do automatically for the
> user.  From a user point of view this is a pita.  I put this file
> here, and then I have to remember which magic tool to run to convert
> the iwi firmware into a kld.  But that's a different tool from the wi
> driver and that's different from the iwp driver, etc.
>
> Each approach has its pro's and con's.  Getting dogmatic about 'X is a
> bad idea, therefor you must do Y' without looking at why 'Y' might be
> a worse alternative to 'X' isn't going to help this discussion.
>
> Warner

Lack of useful content noted.  Thanks.

Scott
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Daniel Eischen
In reply to this post by Warner Losh
On Mon, 21 Nov 2005, Warner Losh wrote:

> > Making the driver aware of the filesystem layout is a bad idea.
>
> Your opinion has been noted.  Repeating it often isn't going to help.
> It has its advantages from the user point of view (I put a file here
> and it is used, my tool of choice can always be "cp").
>
> > Encapsulating data into a KLD is not hard to do.
>
> Neither is a dozen other things that we do automatically for the
> user.  From a user point of view this is a pita.  I put this file
> here, and then I have to remember which magic tool to run to convert
> the iwi firmware into a kld.  But that's a different tool from the wi
> driver and that's different from the iwp driver, etc.

Not taking one side or the other...  Can we use a common
tool to convert firmware to a kld?  Just pass it the
symbol name or whatever else you need?

I guess this makes the assumption that most firmware consists
of just one object file.

--
DE

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
In reply to this post by Scott Long-2
> Lack of useful content noted.  Thanks.

Thank you for your opinion.  It has been noted.

Warner

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
In reply to this post by Scott Long-2
> > None of the other automatic kldload parts of the kernel need
> > this.  Why is driver firmware any different?
>
> Why hack the kernel module loader?

No need to hack the kernel loader.  It already will load a kloader of
a given name.  Driver says load iwi-firmware-ibss and away you go.
That's all I'm saying.  Why make things more complicated than
necessary?

Anyway, your tone has totally pissed me off.  Was that really
necessary?

Warner

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: cvs commit: src/sys/modules/iwi Makefile src/sys/dev/iwi if_iwi.c if_iwireg.h if_iwivar.h

Warner Losh
In reply to this post by Daniel Eischen
> Not taking one side or the other...  Can we use a common
> tool to convert firmware to a kld?  Just pass it the
> symbol name or whatever else you need?

The answer is 'it depends'...

> I guess this makes the assumption that most firmware consists
> of just one object file.

The object files vary somewhat as to what is in the file.  Some are S
records in text format, while otherrs are COFF or some other format in
binary form.  I had assumed that each driver would want to parse
things out into a useful form in the conversion program to keep the
kernel portion as small as possible.  This is inharently driver
dependent.

However, if we wanted to have a raw bits into the kernel conduit, then
we could have a tool, so long as it handled an arbitrary number of
files.  Many drivers have a 'primary' and a 'secondary' firmware to
load.  But presenting an array to the driver, or having the driver
load multiple files is a way around this.

Having the kernel loader be able to load arbitrary files would be even
better...

Warner
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"