CFT: netdump

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

CFT: netdump

Mark Johnston-2
Hi,

In user/markj/netdump I've been working on a feature which allows the
kernel to transmit kernel dumps over a network rather than writing to a
local storage device. This is convenient for VMs, diskless or embedded
devices, and devices with limited storage space. The server component of
netdump is a UDP daemon which is in ports as ftp/netdumpd.

The original code has been around for a long time and wasn't authored by
me. However, I've fixed a number of bugs, made it more reliable, and
integrated it with dumpon(8). I also largely rewrote the userland
daemon. At this point I'd like to propose committing it to HEAD.

The netdump(4) man page included in the patch lists the drivers
currently supported by netdump[*]. To be supported, a driver must be
augmented with a set of routines for use by the generic code. These
allow the generic code to transmit packets and poll for completions. I
added a mechanism to overwrite the mbuf/cluster UMA zone pointers at
panic time so that they instead fetch mbufs from a pre-allocated pool.
This way, existing driver code can be used without modification, and we
don't impose any runtime overhead in the regular mbuf allocation path.

Of course, the netdump data path is not fully reliable since the panic
may have interrupted a thread as it was processing tx/rx ring entries
and thus left them in an inconsistent state. In my experience this
hasn't proved to be an issue, and when testing I will typically have
several iperf streams running over the interface at panic time. That
said, the same issue exists in principle with local storage devices
as well.

A kernel dump will generally contain sensitive data. Thus, netdump
should only be used on trusted networks. It is possible to use the
kernel dump encryption feature (dumpon -k) with netdump, however I
am not certain that the encryption scheme used is well-suited for
this situation. Any feedback on this would be appreciated. Please note
that you need netdumpd-20180411 or later when using this feature, as the
daemon needs to accept and save the symmetric key in addition to the
dump itself and support for this isn't present in earlier versions.

A single large patch for the src tree is available here:
https://people.freebsd.org/~markj/patches/netdump/netdump-20180411.diff
Of course, the user/markj/netdump branch in SVN or its clone in the
GitHub mirror may be used as well. Documentation for configuration is in
the dumpon(8) man page. The gist of it is that one specifies some
additional network parameters to dumpon(8), along with an interface
name. For instance,

# dumpon -c 192.168.2.11 -s 192.168.2.1 vtnet0

tells the kernel to transmit to netdumpd at 192.168.2.1 using a source
address of 192.168.2.11. If there is a router between the two, the
gateway IP for the client must be specified as well, with -g.

I'm interested in any testing reports that folks may have, especially
with ixgbe or bnxt, or with NICs under load. If you have a favourite
NIC driver which does not yet support netdump, I'm happy to help add
support or review patches. Depending on how testing goes, I plan to
create reviews for various components of the patch over the next several
days.

[*] Currently, alc(4), bge(4), bxe(4), cxgb(4), mlx4en(4), re(4) and
vtnet(4) are supported. Drivers implemented using iflib are also
supported in principle, though I've only tested em(4) and igb(4).
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to "[hidden email]"