GPT vs MBR for swap devices

classic Classic list List threaded Threaded
271 messages Options
1234 ... 14
Reply | Threaded
Open this post in threaded view
|

GPT vs MBR for swap devices

bob prohaska
In trying to get a Pi3 at r334939 to run -j4 buildworld it's become clear
that a USB flash (Sandisk Extreme) swap device very quickly triggers OOM
process kills, while a USB mechanical disk swap devices works perfectly.

It's somewhat hard to believe that the Sandisk Extreme is much slower than
a mechanical hard drive for small reads and writes if the rated speeds are
anywhere close to correct, but it happens that the flash device uses MBR
partitioning, while the mechanical hard drive uses GPT.

Is there any reason to think that the partitioning scheme matters?

Thanks for reading,

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Warner Losh
On Wed, Jun 13, 2018 at 9:48 AM, bob prohaska <[hidden email]> wrote:

> In trying to get a Pi3 at r334939 to run -j4 buildworld it's become clear
> that a USB flash (Sandisk Extreme) swap device very quickly triggers OOM
> process kills, while a USB mechanical disk swap devices works perfectly.
>
> It's somewhat hard to believe that the Sandisk Extreme is much slower than
> a mechanical hard drive for small reads and writes if the rated speeds are
> anywhere close to correct, but it happens that the flash device uses MBR
> partitioning, while the mechanical hard drive uses GPT.
>
> Is there any reason to think that the partitioning scheme matters?
>

Usually, partitioning doesn't matter.

USB flash drives are optimized for (a) being big and (b) writing large
files, infrequently.

Neither of these is good for swap space. So, lots of 'small' writes that
the swapper does can often trigger pathological behavior in the drive's FTL
(that's a polite way to say performance goes through the floor). I've seen
on SSDs, which have better FTLs, that can easily do 500MB/s drop to
10-20MB/s when reads and writes are mixed with the wrong workload. Our
swapper is written with the assumption that all writes are equal and small
ones are better than larger ones (since they free up the dirty pages
faster). However, you can often get better performance out of larger writes
and I don't think there's a knob to twist for that.

I should have added (c) and sometimes for FAT file systems. That used to be
the case. And for that MBR used to be a little better. However, that was 5
years ago or so. With the new Sandisk Extreme drives it isn't so much since
they are so large and FAT isn't good enough for most people these days. My
experience with them recently has been they are OK drives. The write speed
is kinda crappy (maybe 50-60MB/s when streaming a file to them, and
20-3MB/s in a mixed workload), but their read speed is surprisingly good at
up to about 200MB/s when there's no flash contention. My main concern,
though, was copying a 16GB video to them and making sure it would play back
OK. For that they are good. For backing up the video project with all it's
little files, not so good, and doing 'du' on it while copying slowed the
copying down a lot, even though it wasn't that much traffic to the drive.
All of this was on macOS, though.

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Wed, Jun 13, 2018 at 10:09:20AM -0600, Warner Losh wrote:
>  The write speed
> is kinda crappy (maybe 50-60MB/s when streaming a file to them, and
> 20-3MB/s in a mixed workload).....

That does not seem horrible compared to a mechanical disk with its
seek delays. A brief web search suggests 4k random write speeds of
1-3 MB/s for mechanical disks more modern than the one I'm using.

Thus far experiements have included swap on microSD (works), swap on
mechanical disk (works), swap on USB flash (causes OOM kills even with
< 20% swap utilization) and mixed USB flash, microSD flash and/or
mechanical swap (causes OOM kills).

It seems odd that adding USB flash swap to either microSD or USB
mechanical swap seems to make mischief. If a device is slow, I
thought the swapper would dis-favor or ignore it.

Some of the combinations trigger the warning:

warning: total configured swap (1048576 pages) exceeds maximum recommended amount (918256 pages).
warning: increase kern.maxswzone or reduce amount of swap.

which I've ignored, thinking the system will ignore excess swap space. Is
this mistaken? It's pretty clear the system does not need more than about
1.5 GB of swap. Available swap devices are presently sized like so:

3 GB mechanical disk
2 GB microSD flash
1 GB microSD flash
1 GB USB flash

The original plan was to use the two 1 GB flash partitions in the hope
they'd interleave, but that does not work at all. The microSD flash
swap seems to work fine. I was wondering if the USB mechanical swap
might point to trouble in the USB side of things, but it does not.

 
Thanks for reading,

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

freebsd-arm mailing list
On 2018-Jun-13, at 11:37 AM, bob prohaska <fbsd at www.zefox.net> wrote:

. . .
> Some of the combinations trigger the warning:
>
> warning: total configured swap (1048576 pages) exceeds maximum recommended amount (918256 pages).
> warning: increase kern.maxswzone or reduce amount of swap.

As I remember, the "increase kern.maxswzone" only applies if one has
previously decreased it: the modern default is the maximum recommended
as I remember (allowing for for half the theoretical maximum swap). If
I remember right the code overrode attempts to set more than the
default, only setting less (when requested). (I've not re-validated
this claim via a code inspection in some time.)

Quoting "man 8 loader" and its kern.maxswzone material,

                 Note that swap metadata can be fragmented, which means that
                 the system can run out of space before it reaches the
                 theoretical limit.  Therefore, care should be taken to not
                 configure more swap than approximately half of the
                 theoretical maximum.

So it will potentially have more fragmentation problems fitting the
extra metadata that is involved in the same amount of total RAM.

I will note that aarch64 and armv7 for the same amount of RAM use
very different "maximum recommended amount"s. (Compare an armv7 rpi2
to an aarch64 rpi3 by adding enough swap to get the notice, for
example. The rpi3 allows far more swap space as I remember.) This
variability is not obvious from the man page's material.

> which I've ignored, thinking the system will ignore excess swap space.

Again: It will potentially have more fragmentation problems fitting the
extra metadata that is involved in the same amount of total RAM.

> Is
> this mistaken?

I expect so because of the metadata issue. (But some later
notes below go in another direction, making this possibly
not a unique source of overall behavior differences.)

> It's pretty clear the system does not need more than about
> 1.5 GB of swap. Available swap devices are presently sized like so:

I'd recommend not using a whole lot more than you need, at least
small enough (total) for it to not report the warning.

One thing I wish FreeBSD had was an (approximate) "Maximum Observed
Swap Used" that could be queried or that some built-in tool would
monitor and report. At times I've made variants of top that displayed
such based on the figures it uses for its swap line. I've made
judgements about -jN choices and swaps space sizes based on what I've
observed for various contexts that I use/do repeatedly and for "large"
things that I do rarely but that can fail in my normal configuration.

> 3 GB mechanical disk
> 2 GB microSD flash
> 1 GB microSD flash
> 1 GB USB flash

To make your experiments comparable, I'd recommend the same total
swap be used across the alternatives being tested and compared.
This may be biased to using a smaller swap space that all the
contexts can support.

Going in another direction: The timing variations between the types
of media may mean that the mix of what is happening at the same time
changes for -j4 from one context to the next. This could get other
issues involved in why there are variations in the overall behavior.


Side note: I've used eMMC on an adapter in some microSD slots. I
had to "adjust" the Pine64+ 2GB case for the adapter that I used
to reach so such is not always an automatically-available option.

> The original plan was to use the two 1 GB flash partitions in the hope
> they'd interleave, but that does not work at all. The microSD flash
> swap seems to work fine. I was wondering if the USB mechanical swap
> might point to trouble in the USB side of things, but it does not.
. . .


Just for completeness for other readers (you may well be using swap
partitions) . . .

I also recommend swap partitions instead of swap files. See, for
example:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048


(OOM process kills were not one of the issues for bugzilla 206048.
But there were other problems for swap files.)

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Wed, Jun 13, 2018 at 02:41:35PM -0700, Mark Millard wrote:

> On 2018-Jun-13, at 11:37 AM, bob prohaska <fbsd at www.zefox.net> wrote:
>
> . . .
> > Some of the combinations trigger the warning:
> >
> > warning: total configured swap (1048576 pages) exceeds maximum recommended amount (918256 pages).
> > warning: increase kern.maxswzone or reduce amount of swap.
>
> As I remember, the "increase kern.maxswzone" only applies if one has
> previously decreased it: the modern default is the maximum recommended
> as I remember (allowing for for half the theoretical maximum swap). If
> I remember right the code overrode attempts to set more than the
> default, only setting less (when requested). (I've not re-validated
> this claim via a code inspection in some time.)
>
> Quoting "man 8 loader" and its kern.maxswzone material,
>
>                  Note that swap metadata can be fragmented, which means that
>                  the system can run out of space before it reaches the
>                  theoretical limit.  Therefore, care should be taken to not
>                  configure more swap than approximately half of the
>                  theoretical maximum.
>
> So it will potentially have more fragmentation problems fitting the
> extra metadata that is involved in the same amount of total RAM.
>
> I will note that aarch64 and armv7 for the same amount of RAM use
> very different "maximum recommended amount"s.

On boot the RPI3 the system reports
warning: total configured swap (1048576 pages) exceeds maximum recommended amount (918256 pages).
warning: increase kern.maxswzone or reduce amount of swap.

I've turned off the 1 GB of microSD swap, leaving only the 3 GB mechanical
disk.

On the RPI2 the report is
warning: total configured swap (524288 pages) exceeds maximum recommended amount (469280 pages).
warning: increase kern.maxswzone or reduce amount of swap

A -j4 buildworld completes without trouble, even with the excess swap.


> > which I've ignored, thinking the system will ignore excess swap space.
>
> Again: It will potentially have more fragmentation problems fitting the
> extra metadata that is involved in the same amount of total RAM.
>
> > Is
> > this mistaken?
>
> I expect so because of the metadata issue. (But some later
> notes below go in another direction, making this possibly
> not a unique source of overall behavior differences.)
>
> > It's pretty clear the system does not need more than about
> > 1.5 GB of swap. Available swap devices are presently sized like so:
>
> I'd recommend not using a whole lot more than you need, at least
> small enough (total) for it to not report the warning.
>
> One thing I wish FreeBSD had was an (approximate) "Maximum Observed
> Swap Used" that could be queried or that some built-in tool would
> monitor and report.

Indeed. Vmstat will log activity, but finding the peak and relating it
to OOM kills or other problems seems difficult without timestamps
>
There are times when slowly-performing swap is infinitely better than
out-of-memory kills. The Raspberry Pi series seem a prime example.

> > 3 GB mechanical disk
> > 2 GB microSD flash
> > 1 GB microSD flash
> > 1 GB USB flash
>
> To make your experiments comparable, I'd recommend the same total
> swap be used across the alternatives being tested and compared.
> This may be biased to using a smaller swap space that all the
> contexts can support.
>
> Going in another direction: The timing variations between the types
> of media may mean that the mix of what is happening at the same time
> changes for -j4 from one context to the next. This could get other
> issues involved in why there are variations in the overall behavior.
>
>
> Side note: I've used eMMC on an adapter in some microSD slots. I
> had to "adjust" the Pine64+ 2GB case for the adapter that I used
> to reach so such is not always an automatically-available option.
>
> . . .
>
>
> Just for completeness for other readers (you may well be using swap
> partitions) . . .
>

All of the swap experiments were with hard partitions. No swap files.

The most glaring (to me) oddity is that both the USB flash and microSD
flash are the same make and model name, all being Sandisk Extreme. The
only obvious difference is the interface to the CPU. There's no hint of
filesystem trouble with any media on either interface.

On the RPI3 USB does not work for flash swap, but seems fine for
mechanical swap. MicroSD flash swap seems to work without issue.
Either kind of swap works on the RPI2. That makes no sense to me.

Thanks for reading!

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Tom Vijlbrief-2
In reply to this post by freebsd-arm mailing list
I am torturing myself by doing build worlds on an original rpi with just
256Mb memory. I used to use an usb hard disk for swap and for
/usr/{src,obj} but that disk has died.

Currently swapping on an NFS swap file (this Linux server also hosts
/usr/{src,obj} ) which works surprisingly well. The user %cpu is much
better when swap is heavily used (often > 90%) compared to the old usb disk
setup. The latency of the NFS server is quite small, even with the slow
100mbit rpi Ethernet.

Op wo 13 jun. 2018 23:42 schreef Mark Millard via freebsd-arm <
[hidden email]>:

> On 2018-Jun-13, at 11:37 AM, bob prohaska <fbsd at www.zefox.net> wrote:
>
> . . .
> > Some of the combinations trigger the warning:
> >
> > warning: total configured swap (1048576 pages) exceeds maximum
> recommended amount (918256 pages).
> > warning: increase kern.maxswzone or reduce amount of swap.
>
> As I remember, the "increase kern.maxswzone" only applies if one has
> previously decreased it: the modern default is the maximum recommended
> as I remember (allowing for for half the theoretical maximum swap). If
> I remember right the code overrode attempts to set more than the
> default, only setting less (when requested). (I've not re-validated
> this claim via a code inspection in some time.)
>
> Quoting "man 8 loader" and its kern.maxswzone material,
>
>                  Note that swap metadata can be fragmented, which means
> that
>                  the system can run out of space before it reaches the
>                  theoretical limit.  Therefore, care should be taken to not
>                  configure more swap than approximately half of the
>                  theoretical maximum.
>
> So it will potentially have more fragmentation problems fitting the
> extra metadata that is involved in the same amount of total RAM.
>
> I will note that aarch64 and armv7 for the same amount of RAM use
> very different "maximum recommended amount"s. (Compare an armv7 rpi2
> to an aarch64 rpi3 by adding enough swap to get the notice, for
> example. The rpi3 allows far more swap space as I remember.) This
> variability is not obvious from the man page's material.
>
> > which I've ignored, thinking the system will ignore excess swap space.
>
> Again: It will potentially have more fragmentation problems fitting the
> extra metadata that is involved in the same amount of total RAM.
>
> > Is
> > this mistaken?
>
> I expect so because of the metadata issue. (But some later
> notes below go in another direction, making this possibly
> not a unique source of overall behavior differences.)
>
> > It's pretty clear the system does not need more than about
> > 1.5 GB of swap. Available swap devices are presently sized like so:
>
> I'd recommend not using a whole lot more than you need, at least
> small enough (total) for it to not report the warning.
>
> One thing I wish FreeBSD had was an (approximate) "Maximum Observed
> Swap Used" that could be queried or that some built-in tool would
> monitor and report. At times I've made variants of top that displayed
> such based on the figures it uses for its swap line. I've made
> judgements about -jN choices and swaps space sizes based on what I've
> observed for various contexts that I use/do repeatedly and for "large"
> things that I do rarely but that can fail in my normal configuration.
>
> > 3 GB mechanical disk
> > 2 GB microSD flash
> > 1 GB microSD flash
> > 1 GB USB flash
>
> To make your experiments comparable, I'd recommend the same total
> swap be used across the alternatives being tested and compared.
> This may be biased to using a smaller swap space that all the
> contexts can support.
>
> Going in another direction: The timing variations between the types
> of media may mean that the mix of what is happening at the same time
> changes for -j4 from one context to the next. This could get other
> issues involved in why there are variations in the overall behavior.
>
>
> Side note: I've used eMMC on an adapter in some microSD slots. I
> had to "adjust" the Pine64+ 2GB case for the adapter that I used
> to reach so such is not always an automatically-available option.
>
> > The original plan was to use the two 1 GB flash partitions in the hope
> > they'd interleave, but that does not work at all. The microSD flash
> > swap seems to work fine. I was wondering if the USB mechanical swap
> > might point to trouble in the USB side of things, but it does not.
> . . .
>
>
> Just for completeness for other readers (you may well be using swap
> partitions) . . .
>
> I also recommend swap partitions instead of swap files. See, for
> example:
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048
>
>
> (OOM process kills were not one of the issues for bugzilla 206048.
> But there were other problems for swap files.)
>
> ===
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Thu, Jun 14, 2018 at 11:24:50AM +0200, Tom Vijlbrief wrote:

> I am torturing myself by doing build worlds on an original rpi with just
> 256Mb memory. I used to use an usb hard disk for swap and for
> /usr/{src,obj} but that disk has died.
>
> Currently swapping on an NFS swap file (this Linux server also hosts
> /usr/{src,obj} ) which works surprisingly well. The user %cpu is much
> better when swap is heavily used (often > 90%) compared to the old usb disk
> setup. The latency of the NFS server is quite small, even with the slow
> 100mbit rpi Ethernet.
>

It's understood that NFS for swap works. It's also a complex
and fragile solution for a seemingly-simple problem. Setting aside a
little bit of a microSD or USB storage device is cheaper, uses less power,
less hardware and has fewer failure points than NFS. I'd submit that it's
a better solution for infrequent needs like buildworld.

My point is the observation that using USB flash for swap does not seem
to work on RPI3, although microSD flash does work, along with USB mechanical
drives.  

The fact the USB flash swap does seem to work on an RPI2 suggests that
there's nothing inherently wrong with USB flash as a swap medium. In
particular, I'm becoming skeptical of claims that modern USB flash
is too slow on write for use as swap.  

Thanks for reading,

bob prohaska


> Op wo 13 jun. 2018 23:42 schreef Mark Millard via freebsd-arm <
> [hidden email]>:
>
> > On 2018-Jun-13, at 11:37 AM, bob prohaska <fbsd at www.zefox.net> wrote:
> >
> > . . .
> > > Some of the combinations trigger the warning:
> > >
> > > warning: total configured swap (1048576 pages) exceeds maximum
> > recommended amount (918256 pages).
> > > warning: increase kern.maxswzone or reduce amount of swap.
> >
> > As I remember, the "increase kern.maxswzone" only applies if one has
> > previously decreased it: the modern default is the maximum recommended
> > as I remember (allowing for for half the theoretical maximum swap). If
> > I remember right the code overrode attempts to set more than the
> > default, only setting less (when requested). (I've not re-validated
> > this claim via a code inspection in some time.)
> >
> > Quoting "man 8 loader" and its kern.maxswzone material,
> >
> >                  Note that swap metadata can be fragmented, which means
> > that
> >                  the system can run out of space before it reaches the
> >                  theoretical limit.  Therefore, care should be taken to not
> >                  configure more swap than approximately half of the
> >                  theoretical maximum.
> >
> > So it will potentially have more fragmentation problems fitting the
> > extra metadata that is involved in the same amount of total RAM.
> >
> > I will note that aarch64 and armv7 for the same amount of RAM use
> > very different "maximum recommended amount"s. (Compare an armv7 rpi2
> > to an aarch64 rpi3 by adding enough swap to get the notice, for
> > example. The rpi3 allows far more swap space as I remember.) This
> > variability is not obvious from the man page's material.
> >
> > > which I've ignored, thinking the system will ignore excess swap space.
> >
> > Again: It will potentially have more fragmentation problems fitting the
> > extra metadata that is involved in the same amount of total RAM.
> >
> > > Is
> > > this mistaken?
> >
> > I expect so because of the metadata issue. (But some later
> > notes below go in another direction, making this possibly
> > not a unique source of overall behavior differences.)
> >
> > > It's pretty clear the system does not need more than about
> > > 1.5 GB of swap. Available swap devices are presently sized like so:
> >
> > I'd recommend not using a whole lot more than you need, at least
> > small enough (total) for it to not report the warning.
> >
> > One thing I wish FreeBSD had was an (approximate) "Maximum Observed
> > Swap Used" that could be queried or that some built-in tool would
> > monitor and report. At times I've made variants of top that displayed
> > such based on the figures it uses for its swap line. I've made
> > judgements about -jN choices and swaps space sizes based on what I've
> > observed for various contexts that I use/do repeatedly and for "large"
> > things that I do rarely but that can fail in my normal configuration.
> >
> > > 3 GB mechanical disk
> > > 2 GB microSD flash
> > > 1 GB microSD flash
> > > 1 GB USB flash
> >
> > To make your experiments comparable, I'd recommend the same total
> > swap be used across the alternatives being tested and compared.
> > This may be biased to using a smaller swap space that all the
> > contexts can support.
> >
> > Going in another direction: The timing variations between the types
> > of media may mean that the mix of what is happening at the same time
> > changes for -j4 from one context to the next. This could get other
> > issues involved in why there are variations in the overall behavior.
> >
> >
> > Side note: I've used eMMC on an adapter in some microSD slots. I
> > had to "adjust" the Pine64+ 2GB case for the adapter that I used
> > to reach so such is not always an automatically-available option.
> >
> > > The original plan was to use the two 1 GB flash partitions in the hope
> > > they'd interleave, but that does not work at all. The microSD flash
> > > swap seems to work fine. I was wondering if the USB mechanical swap
> > > might point to trouble in the USB side of things, but it does not.
> > . . .
> >
> >
> > Just for completeness for other readers (you may well be using swap
> > partitions) . . .
> >
> > I also recommend swap partitions instead of swap files. See, for
> > example:
> >
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048
> >
> >
> > (OOM process kills were not one of the issues for bugzilla 206048.
> > But there were other problems for swap files.)
> >
> > ===
> > Mark Millard
> > marklmi at yahoo.com
> > ( dsl-only.net went
> > away in early 2018-Mar)
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> > To unsubscribe, send any mail to "[hidden email]"
> >
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Rodney W. Grimes-4
> On Thu, Jun 14, 2018 at 11:24:50AM +0200, Tom Vijlbrief wrote:
> > I am torturing myself by doing build worlds on an original rpi with just
> > 256Mb memory. I used to use an usb hard disk for swap and for
> > /usr/{src,obj} but that disk has died.
> >
> > Currently swapping on an NFS swap file (this Linux server also hosts
> > /usr/{src,obj} ) which works surprisingly well. The user %cpu is much
> > better when swap is heavily used (often > 90%) compared to the old usb disk
> > setup. The latency of the NFS server is quite small, even with the slow
> > 100mbit rpi Ethernet.
> >
>
> It's understood that NFS for swap works. It's also a complex
> and fragile solution for a seemingly-simple problem. Setting aside a
> little bit of a microSD or USB storage device is cheaper, uses less power,
> less hardware and has fewer failure points than NFS. I'd submit that it's
> a better solution for infrequent needs like buildworld.
>
> My point is the observation that using USB flash for swap does not seem
> to work on RPI3, although microSD flash does work, along with USB mechanical
> drives.  
>
> The fact the USB flash swap does seem to work on an RPI2 suggests that
> there's nothing inherently wrong with USB flash as a swap medium. In
> particular, I'm becoming skeptical of claims that modern USB flash
> is too slow on write for use as swap.  
>
> Thanks for reading,

I would be very interested in seeing if resizing the swap partition
in the example that greatly exceeds what the system expects as a
max total swap helps to bring the OOM issue under control.

I think the state of things is such that if you use up the
max usable swap space on the first swap device, only that
swap device well ever be used.  I do not believe there is
any attempts what so ever to split the allocation up so
that you use the first fraction of each device.

>
> bob prohaska
>
>
> > Op wo 13 jun. 2018 23:42 schreef Mark Millard via freebsd-arm <
> > [hidden email]>:
> >
> > > On 2018-Jun-13, at 11:37 AM, bob prohaska <fbsd at www.zefox.net> wrote:
> > >
> > > . . .
> > > > Some of the combinations trigger the warning:
> > > >
> > > > warning: total configured swap (1048576 pages) exceeds maximum
> > > recommended amount (918256 pages).
> > > > warning: increase kern.maxswzone or reduce amount of swap.
> > >
> > > As I remember, the "increase kern.maxswzone" only applies if one has
> > > previously decreased it: the modern default is the maximum recommended
> > > as I remember (allowing for for half the theoretical maximum swap). If
> > > I remember right the code overrode attempts to set more than the
> > > default, only setting less (when requested). (I've not re-validated
> > > this claim via a code inspection in some time.)
> > >
> > > Quoting "man 8 loader" and its kern.maxswzone material,
> > >
> > >                  Note that swap metadata can be fragmented, which means
> > > that
> > >                  the system can run out of space before it reaches the
> > >                  theoretical limit.  Therefore, care should be taken to not
> > >                  configure more swap than approximately half of the
> > >                  theoretical maximum.
> > >
> > > So it will potentially have more fragmentation problems fitting the
> > > extra metadata that is involved in the same amount of total RAM.
> > >
> > > I will note that aarch64 and armv7 for the same amount of RAM use
> > > very different "maximum recommended amount"s. (Compare an armv7 rpi2
> > > to an aarch64 rpi3 by adding enough swap to get the notice, for
> > > example. The rpi3 allows far more swap space as I remember.) This
> > > variability is not obvious from the man page's material.
> > >
> > > > which I've ignored, thinking the system will ignore excess swap space.
> > >
> > > Again: It will potentially have more fragmentation problems fitting the
> > > extra metadata that is involved in the same amount of total RAM.
> > >
> > > > Is
> > > > this mistaken?
> > >
> > > I expect so because of the metadata issue. (But some later
> > > notes below go in another direction, making this possibly
> > > not a unique source of overall behavior differences.)
> > >
> > > > It's pretty clear the system does not need more than about
> > > > 1.5 GB of swap. Available swap devices are presently sized like so:
> > >
> > > I'd recommend not using a whole lot more than you need, at least
> > > small enough (total) for it to not report the warning.
> > >
> > > One thing I wish FreeBSD had was an (approximate) "Maximum Observed
> > > Swap Used" that could be queried or that some built-in tool would
> > > monitor and report. At times I've made variants of top that displayed
> > > such based on the figures it uses for its swap line. I've made
> > > judgements about -jN choices and swaps space sizes based on what I've
> > > observed for various contexts that I use/do repeatedly and for "large"
> > > things that I do rarely but that can fail in my normal configuration.
> > >
> > > > 3 GB mechanical disk
> > > > 2 GB microSD flash
> > > > 1 GB microSD flash
> > > > 1 GB USB flash
> > >
> > > To make your experiments comparable, I'd recommend the same total
> > > swap be used across the alternatives being tested and compared.
> > > This may be biased to using a smaller swap space that all the
> > > contexts can support.
> > >
> > > Going in another direction: The timing variations between the types
> > > of media may mean that the mix of what is happening at the same time
> > > changes for -j4 from one context to the next. This could get other
> > > issues involved in why there are variations in the overall behavior.
> > >
> > >
> > > Side note: I've used eMMC on an adapter in some microSD slots. I
> > > had to "adjust" the Pine64+ 2GB case for the adapter that I used
> > > to reach so such is not always an automatically-available option.
> > >
> > > > The original plan was to use the two 1 GB flash partitions in the hope
> > > > they'd interleave, but that does not work at all. The microSD flash
> > > > swap seems to work fine. I was wondering if the USB mechanical swap
> > > > might point to trouble in the USB side of things, but it does not.
> > > . . .
> > >
> > >
> > > Just for completeness for other readers (you may well be using swap
> > > partitions) . . .
> > >
> > > I also recommend swap partitions instead of swap files. See, for
> > > example:
> > >
> > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206048
> > >
> > >
> > > (OOM process kills were not one of the issues for bugzilla 206048.
> > > But there were other problems for swap files.)
> > >
> > > ===
> > > Mark Millard
> > > marklmi at yahoo.com
> > > ( dsl-only.net went
> > > away in early 2018-Mar)
> > >
> > > _______________________________________________
> > > [hidden email] mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> > > To unsubscribe, send any mail to "[hidden email]"
> > >
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "[hidden email]"
>

--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Thu, Jun 14, 2018 at 09:53:57AM -0700, Rodney W. Grimes wrote:
>
> I would be very interested in seeing if resizing the swap partition
> in the example that greatly exceeds what the system expects as a
> max total swap helps to bring the OOM issue under control.
>
The swap partitions at my immediate disposal are
1 GB USB flash
1 GB microSD flash
2 GB microSD flash
3 GB USB mechanical

What combination is apt to be most informative? My original intent
was to use 1 GB USB flash plus 1 GB microSD flash in hopes of a speed
gain from interleaving, but maybe that's no longer realistic. Anything
over 3 GB total causes the "too much swap" warning and I've never observed
more than about 1.2 GB of swap in use.

 


> I think the state of things is such that if you use up the
> max usable swap space on the first swap device, only that
> swap device well ever be used.  I do not believe there is
> any attempts what so ever to split the allocation up so
> that you use the first fraction of each device.
>

Swap usage seem to be spread among active partitions, though
how they're weighted is unclear to me. In days of yore there
was a little note about "interleaved" in swapinfo reports, but
I don't recall seeing that for a loong time. Maybe that feature
has gone away.....

Thanks for reading,

bob prohaska



_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Rodney W. Grimes-4
> On Thu, Jun 14, 2018 at 09:53:57AM -0700, Rodney W. Grimes wrote:
> >
> > I would be very interested in seeing if resizing the swap partition
> > in the example that greatly exceeds what the system expects as a
> > max total swap helps to bring the OOM issue under control.
> >
> The swap partitions at my immediate disposal are
> 1 GB USB flash
> 1 GB microSD flash
> 2 GB microSD flash
> 3 GB USB mechanical
>
> What combination is apt to be most informative? My original intent
> was to use 1 GB USB flash plus 1 GB microSD flash in hopes of a speed
> gain from interleaving, but maybe that's no longer realistic. Anything
> over 3 GB total causes the "too much swap" warning and I've never observed
> more than about 1.2 GB of swap in use.

Well if you do a swapon to the 3 GB USB Mechanical that
should be all that is usable, so we could infact test
the theory by doing that one first and adding
anything to it and find out if it started to
use the others.
Or someone that knows the current code could tell us :-)

It might be interesting to do in order the swapon
commands to 1G USB flash, 1G SD flash, 2G SD flash,
that should if what I think happens yeild a pretty
even 1G of each usable, with 1G wasted on the 2G SD flash.


> > I think the state of things is such that if you use up the
> > max usable swap space on the first swap device, only that
> > swap device well ever be used.  I do not believe there is
> > any attempts what so ever to split the allocation up so
> > that you use the first fraction of each device.
> >
>
> Swap usage seem to be spread among active partitions, though
> how they're weighted is unclear to me. In days of yore there
> was a little note about "interleaved" in swapinfo reports, but
> I don't recall seeing that for a loong time. Maybe that feature
> has gone away.....
>
> Thanks for reading,
>
> bob prohaska
>
>
>
>

--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

freebsd-arm mailing list
In reply to this post by bob prohaska
bob prohaska fbsd at www.zefox.net wrote on
Thu Jun 14 17:56:10 UTC 2018 :

> On Thu, Jun 14, 2018 at 09:53:57AM -0700, Rodney W. Grimes wrote:
> >
> > I would be very interested in seeing if resizing the swap partition
> > in the example that greatly exceeds what the system expects as a
> > max total swap helps to bring the OOM issue under control.
> >
>  
> The swap partitions at my immediate disposal are
> 1 GB USB flash
> 1 GB microSD flash
> 2 GB microSD flash
> 3 GB USB mechanical
>
> What combination is apt to be most informative? My original intent
> was to use 1 GB USB flash plus 1 GB microSD flash in hopes of a speed
> gain from interleaving, but maybe that's no longer realistic. Anything
> over 3 GB total causes the "too much swap" warning and I've never observed
> more than about 1.2 GB of swap in use.

1.2 GB of swap in-use lets out testing just the USB flash unless
one with a larger swap partition were available. Similarly for
the 1 GB microSD flash.

My prior suggestion could be implemented for the other two by
temporarily resizing the 3 GB swap partition to be just 2 GB
and then to try "USB mechanical" by itself for comparison to
the 2GB swap-partition microSD flash by itself.

This does overlap with the Rodney G.'s request for tests that
avoid getting the too-much-swap-space warnings and so might
well cover both directions of investigation if I interpret
Rodney's suggestion correctly.


As for use of multiple swap partitions and which are actually
used, I think the swapinfo command during the time of heavy
use (by size used) should indicate how much is in use for each
partition: it has a "used" field in the output.

# swapinfo -m
Device          1M-blocks     Used    Avail Capacity
/dev/gpt/FBSDUSSDswap     15360       19    15340     0%

(The context has only one swap partition but the above
does show the "used" being about 19 MBytes.)


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
In reply to this post by Rodney W. Grimes-4
On Thu, Jun 14, 2018 at 02:10:21PM -0700, Rodney W. Grimes wrote:
>
> It might be interesting to do in order the swapon
> commands to 1G USB flash, 1G SD flash, 2G SD flash,

It seems clear that USB flash swap, alone or in any
combination, fails early in buildworld. 1 GB of SD
flash swap seemed to work consideraby better, but
even it slowed greatly when 500 MB swap was in use.
3 GB of USB mechanical swap seems to work. 3 GB of SD flash
swap (2 GB plus 1 GB) also seemed to work.

These tests were not with the same kernel and sources,
which makes the comparisons rather suspect.

The machine is now reverting to r334939 using 1 GB + 2 GB of
SD flash swap. If that succeeds I will then run buildworld
for each case using the same kernel, world and sources.

Thanks for reading!

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Warner Losh
On Thu, Jun 14, 2018, 9:52 PM bob prohaska <[hidden email]> wrote:

> On Thu, Jun 14, 2018 at 02:10:21PM -0700, Rodney W. Grimes wrote:
> >
> > It might be interesting to do in order the swapon
> > commands to 1G USB flash, 1G SD flash, 2G SD flash,
>
> It seems clear that USB flash swap, alone or in any
> combination, fails early in buildworld.


I think that's because USB flash can't swap fast enough to keep up with the
page demand. You might be able to confirm this by looking at the write
rates to the swap portions for the various other media with gstat. I
suspect it's FTL is doing more expensive garbage collection under a swap
work load leading to long pauses from time to time that the VM system
responds to by starting OOM too soon.

Warner

1 GB of SD

> flash swap seemed to work consideraby better, but
> even it slowed greatly when 500 MB swap was in use.
> 3 GB of USB mechanical swap seems to work. 3 GB of SD flash
> swap (2 GB plus 1 GB) also seemed to work.
>
> These tests were not with the same kernel and sources,
> which makes the comparisons rather suspect.
>
> The machine is now reverting to r334939 using 1 GB + 2 GB of
> SD flash swap. If that succeeds I will then run buildworld
> for each case using the same kernel, world and sources.
>
> Thanks for reading!
>
> bob prohaska
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Thu, Jun 14, 2018 at 10:00:59PM -0600, Warner Losh wrote:
>
> I think that's because USB flash can't swap fast enough to keep up with the
> page demand. You might be able to confirm this by looking at the write
> rates to the swap portions for the various other media with gstat.

I'll try capturing vmstat -c -w 5 to a file, but will need help
interpreting the results. It seems particularly hard to correlate
vmstat output with corresponding compilation activity.

> I
> suspect it's FTL is doing more expensive garbage collection under a swap
> work load leading to long pauses from time to time that the VM system
> responds to by starting OOM too soon.
>

Wouldn't flash speed issues equally affect SD flash and USB flash swap?
From what I can see SD flash performs far better than USB flash for swap
on the RPI3. Also, using 1 GB USB flash swap together with 1 GB SD flash
swap produced worse performance than the 1 GB SD flash swap alone.

Are there contention issues between USB traffic and SD traffic? I've
always thought they were mostly independent. If they obstruct one
another that might help explain what I'm seeing. It would also make
clear that my goal of "interleaving" swap devices was badly mistaken.

It's worth remembering that USB flash swap (2 GB, in a single partiton)
seems to work quite well on an RPI2 running current. If it works on a Pi2
shouldn't it work on a Pi3?

Thanks for reading!

bob prohaska
 


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

freebsd-arm mailing list
# vmstat -c -w 5
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr da0 ad0   in    sy    cs us sy id
1 0 0 416M  224M  1647   1   0   0  1856  142   0   0  144  1791  1024  4  2 94
0 0 0 416M  224M     9   0   0   0     0    1   0   0    4    85   116  0  0 100
0 0 0 416M  224M    12   0   0   0     0    1   0   0    2    93   113  0  0 100
0 0 0 416M  224M     9   0   0   0     2    1   1   0    4    64   121  0  0 100


On 2018-Jun-14, at 10:15 PM, bob prohaska <fbsd at www.zefox.net> wrote:

> On Thu, Jun 14, 2018 at 10:00:59PM -0600, Warner Losh wrote:
>>
>> I think that's because USB flash can't swap fast enough to keep up with the
>> page demand. You might be able to confirm this by looking at the write
>> rates to the swap portions for the various other media with gstat.
>
> I'll try capturing vmstat -c -w 5 to a file, but will need help
> interpreting the results. It seems particularly hard to correlate
> vmstat output with corresponding compilation activity.
>
>> . . .

When I look at:

# vmstat -c -w 5
procs  memory       page                    disks     faults         cpu
r b w  avm   fre   flt  re  pi  po    fr   sr da0 ad0   in    sy    cs us sy id
1 0 0 416M  224M  1647   1   0   0  1856  142   0   0  144  1791  1024  4  2 94
0 0 0 416M  224M     9   0   0   0     0    1   0   0    4    85   116  0  0 100
0 0 0 416M  224M    12   0   0   0     0    1   0   0    2    93   113  0  0 100
0 0 0 416M  224M     9   0   0   0     2    1   1   0    4    64   121  0  0 100
. . .

and "man vmstat" I do not see any column that is the swap space
usage (nor any combination of columns to do such a calculation
from).

I do not expect that vmstat reports what you are likely/primarily
looking for.

An example is "avm" which for which the man page reports:

             . . . Note that the entire
             memory object's size is considered mapped even if only a subset
             of the object's pages are currently mapped.  This statistic is
             not related to the active page queue which is used to track real
             memory.

The free list size ("fre") is not sufficient either.


===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Thu, Jun 14, 2018 at 11:37:48PM -0700, Mark Millard wrote:

>
> When I look at:
>
> # vmstat -c -w 5
> procs  memory       page                    disks     faults         cpu
> r b w  avm   fre   flt  re  pi  po    fr   sr da0 ad0   in    sy    cs us sy id
> 1 0 0 416M  224M  1647   1   0   0  1856  142   0   0  144  1791  1024  4  2 94
> 0 0 0 416M  224M     9   0   0   0     0    1   0   0    4    85   116  0  0 100
> 0 0 0 416M  224M    12   0   0   0     0    1   0   0    2    93   113  0  0 100
> 0 0 0 416M  224M     9   0   0   0     2    1   1   0    4    64   121  0  0 100
> . . .
>
> and "man vmstat" I do not see any column that is the swap space
> usage (nor any combination of columns to do such a calculation
> from).
>
> I do not expect that vmstat reports what you are likely/primarily
> looking for.
>
> An example is "avm" which for which the man page reports:
>
>              . . . Note that the entire
>              memory object's size is considered mapped even if only a subset
>              of the object's pages are currently mapped.  This statistic is
>              not related to the active page queue which is used to track real
>              memory.
>
> The free list size ("fre") is not sufficient either.
>

That seems astonishing. I imagined that among those columns _had_ to be
reads from and writes to the swap partitions.

It looks as if
top -d 1000 | grep Swap
produces a running list of swap usage, but one must guess how many
times to iterate:
 
bob@www:/usr/src % top -d 1000 | grep Swap
Swap: 3072M Total, 30M Used, 3041M Free
Swap: 3072M Total, 30M Used, 3041M Free
Swap: 3072M Total, 30M Used, 3041M Free
Swap: 3072M Total, 30M Used, 3041M Free
Swap: 3072M Total, 30M Used, 3041M Free
.......

Replacing the "1000" with "0" or "infinite" triggers
a syntax error. Is there a special parameter that makes top run till
it's killed, as in interactive mode? I didn't recognize any hint in the
man page.

Thanks for reading!

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Warner Losh
On Fri, Jun 15, 2018 at 9:43 AM, bob prohaska <[hidden email]> wrote:

> On Thu, Jun 14, 2018 at 11:37:48PM -0700, Mark Millard wrote:
> >
> > When I look at:
> >
> > # vmstat -c -w 5
> > procs  memory       page                    disks     faults         cpu
> > r b w  avm   fre   flt  re  pi  po    fr   sr da0 ad0   in    sy    cs
> us sy id
> > 1 0 0 416M  224M  1647   1   0   0  1856  142   0   0  144  1791  1024
> 4  2 94
> > 0 0 0 416M  224M     9   0   0   0     0    1   0   0    4    85   116
> 0  0 100
> > 0 0 0 416M  224M    12   0   0   0     0    1   0   0    2    93   113
> 0  0 100
> > 0 0 0 416M  224M     9   0   0   0     2    1   1   0    4    64   121
> 0  0 100
> > . . .
> >
> > and "man vmstat" I do not see any column that is the swap space
> > usage (nor any combination of columns to do such a calculation
> > from).
> >
> > I do not expect that vmstat reports what you are likely/primarily
> > looking for.
> >
> > An example is "avm" which for which the man page reports:
> >
> >              . . . Note that the entire
> >              memory object's size is considered mapped even if only a
> subset
> >              of the object's pages are currently mapped.  This statistic
> is
> >              not related to the active page queue which is used to track
> real
> >              memory.
> >
> > The free list size ("fre") is not sufficient either.
> >
>
> That seems astonishing. I imagined that among those columns _had_ to be
> reads from and writes to the swap partitions.
>
> It looks as if
> top -d 1000 | grep Swap
> produces a running list of swap usage, but one must guess how many
> times to iterate:
>
> bob@www:/usr/src % top -d 1000 | grep Swap
> Swap: 3072M Total, 30M Used, 3041M Free
> Swap: 3072M Total, 30M Used, 3041M Free
> Swap: 3072M Total, 30M Used, 3041M Free
> Swap: 3072M Total, 30M Used, 3041M Free
> Swap: 3072M Total, 30M Used, 3041M Free
> .......
>
> Replacing the "1000" with "0" or "infinite" triggers
> a syntax error. Is there a special parameter that makes top run till
> it's killed, as in interactive mode? I didn't recognize any hint in the
> man page.
>
> Thanks for reading!
>

Right, this is why I was suggesting gstat. It's a direct measure of the
read/write performance of the device with some latency numbers. It will
give the kind of data I'm looking for. vmstat won't, top won't. I don't
care about used/free swap usage. I care about performance to the swap
partition. That's what I'm suspecting in the USB thumb drive FTL. I don't
care what the total swap usage is. I suspect that's irrelevant to the issue
at hand since the OOM isn't triggering because we're filling swap, but more
that it's due to not being able to get enough pages to the swap device fast
enough to satisfy the memory shortages, triggering OOM.

As for why it would affect the USB drive and not SD cards, I can only say
that USB drives tend to be first to market with bigger capacities. This has
traditionally made them less well tuned for anything other than large, long
sequential reads or writes that aren't mixed. More so than even SD or uSD
cards which tend to do better than USB drives at that workload. It's the
FTL that's the issue, not the NAND itself. The FTL is the software that
translates the log-style device you have to have for flash to work to the
LBA style devices that people attach to systems. If it can't cope with a
mixed workload, or needs to do too much garbage collection or
read/modify/write operations due to it's poor quality / tuning, that will
show up as long delays. USB flash also tends to suck more with BIO_DELETE
than others, though the swapper doesn't do that, so that's one fewer
wildcards we need to look at.

gstat -Bd -I 10 -f <regexp for your swap partition>  > gstat-swap-data.dat

would be how I'd recommend collecting it. This file may get kinda big
depending how long it takes to trigger the weird state. I'm hoping that if
you put this on a known good device, we'll power through the issues. We
might not get perfect correlation with this, but the data should show all
kinds of crazy before the system drives off the cliff if I'm right, so we
don't need perfect data.

There's some higher fidelity numbers we can get from the I/O scheduler with
dynamic scheduling compiled in, but I don't think we'll need those.

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

bob prohaska
On Fri, Jun 15, 2018 at 09:57:17AM -0600, Warner Losh wrote:

> gstat -Bd -I 10 -f <regexp for your swap partition>  > gstat-swap-data.dat
>
> would be how I'd recommend collecting it. This file may get kinda big
> depending how long it takes to trigger the weird state. I'm hoping that if
> you put this on a known good device, we'll power through the issues. We
> might not get perfect correlation with this, but the data should show all
> kinds of crazy before the system drives off the cliff if I'm right, so we
> don't need perfect data.
>

Apologies,  we spoke of gstat earlier. I lost the distinction between
gstat and vmstat. In the case at hand the possible swap devices are

/dev/da0b (this is the one causing trouble)
/dev/mmcsd0s3b
/dev/mmcsd0s3d
 

what would the correct <regexp> look like?

Is there any indication why problems seen on the RPI3 do not seem
apparent on an RPI2? A 2 GB USB flash swap partition seems to work
successfully on the Pi2.

Thanks very much!

bob prohaska

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

Warner Losh
On Fri, Jun 15, 2018 at 10:42 AM, bob prohaska <[hidden email]> wrote:

> On Fri, Jun 15, 2018 at 09:57:17AM -0600, Warner Losh wrote:
>
> > gstat -Bd -I 10 -f <regexp for your swap partition>  >
> gstat-swap-data.dat
> >
> > would be how I'd recommend collecting it. This file may get kinda big
> > depending how long it takes to trigger the weird state. I'm hoping that
> if
> > you put this on a known good device, we'll power through the issues. We
> > might not get perfect correlation with this, but the data should show all
> > kinds of crazy before the system drives off the cliff if I'm right, so we
> > don't need perfect data.
> >
>
> Apologies,  we spoke of gstat earlier. I lost the distinction between
> gstat and vmstat. In the case at hand the possible swap devices are
>
> /dev/da0b (this is the one causing trouble)
> /dev/mmcsd0s3b
> /dev/mmcsd0s3d
>
>
> what would the correct <regexp> look like?
>

I'd just use -f '^da0b$' for each of the devices.


> Is there any indication why problems seen on the RPI3 do not seem
> apparent on an RPI2? A 2 GB USB flash swap partition seems to work
> successfully on the Pi2.
>

Interesting. The only thing I can think of is different OOM tuning levels.
But if anything, I'd expect more trouble on the RPi2 than the RPi3. So that
might also point to a difference in USB and/or VM things. It would be
interesting to see if we see a significant difference in the graphs between
the two for the same physical drive. I'd have expected, like you, that this
would be reproducible between the two.

Warner
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: GPT vs MBR for swap devices

freebsd-arm mailing list
In reply to this post by bob prohaska


On 2018-Jun-15, at 9:42 AM, bob prohaska <fbsd at www.zefox.net> wrote:

> On Fri, Jun 15, 2018 at 09:57:17AM -0600, Warner Losh wrote:
>
>> gstat -Bd -I 10 -f <regexp for your swap partition>  > gstat-swap-data.dat
>>
>> would be how I'd recommend collecting it. This file may get kinda big
>> depending how long it takes to trigger the weird state. I'm hoping that if
>> you put this on a known good device, we'll power through the issues. We
>> might not get perfect correlation with this, but the data should show all
>> kinds of crazy before the system drives off the cliff if I'm right, so we
>> don't need perfect data.
>>
>
> Apologies,  we spoke of gstat earlier. I lost the distinction between
> gstat and vmstat. In the case at hand the possible swap devices are
>
> /dev/da0b (this is the one causing trouble)
> /dev/mmcsd0s3b
> /dev/mmcsd0s3d
>
>
> what would the correct <regexp> look like?
>
> Is there any indication why problems seen on the RPI3 do not seem
> apparent on an RPI2? A 2 GB USB flash swap partition seems to work
> successfully on the Pi2.

gstat and top/swapinfo are good for different things:

gstat: for various I/O rates and such to the swap partition(s)

top: for samples of how much swap space is in use (total).
swapinfo: per swap partition swap space usage and the total.
(swapinfo requires use of separate looping/sleeping for it to
repeatedly report.)

(swapinfo is a specialization/limitation of pstat.)


There may be times when you want information about both
aspects (I/O rates and usage-size figures from similar time
frames).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-arm
To unsubscribe, send any mail to "[hidden email]"
1234 ... 14