[Bug 222288] g_bio leak after zfs ABD commit

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 222288] g_bio leak after zfs ABD commit

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222288

            Bug ID: 222288
           Summary: g_bio leak after zfs ABD commit
           Product: Base System
           Version: 11.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Many People
          Priority: ---
         Component: kern
          Assignee: [hidden email]
          Reporter: [hidden email]

It looks like the ABD commit may have introduced a leak of geom bio structs.
It's a slow leak so it's not obvious, but after a few weeks of uptime, one of
my 8GB-RAM systems had accumulated a couple million g_bio objects according to
vmstat (over 2GB of RAM) and was starting to swap.  I rebooted it, and after 11
hours, I'm up to 310182:

# uptime
 9:55AM  up 11:08, 11 users, load averages: 0.50, 0.54, 0.55
# vmstat -z | egrep 'ITEM|g_bio'
ITEM                   SIZE   LIMIT     USED     FREE       REQ   FAIL SLEEP
g_bio:                  376,      0,  310182,     818, 38182778,     0,   0

Looking at base r321610 itself, it looks like there's a g_bio_destroy() call
that got relocated from vdev_geom_io_intr() to vdev_geom_io_done(); maybe there
are cases where vdev_geom_io_intr is called, but vdev_geom_io_done isn't?  I
don't know enough about ZFS internals to get any farther than this.

Rolling the kernel back to r321609 makes the leak stop, and updating to r321610
makes it appear again.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

[Bug 222288] g_bio leak after zfs ABD commit

bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222288

Andriy Gapon <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|New                         |Open
                 CC|                            |[hidden email]
           Assignee|[hidden email]    |[hidden email]

--- Comment #1 from Andriy Gapon <[hidden email]> ---
(In reply to Dan Nelson from comment #0)
Thank you very much for the report!
As soon as I read it I noticed that I have the same kind of issue.  I dug
through the biozone to examine the leaked bio-s and they all seem to have
bio_cmd = 5 and bio_flags = 8.  So, they seem to be BIO_FLUSH bio-s.  Their
zio-s must have been ZIO_TYPE_IOCTL.

Now, those zio-s use ZIO_IOCTL_PIPELINE and it is defined as:
#define ZIO_IOCTL_PIPELINE                      \
        (ZIO_INTERLOCK_STAGES |                 \
        ZIO_STAGE_VDEV_IO_START |               \
        ZIO_STAGE_VDEV_IO_ASSESS)

So, ZIO_STAGE_VDEV_IO_START is in the pipeline, but ZIO_STAGE_VDEV_IO_DONE is
not as you have correctly theorised.
The normal I/O pipelines always include ZIO_VDEV_IO_STAGES
#define ZIO_VDEV_IO_STAGES                      \
        (ZIO_STAGE_VDEV_IO_START |              \
        ZIO_STAGE_VDEV_IO_DONE |                \
        ZIO_STAGE_VDEV_IO_ASSESS)
so the problem does not affect them.

I will double-check why the ioctl pipeline omitted ZIO_STAGE_VDEV_IO_DONE and
will test adding that stage.

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "[hidden email]"