Is the BHyVe guest as suitable for high-performance disk IO as the host?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Is the BHyVe guest as suitable for high-performance disk IO as the host?

Tinker
Hi!

For an environment with very heavy parallell IO, should the performance
be just as good in a BHyVe guest as in the FreeBSD host environment?

What I thought of is that I guess within the host environment, the
storage subsystem should have all kinds of optimizations like an
internal work queue that pushes lots of work alinearly/asynchronously to
the disk controller and this way allows it, in turn, to give all its
performance.

Does the virtualized disk interface carry over all that goodness to the
guest?

(https://wiki.freebsd.org/bhyve seems to say yes, presuming you
configure BHyVe to run the virtual disk in AHCI mode?)

Thanks!
Tinker

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Is the BHyVe guest as suitable for high-performance disk IO as the host?

Neel Natu
Hi Tinker,

On Sat, May 9, 2015 at 9:49 AM, Tinker <[hidden email]> wrote:

> Hi!
>
> For an environment with very heavy parallell IO, should the performance be
> just as good in a BHyVe guest as in the FreeBSD host environment?
>
> What I thought of is that I guess within the host environment, the storage
> subsystem should have all kinds of optimizations like an internal work queue
> that pushes lots of work alinearly/asynchronously to the disk controller and
> this way allows it, in turn, to give all its performance.
>
> Does the virtualized disk interface carry over all that goodness to the
> guest?
>

bhyve creates 8 worker threads for each virtual disk controller (both
ahci and virtio-blk).

All guest I/O is handled asynchronously by these worker threads which
provide parallelism.

> (https://wiki.freebsd.org/bhyve seems to say yes, presuming you configure
> BHyVe to run the virtual disk in AHCI mode?)
>

The wiki is out of date.

Since r280037 the virtio-blk emulation also gets the benefits of using
the block_if worker threads.

best
Neel

> Thanks!
> Tinker
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to
> "[hidden email]"
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Is the BHyVe guest as suitable for high-performance disk IO as the host?

Tinker
Hi Neel,

Thank you very much for your response -

That's great!

I guess this should deliver the full capacity for basically any IO
system around, be it a fast SSD or ramdisk. (Since the guest may not
need to flush data immediately to the host, I guess read performance is
the more sensitive point.)

Which disk emulation mode is best for a BSD guest, "virtio-blk" or
"ahci-hd"?

In general, should any other consideration be made for performance
(using the "direct" or "nocache" BHyVe configuration options, particular
consideration for proper sector alignment when using a disk image stored
on the host's SSD-based ZFS, mounting the host ZFS filesystem with
"noatime")?

Kind regards,
Tinker

On 2015-05-10 04:03, Neel Natu wrote:

> Hi Tinker,
>
> On Sat, May 9, 2015 at 9:49 AM, Tinker <[hidden email]> wrote:
>> Hi!
>>
>> For an environment with very heavy parallell IO, should the
>> performance be
>> just as good in a BHyVe guest as in the FreeBSD host environment?
>>
>> What I thought of is that I guess within the host environment, the
>> storage
>> subsystem should have all kinds of optimizations like an internal work
>> queue
>> that pushes lots of work alinearly/asynchronously to the disk
>> controller and
>> this way allows it, in turn, to give all its performance.
>>
>> Does the virtualized disk interface carry over all that goodness to
>> the
>> guest?
>>
>
> bhyve creates 8 worker threads for each virtual disk controller (both
> ahci and virtio-blk).
>
> All guest I/O is handled asynchronously by these worker threads which
> provide parallelism.
>
>> (https://wiki.freebsd.org/bhyve seems to say yes, presuming you
>> configure
>> BHyVe to run the virtual disk in AHCI mode?)
>>
>
> The wiki is out of date.
>
> Since r280037 the virtio-blk emulation also gets the benefits of using
> the block_if worker threads.
>
> best
> Neel
>
>> Thanks!
>> Tinker
>>
>> _______________________________________________
>> [hidden email] mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>> To unsubscribe, send any mail to
>> "[hidden email]"

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Is the BHyVe guest as suitable for high-performance disk IO as the host?

Neel Natu
Hi Tinker,

On Sun, May 10, 2015 at 6:16 AM, Tinker <[hidden email]> wrote:

> Hi Neel,
>
> Thank you very much for your response -
>
> That's great!
>
> I guess this should deliver the full capacity for basically any IO system
> around, be it a fast SSD or ramdisk. (Since the guest may not need to flush
> data immediately to the host, I guess read performance is the more sensitive
> point.)
>
> Which disk emulation mode is best for a BSD guest, "virtio-blk" or
> "ahci-hd"?
>

You'll need to experiment because there are so many moving pieces: the
guest driver implementation, VM exit overhead, capabilities of a
emulated disk controller. From the narrow perspective of VM-exit
overhead 'virtio-blk' is better because the CPU decodes all the
information needed to handle the I/O exit.

> In general, should any other consideration be made for performance (using
> the "direct" or "nocache" BHyVe configuration options, particular
> consideration for proper sector alignment when using a disk image stored on
> the host's SSD-based ZFS, mounting the host ZFS filesystem with "noatime")?
>

Using "nocache" will prevent double-buffering of the virtual disk file
in the host. Using "direct" will guarantee consistency of the virtual
disk file on the underlying storage device. If you are purely focused
on performance then don't use either option but be aware that you'll
be using buffer cache on the host and also that the guest filesystem
could be corrupted on an unplanned host shutdown.

With respect to alignment I am not sure if you can request a specific
alignment when creating a flat file for the virtual disk. However, if
you use a ZVOL as a backing device then it can be created on top of a
partition which has the correct alignment.

best
Neel

> Kind regards,
> Tinker
>
>
> On 2015-05-10 04:03, Neel Natu wrote:
>>
>> Hi Tinker,
>>
>> On Sat, May 9, 2015 at 9:49 AM, Tinker <[hidden email]> wrote:
>>>
>>> Hi!
>>>
>>> For an environment with very heavy parallell IO, should the performance
>>> be
>>> just as good in a BHyVe guest as in the FreeBSD host environment?
>>>
>>> What I thought of is that I guess within the host environment, the
>>> storage
>>> subsystem should have all kinds of optimizations like an internal work
>>> queue
>>> that pushes lots of work alinearly/asynchronously to the disk controller
>>> and
>>> this way allows it, in turn, to give all its performance.
>>>
>>> Does the virtualized disk interface carry over all that goodness to the
>>> guest?
>>>
>>
>> bhyve creates 8 worker threads for each virtual disk controller (both
>> ahci and virtio-blk).
>>
>> All guest I/O is handled asynchronously by these worker threads which
>> provide parallelism.
>>
>>> (https://wiki.freebsd.org/bhyve seems to say yes, presuming you configure
>>> BHyVe to run the virtual disk in AHCI mode?)
>>>
>>
>> The wiki is out of date.
>>
>> Since r280037 the virtio-blk emulation also gets the benefits of using
>> the block_if worker threads.
>>
>> best
>> Neel
>>
>>> Thanks!
>>> Tinker
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>>> To unsubscribe, send any mail to
>>> "[hidden email]"
>
>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "[hidden email]"