Splitting Mellanox ConnectX-4 interface using breakout cables

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Splitting Mellanox ConnectX-4 interface using breakout cables

Andrey V. Elsukov
Hi All,

we are wondering, is it possible to use such configuration under
FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?

--
WBR, Andrey V. Elsukov


signature.asc (565 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Mellanox ConnectX-4 interface using breakout cables

David Horn
I was under the impression that these Mellanox 40G->4x10G breakout cables
were for the Mellanox switch side (not Mellanox NIC) to provide more
flexible utilization options for the switch.  I have never heard of doing
this from the 40G NIC (on any OS).

Please reply if you have seen this working on *any* OS from the Mellanox
40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

-Dave

On Wed, Sep 13, 2017 at 7:00 AM, Andrey V. Elsukov <[hidden email]>
wrote:

> Hi All,
>
> we are wondering, is it possible to use such configuration under
> FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?
>
> --
> WBR, Andrey V. Elsukov
>
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Mellanox ConnectX-4 interface using breakout cables

Andrey V. Elsukov
On 13.09.2017 16:43, David Horn wrote:
> I was under the impression that these Mellanox 40G->4x10G breakout
> cables were for the Mellanox switch side (not Mellanox NIC) to provide
> more flexible utilization options for the switch.  I have never heard of
> doing this from the 40G NIC (on any OS).
>
> Please reply if you have seen this working on *any* OS from the Mellanox
> 40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

We have several Intel-XL710-QDA1 cards that work like this:

ixl0@pci0:6:0:0: class=0x020000 card=0x00028086 chip=0x15848086 rev=0x01
hdr=0x00
ixl1@pci0:6:0:1: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00
ixl2@pci0:6:0:2: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00
ixl3@pci0:6:0:3: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00

This is one card with QSPF28 splited into 4x10G.

--
WBR, Andrey V. Elsukov


signature.asc (565 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Splitting Mellanox ConnectX-4 interface using breakout cables

Matt Joras-2
In reply to this post by David Horn
On Sep 13, 2017 6:43 AM, "David Horn" <[hidden email]> wrote:

I was under the impression that these Mellanox 40G->4x10G breakout cables
were for the Mellanox switch side (not Mellanox NIC) to provide more
flexible utilization options for the switch.  I have never heard of doing
this from the 40G NIC (on any OS).

Please reply if you have seen this working on *any* OS from the Mellanox
40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

-Dave

On Wed, Sep 13, 2017 at 7:00 AM, Andrey V. Elsukov <[hidden email]>
wrote:

> Hi All,
>
> we are wondering, is it possible to use such configuration under
> FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?
>
> --
> WBR, Andrey V. Elsukov
>
>



I was also under the impression that they were exclusively used on the
switch for Mellanox. Mellanox's documentation seems to back this up:

"Important: The 40GbE split options is supported only on Mellanox
switches and not supported on Mellanox adapters (e.g. ConnectX-3) when
equipped with 40GbE ports. In case you wish to limit the 40GbE port on
the adapter to 10GbE you can use QSA or similar 40GbE-10GbE cables
(Refer to Mellanox.com - here)" [1]

That doesn't mention their newer NICs, however, though I don't see a
mention of the feature on their product sheets.

Matt

[1] https://community.mellanox.com/docs/DOC-1450
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"