Allowing a local subnet route to change to a different ifnet

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Allowing a local subnet route to change to a different ifnet

Ryan Stone-2
I'm going to prefix this question by noting that I realize that the
configuration that I am about to describe is quite nonsensical.
Unfortunately, it seems that under older versions of FreeBSD (possibly
FreeBSD 7-vintage), the configuration mostly "worked", and now at
$WORK I am responsible for making the configuration continue to work
in the future.  The customer in this case is completely unwilling to
modify their configuration.

I have a customer that has configured two different IPs on the same
subnet on two different interfaces.  The behaviour that they want is
that if the link on one of the two interfaces goes down, the route to
that subnet will migrate to the other IP on the other interface as a
quasi-failover behaviour.  Under FreeBSD 7, we had a daemon that
accomplished this by detecting the link loss and then using "route
change" to move the route to the up interface.  If the subnet in
question was 192.168.1.0/24, for example, we could run "route change
192.1.68.1.0/24 -ifp em1" to migrate the route.

Running on -head I run into two issues.  The first comes out of
r264986, which changes the behaviour of RTM_CHANGE.  The code path
changed significantly, but the part that impacts me is that now any
RTM_CHANGE command with the gateway set NULL gets EINVAL immediately
where previously it was allowed.  I've hacked around this problem
locally for testing purposes but I really don't understand the code
well enough at this point to see what a real fix would look like.

The second issue is quite complex.  The root cause is that the routing
code sets IFA_ROUTE on any ifaddr that has a local subnet route
associated with it.  If I don't migrate this flag to the new ifaddr,
very bizarre behaviour follows.  I've done a prototype that detects
when this migration is needed in rtrequest1_fib_change() and it works,
but IFA_ROUTE seems to otherwise be handled in individual L3 protocols
like in and in6, so I'm worried that this is a layering violation.  My
patch looks like this:
https://people.freebsd.org/~rstone/patches/route-change-subnet.diff


My first, and most important question, is whether a patch that would
allow a subnet route to be migrated to a different interface be
something that would be acceptable in FreeBSD?  If so, I need guidance
on what a proper fix for both issues would look like so that I can
implement fixes that I can upstream.

Thanks,
Ryan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Allowing a local subnet route to change to a different ifnet

Alan Somers-2
On Wed, Jan 17, 2018 at 2:56 PM, Ryan Stone <[hidden email]> wrote:

> I'm going to prefix this question by noting that I realize that the
> configuration that I am about to describe is quite nonsensical.
> Unfortunately, it seems that under older versions of FreeBSD (possibly
> FreeBSD 7-vintage), the configuration mostly "worked", and now at
> $WORK I am responsible for making the configuration continue to work
> in the future.  The customer in this case is completely unwilling to
> modify their configuration.
>
> I have a customer that has configured two different IPs on the same
> subnet on two different interfaces.  The behaviour that they want is
> that if the link on one of the two interfaces goes down, the route to
> that subnet will migrate to the other IP on the other interface as a
> quasi-failover behaviour.  Under FreeBSD 7, we had a daemon that
> accomplished this by detecting the link loss and then using "route
> change" to move the route to the up interface.  If the subnet in
> question was 192.168.1.0/24, for example, we could run "route change
> 192.1.68.1.0/24 -ifp em1" to migrate the route.
>
> Running on -head I run into two issues.  The first comes out of
> r264986, which changes the behaviour of RTM_CHANGE.  The code path
> changed significantly, but the part that impacts me is that now any
> RTM_CHANGE command with the gateway set NULL gets EINVAL immediately
> where previously it was allowed.  I've hacked around this problem
> locally for testing purposes but I really don't understand the code
> well enough at this point to see what a real fix would look like.
>
> The second issue is quite complex.  The root cause is that the routing
> code sets IFA_ROUTE on any ifaddr that has a local subnet route
> associated with it.  If I don't migrate this flag to the new ifaddr,
> very bizarre behaviour follows.  I've done a prototype that detects
> when this migration is needed in rtrequest1_fib_change() and it works,
> but IFA_ROUTE seems to otherwise be handled in individual L3 protocols
> like in and in6, so I'm worried that this is a layering violation.  My
> patch looks like this:
> https://people.freebsd.org/~rstone/patches/route-change-subnet.diff
>
>
> My first, and most important question, is whether a patch that would
> allow a subnet route to be migrated to a different interface be
> something that would be acceptable in FreeBSD?  If so, I need guidance
> on what a proper fix for both issues would look like so that I can
> implement fixes that I can upstream.
>
> Thanks,
> Ryan
>

I'm sorry for you Ryan; this sounds like a doozy.  I know you said that the
customer is unwilling to change, but would they consider using a lagg(4)
interface?  Using lagg with laggproto=failover is designed to solve exactly
this problem.  They wouldn't have to recable anything, and they could keep
their single IP address.  If not, you should see PR 189088; I think it's
related.

-Alan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Allowing a local subnet route to change to a different ifnet

Adam Vande More
On Wed, Jan 17, 2018 at 4:06 PM, Alan Somers <[hidden email]> wrote:

> I'm sorry for you Ryan; this sounds like a doozy.  I know you said that the
> customer is unwilling to change, but would they consider using a lagg(4)
> interface?  Using lagg with laggproto=failover is designed to solve exactly
> this problem.  They wouldn't have to recable anything, and they could keep
> their single IP address.
>

What single IP address and what does cabling have to do with it?

--
Adam
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Allowing a local subnet route to change to a different ifnet

Rodney W. Grimes-4
In reply to this post by Ryan Stone-2
> I'm going to prefix this question by noting that I realize that the
> configuration that I am about to describe is quite nonsensical.
> Unfortunately, it seems that under older versions of FreeBSD (possibly
> FreeBSD 7-vintage), the configuration mostly "worked", and now at
> $WORK I am responsible for making the configuration continue to work
> in the future.  The customer in this case is completely unwilling to
> modify their configuration.
>
> I have a customer that has configured two different IPs on the same
> subnet on two different interfaces.  The behaviour that they want is
> that if the link on one of the two interfaces goes down, the route to
> that subnet will migrate to the other IP on the other interface as a
> quasi-failover behaviour.  Under FreeBSD 7, we had a daemon that
> accomplished this by detecting the link loss and then using "route
> change" to move the route to the up interface.  If the subnet in
> question was 192.168.1.0/24, for example, we could run "route change
> 192.1.68.1.0/24 -ifp em1" to migrate the route.

"route change 192.1.68.1.0/24 -ifp em1" does not appear to be
valid syntax to me, -ifp is not a route option?
Did you mean -interface?

And I think this well work if you use the local IP of the em1 interface
for the gateway?  That should get past the null check

>
> Running on -head I run into two issues.  The first comes out of
> r264986, which changes the behaviour of RTM_CHANGE.  The code path
> changed significantly, but the part that impacts me is that now any
> RTM_CHANGE command with the gateway set NULL gets EINVAL immediately
> where previously it was allowed.  I've hacked around this problem
> locally for testing purposes but I really don't understand the code
> well enough at this point to see what a real fix would look like.
>
> The second issue is quite complex.  The root cause is that the routing
> code sets IFA_ROUTE on any ifaddr that has a local subnet route
> associated with it.  If I don't migrate this flag to the new ifaddr,
> very bizarre behaviour follows.  I've done a prototype that detects
> when this migration is needed in rtrequest1_fib_change() and it works,
> but IFA_ROUTE seems to otherwise be handled in individual L3 protocols
> like in and in6, so I'm worried that this is a layering violation.  My
> patch looks like this:
> https://people.freebsd.org/~rstone/patches/route-change-subnet.diff
>
>
> My first, and most important question, is whether a patch that would
> allow a subnet route to be migrated to a different interface be
> something that would be acceptable in FreeBSD?  If so, I need guidance
> on what a proper fix for both issues would look like so that I can
> implement fixes that I can upstream.

From a fundemental standpoint this should work,
that it is now broken is a regression that needs fixed.

You may also run into issues with the kernel
maintain_loopback_route functions.

> Thanks,
> Ryan

--
Rod Grimes                                                 [hidden email]
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Allowing a local subnet route to change to a different ifnet

Bjoern A. Zeeb
On 18 Jan 2018, at 0:09, Rodney W. Grimes wrote:

>> I have a customer that has configured two different IPs on the same
>> subnet on two different interfaces.  The behaviour that they want is
>> that if the link on one of the two interfaces goes down, the route to
>> that subnet will migrate to the other IP on the other interface as a
>> quasi-failover behaviour.  Under FreeBSD 7, we had a daemon that
>> accomplished this by detecting the link loss and then using "route
>> change" to move the route to the up interface.  If the subnet in
>> question was 192.168.1.0/24, for example, we could run "route change
>> 192.1.68.1.0/24 -ifp em1" to migrate the route.
>
> "route change 192.1.68.1.0/24 -ifp em1" does not appear to be
> valid syntax to me, -ifp is not a route option?
> Did you mean -interface?

The proper syntax should be route change -ifp <if_name> <network>
but that’s not the issue (the other syntax works or used to work but
not according to specification).


>> Running on -head I run into two issues.  The first comes out of
>> r264986, which changes the behaviour of RTM_CHANGE.  The code path
>> changed significantly, but the part that impacts me is that now any
>> RTM_CHANGE command with the gateway set NULL gets EINVAL immediately
>> where previously it was allowed.  I've hacked around this problem
>> locally for testing purposes but I really don't understand the code
>> well enough at this point to see what a real fix would look like.

Running route -n monitor & while doing the change I get very weird
results (without your patch).

[ignore the patch for the moment]

>> My first, and most important question, is whether a patch that would
>> allow a subnet route to be migrated to a different interface be
>> something that would be acceptable in FreeBSD?

Yes.

>> If so, I need guidance
>> on what a proper fix for both issues would look like so that I can
>> implement fixes that I can upstream.
>
> From a fundemental standpoint this should work,
> that it is now broken is a regression that needs fixed.

+1

/bz
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"