Packets passed by pf don't make it out?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Packets passed by pf don't make it out?

J David
We have pf running on a FreeBSD 11.4 system acting as a load balancer,
mapping a set of 8 external DNS service IP addresses to a set of
internal DNS servers, any of which can handle those requests.

When UDP packets from one source IP/port arrive for multiple external
IPs in a short period of time, pf claims they all pass, but only the
ones for the first IP actually make it out the outbound interface.

Redirect rule:

rdr inet proto udp from any to { 172.17.53.1, 172.17.53.2,
172.17.53.3, 172.17.53.4, 172.17.53.5, 172.17.53.6, 172.17.53.7,
172.17.53.8 } port 53 -> { 10.53.0.1, 10.53.0.2, 10.53.0.3 }
round-robin sticky-address

Pass rule:

pass in log quick proto udp to { 10.53.0.1, 10.53.0.2, 10.53.0.3 } port 53

(The pass rule isn't technically necessary, it's only there to log the
packets to debug this issue.)

With tcpdumps running simultaneously on ix1, all packets show up the
inbound interface:

16:32:39.183168 IP 149.20.1.48.56246 > 172.17.53.1.53: 3215 SOA?
example.com. (29)
16:32:39.183761 IP 149.20.1.48.56246 > 172.17.53.2.53: 2934 SOA?
example.com. (29)
16:32:39.184368 IP 149.20.1.48.56246 > 172.17.53.3.53: 52875 SOA?
example.com. (29)
16:32:39.185618 IP 149.20.1.48.56246 > 172.17.53.4.53: 36289 SOA?
example.com. (29)
16:32:39.186067 IP 149.20.1.48.56246 > 172.17.53.5.53: 44049 SOA?
example.com. (29)
16:32:39.186422 IP 149.20.1.48.56246 > 172.17.53.6.53: 34410 SOA?
example.com. (29)
16:32:39.186494 IP 149.20.1.48.56246 > 172.17.53.7.53: 30923 SOA?
example.com. (29)
16:32:39.188541 IP 149.20.1.48.56246 > 172.17.53.8.53: 48814 SOA?
example.com. (29)

and on pflog0:

16:32:39.183189 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 3215 SOA? example.com. (29)
16:32:39.183780 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 2934 SOA? example.com. (29)
16:32:39.184375 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 52875 SOA? example.com. (29)
16:32:39.185625 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 36289 SOA? example.com. (29)
16:32:39.186074 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 44049 SOA? example.com. (29)
16:32:39.186425 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 34410 SOA? example.com. (29)
16:32:39.186499 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 30923 SOA? example.com. (29)
16:32:39.188548 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
10.53.0.1.53: 48814 SOA? example.com. (29)

but only the first one appears on ix0, the outbound interface:

16:32:39.183211 IP 149.20.1.48.56246 > 10.53.0.1.53: 3215 SOA? example.com. (29)

The actual query order is random, so if the test is repeated a minute
later, then 172.17.53.3 might be hit first, and then that one will
make it through and the rest will disappear.  So it is not specific to
any destination IP.

It also only appears to occur when the UDP source port is the same
across the connections.  (This is probably why TCP is not affected.)

It does not appear related to state entries ("no state") doesn't help.

If "sticky-address" is removed from the rdr, then one packet will make
it through for each backend IP, instead of one total.

What could be causing this?  It seems somehow related to 5-tuple
non-uniqueness after the rdr, but that shouldn't be an issue for UDP;
it should be treated as two connectionless packets from the same
source for the same destination.

(The query test source, 149.20.1.48 is an EDNS checker found at
https://dnsflagday.net/2020/ .)

Thanks for any advice!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

J David
To investigate this issue, I've created a greatly simplified and
reproducible test case.  The code is available at:

https://github.com/jdavidlists/pfudpbug

It includes the pf.conf, the source code for client and server, and
the rc.conf from all three machines.

The test uses three FreeBSD systems (client, gateway, and server) to
demonstrate that if a client with a bound UDP source port sends UDP
packets to multiple server addresses that a gateway running pf
redirects to one backend server, only the first such packet will be
delivered.  The remaining packets never leave the gateway; they get
lost somewhere after being logged as passing pf via pflog0 but before
a tcpdump running on the gateway's server-facing interface.  They also
do not increase the outbound packet count on the server-facing
interface.

(More detail is available in the repository's README.md.)

This may indicate a bug, but I'm not sure whether it is happening
inside pf or farther along the output path.  Nor do I know how to
explore this further.

Is anyone able to point me in the right direction here?

Thanks!

On Fri, Oct 2, 2020 at 1:35 PM J David <[hidden email]> wrote:

>
> We have pf running on a FreeBSD 11.4 system acting as a load balancer,
> mapping a set of 8 external DNS service IP addresses to a set of
> internal DNS servers, any of which can handle those requests.
>
> When UDP packets from one source IP/port arrive for multiple external
> IPs in a short period of time, pf claims they all pass, but only the
> ones for the first IP actually make it out the outbound interface.
>
> Redirect rule:
>
> rdr inet proto udp from any to { 172.17.53.1, 172.17.53.2,
> 172.17.53.3, 172.17.53.4, 172.17.53.5, 172.17.53.6, 172.17.53.7,
> 172.17.53.8 } port 53 -> { 10.53.0.1, 10.53.0.2, 10.53.0.3 }
> round-robin sticky-address
>
> Pass rule:
>
> pass in log quick proto udp to { 10.53.0.1, 10.53.0.2, 10.53.0.3 } port 53
>
> (The pass rule isn't technically necessary, it's only there to log the
> packets to debug this issue.)
>
> With tcpdumps running simultaneously on ix1, all packets show up the
> inbound interface:
>
> 16:32:39.183168 IP 149.20.1.48.56246 > 172.17.53.1.53: 3215 SOA?
> example.com. (29)
> 16:32:39.183761 IP 149.20.1.48.56246 > 172.17.53.2.53: 2934 SOA?
> example.com. (29)
> 16:32:39.184368 IP 149.20.1.48.56246 > 172.17.53.3.53: 52875 SOA?
> example.com. (29)
> 16:32:39.185618 IP 149.20.1.48.56246 > 172.17.53.4.53: 36289 SOA?
> example.com. (29)
> 16:32:39.186067 IP 149.20.1.48.56246 > 172.17.53.5.53: 44049 SOA?
> example.com. (29)
> 16:32:39.186422 IP 149.20.1.48.56246 > 172.17.53.6.53: 34410 SOA?
> example.com. (29)
> 16:32:39.186494 IP 149.20.1.48.56246 > 172.17.53.7.53: 30923 SOA?
> example.com. (29)
> 16:32:39.188541 IP 149.20.1.48.56246 > 172.17.53.8.53: 48814 SOA?
> example.com. (29)
>
> and on pflog0:
>
> 16:32:39.183189 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 3215 SOA? example.com. (29)
> 16:32:39.183780 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 2934 SOA? example.com. (29)
> 16:32:39.184375 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 52875 SOA? example.com. (29)
> 16:32:39.185625 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 36289 SOA? example.com. (29)
> 16:32:39.186074 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 44049 SOA? example.com. (29)
> 16:32:39.186425 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 34410 SOA? example.com. (29)
> 16:32:39.186499 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 30923 SOA? example.com. (29)
> 16:32:39.188548 rule 16/0(match): pass in on ix1: 149.20.1.48.56246 >
> 10.53.0.1.53: 48814 SOA? example.com. (29)
>
> but only the first one appears on ix0, the outbound interface:
>
> 16:32:39.183211 IP 149.20.1.48.56246 > 10.53.0.1.53: 3215 SOA? example.com. (29)
>
> The actual query order is random, so if the test is repeated a minute
> later, then 172.17.53.3 might be hit first, and then that one will
> make it through and the rest will disappear.  So it is not specific to
> any destination IP.
>
> It also only appears to occur when the UDP source port is the same
> across the connections.  (This is probably why TCP is not affected.)
>
> It does not appear related to state entries ("no state") doesn't help.
>
> If "sticky-address" is removed from the rdr, then one packet will make
> it through for each backend IP, instead of one total.
>
> What could be causing this?  It seems somehow related to 5-tuple
> non-uniqueness after the rdr, but that shouldn't be an issue for UDP;
> it should be treated as two connectionless packets from the same
> source for the same destination.
>
> (The query test source, 149.20.1.48 is an EDNS checker found at
> https://dnsflagday.net/2020/ .)
>
> Thanks for any advice!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Andreas Longwitz
> To investigate this issue, I've created a greatly simplified and
> reproducible test case.  The code is available at:
>
> https://github.com/jdavidlists/pfudpbug

A similar setup works for me without any problems, so there may be
something special in your environment.

Please look at the output of "pfctl -vsn" on fb2 during your test.
With "netstat -ss | grep drop" you can check for packets dropped by the
kernel for what reason ever.

It seems your routing table on fb2 is empty, please try to set a
defaultroute, e.g.: "route add default 10.0.0.NN" with any NN.


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

J David
On Sun, Oct 11, 2020 at 12:46 PM Andreas Longwitz <[hidden email]> wrote:
> Please look at the output of "pfctl -vsn" on fb2 during your test.
> With "netstat -ss | grep drop" you can check for packets dropped by the
> kernel for what reason ever.

Here's the complete diff of the output from netstat -ss from before to
after running the test:

--- nss.pre 2020-10-11 17:10:19.932002000 +0000
+++ nss.post 2020-10-11 17:10:21.999823000 +0000
@@ -48,9 +48,9 @@
  Packet drop statistics:
  Timeouts:
 ip:
- 66578 total packets received
+ 66582 total packets received
  66531 packets for this host
- 16 packets forwarded
+ 17 packets forwarded
  1 packet not forwardable
  31675 packets sent from this host
  10 packets sent with fabricated ip header

No drops of any kind (nor anything else) recorded during the test.

4 packets in, 1 packet forwarded, which exactly matches the observed
behavior of only one packet reaching the server.

The results of "pfctl -vsn" are a bit more interesting, and also inconsistent.

Before, after a full flush to zero states and counters:

rdr inet proto udp from any to 172.16.0.0/12 port = 12345 -> 10.255.255.3
  [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
  [ Inserted: uid 0 pid 1044 State Creations: 0     ]

After:

rdr inet proto udp from any to 172.16.0.0/12 port = 12345 -> 10.255.255.3
  [ Evaluations: 4         Packets: 1         Bytes: 44          States: 1     ]
  [ Inserted: uid 0 pid 1044 State Creations: 4     ]

So it says it created four states, but only matched one packet out of
the four it evaluated.  And it didn't create 4 states, either. "pfctl
-s state" shows only 1:

all udp 10.255.255.3:12345 (172.16.0.1:12345) <- 10.0.0.1:23456
NO_TRAFFIC:SINGLE

and pflog0 reported all four packets as matching the pass rule which,
important, is based on the destination address after redirection:

17:23:39.039641 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16
17:23:39.039751 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16
17:23:39.039769 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16
17:23:39.039780 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16

If I repeat the test, this happens:

rdr inet proto udp from any to 172.16.0.0/12 port = 12345 -> 10.255.255.3
  [ Evaluations: 7         Packets: 2         Bytes: 88          States: 1     ]
  [ Inserted: uid 0 pid 1044 State Creations: 7     ]

But still just the one state:

all udp 10.255.255.3:12345 (172.16.0.1:12345) <- 10.0.0.1:23456
NO_TRAFFIC:SINGLE

But only three passes appear in pflog0:

17:29:19.857174 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16
17:29:19.857193 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16
17:29:19.857226 rule 0/0(match): pass in on em1: 10.0.0.1.23456 >
10.255.255.3.12345: UDP, length 16

And, to confirm, the first packet from each of these tests did reach
the server, the remaining three from each test did not.

> A similar setup works for me without any problems, so there may be
> something special in your environment.

This has been tested on fresh installs of both FreeBSD 12.1 and 11.4
on both physical hardware and virtual machines including both Xeons
and AMD Epyc.  So it seems like most environmental factors have been
controlled for.

> It seems your routing table on fb2 is empty, please try to set a
> defaultroute, e.g.: "route add default 10.0.0.NN" with any NN.

fb2 does have a default route, it is obtained from DHCP on the first
interface.  But that is not relevant; the client machine (fb1) is
directly connected to fb2's second interface, and the server (fb3) is
directly connected to fb2's third interface.

No additional routes are necessary for this test, and the default
route is never consulted.

Perhaps there's some detail of the scenario that I have omitted
without which it's not clear?

Thanks!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Andreas Longwitz
Hello,

now I can confirm (on FreeBSD 10 Stable) what you see on fb2 when your
program udp_client is running on fb1. pf creates a state for the first
packet only, for the other packets pf failes to create a state with
messages like

pf: stack key attach failed on re0: UDP in wire: 192.168.14.10:23456
172.16.0.2:12345 stack: 192.168.14.10:23456
192.168.14.100:12345 1:0, existing: UDP in wire: 192.168.14.10:23456
172.16.0.1:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0

pf gives this messages in debug mode (pfctl -x loud).

I do not know if we see a bug in pf or if your program udp_client does
something illegal, I think Kristof can tell us.

Regards Andreas
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Kristof Provost
On 12 Oct 2020, at 23:48, Andreas Longwitz wrote:

> Hello,
>
> now I can confirm (on FreeBSD 10 Stable) what you see on fb2 when your
> program udp_client is running on fb1. pf creates a state for the first
> packet only, for the other packets pf failes to create a state with
> messages like
>
> pf: stack key attach failed on re0: UDP in wire: 192.168.14.10:23456
> 172.16.0.2:12345 stack: 192.168.14.10:23456
> 192.168.14.100:12345 1:0, existing: UDP in wire: 192.168.14.10:23456
> 172.16.0.1:12345 stack: 192.168.14.10:23456 192.168.14.100:12345 1:0
>
> pf gives this messages in debug mode (pfctl -x loud).
>
> I do not know if we see a bug in pf or if your program udp_client does
> something illegal, I think Kristof can tell us.
>
Your confidence is both flattering and misplaced :)

I think I can reproduce the problem on CURRENT and with VNET jails,
which is convenient.

I see the same ‘stack key attach failed’ error message. My current
thinking is that we’re hitting a state collision, because post-RDR our
connection information is the same (192.168.14.10:23456
192.168.14.100:12345). That means we can’t create a new state, and the
packet gets dropped.

It’s a little unusual for a client to keep re-using src ports like
that, but it’s not actually wrong.
I’m not sure how we can fix this.

Best,
Kristof
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

J David
On 12 Oct 2020, at 23:48, Andreas Longwitz wrote:
> pf gives this messages in debug mode (pfctl -x loud).

Yes, with that setting I'm also seeing those messages.

On Tue, Oct 13, 2020 at 5:35 PM Kristof Provost <[hidden email]> wrote:
> I see the same ‘stack key attach failed’ error message. My current
> thinking is that we’re hitting a state collision, because post-RDR our
> connection information is the same (192.168.14.10:23456
> 192.168.14.100:12345). That means we can’t create a new state, and the
> packet gets dropped.

This is probably a dumb question because I know less than nothing
about pf internals, but why wouldn't it match the existing state?

Yes, the original destination was different before the redirect, but
after the redirect they're going from the same host/port to the same
host/port.  What's the definition of state equality, and does that
satisfy it?

In order to say that they're not the same state, wouldn't the packets
have to bear some property that survived the redirect that would
distinguish them as state-unequal?  If it did, would/should that
property not then be part of the computation of state entry
uniqueness?

Like I said, probably a dumb question.

> It’s a little unusual for a client to keep re-using src ports like
> that, but it’s not actually wrong.

After further study, I think the DNS validators used by Let's Encrypt
to control TLS certificate issuance may also be doing this, which
would make it a little more urgent.  (But that bears confirmation.)

Thanks!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Kristof Provost
On 14 Oct 2020, at 18:52, J David wrote:

> On 12 Oct 2020, at 23:48, Andreas Longwitz wrote:
>> pf gives this messages in debug mode (pfctl -x loud).
>
> Yes, with that setting I'm also seeing those messages.
>
> On Tue, Oct 13, 2020 at 5:35 PM Kristof Provost <[hidden email]>
> wrote:
>> I see the same ‘stack key attach failed’ error message. My
>> current
>> thinking is that we’re hitting a state collision, because post-RDR
>> our
>> connection information is the same (192.168.14.10:23456
>> 192.168.14.100:12345). That means we can’t create a new state, and
>> the
>> packet gets dropped.
>
> This is probably a dumb question because I know less than nothing
> about pf internals, but why wouldn't it match the existing state?
>
“It’s complicated”.

In essence, pf tracks both the pre- and post-translation tuple, so what
we’re seeing here is one of those conflicting with an existing session
and that’s causing the failure.
There’s good reason to do this, as we have to be able to match state
on both the pre-translation side (when processing LAN -> WAN traffic)
and post-translation (WAN -> LAN).

Best regards,
Kristof
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

J David
On Wed, Oct 14, 2020 at 1:59 PM Kristof Provost <[hidden email]> wrote:
> There’s good reason to do this, as we have to be able to match state
> on both the pre-translation side (when processing LAN -> WAN traffic)
> and post-translation (WAN -> LAN).

So, basically, pf would need separate states for each pre-redirect
destination address in order to have the information needed to map the
reply packet back to the original destination address.

But even if pf did that, the problem does not go away.  It just moves
to the reply packet coming back with only the post-redirect info.
That info matches multiple states, leaving pf no way to pick the right
one.

Is that about right?

Thanks!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Kristof Provost


On 14 Oct 2020, at 21:16, J David wrote:

> On Wed, Oct 14, 2020 at 1:59 PM Kristof Provost <[hidden email]>
> wrote:
>> There’s good reason to do this, as we have to be able to match
>> state
>> on both the pre-translation side (when processing LAN -> WAN traffic)
>> and post-translation (WAN -> LAN).
>
> So, basically, pf would need separate states for each pre-redirect
> destination address in order to have the information needed to map the
> reply packet back to the original destination address.
>
> But even if pf did that, the problem does not go away.  It just moves
> to the reply packet coming back with only the post-redirect info.
> That info matches multiple states, leaving pf no way to pick the right
> one.
>
> Is that about right?
>
Pretty much, I think.

I’ve not dug very deep yet, but I wonder if we shouldn’t have to
teach pf to change the source port to avoid conflicting states in the
first place.
It’s a non-trivial problem in any case.

Regards,
Kristof
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

J David
On Wed, Oct 14, 2020 at 3:20 PM Kristof Provost <[hidden email]> wrote:
> I’ve not dug very deep yet, but I wonder if we shouldn’t have to
> teach pf to change the source port to avoid conflicting states in the
> first place.

That was my first thought as well, framed mentally as some sort of
port-only Frankenstein's binat because my level of understanding is
clearly more cartoonish than yours. ;-)

My second thought was to wonder if my approach is architecturally
wrong.  Would it make sense for the many-to-many case to use route-to
instead of rdr, leave the packet unmodified, and expect every machine
in the server pool to catch all the public IPs?

That might still be tricky.  Using rdr would presumably hit the same
problem.  Maybe something gross like ifconfig'ing the public pool
addresses as /32's on lo0, then binding on those, maybe?

Thanks!
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Packets passed by pf don't make it out?

Kristof Provost
On 14 Oct 2020, at 21:35, J David wrote:

> On Wed, Oct 14, 2020 at 3:20 PM Kristof Provost <[hidden email]>
> wrote:
>> I’ve not dug very deep yet, but I wonder if we shouldn’t have to
>> teach pf to change the source port to avoid conflicting states in the
>> first place.
>
> That was my first thought as well, framed mentally as some sort of
> port-only Frankenstein's binat because my level of understanding is
> clearly more cartoonish than yours. ;-)
>
> My second thought was to wonder if my approach is architecturally
> wrong.  Would it make sense for the many-to-many case to use route-to
> instead of rdr, leave the packet unmodified, and expect every machine
> in the server pool to catch all the public IPs?
>
> That might still be tricky.  Using rdr would presumably hit the same
> problem.  Maybe something gross like ifconfig'ing the public pool
> addresses as /32's on lo0, then binding on those, maybe?
>
I honestly don’t know. The pf NAT/RDR/… code is complex, and I
certainly don’t understand all edge cases.
It may be worth experimenting with such options though, because this is
unlikely to be fixed short-term.

Best regards,
Kristof
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "[hidden email]"