A web server behind two gateways?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

A web server behind two gateways?

Grzegorz Junka-2
Hello,

I have a jail running a web server in LAN. There are two routers/WANs
that can connect LAN to the internet. I enabled NAT and port forwarding
to the web server on both routers.

The problem is that the web server responds to requests only from one
router at a time depending on the default gateway set on the jail's
host. If the default gateway is set as router 1 then the web page can be
opened only through WAN1 and vice versa.

Can I configure either router/host/jail so that the web server sends the
response back to the IP that sent the request packet rather than to the
default gateway?

And a bonus question, how can I configure two jails so that each jail
sends packets to a different gateway (which may or may not be the same
as the jails' host's default gateway)?

Thanks
Grzegorz

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Eugene Grosbein-10
On 16.07.2017 19:48, Grzegorz Junka wrote:

> Hello,
>
> I have a jail running a web server in LAN. There are two routers/WANs
> that can connect LAN to the internet. I enabled NAT and port forwarding
> to the web server on both routers.
>
> The problem is that the web server responds to requests only from one
> router at a time depending on the default gateway set on the jail's
> host. If the default gateway is set as router 1 then the web page can be
> opened only through WAN1 and vice versa.
>
> Can I configure either router/host/jail so that the web server sends the
> response back to the IP that sent the request packet rather than to the
> default gateway?

This is the job of external NAT box to route translated replys to right WAN
based on external source IP address produced during translation of the reply.
The jail or internal NAT have nothing to do with the problem.

So, the solution depends of kind of NAT you use.

> And a bonus question, how can I configure two jails so that each jail
> sends packets to a different gateway (which may or may not be the same
> as the jails' host's default gateway)?

Read "man jail" for "vnet" feature.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Alan Somers-2
On Mon, Jul 17, 2017 at 5:33 AM, Eugene Grosbein <[hidden email]> wrote:

> On 16.07.2017 19:48, Grzegorz Junka wrote:
>> Hello,
>>
>> I have a jail running a web server in LAN. There are two routers/WANs
>> that can connect LAN to the internet. I enabled NAT and port forwarding
>> to the web server on both routers.
>>
>> The problem is that the web server responds to requests only from one
>> router at a time depending on the default gateway set on the jail's
>> host. If the default gateway is set as router 1 then the web page can be
>> opened only through WAN1 and vice versa.
>>
>> Can I configure either router/host/jail so that the web server sends the
>> response back to the IP that sent the request packet rather than to the
>> default gateway?
>
> This is the job of external NAT box to route translated replys to right WAN
> based on external source IP address produced during translation of the reply.
> The jail or internal NAT have nothing to do with the problem.
>
> So, the solution depends of kind of NAT you use.

That's not 100% true.  The web server is choosing which gateway to
use.  As Grzegorz said, it's only configured to use a single gateway
at a time.  To do what Grzegorz wants, he'll need to use multiple
fibs.  Set "net.fibs=2" and "net.add_addr_allfibs=0" in
/boot/loader.conf and reboot.  You'll be able to configure a separate
gateway for each fib.  The hard part, though, is configuring your web
server to use multiple fibs.  I don't know if any common web server
has that kind of support builtin.  But your next guess was good.

>
>> And a bonus question, how can I configure two jails so that each jail
>> sends packets to a different gateway (which may or may not be the same
>> as the jails' host's default gateway)?
>
> Read "man jail" for "vnet" feature.

This is definitely the path of least resistance.  Basically, you'll
assign each jail to a separate fib, so you'll still need the
loader.conf settings I mentioned.  Unfortunately, VNET/VIMAGE isn't in
the standard kernel.  If you're unable to run a custom kernel on this
machine, you can still create two jails on separate fibs.  The biggest
downside compared to VNET/VIMAGE is that they'll share a single DNS
resolver.  Here's how to do it:

* Make the loader.conf settings I mentioned earlier.
* Create a separate static IP address for each jail, and associated
each with a separate fib.  Your rc.conf should contain something like
this:
  ifconfig_igb1_alias0="inet 10.1.2.76/20 fib 0"
  ifconfig_igb1_alias1="inet 10.1.18.76/20 fib 1"
* Add the default routes in /etc/rc.local like this:
  /sbin/route add default 10.1.0.1 -fib 0
  /sbin/route add default 10.1.16.1 -fib 1
* Assign one address to one jail and the other address to the other jail
* Ensure that in each jail, the web server starts with the correct
fib.  For example, if you're using apache24, I think you can put
"apache24_fib=1" in /etc/rc.conf.  Other web servers may require
something different, depending on how their RC scripts are written.

Happy hacking!
-Alan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Sami Halabi
In reply to this post by Eugene Grosbein-10
Hi,
simple solution i can think about is:
1. launch 1st jail apache/nginx with db (mysql?) ve sure to use mysql
address accesible vian jail2 (maybe epair), this jail will use default
route, lets say wan1.
2. launch 2nd jail with vnet, default route wan2, mount the same data
directories as jail1, and apache/nginx, since the ip of the db is the
internal ip between jails it'll connect to the 1st db.

this way you have 2 jails that share same data dir but service users vian
different wans behind nat.

Hope the idea helps.

Sami

בתאריך 17 ביולי 2017 02:34 PM,‏ "Eugene Grosbein" <[hidden email]> כתב:

> On 16.07.2017 19:48, Grzegorz Junka wrote:
> > Hello,
> >
> > I have a jail running a web server in LAN. There are two routers/WANs
> > that can connect LAN to the internet. I enabled NAT and port forwarding
> > to the web server on both routers.
> >
> > The problem is that the web server responds to requests only from one
> > router at a time depending on the default gateway set on the jail's
> > host. If the default gateway is set as router 1 then the web page can be
> > opened only through WAN1 and vice versa.
> >
> > Can I configure either router/host/jail so that the web server sends the
> > response back to the IP that sent the request packet rather than to the
> > default gateway?
>
> This is the job of external NAT box to route translated replys to right WAN
> based on external source IP address produced during translation of the
> reply.
> The jail or internal NAT have nothing to do with the problem.
>
> So, the solution depends of kind of NAT you use.
>
> > And a bonus question, how can I configure two jails so that each jail
> > sends packets to a different gateway (which may or may not be the same
> > as the jails' host's default gateway)?
>
> Read "man jail" for "vnet" feature.
>
> _______________________________________________
> [hidden email] mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-jail
> To unsubscribe, send any mail to "[hidden email]"
>
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Eugene Grosbein-10
In reply to this post by Alan Somers-2
17.07.2017 23:46, Alan Somers wrote:

>> So, the solution depends of kind of NAT you use.
>
> That's not 100% true.  The web server is choosing which gateway to
> use.  As Grzegorz said, it's only configured to use a single gateway
> at a time.  To do what Grzegorz wants, he'll need to use multiple
> fibs.  Set "net.fibs=2" and "net.add_addr_allfibs=0" in
> /boot/loader.conf and reboot.

This will work for a server directly connected to both external
gateways but won't work for a server behind two NAT boxes.

Eugene Grosbein

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Kurt Jaeger-14
In reply to this post by Grzegorz Junka-2
Hi!

> I have a jail running a web server in LAN. There are two routers/WANs
> that can connect LAN to the internet. I enabled NAT and port forwarding
> to the web server on both routers.
[...]
> Can I configure either router/host/jail so that the web server sends the
> response back to the IP that sent the request packet rather than to the
> default gateway?

I have a vague idea:

If you set a tag (or a keep-state :flowname) using a ipfw rule that matches
the incoming gateway MAC and match that tag/check-state flowname and
the connection (keep-state) to fwd the answer packet back to that gateway ?

--
[hidden email]            +49 171 3101372                         3 years to go !
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Eugene Grosbein-10
18.07.2017 0:26, Kurt Jaeger wrote:

> I have a vague idea:
>
> If you set a tag (or a keep-state :flowname) using a ipfw rule that matches
> the incoming gateway MAC and match that tag/check-state flowname and
> the connection (keep-state) to fwd the answer packet back to that gateway ?

In fact, the NAT engine already keeps state track of packet flows
and uses that to correctly translate answers back to public IP address.

All you need is to forward translated outgoing answers to correct channel
based on translated external source IP address (read: do policy based forwarding).


_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Alan Somers-2
In reply to this post by Eugene Grosbein-10
On Mon, Jul 17, 2017 at 11:19 AM, Eugene Grosbein <[hidden email]> wrote:

> 17.07.2017 23:46, Alan Somers wrote:
>
>>> So, the solution depends of kind of NAT you use.
>>
>> That's not 100% true.  The web server is choosing which gateway to
>> use.  As Grzegorz said, it's only configured to use a single gateway
>> at a time.  To do what Grzegorz wants, he'll need to use multiple
>> fibs.  Set "net.fibs=2" and "net.add_addr_allfibs=0" in
>> /boot/loader.conf and reboot.
>
> This will work for a server directly connected to both external
> gateways but won't work for a server behind two NAT boxes.
>
> Eugene Grosbein

I think what you meant to say is "this will work for a server directly
connected to two external gateways (whether or not NAT is involved),
but won't work if the server is not on the same subnet as the
gateways".  That's true.  But judging by the OP, I think they're all
on the same subnet.

-Alan
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Eugene Grosbein-10
18.07.2017 0:48, Alan Somers wrote:

> I think what you meant to say is "this will work for a server directly
> connected to two external gateways (whether or not NAT is involved),
> but won't work if the server is not on the same subnet as the
> gateways".  That's true.  But judging by the OP, I think they're all
> on the same subnet.

Yes. Anyway, as long as there is NAT involved, one already has stateful engine
and simpliest and universal solution for this situation is PBR after NAT for outgoing packets.

It works no matter whether gateways are directly connecter or not
and does not require multiple routing tables nor complex FIB or VNET configurations:

# remove "default" NAT rule
ipfw delete 50

# translate incoming traffic and create NAT states
ipfw add 40 nat 123 ip from any to any in recv $iface1
ipfw add 50 nat 124 ip from any to any in recv $iface2

# insert normal filtering here
...
# translate outgoing replies using existing NAT states
ipfw add 50020 nat global ip from $LAN to any out xmit $iface1
ipfw add 50030 nat global ip from $LAN to any out xmit $iface2

# translate new outgoing connections not having a state yet
ipfw add 50040 nat 123 ip from any to any out xmit $iface1
ipfw add 50050 nat 124 ip from any to any out xmit $iface2

# perform Policy Based Routing for packets going to "wrong" route
ipfw add 50140 fwd $gateway2 ip from $extip2 to any out xmit $iface1
ipfw add 50150 fwd $gateway1 ip from $extip1 to any out xmit $iface2

# that's all, folks!

This works no matter where default route points to ($gateway1 or $gateway2).
All you need is working default route and net.inet.ip.fw.one_pass=0.

This can be extended to any number of external channels/interfaces
and optimized with ipfw tables but for two channels I prefer write it so
for readability. I use this for many installations and it just works.

_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A web server behind two gateways?

Grzegorz Junka-2

On 17/07/2017 18:22, Eugene Grosbein wrote:
> 18.07.2017 1:19, Eugene Grosbein пишет:
>> 18.07.2017 0:48, Alan Somers wrote:
>>
>>

Not answering any particular email in this thread, many thanks for your
help. That;s plenty of ideas to try so may take some time!

Just one more question, since VNET was mentioned. Is it production ready
now? I remember there used to be problems with memory leaks. And why
isn't it the kernel, yet? Any plans for that?

Grzegorz J
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "[hidden email]"
Loading...