Skip to main content

Backup ISP with some traffic selection

(2013-08-26)

Problem:

My scenario is that I have a FortiGate 60D with two ISPs: a static DSL, and a dynamic high speed cable. So I have VIP objects defined on my DSL line going to a couple of systems, including a mail system.

I want:

  • The mail system to send traffic out through the DSL
  • All other outbound traffic to go out through the cable, except if the cable isn’t up, in which case I want it sent out the DSL (so a failover)
  • The VIP objects on the DSL line to be usable.

Solution:

So I have set things up as:

  • Change the cable interface so that the default route has the same distance as the static DSL default route
  • Changed the priority on the DSL default gateway to be non-zero (since there doesn’t appear to be a way to set the prior ity on a DHCP-provided route, only the distance)
  • Set a policy-route to force outbound traffic from the mail server to the DSL default-route
  • Added dead-gateway-detection to both the cable and DSL default-routes
  • Ignored ECMP because my priorities are different

It seems to be working:

  • I can contact VIPs on the DSL link that go to either the mail server or the non-mail server
  • Whatismyip shows the correct outbound IP when tested from the mail server and from a non-mail server
  • Mail appears to be flowing properly

Discussion

The trick seems to be to mess with the static route priority, not distance. If you mess with the route's distance, the route with the further distance does not appear in the routing table. This would be fine if we wanted the DSL only for redundancy purposes.

Also, this means that if you have VIP objects on the interface with the further distance, they will not be reachable from the internet while the lower-distance route is available to the firewall:

When two different distances are used on the wan1 and wan2 default routes, traffic originating from the Internet can only be responded to by the interface with the default route with the lowest distance metric.

If you mess with the priority, but the distances are the same, both routes appear in the routing table and are usable, but unless specified by policy only the first entry in the table will get used. In my case, the default had the DSL route first in the routing table and was preferred by implication. (This might possibly be because the DSL is defined as WAN1 and the cable is defined as WAN2. Possibly swapping them would change the implied order.)

Things I'm not sure about with this set-up:

  • If my cable DHCP gets changed such that the gateway changes, the dead-gateway-detection is going to detect the link as down. While unlikely on this particular service, it isn't impossible. This means the change will require some manual hand-holding. Is there a better way?
  • If the DSL dies, will the policy route effectively block the mail server from communicating to the outside world? Will it restrict outbound mail traffic to a connection, even if it is dead?