In my situation, there are three different sites in the same city
(Amsterdam), interconnected using a shared L2 vlan, and six routers
(A1, A2, B1, B2, C1, C2) on that vlan, two per site for redundancy
reasons. Each router runs ospf.
The vlan is provided to us by a telco that we do not necessarily trust.
So ideally, we'd like all traffic that goes over the vlan (modulo ARPs
and STP and stuff) to be encrypted. ("-o eth3 -j ENCRYPT")
The problem I kept running into with tunnel mode is that tunnel
mode SPD rules appear to dictate routing policy in a way that's not
compatible with dynamic routing.
I.e., a line like:
spdadd 10.10.1.0/24 10.0.1.0/24 any -P out ipsec
esp/tunnel/1.2.3.4-5.6.7.8/require;
effectively says "All traffic from 10.10.1.0/24 to 10.0.1.0/24 will
be sent over a tunnel with local endpoint 1.2.3.4 and remote endpoint
5.6.7.8", but:
- I have no idea beforehand what address ranges are going to be routed
over this vlan. (Customers might send traffic with source addresses
in address ranges that they don't announce to us (asymmetric routing),
and I don't want those packets to remain unencrypted just because they
don't match the SPD rule.) A 0.0.0.0/0 0.0.0.0/0 rule would not be
appropriate either since that'd suck _all_ traffic into this tunnel
- I have no idea beforehand what the remote nexthop is going to be. A1
might ordinarily send its traffic for site B to B1, but if B1 fails
it'll want to start using B2 instead, which would be prevented by the
SPD rule hardcoding the remote tunnel endpoint to B1.
The workaround we tried at first was to create GRE tunnels between each
pair of routers on the vlan, and to run ospf over the tunnels instead of
directly over the vlan interface. That gave MTU problems, though, which
made us just forget about ipsec altogether and use vtund instead, hardly
better than nothing. (Now that Herbert has submitted a number of fixes
for ipsec MTU issues with tunnels, I guess I should go and give the
GRE-over-ipsec setup a go again.)