GRE over IPsec

脚本：建立一个IPsec tunnel之后运行这个shell，可以建立GRE tunnels over IPsec
-----------------------------------------------------------------------------------------------
#!/bin/sh

# left is "18", right is "20"

#

# diff DEVnames are not needed, but make things more clear

DEV_LEFT=tun18

DEV_RIGHT=tun20

LEFT_IP=192.168.2.18

LEFT_NET=10.1.18.1/32

RIGHT_IP=192.168.2.20

RIGHT_NET=10.1.20.1/32

 

setup_left() {

        DEV=$DEV_LEFT

        local_ip=$LEFT_IP

        local_net=$LEFT_NET

        remote_ip=$RIGHT_IP

        remote_net=$RIGHT_NET

}

setup_right() {

        DEV=$DEV_RIGHT

        local_ip=$RIGHT_IP

        local_net=$RIGHT_NET

        remote_ip=$LEFT_IP

        remote_net=$LEFT_NET

}

 

case "`/sbin/ip -4 -o addr show dev eth0`" in

*192.168.2.18*) setup_left

                ;;

*192.168.2.20*) setup_right

                ;;

*)     

        echo "ERROR";;

esac

 

case "$1" in

start) 

        modprobe ip_gre

        (set -x

        ip tunnel add $DEV mode gre remote $remote_ip local  $local_ip ttl 255

        ip link set $DEV  up multicast on  ### DOESNT SEEMS to work for zebra :(

        ip addr add $local_net peer $remote_net dev $DEV

        #ip route add $remote_net dev $DEV # Needed if you run BGP instead of OSPF

        )

        ;;

stop)  

        (set -x

        ip tunnel del  $DEV

        )

        modprobe -r ip_gre

        ;;

esac

 

-----------------------------------------------------------------------------------------
Well, start with your normal PSK ipsec.conf file, with only a single
tunnel defined (host to host) between your two gateways.  Or use X.509, or
rsasig - it doesn't matter.  Just get a tunnel between your two sites up,
host to host.

Get your SA established, and the eroute happy.  In other words, *make sure
FreeS/WAN is working* before going any further.  Otherwise, it's a pain to
debug.

Next up, it's GRE time:

$remote_ip = remote side of tunnel (what ipsec# IP is on remote GW is)
$local_ip = local IP address (what ipsec# IP is on is)

ip tunnel add site1tosite2 mode gre remote $remote_ip local  $local_ip ttl 255
ip link set site1tosite2  up
ip addr add 192.168.0.1 dev site1tosite2
ip route add 192.168.0.2/32 dev site1tosite2

Remember to reverse this on the other side... eg:

ip tunnel add site1tosite2 mode gre remote $local_ip local  $remote_ip ttl  255
ip link set site1tosite2  up
ip addr add 192.168.0.2 dev site1tosite2
ip route add 192.168.0.1/32 dev site1tosite2

Note: I'm using only 2 IP addresses... so it's a point to point style
link.  Handy if you have lots of these and don't wanna waste /24's each
time.

Make sure it works - ping the other end of the GRE tunnel.  Check iptables
rules so traffic won't be dropped.  You've probably done this already if
you're adding this to a working FreeS/WAN setup :)

Then, route whatever you want over the tunnel - eg:

ip route add 172.16.0.0/16 via 192.168.0.2 dev site1tosite2

Or, do something completely insane (like me) and run zebra/bgpd on both
ends, and let them exchange routes dynamically.  Never issue an "ip route
[add|delete]" command again!

And that's about it.  The exercise of putting all of this into custom
_updown scripts is left to the reader :)
-------------------------------------------------------------------------------------------
I must be missing something.

Why would you want to establish an IPSec tunnel from site to site and then
run a GRE tunnel?

TIA.
--------------------------------------------------------------------------------------------
Several reasons.  The ones that come to mind are:

1) Simplified config.  If you have gateway A and gateway B, and behind A
there are N networks and behind B there are M networks, then you will need
to define N*M connections with straight ipsec.  If you use a gre tunnel,
then you will only need to define one connection, build one tunnel, and then
add M routes on A and N routes on B.  With reasonably large networks, a
linear increase in complexity with network size is *significantly* easier to
manage than an exponential increase in complexity with network size.  Once N
and M get to be above 10 or so, it can start to get painful.

2) Non-IP protocols.  Say, for instance, you've got a nice IPX network at
two sites with internet connections.  You *could* buy a leased line between
the two for all IPX traffic, but that would be expensive.  You *could*
tunnel it all in the clear, but that would be insecure.  You *could* write
your own IPX-over-IP-with-spiffy-encryption protocol, but that would be
difficult.  Or, you could use an ipsec encrypted gre tunnel, and everything
would (theoretically) work.

3) fault-tolerant routing.  There's very little built in to the ipsec
protocols to either detect when a remote peer is no longer responding, or to
find a better way to get to the networks that were previously reachable
through that dead peer.  gre tunnels, when combined with standard routing
protocols, make it easier to do this.

On that note, I have done the above (ipsec+gre+ospf/eigrp) extensively on
cisco boxes.  However, as much as I have been saying that it's a cool thing
to do, I have just today finished setting up my first combination of
freeswan/ipsec+gre+zebra/ospf.  I know that Ken has done the same thing with
zebra/bgp, and there have been some questions about how this sort of thing
is done, so I wanted to share some of my experiences/gotchas.

First, my biggest moment of "d'oh!", make sure your firewall rules allow
ospf traffic on your gre interfaces.  :)
Second, zebra looks a lot like cisco, but acts somewhat differently.  There
are three things that need to be done in ospfd.conf to make this work:
1) for each tunnel interface, do:
 interface {tunnel-name}
  ip ospf network non-broadcast
because zebra doesn't seem to like listening for multicast messages on
virtual interfaces (which sucks).  A tunnel interface *should* be a
point-to-point.  but it doesn't seem to work that way.
2) because of 1, you need to define neighbors with statements like:
 router ospf
  neighbor {tunnel ip address of remote peer}
3) for some incredibly wierd reason, normal network statements like one
would use for other interfaces don't work.  network statements for gre (or
actually any point-to-point) interfaces have to be the ip address of the
local tunnel interface and have to be /32, like:
 router ospf
  network 192.168.1.1/32 area 0
even if the tunnel is 192.168.1.1/30 and you would expect a statement like
"network 192.168.1.0/30 area 0" or even "network 192.168.0.0/16 area 0" to
work, it won't work, zebra will not activate ospf on the interface, and
nothing will happen.  Which is *really* frustrating.

-Joe
------------------------------------------------------------------------------
On Tue, 15 Oct 2002, Joe Patterson wrote:

> Several reasons.  The ones that come to mind are:
>
> 1) Simplified config.  If you have gateway A and gateway B, and behind A
> there are N networks and behind B there are M networks, then you will need
> to define N*M connections with straight ipsec.  If you use a gre tunnel,
> then you will only need to define one connection, build one tunnel, and then
> add M routes on A and N routes on B.  With reasonably large networks, a
> linear increase in complexity with network size is *significantly* easier to
> manage than an exponential increase in complexity with network size.  Once N
> and M get to be above 10 or so, it can start to get painful.

That was my primary reason for doing this.  I run what I term an
Enterprise Class VPN between two datacenters.  I have 2 GW's @ each side,
running Heartbeat for IP Address takeover, ospfd for internal routing, and
now BGP between 4 ISPs and between my own sites.  My config is nessecarily
complex, so I welcome simplicity.  I also have several business partner
VPN's, run off various devices.  I static-route thier remote networks on
my GW's, and ospf/bgp redistribute them to my other sides, so all traffic
to the business partner goes through 1 link.  Much simpler than giving a
partner connections to all of your sites, and much simpler to secure.

> 3) fault-tolerant routing.  There's very little built in to the ipsec
> protocols to either detect when a remote peer is no longer responding, or to
> find a better way to get to the networks that were previously reachable
> through that dead peer.  gre tunnels, when combined with standard routing
> protocols, make it easier to do this.

Yup.  My Heartbeat+FreeS/WAN combo needs dynamic routing to work
effectivly.

> On that note, I have done the above (ipsec+gre+ospf/eigrp) extensively on
> cisco boxes.  However, as much as I have been saying that it's a cool thing
> to do, I have just today finished setting up my first combination of
> freeswan/ipsec+gre+zebra/ospf.  I know that Ken has done the same thing with
> zebra/bgp, and there have been some questions about how this sort of thing
> is done, so I wanted to share some of my experiences/gotchas.
>
> First, my biggest moment of "d'oh!", make sure your firewall rules allow
> ospf traffic on your gre interfaces.  :)
> Second, zebra looks a lot like cisco, but acts somewhat differently.  There
> are three things that need to be done in ospfd.conf to make this work:
> 1) for each tunnel interface, do:
>  interface {tunnel-name}
>   ip ospf network non-broadcast
> because zebra doesn't seem to like listening for multicast messages on
> virtual interfaces (which sucks).  A tunnel interface *should* be a
> point-to-point.  but it doesn't seem to work that way.
> 2) because of 1, you need to define neighbors with statements like:
>  router ospf
>   neighbor {tunnel ip address of remote peer}
> 3) for some incredibly wierd reason, normal network statements like one
> would use for other interfaces don't work.  network statements for gre (or
> actually any point-to-point) interfaces have to be the ip address of the
> local tunnel interface and have to be /32, like:
>  router ospf
>   network 192.168.1.1/32 area 0
> even if the tunnel is 192.168.1.1/30 and you would expect a statement like
> "network 192.168.1.0/30 area 0" or even "network 192.168.0.0/16 area 0" to
> work, it won't work, zebra will not activate ospf on the interface, and
> nothing will happen.  Which is *really* frustrating.
>
> -Joe
>

Thanks for doing this and sharing the knowledge.  Pat Felt (see messages
from the past few days) has also attempted this, and got it working today. 
His experience matches yours 100%, down the network 192.168.1.1/32 area 0
config lines.  I will be including both your's and his information in my
FreeS/WAN + Dynamic Routing doc (forthcoming)
------------------------------------------------------------------------------------
> I must be missing something.
>
> Why would you want to establish an IPSec tunnel from site to site and then
> run a GRE tunnel?
>
> TIA.
>

Many reasons.  Quickly:

* GRE supports Broadcast + Multicast.  IPSec does not.
* Use dynamic routing protocols to add/remove routes over the ipsec
interface
* Tunnel over multiple IPSec "hops" in a fully encrypted mutli-hop network
* Because you can :)
-------------------------------------------------------------------------------------