Choosing how to route inside the DMVPN is such a massive topic with so many considerations it'd be impossible to pack into a single post. This post is a start to my lab studies of DMVPN and I will use the same model to continue those studies into other protocols and technologies.
It is important to keep in mind that DMVPN is a virtual circuit facilitation. There aren't a lot of routing limitations that directly relate to DMVPN, but rather are an effect of how routing functions in general. An example of this would be EIGRP split horizon on a hub. It is a routing loop function to EIGRP, telling it to not advertise a network out the same interface the original advertisement was received. Disabling split horizon fixes this issue with distance vector protocols.
DMVPN, or more precisely NHRP, is relatively simple in terms of configuration and there aren't really a lot of deviations you can do. The design considerations will be related to choosing your routing protocol and knowing and understanding how that protocol behaves. There are also a myriad of considerations in terms of hardware, especially the platform may dictate how you have to design certain things. An old example would be the C6500/7600 platform and Cisco stating using VRF and tunnel keys will cause the routes to be process switched. The [documentation](http://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/WAN_and_MAN/ngwane/ngwanedmvpn.pdf](http://) is of older IOS version, but these are the kind of platform limitations you may find.
OSPF, EIGRP and more
You can use both and it will definitely work, but OSPF is more restricting and you may find a scalability issue sooner than with EIGRP. Once again this doesn't relate directly to DMVPN. With OSPF you have to connect all nodes through area 0. This means area 0 can become quite big and you can't filter route information the same way you'd be able to with EIGRP. With EIGRP you can use route summarization or apply a distribute list on a router and have it ignore networks. You can't do this within the same area in OSPF due to the way LSA works.
You can design the OSPF with every node having their own networks in another area and you most likely want to do it this way. Now you can do ABR summarization, but we are still forced to follow the hierarchy design of OSPF, which makes things less flexible and we need a greater foresight of future growth.
EIGRP doesn't have the hierarchy design and you can summarize anywhere in the network you want to. It makes route filtering very easy.
These two protocols are not our only choices. BGP, ODR and IS-IS can be used as well. VRF and MPLS is supported too.
I decided to explore OSPF as the first overlay protocol. The goal is to have a failover capable DMVPN network. I've made a topology that I cut to match the lab. From a testing perspective there is an unnecessary amount of routers, but it is slightly easier for me to keep the same topology and just cut a little. I actually managed to kill my laptop with GNS3. Turns out 8GB memory is not a lot with Chrome and GNS3 open.
Logical overlay topology
ISP underlay topology. Nothing special has been made of the BGP between the providers. It's not the focus for now. They simply connect and provide routing information for the 184.108.40.206 networks.
Configurations: I decided the easiest way to keep track and share the configurations would be using GIST. The only thing I miss is a way to organize the configurations, but they are all available here. https://gist.github.com/KasperJak
IOS used c7200-adventerprisek9-mz.152-4.M6.
It's time to deploy some configuration and see the effect of the different choices. I aim to be as thorough as possible when comparing differences of the configurations. I am not going through the exact configuration that has been applied, only when it makes sense for comparison. Github will contain the configurations used for every lab.
Pre DMVPN configuration. What does the spoke know in the routing table:
All thats needed to connect, is a default route to our ISP. A ping to the NBMA address of our hub:
I can create the DMVPN now.
Checking the spoke-to-spoke function.
At this point I have a functioning DMVPN network and from here on, the rest of the configuration is based on OSPF design. Let's start with the consideration of how to establish OSPF peers. The peers are going to be established through a single interface and OSPF uses point-to-point link type by default. If I don't change the link type, there'll be a hub trying to establish a single peer relationship while it has multiple peer options. It'll look like this:
Using broadcast network type we get the lovely DR/BDR feature. I'm not interested in this, though. It doesn't make sense to have a spoke router becoming BDR or even DR. Disable the spoke router participation in the election by using ip ospf priority 0 on the tunnel interface. How does the neighbors/RIB look?
Well that looks pretty alright. I had to do several different configurations to test the behavior of NHRP in OSPF. The RIB and CEF information is nearly "too perfect". Looking at the 172.16.x.x networks (the local spoke networks), the correct routing information is already in the RIB. Looking at the CEF table:
Sending traffic to the 172.16.2.1 network to get the correct CEF entry is not necessary when using broadcast network type. There'll still have to be sent an initial request to create the tunnel, though.
Of course I tried doing this with point-to-multipoint network type. A look at the routing table after neighbor establishment, but before having done any extra NHRP resolutions.
Resolving 172.16.2.1. CEF entry before trying to establish connection is sending traffic to the hub.
And the routing table now shows our override % at the 172.16.2.1 entry. Look at the "via 10.172.0.1", which is the discrepancy between the CEF and RIB, common to DMVPN.
The last thing I want to show about P-to-MP is the special phase 3 NHRP mapping. This is the "end network to NBMA" mapping instead of "Next-Hop to NBMA". You won't see this using broadcast type.
Which network type should I use? Seemingly, the only downside to broadcast is the need to control DR/BDR selection. Oh, and this bug from Cisco. I can't test if this has any real effect, but the NHRP, CEF, RIB and what else are all correct to the router and traffic routes properly.
With P2MP I have to deal with RIB/CEF information changing, but no Resolution bug with unknown effect. There are also more routes in the table pointing towards the other tunnel interfaces in the area. Both broadcast and P2MP can also use be done with "non-broadcast". You statically map the neighbors on the hub. The spokes only need to have the correct link type on the tunnel interface and they'll get discovered automatically by the hub. It adds control to your area.
Areas, summarization and default route
I have my foundation for DMVPN and OSPF. How scalable is this configuration? I can summarize on every node, but only from their local OSPF area into area 0. This is just fine with a lesser amount of DMVPN nodes and if you have a lot of networks on those nodes, then it might even be great. But every node has to know about every other node's advertisement, whether these advertisements are summarized or not. With this area 0 design you can't do something like sending a full summarization of the entire network from the hub to all spokes. You can do that with EIGRP and let NHRP map the more precise prefixes needed, which I'll explore in another post. If you have 500 routers you'll have at least 500 routes in every RIB on every router, assuming every node has just one subnet in OSPF and that's a scaling issue.
Using a default route is possible. I made a mistake by applying a default route pointing to the internet for DMVPN purposes. I can fix this by replacing the default route with one pointing to 220.127.116.11 255.255.255.0 100.0.0.x and my DMVPN will still be able to form across the service providers. Better not make this mistake and realize it after deploying 100 routers.
Or semi non-conclusion. I underestimated how much time it took to write these posts and I have been pretty busy with other stuff as well. I had to severely cut the scope of my initial post here. I will continue the DMVPN studies and create more posts, but I wanted to get something out the door now. It'll also be easier for me to create my next post using the base of this.