The CRXN/DN42 interconnection is up!

After about two months there is finally an interconnection between the CRXN network and the dn42 network.

• Start / Idea
• Consensus
• Synchronization
• Routes from CRXN into dn42
• Routes from dn42 into CRXN

Start

The idea came up in the matrix room of the CRXN community. The CRXN network is quite small and therefore has both few routes and little content. The dn42 on the other hand has a lot of content to offer. So it would be nice to be able to surf the dn42 from CRXN. Furthermore, this would also allow one to access content in the CRXN from the dn42. Beyond that, it’s fun :-)

Consensus

A rather annoying, but necessary step. To implement bigger things, you need the consensus of the community. The CRXN community (consisting of about 5 people) agreed, but the dn42 community is much larger. Therefore, the interconnection was proposed on the dn42 mailing list for discussion. Furthermore, there was a meeting on Mumble. After about a month a consensus was reached: the interconnection may take place.

Synchronization

The “problem” with this interconnection is that dn42 and CRXN use different routing protocols. dn42 uses the EGP BGP and CRXN the IGP babel. A detailed description of how the interconnection works can be found on CRXN / dn42 gateway documentation. The summary is that the CRXN routes in dn42 are announced under the AS number 4242423182.

Accordingly, the prefixes are added to the dn42 registry. For each prefix an inet6num object and a route6 object are created. If specified, the max-len is also included in the route6 object. Furthermore - if available - also person objects are taken over. Furthermore there is a community object (COMMUNITY-CRXN) in the dn42 registry. This object contains the different places (Matrix and IRC) where you can get in contact with the CRXN community.

The whole synchronization was implemented by jrb0001. Many thanks for that!

Routes from CRXN to dn42

To act as a gateway, one must be present in both networks. I, for example, meet this requirement. Then you take the CRXN routes and “pretend they came from AS4242423182” and propagate them further to your peers. So when I receive the CRXN routes at one of my gateways, I take them, attach 4242423182 to the (still empty) AS path and propagate them further into dn42. This makes it appear from the outside that all CRXN routes would come from one AS.

In bird you can easily do this with the following paragraph in the export filter:

if ( is_crxn_net() ) then {
    bgp_path.prepend(CRXNAS);
}

It is not possible to distinguish by prefix to which network, which route belongs, because the dn42 and the CRXN network share the ULA space fd00::/8. Here is_crxn_net is a function that checks if a network belongs to the CRXN network and if it does, returns true.

I have extended the function with the following:

if ( is_crxn_net() ) then {
    bgp_path.prepend(CRXNAS);
    if (babel_metric > 200) then {
        bgp_path.prepend(CRXNAS);
    }
}

If the Babel metric is greater than 200 - so I have a rather poor connection to the prefix - I do a Path Prepending. The longer AS path makes me less attractive. If another gateway now receives the prefix with a metric below 200, one should go through the other gateway. By the shorter AS path from the other gateway this is achieved.

Routes from dn42 into CRXN

The obvious way to do this is to take the prefixes from the dn42 - which were transmitted via BGP - and send them into the CRXN - which uses babel. This means you then also propagate the dn42 prefixes and pretend they are your own routes. Using the babel metric, CRXN subscribers can then decide which gateway is better. I also tried this method first, but it brought some problems:

1. enormously, enormously large tables. Large is of course releative, but the dn42 routing table over 700 IPv6 entries is larger than the CRXN routing table with about 20 entries. Accordingly the CRXN routing table “exploded”. babelweb2 - a monitoring and debugging software which is accessible via a web interface - took several minutes to load the table. This is very annoying in the long run. In the CRXN Matrix room the suggestion was mentioned to optimize the software, but there is a lack of knowledge, capacity and the benefit. So the suggestion was discarded.
2. it is enormously difficult to create simple filters for this scenario. Without dn42 one has about 20 prefixes one can filter. With dn42 you have over 700 prefixes. If one now wants to create a filter for this, one needs both the CRXN prefixes and the dn42 prefixes. However, it is desirable to outsource the dn42 filtering to the gateways. This means the CRXN members should filter CRXN routes and not dn42 routes. This means one must distinguish the prefixes of the two networks. This information cannot be communicated by the gateways due to the nature of Babel. However, to solve this, the CRXN members would need an extra list of all dn42 routes. However, this is exactly what needs to be avoided. Furthermore, more complex non-yes-no filters are not possible in all implementations. bird provides extensive methods to create filters. Like in programming you can include other files, use if-else if-else statements and variables. In babeld, however, only yes-no filters are possible. babeld works similar to a firewall in this aspect. babeld reads the rules from top to bottom and executes the corresponding action (allow/deny). Saving intermediate results is not possible. So in order to distinguish CRXN and dn42 routes in babeld, you would have to write all dn42 routes to the configuration file. If you just accept all routes instead, you can’t do any (maxlen) filtering of CRXN routes anymore.
   • The entitydb (the registry of CRXN) is freely available on the Internet. Anyone can view the information there. To get the data from the entitydb, you can clone the repository with Git or download it as a ZIP file. With the dn42 registry it is unfortunately somewhat different. The dn42 registry is also based on Git, so in principle the same methods are available to fetch the information. However, the dn42 registry is behind an account wall. So you have to log in first to access the registry. If you want to clone the git repository, you first have to store an SSH key and then you can access it with that key. In my opinion, the whole measure is pointless since you can get a JSON dump of the registry via the APIs: here or here.

There are several ways to solve this problem:

1. prefix-based filtering: the CRXN network picks a /32 prefix in the ULA range and allocates only from this range. This would result in costly renumbering, which is why this option is rejected by parts of the CRXN community.
2. default route: Instead of the individual dn42 prefixes, it is also possible to advertise a general route - so-called default route - ‘fd00::/8’ in CRXN. If no more specific prefix is known, the request will then be sent to this address. fd00::/8 is only propagated by the gateways. They can then forward the request to dn42 accordingly.
   • This method has two disadvantages: Once the router from a CRXN member cannot decide for itself if there is a path. So an ICMP Unreachable message would have to come from the gateway. Then, if a gateway is not miskonfigured, it could not generate an ICMP message and the request would timeout. Furthermore, it should be the job of routers to have an overview of the entire network and thus be able to decide for themselves whether a particular prefix is reachable or not - this should not be the job of gateways. The gateways are rather meant as a kind of “translator”, which mediates between the different routing protocols.

Assuming the gateways feed the dn42 into CRXN, it is always possible for a CRXN member to find the best path to the next gateway because of the Babel metric, but this does not necessarily mean that this is the best path to the destination. For example, gateway A may be peered with the destination AS in dn42 and gateway B may reach the destination AS only via several other ASs, but the CRXN member chooses gateway B based on the Babel metric. This problem could be solved by calculating the Babel metric based on the AS path length, but I am unsure if this is technically possible, on the other hand this may compromise the loop-free nature of Babel. This problem, but with no potential solution, also occurs with the Default Route.

Trade-off

It is difficult to decide between Full Table and Default Route. Both methods have advantages and disadvantages. In principle, a mixture would also be possible - so Full Table + Default Route. However, for CRXN nodes that choose Default Route and filter other non-CRXN prefixes, filtering may lead to higher CPU performance. Furthermore, such a node would not propagate the non-CRXN prefixes. The same is true for Full Table: a node that takes Full Table but rejects the default route would not propagate the default route.

I think that an optimal interconnection is not possible due to the different nature of EGP BGP and IGP Babel. Therefore one should ask how the interconnection could look like then. Because of the low effort, I am a friend of the solution with the default route - but as described above, not all problems disappear.

Either way, these are CRXN and dn42 projects to learn and have fun with. So have fun with the interconnection!

Addendum 1: Tristan aka deavmi, initiator of CRXN, has also written a blog entry: Announcing the CRXNxDN42 inter-connect!