DNMS - Distributed Network Management System

The Distributed Network Management System (DNMS) is a radically new approach to managing IP addresses that can handle both IPv4 and IPv6 addreses. It replaces DHCPv4, DHCPv6, SLAAC and traditional IPAMS.

IPAMS stands for Internet Protocol Address Management System. Traditional designs were developed for IPv4, which has a very different address management model than IPv6.

With IPv4, addresses are assigned manually, or assigned from a central DHCPv4 server. IPv4 nodes do not autonmously generate addresses (except for APIPA "Link-Local" addresses in the 169.254/16 block, which are not fully realized compared to IPv6 Link-Local addresses). Virtually all IPv4 nodes have DHCPv4 clients, and they work well.

With IPv6, addresses can be assigned manually, or autonomously. All IPv6 nodes will autonomously generate a Link-Local unicast address, and in the presence of a source of Router Advertisement messages (that inlcude at least one Prefix Information option), they will also autonomously generate a regular global unicast address for each advertised prefix. If they have the Temporary Address option enabled, they will also generate a Temporary global unicast address for each prefix. Traditional IPAMS has no way to capture autonomously generated addresses, since it uses DHCP to propagate configuration information to nodes, and DHCP is a one way protocol (it sends information from the server to nodes - it has no way to collect configuration information from nodes).

Some Operating Systems (e.g. Windows, since Vista) have good DHCPv6 clients. Other Operatign Systems (e.g. FreeBSD, Linux) have very poor DHCPv6 client support that is difficult to install and configure. Many operating systems, like Android, have no DHCPv6 support at all. DHCPv6 is very limited compared to DHCPv4. It can assign a unique unicast address (even with address reservation), and it can advertise IPv6 addresses of DNS. It cannot advertise the IPv6 default gateway address.

One of the main functions of traditional IPAMS is keeping track of scarce IPv4 addresses. Public IPv4 addresses are very scarce now, but even private ones are scarce in some situations (e.g. ISP with more than 16M customers). This means when one is released, you must be able to recover and reuse it. A lot of the mechanisms of traditional IPAMS are related to this function. In IPv6 there is no shortage of addresses (even globally routable ones). There is no need to recover and reuse old IPv6 addresses. They are disposable ("use once and throw away"). The only thing you need to worry about with IPv6 is not accidentally assigning an address that is currently in use to another node.

Traditional IPAMS keeps track of entire address blocks with a bit map (with one bit per address). Even in the largest managed IPv4 block (a /8) this only results in a 2 megabyte array. This is entirely manageable with today's PCs.

In IPv6, the smallest managed address block is a /64. If you had a memory array with one bit per address (like with traditional IPAMS), it would take 2,097,152 terabytes of memory for each managed /64 block. My PC only has 32 GB of RAM - it would take 67 million PCs like mine to hold one such bit map. IPv6 clearly requires a very different approach to IPAMS.

If you use SLAAC, you are going to have a lot of IPv6 unicast addresses on each node:

  • Autonomously generated Link-Local address
  • Autonmously generated regular global unicast address (per advertised prefix)
  • Autonomously generated temporary global unicast addres (per advertised prefix - if Temporary Address option is enabled)
  • One or more manually assigned global unicast address
  • Possibly a ULA unicast address

If you are using temporary addresses, those will even change from time to time. The rules for choosing which of those addresses to use as the source address in outgoing packets are quite complex. Realistically, you have little or no control over this. This makes it difficult to use "penalty boxes" to block access to sites by IP address, or a "white list" of acceptable addresses to access a secure site. If you restrict access to some of a bad guy's global addresses, they might have more that you missed. If you restrict access to a site to only some of an administrator's global addresses, his node might choose to use another address to connect, and he will be denied access. If temporary addresses are used, this is even more complex.

There are two ways for an IPv6 node to learn the IPv6 addresses of DNS automatically: stateless DHCPv6 and RFC 6106 (DNS advertisement in SLAAC). You can certainly deploy a DHCPv6 server and configure it, but you also have to either advertise its presence in Router Advertisement messages, or enable the Managed Address option on your node (which requires disabling Router Discovery, which turns off SLAAC).

You don't want to completely disable Router Advertisements in your subnet, since there may be simple IPv6 nodes that can only configure networking using SLAAC. If you provide Router Advertisements, but don't include a Prefix Information option in them, again, simple nodes can't configure global addresses.

Most nodes have an option to disable SLAAC (ignore Router Advertisement messages). If you do that on a node, that node can no longer discover the IPv6 default gateway using Router Discovery. Unfortunately, DHCPv6 cannot advertise the default gateway. So if you disable SLAAC on your managed nodes, you need to manually configure the IPv6 default gateway address on every node.

What is needed is a distirbuted, agent based system. A Node Agent must be deployed on every managed node, and a Subnet Agent must be deployed in each subnet, that can collect the network configuration of all managed nodes in a central location, and provide both stateless (e.g. default gateway and DNS addresses) and stateful (e.g. node address) information to the managed nodes. The Node Agent must be able to get and set all network configuration items. The Subnet Agent must allow easy setting of stateless information for the subnet and creation and management of address allocation pools from which it can assign managed node addresses.

One of the problems in IPv6 address management is that both EUI-64 and Random Interface Identifier generation during SLAAC results in addresses effectively randomly distributed over the entire /64. You certainly can't cluster Interface Identifiers into address ranges based on group membership. This is required in order to have simple firewall or Network Access Control rules. For example, it is useful to have a single firewall rule (based on an address range) that allows incoming VoIP to all members of the group "sales". Without clustering into address ranged by group, you either have to enable VoIP for everyone, or have a separate firewall rule for every member of the sales group.

To control outgoing is even more complex when using SLAAC, since you would need to have a rule for every node address, or the node would just use an address you are not restricting. Again, the alternative is to block all outgoing connections of a given protocol.

The key to effective IPv6 address management is to disable SLAAC on the managed nodes. You then need to supply all network configuration to those nodes from a central source: default gateway, node address, IPv6 addresses of DNS, etc. As long as you are doing that, you could also manage IPv4 by disabling DHCPv4 on the node, and supplying the same information for IPv4 over the same mechanism.

A DHCPv4 client finds the DHCPv4 server or relay agent through broadcast, and that server sends the configuation information back to the node via broadcast. That solves the "chicken and the egg" problem of how the node communicates with the address management system before it gets an address, for IPv4.

A DHCPv6 client finds the DHCPv6 server or relay agent through link-local multicast (ff02::1:2), using its automously generated Link-Local address as source. The DHCPv6 server or relay agent can reply to the node's Link-Local address. This solves the "chicken and the egg" problem of how the node can communciate with the address management system before it gets and address, for IPv6.

Actually, using the autonomously generated IPv6 Link-Local address is key to DNMS working.  I use it for a node to connect to the Subnet Agent using a link-local multicast address. Of course, multicast only works over UDP, and TCP provides better reliability, so the first step is for the Subnet Agent to accept a request for Agent Discovery over UDP multicast and reply with all the addresses and ports it is listening on. The Node Agent can then pick one of those (usually the Link-Local address) and switch to TCP.

The Node Agent can obtain all network configuration and register it with the Subnet Agent, which has a database to store it. The Subnet Agent also keeps the subnet stateless information, and allocation pools in the database, and can supply configuration to the Node Agent upon request (and update the database with the new information). The Node Agent can also disable SLAAC, remove any existing IP addresses (except for the autonomous Link-Local address), configure the managed IP addresses, default gateways and IP addresses of DNS.

- More details and screen shots to follow -