Adding IPv6 to teensy-ntp

I ported my NTP server from the stm32f407 to the Teensy 4.1. Because I recently switched from a cable modem to fiber, I wanted to use my home stratum 1 NTP servers with my public servers in the NTP pool. To make this easier, I wanted to enable IPv6 on my home stratum 1 NTP servers.

My teensy-ntp server uses the LWIP library, which already has IPv6 support built-in. However, I did need to make some changes to get it working for my specific setup.

Outline of Changes

  • Basics
  • MLD & multicast filter
  • Timestamp v6 packets
  • IP address data type
  • Address autoconf
  • Update NTP interleaved mode for v6

Basics

The first thing that needs to happen is LWIP_IPV6 + LWIP_IPV6_MLD need to be enabled in lwipops.h:

// Enable IPv6
#define LWIP_IPV6	1
// Needed to manage multicast groups for v6
#define LWIP_IPV6_MLD   LWIP_IPV6

Then in the interface init code, the generic "ethip6_output" function is used (which depends on netif->linkoutput):

#if LWIP_IPV6
    netif->output_ip6 = ethip6_output;
#endif

MLD & multicast filter

In IPv4 neighbor discovery (ARP) is done via broadcast, but in IPv6 neighbor discovery is done via multicast. In order to receive multicast packets, the NIC has to be told which multicast addresses to accept. In this hardware, the NIC uses a 64 bit hash filter to determine which addresses to forward on to the software. "MLD" stands for Multicast Listener Discovery and is how LWIP communicates which multicast addresses are needed to switches. My home network doesn't use MLD snooping, so I'm only using MLD to update the NIC's filter.

I created a function to manage the filter. Right now I'm just adding bits and not removing them. This is fine for my application because it will only join two multicast addresses and they won't change. But if they did, I would probably implement a reference counter per hash bit to determine when to remove it from hardware.

#if LWIP_IPV6_MLD
// multicast filter is 64 bit hash based on the crc32 of the mac address
err_t t41_mld_mac_filter(struct netif *netif, const ip6_addr_t *group, enum netif_mac_filter_action action)
{
    // TODO: remove as well as add
    if (action != NETIF_ADD_MAC_FILTER)
      return ERR_OK;

    // multicast mac address is 33:33:[lowest 32 bits of v6 address]
    uint8_t address[6] = { 0x33, 0x33 };
    memcpy(address + 2, group->addr + 3, 4);

    uint32_t crc = crc32(address, sizeof(address));

    // take the top 6 bits to determine which of the 64 bits to set
    uint32_t hash = (crc >> 26) & 0x3f;

    // if the upper bit is set, use GAUR otherwise GALR
    if (hash & 0x20)
        ENET_GAUR |= 1 << (hash & 0x1f);
    else
        ENET_GALR |= 1 << (hash & 0x1f);

    return ERR_OK;
}
#endif

In the netif_init function, I enable MLD on the network interface:

#if LWIP_IPV6_MLD
    netif->flags |= NETIF_FLAG_MLD6;
    netif->mld_mac_filter = t41_mld_mac_filter;
#endif

And then I join the all nodes multicast group after the low_level_init in order to receive the Router Advertisement periodic messages.

#if LWIP_IPV6_MLD
    // we need to listen to the all nodes multicast address for route advertisements
    ip6_addr_t ip6_allnodes_ll;
    ip6_addr_set_allnodes_linklocal(&ip6_allnodes_ll);
    netif->mld_mac_filter(netif, &ip6_allnodes_ll, NETIF_ADD_MAC_FILTER);
#endif

Timestamp v6 packets

In the low level ethernet output function, I was limiting transmit timestamping to only IPv4 packets so that ARP packets wouldn't be included. I needed to extend that to also timestamp IPv6 (but not ICMPv6) packets:

if(txTimestampEnabled && (ethhdr->type == PP_HTONS(ETHTYPE_IP) || ethhdr->type == PP_HTONS(ETHTYPE_IPV6))) {
  bdPtr->extend1 |= kEnetTxBdTimeStamp;
  txTimestampEnabled = 0;
}

Address autoconf

I'm using IPv6 stateless configuration (RFC4862), which works in most every IPv6 network. In the link state callback, I check to see if link is up. If it is, then I set a link local address on the network interface and tell LWIP to start autoconfig:

  if (netif_is_link_up(netif)) {
    netif_set_up(netif);
    dhcp_start(netif);
#if LWIP_IPV6
    netif_create_ip6_linklocal_address(netif, 1);
    netif_set_ip6_autoconfig_enabled(netif, 1);
#endif
  }

IP address data type

When receiving NTP packets, I needed to handle both the v4 and v6 cases. LWIP extends the ip_addr_t type to include the field "type".

 if (addr->type == IPADDR_TYPE_V4) {
    // handle v4 addresses here, use ip_2_ip4(addr) to access the IP
  } else {
    // handle v6 addresses here, use ip_2_ip6(addr) to access the IP
  }

Some useful functions:

  • ip4addr_ntoa_r - ipv4 address to string
  • ip6addr_ntoa_r - ipv6 address to string
  • ip6_addr_cmp/ip4_addr_cmp - compare two addresses
  • ip6_addr_set/ip4_addr_set - set address

Update NTP interleaved mode for v6

NTP interleaved mode needs to keep a list of NTP client IPs and their transmit timestamps. In IPv4+IPv6 mode, I'm storing the IPv4 mapped address. This makes it possible to write generic code using defines that automatically switches between using ip6_addr_t/ip6_addr_cmp/ip6_addr_set and ip4_addr_t/ip4_addr_cmp/ip4_addr_set depending on if LWIP has v6 enabled or not.

End result

First comparison is flash+ram usage:

IPv4 only:

Memory Usage on Teensy 4.1:
  FLASH: code:82364, data:17116, headers:9060   free for files:8017924
   RAM1: variables:86368, code:79688, padding:18616   free for local variables:339616
   RAM2: variables:12384  free for malloc/new:511904


IPv4+IPv6:

Memory Usage on Teensy 4.1:
  FLASH: code:99836, data:17116, headers:8996   free for files:8000516
   RAM1: variables:90176, code:97160, padding:1144   free for local variables:335808
   RAM2: variables:12384  free for malloc/new:511904

About +17KB code and +4KB memory, but there's plenty of room to grow into.

Second comparison is running v4 code side-by-side with the v4+v6 code:

$ chronyc sources ; chronyc sourcestats
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* teensy-2.lan                  1   4   377    27  -2858ns[-2820ns] +/-   39us
^+ teensy-1.lan                  1   4   377    18    +15ns[  +15ns] +/-   38us
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
teensy-2.lan               11   7   161     -0.001      0.006   -521ns   202ns
teensy-1.lan               21  13   243     +0.000      0.002   +519ns   156ns
chrony as NTP client against teensy-1.lan (v4+v6) and teensy-2.lan (v4)

Lastly, the goal was to use these as another source of time for my NTP pool servers:

server1$ chronyc sources ; chronyc sourcestats
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^- 2600:1700:x                   1   6   377   186   -113us[ -113us] +/- 5317us
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
2600:1700:x                51  23   59m     -0.015      0.186    -34us   433us

server2$ $ chronyc sources ; chronyc sourcestats
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^- 2600:1700:x                   1   6   377   129   -344ns[ -344ns] +/- 5048us
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
2600:1700:x                25  17   31m     +0.106      0.189   +283us   139us

server3$ chronyc sources ; chronyc sourcestats
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^- 2600:1700:x                   1   6   377   126   -468us[ -423us] +/- 5574us
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
2600:1700:x                64  29   71m     -0.001      0.054   +503us   141us

I am satisfied with these results.

See also: