Overview

RoutedBits is operated by a small group of professional network engineers. We run the infrastructure primarily for experimentation and design research, using DN42 as a hobby platform to evaluate real-world networking architectures and automation practices.

Our infrastructure consists of routers deployed across cloud provider networks. Rather than using a full-mesh design, we interconnect routers in a manner that resembles a service-provider backbone. Each router maintains multiple redundant paths to other routers, selected using hop count and observed latency.

The goal of RoutedBits is to simulate the constraints and design trade-offs that real service providers routinely face. We explore emerging technologies such as Segment Routing over IPv6 (SRv6) and pursue automation that spans the full lifecycle—from initial provisioning to self-service peering and ongoing configuration management.

Network design

RoutedBits is designed to evaluate service-provider-style routing and control-plane decisions under realistic constraints, with a primary focus on operational scaling. We want to observe how policy, routing updates, and configuration behaviors hold up as the network and its peering set grow.

To achieve this, we use an SRv6 IS-IS underlay with a weighted, latency-based metric. This models transport cost explicitly, helping us study how path selection changes when performance and operational preferences are both part of the design space (traffic engineering), and how those choices behave under failure or suboptimal conditions (resiliency).

On top of the underlay, we use multi-protocol BGP across a geographically diverse route reflector cluster. This provides a practical BGP control plane for examining scaling characteristics such as policy distribution, update dynamics, and reflection behavior at distributed scale.

Finally, DN42 is run as an overlay service on top of an SRv6 L3VPN control plane, with signaling carried via BGP. This VRF separation provides a clean boundary between DN42 and the operating system's default table.

Multiple services are hosted from each node. Containerized workloads provide metrics collection and anycasted DN42 services such as our authoritative DNS and RPKI RTR servers, enabling end-to-end validation of performance, routing behavior and service reachability.

Platform

Our routers run a customized version of VyOS. Over time, we've contributed bug fixes, features, and targeted customizations upstream. This keeps the platform maintainable for our needs while ensuring the community can benefit from our improvements.

We chose VyOS over a hand-rolled approach for practical reasons. It is an off-the-shelf platform with a familiar CLI and operator workflow, similar in spirit to Juniper's Junos-style operational model. By building on a maintained baseline, we avoid the overhead of sustaining an entire self-built network OS ecosystem and can focus engineering effort where it matters most.

We also value VyOS as a realistic automation target. Automating third-party platforms—handling versioning, operational quirks, and consistent interfaces—is a common constraint in real service-provider environments, and we design our automation practices to work with those realities.

Automation

All router management and configuration is automated to reduce manual effort and keep operational changes repeatable. When a peering is added or updated, an automated job triggers a workflow that renders the appropriate configuration elements and applies the update to a selected router.

Nightly, a reconciliation workflow ensures ongoing correctness. It compares the live configuration against an expected configuration generated from our source of truth, which is a combination of Git and a custom web service API. Discrepancies are identified and automatically remediated by reconciling the live state to the expected, known-good configuration.

This combination of automated change delivery and continuous verification is central to our design and allows for ongoing experimentation.