OpenConfig Public Projects

   2020-05-17      tech      Tech · OpenConfig · YANG

There are a number of public projects that we’ve been working on over the last few years in OpenConfig, and published from Google. It seemed like it might be worth giving a brief “hitchhikers guide” that glues together some of the different projects that we’ve published on GitHub.

Of course, the initial output of OpenConfig, which has motivated much of this ecosystem is the data models — which are publicly available on GitHub.

gNMI - Streaming Telemetry in Production

Whilst working on streaming telemetry, we observed that there were many different implementation decisions going in to the way that telemetry was being sent from a device. Initial implementations had tried to use UDP-based streaming from a network device to a collector, but our initial experiments with this in a production environment showed that there were a number of problems with this approach. When we moved on to implementations that streamed data over gRPC, we initially gave an abstract specification — however, our observation was that this was driving complexity in the stack, and making it harder for us to deliver a consistent telemetry experience across different implementations. To that end, we decided to define a standard gRPC service for interacting with network devices - and called this gNMI, the gRPC Network Management Interface. Since then, we’ve published specifications, extensions, and implemented code to the industry:

• gNMI Specification — this repo contains the core specification for the gNMI protocol, and extensions to that base specification - for example, describing paths with gNMI, and different means of carrying binary encoded values in the protocol.
• Go gNMI Collector and Core Libraries — this codebase stores the actual gNMI protobuf as well implementations of a gNMI collector and client CLI utility - and their constituent libraries. This implementation is used in production in Google for our fleet, and the client libraries are leveraged by a significant number of our Go clients.
• gNMITest — as we implemented the gNMI collector in production, we observed that there were two gaps that we wanted to fill — particularly, we needed to be able to test devices for schema and protocol compliance, and we needed to be able to perform validation of data that was in the collector in production. To this end, we wrote a gNMI test framework which we subsequently open sourced. The framework works by allowing assertions to be made against the data that received from a particular target.
• grpctunnel — since we started out with gNMI, we’ve had a number of questions about how to implement a dial-out model for telemetry. Particularly, this helps with devices that need to make the initial TCP connection to a collector, rather than the collector being able to “dial in” and connect to the target. @gcsl (who is responsible for a significant amount of the code discussed in this section) and I spent some time thinking about this problem - and concluded that we thought the best approach was to have a dial-out tunnel which existing management plane services could be run over. This resulted in this POC implementation of a tunnel that can carry TCP over gRPC.
• Some small miscellaneous helper libraries for streaming — for example, lsdbparse — which is an implementation of a binary IS-IS PDU parser, with serialisation into the OpenConfig-modelled IS-IS LSDB. This has been deployed in production at Google.

On top of these code artifacts, we’ve also spoken publicly about a few of these projects, particularly:

• Carl and I spoke at NANOG73 about our experience removing SNMP from some of Google’s networks, the slides and video are publicly available.
• I lectured at Ecole Polytechnique comparing our approach for ST with SNMP, as part of the series of lectures that I’ve given using the framework of RFC5218 to discuss protocol design. The slides are published on research.google.

gNMI is now widely implemented across the industry. A number of network devices now implement gNMI - for example, Cisco IOS XR and IOS XE, JUNOS, Arista EOS (both as OpenConfig native, and sysdb paths), Nokia SROS, SONiC, Mojo/Arista WiFi, Ciena optical and others. There’s support in closed source (e.g., Arista CloudVision, Kentik’s platform, YumaWorks and open source management solutions,

ygot - Making .yang files actually do something

As I wrote way back in 2015, YANG models themselves are not particularly useful without a toolchain. Prior to joining Google, I’d written and open sourced pyangbind — which provided generated Python code from a YANG model. This code was adopted by a good number of folks across the industry, and continues to be used (although I haven’t quite had the time that I’d like to dedicate to it of late). After joining Google, where our management stack is written mainly in Go, I picked up and refactored some of the work that had been started internally to extend this approach to generate Go code from YANG models, and provide a runtime around them. Subsequently, this was further extended to generate protobufs from YANG models programmatically — which allows other languages to get at least the data structure side of the problem solved. Additionally, this provides a means to have a binary-encoding of the a YANG modelled schema on the wire, giving significant reductions in data volumes.

The toolchain that we have published is broken down into a few subcomponents:

• goyang is a YANG parser that reads YANG models and generates an in-memory representation of the schema tree that they represent. This library is the basis of our code generation tools. Originally written by Paul Borman, it’s now co-maintained by Arista and Google (particularly, Andrew Fort and myself).
• ygot’s go generator - this tool takes an input set of YANG schemas and generates Go from the input. The generated Go uses subcomponents of ygot to validate against the schema, and serialise these datastructures to JSON (RFC7951 and an internal format that is also supported by pyangbind), and gNMI Notifications.
• ygot’s proto generator - alongside the go generator, this tool generates protobufs from a set of YANG schemas.
• ygot’s pathgen package generates a set of nested methods representing a YANG schema. It can be leveraged by projects to create a way to bind methods to a particular path within the schema. Whilst not open source, currently this is used inside Google to generate schema-aware telemetry client libraries in Go.
• ygot also has a number of other tools — for example, it has methods that can be combined to implement a backend for a gNMI implementation (e.g., ytypes’ SetNode and GetNode features), testing utilities for gNMI, and path handling features.

This toolchain has been in use in production in Google for a number of years — we’ve been using it both for configuration generation, as well as handling telemetry. It continues to be actively extended for additional use cases. It’s also in use across a number of open source projects — for example, SONiC and ONOS.

YANG Modelling Helpers

As we’ve developed YANG models, we’ve found that there are a tonne of different things that aren’t very mature in the industry. We’ve tried to put together tools to help with this:

• oc-pyang - This repo stores a number of tools that are helpful for model developers, particularly:
  • an OpenConfig-focused linter — this plugin makes an extensible framework that can be used to validate models according to a specific set of rules. This implementation implements checks for the OpenConfig-style guide rules. We’ve been using this for a few years in various forms to ensure that our models conform before commits.
  • a documentation output plugin — we found that the default jstree that is produced by pyang is helpful, but not that useful for a documentation resource. @aashaikh put together a plugin that generates a pretty set of documentation. This is run automatically as part of our post-commit CI hooks, and publishes the documentation that you see on the ops.openconfig.net site.
  • a path output plugin - this plugin has been useful for outputting the set of paths that are within a model in a form that can be programmatically consumed in a number of different contexts — again, developed by @aashaikh
• We have a complete implementation of a toolchain for running a series of different validation tools and YANG parsers that is deployed on Google Cloud Build, put together by @wenovus, which we’ll definitely open source at some point.

Other OpenConfig Projects

Alongside these projects, there are a number that I haven’t mentioned — and don’t fit into the categories above:

• gNOI and the associated gNxI tooling that provides clients. gNOI is a set of microservices that formalise the API for operational commads between network devices and a management system.
• gRIBI - which is a gRPC service for RIB programming. Vendor devices often have custom APIs for programming the RIB — Cisco’s SLAPI, Juniper pRPD, etc. — gRIBI is a project to be align these different APIs to expose a RIB interface to control-plane systems that run off box.

Closing Thoughts

It’s funny looking back at the almost 5 years since the initial OpenConfig commits — although Anees’ first commit pre-dates this by a year (September 2014) and mine is a few weeks later in October 2014. In this time - we’ve published a lot of different artifacts, and even done a historical podcast. Yikes, how time flies.

I think this work represents a significant change to how we’re interacting between off-box and on-box components of a networking system — but there’s still more to go, it’ll be interesting to see how things continue to evolve.