Working with YANG Models - A brief intro to 'pyangbind'

   2015-04-24      imported      Code · IETF · SDN · python

Both the Hawaii IETF meeting (IETF91) and the subsequent meeting we had a few weeks ago in Dallas were somewhat YANG-heavy. Following work to move towards YANG as a standard modelling language for network configuration, and the subsequent IESG statement effectively deprecating SNMP as the way that we present network configuration - the IETF, and especially the routing area, has dived head-first into YANG.

Indeed, I’ve been occupied somewhat with some really great collaborative work with a number of awesome engineers from Google, Microsoft, AT&T, Level3, Yahoo!, Facebook, Cox, Verizon and others on the OpenConfig initiative. We’re trying to take an operator and use-case driven approach to developing YANG modules for both configuration and defining the schema for telemetry. This work has turned up a few times in the press, and I should probably write something separate about it in the near future.

However, one observation that a number of people have made, is that there’s really limited tooling available to work with YANG modules. We have (the rather excellent) pyang, which provides a validation tool for YANG modules and the corresponding JNC plugin that creates Java classes – but after that, options start to run pretty dry for what one might use, other than commercial products such as tail-f NCS. In some cases, the way that these modules work is also a bit esoteric, requiring quite a lot of care around what the YANG types are in the consuming code.

To drive adoption of YANG and NETCONF for making the network more programmable – we need to make it easy to program the network. To this end, I started some work, with the aim of:

• Automatically generating a set of Python classes from a YANG module, that hang together exactly as per the configuration hierarchy described in a YANG module - such that a developer can take the YANG modules that they get from a vendor, or ideally standardised like the OpenConfig models - and generate bindings that they can use in their application.
• Ensuring that the generated bindings act like native types in Python – it should not be that a developer needs to learn something specific to be able to use these modules - something that looks like a dict should work just like a normal dict - so methods like keys() and iterators just work transparently.
• Supporting enough of the YANG types that real modules can be worked with. The aim is to provide something that at the very least works with the OpenConfig models – but ideally more. 

…

TypeError: peer_type must be INTERNAL or EXTERNAL

>>> oc.bgp.neighbors.neighbor["192.168.1.2"].config.peer_type = "EXTERNAL" 
 

The tool also tracks what has changed from the initial values (which can be populated from any source) - and has an output that can be serialised in a fashion that it could be used as input to a NETCONF or RESTCONF library to commit to a router:

>>> pp.pprint(oc.get(filter=True))

{   ‘bgp’: {   ‘global’: {   ‘config’: {   ‘as’: 2856,

                              ‘router-id’: ‘10.152.0.4’}},

               ‘neighbors’: {   ‘neighbor’: {   ‘192.168.1.2’: {   ‘config’: {   ‘description’: ‘a fictional transit session’,

                                                            ‘peer-as’: 5400,

                                                                                 ‘peer-type’: ‘EXTERNAL’},

                                                                   ‘neighbor-address’: ‘192.168.1.2’}}}}}
 

Clearly, there are some baby steps happening here –  as such, this just gives the data structures that one might interact with to be able to build policy - but configuring peers for any platform using loops like this is definitely something that starts to make programming the network easier from my perspective!

  global_config = {"my_as": 2856,}

  peer_group_list = ["groupA", "groupB"]

  peers = [("1.1.1.1", "groupA", 3741), ("1.1.1.2", "groupA", 5400,),

          ("1.1.1.3", "groupA", 29636), ("2.2.2.2", "groupB", 12767)]

  bgp = openconfig_bgp_juniper()

  bgp.juniperconfig.bgp.global.as_ = global_config["my_as"]

  for peer_group in peer_group_list:

    bgp.juniper_config.bgp.peer_group.add(peer_group)

  for peer in peers:

    bgp.juniper_config.bgp.peer_group[peer[1]].neighbor.add(peer[0])

    bgp.juniper_config.bgp.peer_group[peer[1]].neighbor[peer[0]].peer_as = peer[2]

 
 There’s some work to go – and as Dave Freedman and Ignas Bagdonas noted back at RIPE69 - it’d be great to have some abstraction away from the base configuration. However, as long as one can express that higher-level abstraction in something that can be written in Python - it should be possible to transform from the abstracted view, into the base configuration, with a set of fairly simple transformation models (or templates)… more coming on that as I’ve committed the code.

Hopefully, I’ll get to the stage where I can release this code to the wider world – and encourage its use. The focus on the management plane of the network has been lacking for years - and we finally have a chance to be able to fix it.

I’ll leave this post with a link to the talk that Anees Shaikh did at facebook’s networking@scale event. Anees did an awesome job of explaining what we’re trying to do with OpenConfig, and gives some cool insight into what the guys at Google are working on too:


 As usual – thoughts/comments are very welcome to rjs@rob.sh :-)