As it turns out - it's pretty easy to code up a very basic blog application and integrate it with your existing flat-file site in Django. In fact, it fits very nicely into the 20 minute gap that you're taking to try and make sure that you can sleep!

There's also some semblance of a design here now.

Since I'm taking the Computational Physics module in my third year at IC, we're using some libraries like GSL to provide "better-than-default" random number generators and such like. It turns out that those of us using a Mac don't get GSL installed by default with Xcode, or under OS X - and unlike Windows and Linux - there's no instructions on the course site. Here's a really quick way to ensure that you can link against it:

• Install MacPorts from MacPorts.org.

• $ sudo /opt/local/bin/port install gsl

• In the directory that you're writing your code from, perform the following:
  

  $ g++ -I/opt/local/include -L/opt/local/lib -lgsl -o binary_name source_file.cpp

Which should let you link against GSL just fine :-)

Further to the previous comment, you can also link against GSL from within an Xcode project:

• Create an Xcode "C++ Tool" project from the command line projects options.
  
  
  
• Within the project Ctrl+Click on the project name, and select "Get Info"
  
  
  
• Use the Spotlight-esque search at the top of this window to search for Compiler, and select the "Other C++ Flags" text box.
  
  
  
• Into the box enter

  -I/opt/local/include -L/opt/local/lib -lgsl

  leave the

  $(OTHER_CFLAGS)

  existing input there.
  
  
  

You should then be able to use the Build command in Xcode to build the project linked against GSL. This assumes that you install GSL from MacPorts as documented previously.

Since I've got a few moments, and I've decided to actually write down some rants rather than deciding that I can't be bothered to - I'm going to use some space to single the praises of Django.

I've been using Django for a couple of years now - since around the autumn of 2005, and as such, feel that I've got a pretty good grasp of how the framework works. I haven't really hacked around that much with the innards of Django (although I did propose a patch), however, what I really like about this framework isn't particularly the internals, but just the whole philosophy that there seems to be in terms of building a web application.

Let's face it, when you sit down to write a web application, especially as a sysadmin or a network engineer - your primary motivation isn't to write a web application, your motivation is to get some data out in a very easily readable format to some audience. So, you start looking at how your database should be designed (or retrofitting your code around an existing database), and you start writing code for updating one table, or grabbing data from another. Stop. This is where Django comes in. With Django, you just rapidly develop your data into a set of models, which can be related in a number of different ways - almost everything that you want to do (for generic data handling) can then be done straight from a generic Django view - without you having to go around writing a lot of code for really generic things. It really speeds up web application development.

I love the fact that Django has things like overloadable save(), and delete() methods for each model - it sounds fairly trivial, but it's great to be able to have the application create the ports on a switch when it saves it into the database the first time - and remove them when the user removes the switch. I've found myself able to work on really quite complicated model layouts without getting too tied up in it, because it's just a case of breaking it down simply into models.

I've got a couple of Django based projects that I'm maintaining (hint: you're reading one of them). The other is a portal system for Catalyst2. I've been working on this system for almost 2 years now - it started out as a PHP application, and then morphed into Django when I discovered it. As a PHP app it was just unmanageable, because of the amount of code that I was having to write. Django's inbuilt admin application has let me just worry about actually presenting data to users - and keeping the boring stuff (like adding data to simple database tables) out of the way of the application. It's become a pretty complicated application.

The system tracks ISP assets, things like racks, servers, switches, routers, cabling...the list goes on. For servers it can control APC MasterSwitches via SNMP, it integrates into cacti for network graphs, it can access RANCID SVN repositories to obtain device configurations...I built an SNMP poller that collects traffic data, and produces billing data as both HTML and PDF. I've integrated it with our automated DNS and MX secondary system. The list is pretty huge (hence why this project is almost two years old!).

Before it sounds like I'm just trying to give a list of features that I've written into an application - I'll get to the point. Django and python's flexibility means that I've been able to sit down and write features, I haven't had to sit down and write a lot of generic type functions that add entries to a database, allow them to be updated, and then save them, I haven't had to handle how my site is going to be templated - I've written features. This is the massive difference for me using Django.

Sure, I could get this functionality with just about any framework out there - BUT, Django does this really well - there are very, very few things that you come across and say "Oh, I don't like that" that you can't change. I didn't really like the authentication system that it uses by default - no worries, I can just replace it with an LDAP authentication system for our users, and use the Django one to provide application-specific privileges. Django does things easily, in a sane manner, that you can just code for. What's even better, is it fits in with people like me - who just want to get an application out - but also it fits in with those guys who want to produce just the frontend part of the system - the templating language allows really simple creation of complex pages, without a steep learning curve.

I didn't really plan this entry before I started, but I hope I've got across what I actually wanted to say, Django is a really great framework that's very, very flexible. It's also getting better, they're carefully considering what's added to it, and keeping it so that its very database neutral, and can fit into many, many development styles.

I've written >5,000 lines of just python code in my Django applications, and I'm still finding new and cool things that the framework can do - I really do recommend it for web application RAD!

Further to my previous post - I presented this issue at LINX65 - video and slides can be found below.

Video
Fixed Slides - LINX's PowerPoint install seems to have corrupted my slides on the day.




Comments and feedback are most welcome.

For all network deployments, there is a requirement to present information relating to both topology, and various utilisation statistics to some human operator. In many cases, this process has become so ingrained in network requirements that there are almost ubiquitous solutions to the visualising data - for example, link utilisation is almost always presented via some framework or tool powered by RRDTool. Other tools, such as network "weathermap" diagrams linking this utilisation information into an overview of a network topology are also seen in many NOCs. In most cases, the problem of visualising data relating to a flat MPLS or IP network is solved for most common deployments.

The problem of data presentation is somewhat altered in the case of an MPLS-TE network. Whilst link utilisation graphs, and weathermap diagrams may continue to provide useful information, suddenly the overlaid tunnels within a network are of interest to operators. Whilst a first-line NOC engineer may not be aware of what is indicated by a certain LSR becoming a midpoint for a large number of tunnels within a network, this almost certainly identifies some form of failure, or congestion that should be reacted to. In addition to identifying events such as TE LSP path changes to particular nodes, it is often useful to retain some insight into the constraints within which CSPF is currently selecting paths for LSPs. In order to solve these problems, there's a requirement for an additional set of visualisation tools. In some cases, these tools may be implemented using existing visualisation tools - however, this often results in overly-complex presentations. The intent of this post is to outline some of the challenges, and interesting methods that appear to be available for presenting data relating specifically to MPLS-TE networks - a number of beta-quality solutions in use within AS5413 are presented, however, this post does not intend to define a complete solution.

In order to properly consider the available solutions to any visualisation problem, the data to be presented to an operator needs to be defined. Ideally, the information that is required by an operator is likely to include the following:

1. The path taken by specific TE LSPs throughout a network -- within an straight MPLS or IP network, it is likely that there is a relatively easy set of nodes through which traffic is forwarded, since within a TE deployment paths are dynamic according to constraints, rather than following the shortest-path, then the presentation of path information to a human is of operational benefit.
2. The resource consumption on any TE-enabled path -- this requirement is likely to include the presentation of resource utilisation over a specific link by reservations associated with a TE LSP. However, it should also view of the resources consumed by a specific TE LSP, both in terms of reserved bandwidth, and actual traffic forwarded in the interface
3. Efficiency of path selection across a MPLS-enabled topology -- in many cases, TE is deployed in order to ensure that service level guarantees are upheld. Since typically, these guarantees are made in terms of loss, latency and jitter, the effect of TE path selection on forwarding between specific points in a TE network is of interest to an operator. Where the 'efficiency' of path selection is referred to, this should be interpreted as the proximity of path selection in the network to the optimal path throughout a network (this should be assumed to be the shortest-path as would be selected typically by an IGP).

In order to address requirement 1 above - there are two types of visualisation tool that is of direct interest to an operator - a view based on a specific node, and a whole path view. For a specific LSP, a per-node view showing the egress interface for each LSP is relatively trivial to produce from the output of a command such as show ip rsvp reservation or similar. An example of this form of diagram (as produced by one of the tools built for AS5413 use) is below.

A significant limitation of producing static images such as the example above, is that it is difficult to visualise changes between egress interfaces. The node in the example is a midpoint for a small number of nodes running MPLS TE - during the time at which the example output was gathered, a small number of test PEs had been deployed with automatic bandwidth adjustment on each TE LSP, and hence the number of TE reservations is relatively small. In order to determine egress paths for each LSP on the node, and hence determine the cause of any imbalance across equal-cost paths, a diagram such as this can be useful. However, to properly view the changes in paths over time, a relatively large number of diagrams of this nature need to be examined. It is likely that a better result would be achieved by having some form of dynamically updating graphic representation.

The second type of diagram requires a topological view of the network - it is almost impossible to represent all TE paths for which a specific node is the head-end in a network on a single diagram whilst retaining a clear, easy-to-parse image. Producing per-LSP diagrams is possible, but again produces some difficulty in representing the changes of path for an LSP. Diagrams can quickly become convoluted if all possible paths are included. For example, the diagram below shows all possible NHOP and NNHOP LSRs for a specific node within the AS5413 network topology. Where there is a secondary device via which all nodes can be seen, the topology becomes relatively complex very quickly. The problem of finding a computational method to produce layouts that are clear for human interpretation is relatively complex, with Graphviz being the standard OSS tool utilised for this purpose.

Should there be a requirement to scale a diagram such as the example above to show all possible nodes via which a TE LSP may be routed, and overlay the actual path taken, then a large number of diagrams are likely to be required. In addition, the information presented to an human is unlikely to be easy to parse. Again, a requirement would appear to exist for dynamic means of displaying information, and the ability to adjust an image to show specific nodes or paths of interest would be advantageous.

The second requirement in the list above is somewhat easier to solve, as it mirrors existing visualisations that are utilised for a straight IP/MPLS network. A weathermap-type topology diagram, where each link is marked according to the reservations made on it can easily display information relating to the size of reservations on specific interfaces. Additional information can be shown in such a topology diagram by adjusting the display of specific nodes to reflect the number of LSPs carried. Such a solution can be implemented with relatively minor changes to existing tools such as Network Weathermap. It should be noted that this is likely to require human interaction to develop a set of topologies with a network that are of interest to those using such a weathermap - for instance, certain NOC engineering teams may require specific topologies - or specific subsets of nodes. Displaying every PE within a relatively large network on a single diagram is unlikely to provide useful information to an engineering team.

To monitor SLGs within a network (as per the third requirement above), typically, some form of graphing of an IP SLA probe, or some end-to-end ICMP graph (via a tool such as SmokePing) is used. It is likely that such implementations can continue to be utilised to support SLAs. However, the problem of displaying efficiency of path selection within a network is not as trivial to solve, and indeed is unlikely to ever be a requirement in a network where a packet is routed according to an SPF algorithm. This is not something that has been implemented within AS5413 currently, however, it would appear that some interesting approaches do exist - for example, Karol Kowalik and Martin Collier from Dublin City University present an eye-diagram approach to displaying the efficiency of path selection throughout a TE network. Within their paper, they note that this approach does not scale easily to TE networks above 30 nodes, however, since there appears to be no open reference implementation for this method, it hard to evaluate its performance. This paper appears to be one of the few approaches to showing path efficiency through a TE network that has been discussed.

The post's intention, as discussed, is to present the small number of solutions, and their limitations that have currently been implemented at AS5413 - clearly, these tools are not production quality, or complete at the current time. If there is interest in providing a non-commercial solution (as an alternative to solutions like OPNET, and the Cisco TE tools) that provides information for small to medium SPs, where such an investment does not seem justified, then there may be a possibility to push some collaborative OSS visualisation tools. If you, or your organisation have any such interest, please let me know via the comments.

It looks like, once again, there's another attribute flying around the global BGP table causing Quagga instances to crash (if based on 0.99.9 - I believe the bug is fixed in 0.99.10). This relates to the 2007 draft that introduced AS_PATHLIMIT - see ietf.org - draft-ietf-idr-as-pathlimit. This attribute is actually relatively interesting, from an operator's point of view, where control that is more granular than setting the common no-export or no-advertise communities does not suffice.

As far as I could see, there's only one BGPd that supports this draft (which does appear to have fallen by the wayside in the IDR WG), which is quagga. Whilst looking for this, I found the patch in which this was introduced. Looking at the code, it really reminded me why John Scudder's optional-transitive draft is very useful. If we look at the code in the patch mentioned above:

+  if (flag != BGP_ATTR_FLAG_TRANS)
+    {
+      zlog (peer->log, LOG_ERR, 
+	    "AS-Pathlimit attribute flag isn't transitive %d", flag);
+      bgp_notify_send_with_data (peer, 
+				 BGP_NOTIFY_UPDATE_ERR, 
+				 BGP_NOTIFY_UPDATE_ATTR_FLAG_ERR,
+				 startp, total);
+      return -1;
+    }
+  
+  if (length != 5)
+    {
+      zlog (peer->log, LOG_ERR, 
+	    "AS-Pathlimit length, %u, is not 5", length);
+      bgp_notify_send_with_data (peer, 
+				 BGP_NOTIFY_UPDATE_ERR, 
+				 BGP_NOTIFY_UPDATE_ATTR_FLAG_ERR,
+				 startp, total);
+      return -1;
+    }

Note here that in both these cases, we're checking for errors in AS_PATHLIMIT, that's an optional transitive. This is another "routing metadata" element, or really just a protocol enhancement, but - when an error is found - we're sending a NOTIFICATION message to the other side. Once again, a bug, or incorrect population of this element is going to affect a whole session (which may carry multiple AFIs). I understand that there's no standard, other than NOTIFICATION, available to implementers right now, but this really does seem quite harmful.

I hope that this will be solved as it's pushed through the IDR WG, as usual, operator support for this is useful, as it ensures that vendors, and the WG is well aware of the importance of such a draft!