Using Net-SNMP and IPython
Interactively explore Net-SNMP with Python bindings with IPython to manage UNIX and Linux systems
Most systems administrators have had some experience with Simple Network Management Protocol (SNMP), or at least have heard of it. If you are working in a data center, then you probably interact with SNMP, in some way, on a daily basis. There are many impressive, full-scale Network Management Systems (NMS) or Network Monitoring Systems that do notable SNMP monitoring, but this article is not about those systems. This article is about exploring SNMP through the Python™ language and writing the code yourself.
A friend recently told me that there are times when you just want to walk down the street to grandma's house, and you don't need a Saturn V rocket to do that. Many tasks, such as utilizing or configuring a massive NMS, are like a Saturn V rocket, and you would be better served, holding off on filling up the liquid oxygen tanks, until you try a little Python first. Knowing how to write agile Python code to interact with SNMP can be one of the most interesting and productive skills a systems administrator can pick up. Although SNMP is frighteningly complex to set up and use, the stack discussed in this article makes it fun.
Installing and configuring Net-SNMP
For this article, it's necessary to have a reasonably up-to-date Python installed on your *nix machines, which means Python 2.3 or greater. Python 2.5.1 was the most current version of Python at of the date of this writing. You also need IPython to interactively work with the Net-SNMP library with Python bindings. The Net-SNMP team has done a decent job of testing support on AIX®, HP-UX®, GNU/Linux® distributions, such as Red Hat, Windows®, and even OS X™.
Getting a copy of IPython installed is quite easy. A good option is to use Easy Install to manage Python packages. You can install any Python package quite "easily" by running the ez_setup.py script. For example, just type:
Other options for installation include, using your favorite package management system, or just downloading IPython and typing:
python setup.py install
For this article, you want to make sure your client machine or the machine that runs all of the code has NET-SNMP Version 5.4.x or higher, as this is when the Python bindings were included in the source distribution. The installation of the bindings requires compiling from source in most cases; however, there are Red Hat Package Managers (RPMs) available. If you are especially brave, feel free to check out the latest copy off of the trunk, available from the Net-SNMP Web site.
There are many compile options, but the main task is to get NET-SNMP to compile correctly, and to then run the separate Python installer in the included Python directory. One other thing to note is that when you do a compile and run ./configure, it runs a configuration script for the local machine on which the agent is being compiled. You shouldn't use the configuration script, so just create a simple configuration for the sake of this article.
Make a backup of whatever configuration file lives in
/etc/snmp/snmpd.conf, and build this extremely basic
syslocation "My Local Machine" syscontact email@example.com rocommunity public
Save it and restart the
snmpd daemon. On most *nix
/etc/init.d/snmpd restart will do the trick.
Unless you absolutely have to, a good practice to follow is not to compile software off of trunk in active development, as you might be put in a situation of fixing broken code yourself. On CentOS 5 or Red Hat Enterprise Linux 5 (RHEL 5), you can download the latest stable source RPM, which at the time of this writing was 5.4.1. Please note that you also need the Python source to build the bindings. So if you are on a Red Hat-based machine, make sure you install python-dev (or the equivalent) and Python header files for your specific *nix OS. If you need any help building RPMs from source, please consult the official Red Hat documentation. Building packages from source can be a very complex subject and is out of the scope of the article. If you do have trouble getting the Python bindings working, you should ask for help on the Net-SNMP mailing list.
Jumping into the code
Why wait any longer, let's assume you have gotten the Python binding to install and have IPython working. Now you're ready to fire up IPython and get to work. At some point though, you need to read through the IPython documentation. There are also some very good screencasts on IPython by the current Python Advocacy Coordinator, Jeff Rush. Okay, let's proceed with the coding.
Let's do a simple query to identify a machine by using its object identifier
(OID) value, sysDescr. Start IPython by typing in
ipython and then perform this interactive session:
Listing 1. IPython example
In : import netsnmp In : oid = netsnmp.Varbind('sysDescr') In : result = netsnmp.snmpwalk(oid, ...: Version = 2, ...: DestHost="localhost", ...: Community="public") In : result = netsnmp.snmpwalk(oid, Version = 2, DestHost="localhost", Community="public") In : result Out: ('Linux localhost 2.6.18-8.1.14.el5 #1 SMP Thu Sep 27 18:58:54 EDT 2007 i686',)
Note that your result value is different than the result value show here. If you have followed the configuration shown in Listing 1 above, everything else should work for you. If you are an old hand at SNMP, then you might immediately grasp the power behind the stack.
One of the nice things about using IPython to test out snippets of SNMP code is that it also acts like a normal shell, and many basic, interactive shell concepts “just work”—but in a pythonic way so to speak. Writing SNMP code can be a very tedious activity, but the Net-SNMP library with IPython makes it quite fun.
As you can see, it is quite simple to get a result back as a Python data type. This is why IPython and Net-SNMP go so well together. Now it is just a matter of interactively exploring combinations of OIDs to query for the purpose of writing a custom script. In a perfect world, there would be a massive, easy-to-configure NMS setup script to run that automatically integrates a new machine to the network.
This of course is not a perfect world and, as a systems administrator, knowing how to hack together some clever SNMP code can be quite handy. One example situation is that you just converted a high-speed DDR into a 2 TB RAID 0 server running Ubuntu Linux, because that is all you had time to do in the one hour you were given to solve the crisis.
Now you are in big trouble, and you only have minutes to monitor the situation
for trouble to know if you need to start sending out resumes, or if you should
start preparing a speech and asking for a raise. Let's use the edit function in
IPython to write a script out to a file, and then run it in one session without
Listing 2. IPython module creation
import netsnmp class snmpSessionBaseClass(object): """A Base Class For a SNMP Session""" def __init__(self, oid="sysDescr", Version=2, DestHost="localhost", Community="public"): self.oid = oid self.Version = Version self.DestHost = DestHost self.Community = Community def query(self): """Creates SNMP query session""" try: result = netsnmp.snmpwalk(self.oid, Version = self.Version, DestHost = self.DestHost, Community = self.Community) except: import sys print sys.exc_info() result = None return result
Go ahead and cut and paste the code below into the file you just created. When you save this file, IPython automatically runs it and places the class inside the module in your environment. If you type in who, you will see something like:
In : who netsnmp snmpSessionBaseClass
This is extremely powerful, because you can get all of the benefits of using your favorite text editor, perhaps Vim or Emacs, and then use that code immediately in an interactive IPython shell session. Note, if you already have a module you wrote, you can also just type in and run it to get the same exact result. Executing and running the module in IPython literally runs through the code and puts it into the IPython environment.
By using IPython, it is now possible to combine the best features of a Python shell, UNIX shell, and your favorite text editor. When interacting with something as complex as a library for SNMP, you need all the help you can muster and, in this case, it really shows the power of IPython.
You can write modules on the fly and you can test and use them later. IPython also works quite well with any programming style, including Test Driven Development (TDD) or Test Enhanced Development (TED). Let's jump right into the module you just wrote to demonstrate the convenience.
Now that you have an object-oriented interface to SNMP, you can start interrogating your local machine a bit:
Listing 3. IPython iterative coding
In : run snmpinput In : who netsnmp snmpSessionBaseClass In : s = snmpSessionBaseClass() In : s.query() Out: ('Linux localhost 2.6.18-8.1.14.el5 #1 SMP Thu Sep 27 18:58:54 EDT 2007 i686',) In : result = s.query() In : len(result) Out: 1
It is quite easy to get results by using this module, but you are basically just running a hardcoded script, so change the value of the OID object to walk the system subtree:
Listing 4. Changing the value of the OID object
In : s.oid Out: 'sysDescr' In : s.oid = ".18.104.22.168.2.1.1" In : result = s.query() In : print result ('Linux localhost 2.6.18-8.1.14.el5 #1 SMP Thu Sep 27 18:58:54 EDT 2007 i686', '.22.214.171.124.4.1.8072.3.2.10', '121219', 'firstname.lastname@example.org', 'localhost', '"My Local Machine"', '0', '.126.96.36.199.188.8.131.52.1.1', '.184.108.40.206.220.127.116.11.1.1', '.18.104.22.168.22.214.171.124.1.1', '.126.96.36.199.6.3.1', '.188.8.131.52.2.1.49', '.184.108.40.206.2.1.4', '.220.127.116.11.2.1.50', '.18.104.22.168.22.214.171.124.2.1', 'The SNMP Management Architecture MIB.', 'The MIB for Message Processing and Dispatching.', 'The management information definitions for the SNMP User-based Security Model.', 'The MIB module for SNMPv2 entities', 'The MIB module for managing TCP implementations', 'The MIB module for managing IP and ICMP implementations', 'The MIB module for managing UDP implementations', 'View-based Access Control Model for SNMP.', '0', '0', '0', '0', '0', '0', '0', '0')
As you can tell, it is quite easy to take this module and start investigating the whole network, one machine at a time. Take your time and figure out what you would like to query on your network. There is another interesting feature of IPython that is worth mentioning. IPython has an incredible feature that lets you run snippets of Python code as background processes. Fortunately, it is incredibly simple to do. Let's run that same query again, but as a background process this time (see Listing 5).
Listing 5. IPython iterative coding example—background processes
In : bg s.query() Starting job # 0 in a separate thread. In : jobs.status Out: 'Completed' In : jobs.result Out: ('Linux localhost 2.6.18-8.1.14.el5 #1 SMP Thu Sep 27 18:58:54 EDT 2007 i686', '.126.96.36.199.4.1.8072.3.2.10', '121219', 'email@example.com', 'localhost', '"My Local Machine"', '0', '.188.8.131.52.184.108.40.206.1.1', '.220.127.116.11.18.104.22.168.1.1', '.22.214.171.124.126.96.36.199.1.1', '.188.8.131.52.6.3.1', '.184.108.40.206.2.1.49', '.220.127.116.11.2.1.4', '.18.104.22.168.2.1.50', '.22.214.171.124.126.96.36.199.2.1', 'The SNMP Management Architecture MIB.', 'The MIB for Message Processing and Dispatching.', 'The management information definitions for the SNMP User-based Security Model.', 'The MIB module for SNMPv2 entities', 'The MIB module for managing TCP implementations', 'The MIB module for managing IP and ICMP implementations', 'The MIB module for managing UDP implementations', 'View-based Access Control Model for SNMP.', '0', '0', '0', '0', '0', '0', '0', '0')
Before you get too excited, background threading works like a charm in IPython, but it only works with libraries that support asynchronous threading. Unfortunately, Net-SNMP is synchronous. If you are curious, test this out by changing the s.oid value to .iso. You should notice that the IPython interpreter "blocks" or "hangs" until the query is done. Just a warning though, doing an SNMP walk of the whole .iso tree can take quite a while, so you might want to just take my word for it.
There is another solution of course. You can use one of the many processing
libraries available in Python to fork this blocking process. Several third-party
libraries are available at the Python cheese shop. If you are using
easy_install, then it would be trivial to install a
package, such as Parallel Python, and test out this library with Net-SNMP, so it's
up to you.
One final feature to demonstrate is running a unittest within an IPython shell.
It is quite handy to frequently run a unittest when making changes to a module.
You need to add a flag to run
run -e so that you can
avoid getting a traceback to the unittest module in the IPython shell. You can
download this unittest in the source files that come with the article.
Please note that IPython 0.8.2 also has a new doctest feature, which allows you to generate doctests within IPython. Doctests are a wonderful feature of Python, because among other things, it allows for a way to create testable documentation for an API. Here is an example of how to run doctests for our module inside of IPython:
Listing6. IPython running in doctest mode
In : %doctest_mode *** Pasting of code with ">>>" or "..." has been enabled. Exception reporting mode: Plain Doctest mode is: ON >>> from snmpinput import snmpSessionBaseClass >>> s = snmpSessionBaseClass() >>> s.query() ('Linux devmws2.racemi.com 2.6.9-55.0.2.EL #1 Tue Jun 26 14:08:18 EDT 2007 i686',)
Since doctest blindly executes Python statements, you must be careful not to use a value that might change in a doctest, like the value shown above. If you paste lines of code inside of the docstring of your module, you can then test your API documentation by using the idiom:
def _test(): import doctest doctest.testmod() if __name__ == "__main__": _test()
In this article, you've learned how using Net-SNMP and IPython together can be a powerful combination. The following main concepts were covered:
- Python bindings: Net-SNMP now has Python bindings, which allows the power and elegance of Python to leverage the SNMP protocol.
- Processing library: The Python bindings are synchronous at this point in time, but using a processing library to fork each request can solve this problem.
- Agile techniques: IPython is an incredibly sophisticated and powerful tool for systems administrators and software engineers. Even though the article only briefly mentioned agile techniques, such as doctests and unittests, you can apply these techniques to do any test-centric development or to just interactively write and explore code.
- SNMP and IPython: This article barely scratched the surface of what SNMP and IPython can do separately or together.
SNMP is a tremendously complex beast, and it is almost overwhelming to think about writing any code of significance, but the techniques covered here hopefully spur some ideas. If you are curious about just how far a Python implementation of SNMP can go, take a look at Zenoss, download a Virtual Machine, and test it out. There is also an API that you can script against, so you can combine what you've learned here with a full-blown Python NMS. Of course, the same applies for any other NMS as well.
- See the IPython documentation page for information about the usage and configuration of the IPython shell.
- Enhanced interactive Python with IPython: Check out this comprehensive article on using IPython by Jeremy Jones.
- Net-SNMP Wiki: Visit this wiki to find information documenting the usage of the Python bindings for Net-SNMP.
- The Python tutorial: Take this tutorial and get an introduction to the basic concepts and features of the Python language and system.
- Easy Install for Python: This Python module lets you automatically download, build, install, and manage Python packages.
- AIX and UNIX: The AIX and UNIX developerWorks one provides a wealth of information relating to all aspects of AIX systems administration and expanding your UNIX skills.
- AIX Wikis: Discover a collaborative environment for technical information related to AIX.
- Search the AIX and UNIX library by topic:
- IBM trial software: Build your next development project with software for download directly from developerWorks.