Introduction

There has been a good deal of cross pollination between the Java language and Python camps over the years. Probably the most prominent example of this is Jython, the pure Java implementation of the Python runtime. In that tradition, you will explore the use of the Eclipse IDE and the Ant build and deployment tool for use in Python development. Eclipse and Ant are extremely popular, feature-rich, extensible, and open source; qualities shared with Python. What makes it possible to use these Java tools for Python development are the PyDev and PyAntTasks extensions to Eclipse and Ant, respectively. This article will start with pointers on downloading and installing the required tools and extensions. I'll use a working example of Python code which reads RSS feeds in order to demonstrate the use of Eclipse and Ant for Python development.

This article won't get into the details of Eclipse, Ant, or Python. See the links in the Resources section for in-depth coverage of these.

Flavors of Python supported

The software covered in this article has been tested against CPython 2.3. With some exceptions, it should also work with Jython. Specifically, the PyDev debugger does not currently support Jython. Another difference is that scripts executed via Jython go into interactive mode after being run from within PyDev, forcing you to have to kill them manually. The PyDev editor works for Jython source code, and the Python Ant tasks work with Jython with the exception of the py-doc task.

Back to top

Using Eclipse for Python development

Overview of Eclipse

Eclipse is a Java technology integrated development environment that was developed and open sourced by IBM. It is the basis for IBM's commercial WebSphere Application Development environment, and various other tools. Eclipse has an extremely active community of developers who develop Eclipse itself and the large number of plug-ins available for it. See the Resources section for links to the Eclipse and Eclipse plug-ins Web sites. Although traditionally a tool for Java development, several plug-ins exist for developing with other languages within Eclipse, including C/C++, Python, and Perl.

Within Eclipse, source code is organized into projects. Projects may be loaded, unloaded, and imported. The Eclipse user interface is structured into views and editors. Examples of views and editors include the source code outline view, Java source editor, Python source editor, and filesystem navigator view. One of the key metaphors within the Eclipse user interface is the perspective. Perspectives are an organization of views that would typically be used together when carrying out some type of activity. The perspectives that are standard with Eclipse are: Debug, Java Browsing, Java, Java Type Hierarchy, Plug-in Development, CVS Repository Exploring, Resource, and the Install/Update perspective. A separate Python perspective does not currently exist. When doing Python development, I typically use the Resource perspective and Debug perspective.

Installing PyDev

To get started, download and install Eclipse from the Eclipse Web site (see link in Resources section), following the installation instructions appropriate for your platform.

The Eclipse update facility makes it easy to install the PyDev plug-in. From within Eclipse, select Help > Software Updates > Update Manager to start the Install/Update perspective. In the Feature Updates view at the bottom left, add the PyDev plug-in update site as a new Site Bookmark under the folder "Sites to Visit". The Eclipse update site URL for PyDev is http://pydev.sf.net/updates/. "PyDev" should now be displayed as a feature in the Feature Updates editor. In the Feature Updates editor, expand PyDev > Other, and select the PyDev feature which is displayed (should be at least 0.4.1). Then select "Install Now" to install the feature. Eclipse will download the PyDev plug-in and install it into Eclipse.

Importing the sample project

To access the sample code used within this project, download the zip file (see Resources section), expand the zip file into your filesystem, and then import the contained project into Eclipse. To import the project, switch into the Resource perspective, select File > Import, select "Existing Project into Workspace", and select the location where you expanded the zip file. The feedParserTest project should now appear in the Navigator view.

For the sample project, I've included the Feed Parser universal feed parser library, which is distributed under the Python open source license. See the Resources section for a link to the Feed Parser project Web site.

Back to top

A tour of PyDev

You'll now learn how to use the imported project to explore PyDev's features. PyDev is a work in progress, but is already a very effective development environment for Python development. The current list of PyDev features includes:

• Python editor including Python syntax highlighting
• Python syntax analysis, highlighting errors in the Python editor and in the Tasks View
• Option to convert Tabs into spaces
• Outline view showing imports, classes, and functions
• Python stack traces in the console view are hyperlinked to the source code
• Hyperlinking within the source code;Imports and function calls in the same module can be navigated via hyperlinks
• Ability to run a Python script from the Navigator view
• Debugger supporting breakpoints, stepping through code, and displaying variables

PyDev preferences window

The PyDev preferences can be reached via Window > Preferences and selecting Pydev (see Figure 1). The first set of preferences lets you change settings on how PyDev treats tabs within source code, and to change syntax coloring.

Setting the Python interpreter

The PyDev Debug preferences let you choose the Python interpreter that PyDev will use to execute your Python code. If PyDev is not able to find your Python interpreter, or if you want to use a different one, configure it here (see Figure 2).

Working with source code

I do most of my Python work within the Resource perspective. Switch into the Resource perspective, and in the Navigator view at the top left, double click on the file feedParserTest/src/feedparserTest/FeedparserTest.py. The Python editor opens the file, parsing the Python syntax for syntax coloring and syntax checking (see Figure 3).

If any errors are found in the source, they are displayed in the Tasks view at the bottom right. Double clicking on an error in the Tasks view will take you to the offending line of code.

The Outline view at the bottom left displays a browsable structure of the file currently being edited. Imports, classes, and functions are displayed, and may be navigated to by clicking on the item in the Outline view. PyDev re-parses the Python file as it is being edited, and updates the Outline view, performs syntax checking, and syntax coloring.

Editor features

The PyDev 0.4 release added a function and import hyperlink feature to the Python source editor. If you hold down the control key while hovering over an import or a function call that is on the PYTHONPATH, PyDev will display a hyperlink, allowing you to navigate to the source of the import or function. Note that in order for this to work for your source code across modules (linking from one module to another), you'll have to change your PYTHONPATH environment variable to include those modules so that PyDev can find them.

Some nice-to-have source editing features have started being added to PyDev in the latest release, including block comment/uncomment, and shift code right/left (see Figure 4).

Running a Python script

An IDE isn't very useful if you're not able to execute code. To execute Python code, select the feedparser.py file within the Navigator view, right click, and select Python > Run. The Python launch configuration window will be displayed (see Figure 5).

The Python launch configuration window gives you the ability to specify the current working directory from which the script will be executed, arguments to pass to the script, and what Python interpreter should be used to run the script. feedparser.py takes an RSS URL as an argument, so fill one in the arguments field, such as http://www-106.ibm.com/developerworks/news/dw_dwtp.rss. The rest of the default options are fine, so click Run.

The script is executed and the output is displayed in the Console window. If any errors occurred, a stack trace is displayed in the Console window and the lines within the stack trace are hyperlinks to the Python source code.

Python debugger

A Python debugger was recently added to the PyDev plug-in. To use it, set a breakpoint by clicking to the left of the line of code you'd like to break on within the Python editor. In Figure 6, I set a breakpoint at line 1830 of feedparser.py. Then select the Python module within the navigator view, right click and select "Python > Debug..." A launch configuration similar to before will be displayed. Click Debug to enter the Debug perspective and to launch the debugger.

The Debug view in the top left shows the processes and threads that are currently executing, the Variables view in the top right displays variables that are currently in scope, the Python editor shows at which line of code the debugger is currently stopped, and any output is displayed in the Console view. The debugger can be controlled via the buttons at the top of the Debug view or via the Run menu.

Other Eclipse features

Eclipse and its plug-ins have many other features which are applicable to Python development, including XML editors, UML editors (although most are Java code-centric), and source control plug-ins. Currently, almost 500 plug-ins are listed on the Eclipse plug-ins site (see link in the Resources section). Without going into detail, I'll highlight one plug-in in particular that would be useful to many Python developers: the CVS plug-in which is included as a part of the Eclipse distribution.

Eclipse includes a very feature-rich integration with CVS:

• Supports the SSH, pserver, and ext connection methods
• Basic CVS commands support: checking out projects, committing changes, updating, adding files/patterns to .cvsignore
• File merge view
• Compare differences in source control
• Project synchronization with a repository showing outgoing and incoming changes

Support for other source control systems such as ClearCase, Subversion, and Visual SourceSafe is provided by other plug-ins.

Back to top

Using the Python interactive shell within Eclipse

The Python interpreter supports interactive execution of Python code. This can be very handy for trying out pieces of code without putting them into a Python script and executing the script. And the Python interpreter interactive mode can be very easily integrated into Eclipse.

To add support for interactive execution of Python, add an External Tool launcher via Run > External Tools > External Tools. This will bring up the External Tools launcher configuration window. Create a new configuration by selecting "Program" in the Configurations list, and click "New". Name the configuration something like "pythonInteractive", set the Location to point to your Python interpreter, and pass "-i" as the single argument (see Figure 7).

Under the Common tab, check the box to display this configuration in the External Tools favorites menu.

To run the launcher you just created from within Eclipse, select Run > External Tools > pythonInterpreter. The Python interpreter output is displayed in the Console view. Python commands may be typed into the Console and executed just as they are when executing Python from the command line. In order to import and use your modules within Python in interactive mode, you'll need to add the location of your modules to the PYTHONPATH environment variable.

The one difference in executing Python within the Eclipse Console from when executing on the command line is that the command history feature (accessible via the up arrow and down arrow) isn't enabled because the Eclipse Console interprets these keys itself.

Back to top

Using Ant for Python development

Python automatically compiles modules when it needs to. This means that Python developers don't typically have to explicitly compile modules. Nonetheless, there are cases when it’s useful to manually compile Python code, and there are many other aspects of the build and deployment process that could be automated. This is what build tools are for.

I'll highlight a build tool from the Java programming world, Apache Ant, which is largely applicable to Python development. Apache Ant is the de-facto standard build tool within the Java programming world. It was created as a more portable and Java technology-friendly alternative to other build tools. Ant will run on any platform that supports Java programming language. Although most of what was required was already provided by Ant, there were some key Python-specific features that were needed if Ant was going to serve as a Python build tool. I've developed some custom Ant plug-ins (tasks in Ant lingo) which provide Python-specific features needed to do Python builds.

Ant uses XML as its format for describing builds. A build file is organized into targets to be carried out. Each target may depend on other targets. Ant will execute whatever targets necessary based on the target you request be executed, and the set of depended-upon targets. Each target may contain any number of Ant tasks which actually carry out the work of the target. Ant has many built-in tasks for things such as compiling Java code, generating documentation, manipulating files and directories, and there are many add-on tasks provided by third parties.

Installing the Python Ant library

I'll introduce the basics of an Ant build script and the Python Ant tasks by working through creating a build script for the feedparser test project. In order to use the Python Ant tasks, you'll need to download and install the Java library that contains these tasks. First, download the Python Ant tasks library (pyAntTasks.jar) from the URL listed in the Resources section. Then, copy the JAR file into the lib directory of the Ant plug-in within Eclipse. Under your Eclipse install directory, it should be in a directory like plugins/org.apache.ant_1.5.3.

Once you've copied in the Python Ant tasks library, it must be enabled within Eclipse. Select Window > Preferences, and thenAnt > Runtime. Expand out the Ant Home Entries to see the list of libraries (JAR files) that Eclipse uses. Add the Python Ant JAR file you just copied by selecting "Add JARs" and selecting the Python Ant JAR file within the Eclipse Ant plug-ins lib directory (see Figure 8).

You should now be able to create and run Ant build scripts that contain the Python tasks. So onto creating the build script!

Creating the build script

I'll step through the creation of a simple Python build script (see Listing 1). The complete build script, build.xml, can be found in the top directory of the feedParserTest project.

<project name="feedParserTest" default="compile">

  <taskdef resource="pyAntTasks.properties"/>

  <property name="src.dir" value="src"/>

  <target name="compile">
    <py-compile dir="${src.dir}" pythonpath="${src.dir}" optimize="0"/>
  </target>

</project>

Let's start with a build script that just compiles the example Python code. The <project> tag is always the root tag of the build script. The <taskdef> tag declares the Python tasks you'll be using throughout the build script. Near the bottom of the build script, you will define the compile target. Inside of the target element are the tasks that are carried out when compile is run. In particular, the py-compile task is used to compile all Python code starting in the src directory. The task will recursively go into any subdirectories and compile any Python modules. Rather than hard-coding the path to the src directory everywhere it will be needed, you've defined it as a property called src.dir within the build script. Then, wherever it is needed, it can be referred to via ${src.dir}.

To run the build script, open the build script within Eclipse. Eclipse has built-in support for editing and navigating Ant build scripts. The Outline view should display the structure of the build script. In the Navigator view, select the build script, right click and select "Run Ant...". Select the compile target, and click on "Run". The output of the build script execution should be displayed on the Console view, showing a successful run.

Python script execution task

Next, you'll add a target to the build script to execute a Python script (see Listing 2). In this case, you'll execute the feedparser.py script, which takes an RSS URL as an argument.

  <target name="run.feedparser" depends="compile">
    <py-run script="src/feedparser/feedparser.py" pythonpath="${src.dir}" optimize="0">
      <arg value="http://www-106.ibm.com/developerworks/news/dw_dwtp.rss">
    </py-run>
  </target>

The above target executes the feedparser.py script, passing the RSS URL as the single argument. The target is declared to depend on the compile target, so it will be executed first. This actually isn't really necessary because Python will automatically compile the source code if needed. If you execute the run.feedparser target, the feedparser.py script should run, outputting the RSS content to the Console.

Python documentation task

Python has an API documentation facility similar to Java technology’s JavaDoc system, called PyDoc. The following snippet of XML in Listing 3 added to your build script will generate PyDocs for all of your Python modules.

 1:  <property name="pydoc.dir" value="pydoc"/>
 2:
 3:  <target name="init">
 4:    <mkdir dir="${pydoc.dir}"/>
 5:  </target>
 6:
 7:  <target name="pydoc" depends="init,compile">
 8:    <py-doc pythonpath="${src.dir}" destdir="${pydoc.dir}">
 9:      <fileset dir="${src.dir}">
10:        <include name="**/*"/>
11:      </fileset>
12:    </py-doc>
13:  </target>

Deconstructing the pydoc target above, line 7 declares the target name, and indicates it depends on the init and compile targets. This means that prior to running the pydoc target, Ant will make sure that the init and compile targets have already been run, and will run them if not.

The init target, which the pydoc target depends on, is defined in lines 3-5. The init target simply creates a directory to hold PyDoc API documentation files. As before, you've defined a property, named pydoc.dir, for the location of the generated documentation.

Line 8 is the start of the py-doc task. As before, you pass the PYTHONPATH you want to use in generating the pydocs. The destdir attribute tells the py-doc task where to output the generated HTML documentation.

Lines 9-11 specify what Python source files should be processed when generating documentation. The fileset is a common construct in Ant scripts, used to specify a collection of files to operate on. This is a powerful feature allowing you to use naming patterns, boolean logic, and file attributes to select what files to operate on. This is fully documented in the Ant documentation. In this case, you're recursively selecting all files starting with the 'src' directory.

Python unit test task

Python comes standard with a unit testing framework (as of Python 2.3. In Python 2.2, it was an optional module), much like the Java jUnit framework. Test cases are structured in the same manner as jUnit. Each class/module to be tested would typically have its own test class. The test class contains test fixtures, which are initialized in a setUp function. Each test is written as a separate test function within the test class. The unittest framework will cycle through the test functions, calling setUp, the test function, then tearDown for each test function. See Listing 4 for an example.

import unittest
from pprint import pprint

import feedparser

class FeedparserTest(unittest.TestCase):
    """
    A test class for the feedparser module.
    """
    
    def setUp(self):
        """
        set up data used in the tests.
        setUp is called before each test function execution.
        """
        self.developerWorksUrl = "testData/developerworks.rss"       

    def testParse09Rss(self):
        """
        Test a successful run of the parse function for a
        0.91 RSS feed.
        """
        print "FeedparserTest.testParse09RSS()"
        
        result = feedparser.parse(self.developerWorksUrl)
        pprint(result)

        self.assertEqual(0, result['bozo'])
        
        self.assert_(result is not None)
        channel = result['channel']
        self.assert_(channel is not None)
        chanDesc = channel['description']
        self.assertEqual(u'The latest content from IBM developerWorks',
            chanDesc)
        
        items = result['items']
        self.assert_(items is not None)
        self.assert_(len(items)> 3)
        firstItem = items[0]
        title = firstItem['title']
        self.assertEqual(u'Build installation packages with 
            solution installation and deployment technologies',
            title)
  
    def tearDown(self):
        """
        tear down any data used in tests
        tearDown is called after each test function execution.
        """
        pass
                
if __name__ == '__main__':
    unittest.main()

The above listing is a test class which performs basic tests on the feedparser module. The complete test class can be found in the feedParserTest project under src/feedparserTest/FeedparserTest.py. The setUp function prepares the test fixtures used throughout the tests, in this case just the path to the test RSS file you'll parse in the test function. testParse09Rss is the actual test function. This function calls the feedparser.parse function with the test RSS file, outputs the parsed results, and performs some basic checks via the TestCase class assert functions. If any assertion does not evaluate to true, or if any exceptions are thrown as a part of processing, unittest will report it as a test failure or error. Finally, the two lines at the bottom allow you to run the tests within this test class by simply running this module.

To run this test class by itself, run the FeedparserTest.py module in the same manner as before. Select the FeedparserTest.py module in the Eclipse Navigator view, and run via Python > Run. The launch configuration window will be displayed. The default values are fine except for the Base directory. The Base directory has to be the feedParserTest project directory, so that the RSS file (testData/developerworks.rss) can be found from the current directory. Change the base directory setting and click 'Run'. The output will be displayed on the Console.

You'd like any unit tests we write to be executed automatically as a part of the build. The following build fragment in Listing 5, added to your build script, will do just that.

1:  <target name="tests" depends="compile">
2:	<py-test pythonpath="${src.dir}" dir=".">
3:  	  <fileset dir="${src.dir}">
4:		<include name="**/*Test.py"/>
5:	  </fileset>
6:  	</py-test>
7:  </target> 

The first line is the target declaration, the same as the others. Lines 2 through 6 invoke the py-test task. These lines will run any tests found anywhere under the 'src' directory, looking for any files that end in 'Test.py'. The PYTHONPATH will be set to 'src', and the tests will be executed with the current working directory set to the current directory ('.').

To run the target, run the build script, and select the 'tests' target to execute. The target will run all test cases that end with 'Test.py', in this case just FeedparserTest.py.

Back to top

Summary

The combination of Eclipse and the PyDev plug-in, and Apache Ant along with the Python Ant tasks provide a complete integrated development environment and build/deployment tool for Python development. These tools are undergoing development, so check for updates periodically, and if you feel an itch for a feature you'd like to see, consider rolling up your sleeves and pitching in.

Back to top

Download

Information about download methods

Resources

Learn

• Visit Eclipse.org for Eclipse downloads and documentation.
  
  
• The PyDev SourceForge project site provides downloads, documentation, and user forums.
  
  
• The Python Ant tasks Web site contains the latest version of the Python Ant tasks.
  
  
• Feed Parser, the universal feed parser implemented in Python, can be found at the Feed Parser Web site.
  
  
• Python in a Nutshell (O'Reilly & Associates; 2003), by Alex Martelli, is an excellent reference on the Python language.
  
  
• Find more articles for Eclipse users.
  
  
• Visit the developerWorks Open source zone for extensive how-to information, tools, and project updates to help you develop with open source technologies and use them with IBM's products.
  
  

Get products and technologies

• The Eclipse Plug-ins Web site contains hundreds of commercial and open source plug-ins that you can install into Eclipse.
  
  
• The Apache Ant Web site has Ant downloads and documentation on Ant.
  
  
• For downloads and information on Python, see the Python Web site.
  
  
• See the latest Eclipse technology downloads on alphaWorks.
  
  
• Innovate your next open source development project with IBM trial software, available for download or on DVD.
  
  

Discuss

• Get involved in the developerWorks community by participating in developerWorks blogs.
  
  

About the author

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• Using Eclipse for Python development
• A tour of PyDev
• Using the Python interactive shell within Eclipse
• Using Ant for Python development
• Summary
• Download
• Resources
• About the author
• Comments

Next steps from IBM

Spring, JRuby, and Ajax development is easier with WebSphere Application Server - a smart Java 5 and J2EE Web services-based application server.

---

• Try: The no-charge WebSphere Application Server Community Edition is a pre-integrated, lightweight Java 5 application server built on Apache Tomcat and other best-of-breed open source software such as OpenEJB, Apache Axis, and Apache Derby.
• Article: The article leverage the Spring Framework and the WebSphere Application Server to improve your J2EE project productivity.
• Tutorial: See how the free WebSphere Application Server and XML can improve the efficiency of your JRuby on Rails and Ajax development.
• Buy: WebSphere Application Server - Express

My developerWorks community

Dig deeper into Open source on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers