Although Python is commonly equated with CPython, its specification has been implemented elsewhere several times, including in applications for Java and .NET. JPython compiles Python source to Java bytecode and provides transparent access to Java classes. Python for .NET is an application in the works for Microsoft's forthcoming cross-language technology platform. In my interview with Mark Hammond, Finn Bock, and Barry Warsaw, I found out more about how JPython and Python for .NET were developed and what's in store for the future of these alternative Python implementations.
Mark Hammond is familiar to most Python programmers because of his excellent development of the PythonWin environment and the PythonCOM extensions. And for the same reasons that we look up to Mark, Microsoft also looks up to Mark. They decided to contact him for help on their implementation of Python for .NET. According to Mark, a working version of Python for .NET should be available Real Soon Now and you should already be able to get an alpha or beta version from ActiveState (see Resources).
David Mertz: Exactly what is Python for .NET? I guess I'm especially curious about how Python for .NET relates to your own PythonWin and PythonCOM extensions to CPython, which already seem to enable control of Windows' guts.
Mark Hammond: Python for .NET is a compiler and runtime that implements Python on the Microsoft .NET platform. The .NET platform provides a runtime and metadata system designed to allow complete language interoperability, but to achieve that, the languages have to work in that runtime.
For example, if a Python class is made public so that a Visual Basic programmer can inherit from it, the Python class has to be implemented and described in .NET terms, not in CPython terms.
The advantage of Python .NET is simply that you can interoperate with the .NET framework. There are still a lot of disadvantages here, mainly due to the immaturity of the implementation. But it's really only a matter of time. We're still in the beta stage in terms of development and fine-tuning.
Mertz: What do you think the incompatibilities are right now between Python for .NET and CPython?
Hammond: Well, most modules haven't been implemented yet, so existing modules written in C definitely can't be used. If you aren't targeting the .NET framework, you probably wouldn't want to use Python .NET at this point.
Mertz: Nonetheless, there are definitely some major advantages of Python for .NET, like easy inter-language communication and multi-language application development. But why would you say it's better (hypothetically of course) than what's already out there -- Python+C+SWIG, for example?
Hammond: Well, as far as Python+C+SWIG goes, it should be obvious. Inter-language calling should never be as hard as it is with Python+C+SWIG. But SWIG is a great product in a lot of other respects. It's taken writing Python C extensions from a black art and moved it into the merely difficult range!
Comparing .NET with COM or Corba would be more reasonable. COM and Corba both offer a solution in which cross-language calling "just works," without any handcrafting or compiling. .NET takes this one step further, and offers cross-language inheritance and exception capabilities, among other things. These advantages are very similar to what you would find with multiple language implementations under the Java Virtual Machine.
Mertz: Python for .NET compiles Python scripts to the format of an external Virtual Machine. Do you have any early guesses about whether the .NET VM will support some of the exotic features of Stackless and Vyper, like continuations, generators, coroutines, tail-recursion, or lazy-evaluation?
Hammond: Yes, theoretically it will. But the terms of the Microsoft Beta agreement prevent me from talking about performance. Let's say we'll only be targeting features defined in the core Python language reference. Garbage collection is inherent, just like it is in JPython and JVMs.
Mertz: Moving on to the topic of politics, why do you think Microsoft is working on Python for .NET's development?
Hammond: So that people who choose the target .NET can do so using Python. Microsoft made sure to involve itself with Python and a lot of other languages in the early days to make sure that their VM really was language-agnostic. Now they've made a number of changes to their VM, based on feedback from the various language implementers.
Mertz: So what's their financial relationship to Python for .NET? Do you pay them? Do they pay you??
HammondGreg Stein and I have a contract with Microsoft to build Python for .NET. The terms of that contract remain confidential. But I'm basically working for ActiveState (who did the Perl for .NET implementation). I expect they'll eventually get a similar contract with Microsoft for the completion of the port.
Mertz: What does this mean in terms of the license that Python for .NET will carry?
Hammond: It'll carry a "(c) Microsoft" note, but it'll definitely be freely available.
Mertz: I can't help but worry that Microsoft will try to use proprietary extensions and enhancements in a familiar "embrace, extend, extinguish" strategy. In other words, I fear that Python for .NET might not actually be very good for Python in the long term.
Hammond: Well, if .NET becomes a significant force, then having a Python implementation that targets it can only help Python, much in the same way that JPython helps Python by targeting the JVM.
Barry Warsaw and Finn Bock are currently the two most active JPython developers, although it's really a very community-oriented effort. Unfortunately Jim Hugunin, the original JPython developer, is no longer able to be actively involved with it. The unabridged (and more technical) version of this interview is available at my Web site (see Resources).
David Mertz: What exactly is JPython?
Barry Warsaw: I'll give you my standard marketing speech:
JPython is a 100% Pure Java implementation of the Python programming language. It allows users to compile Python source code to Java bytecode, and run the resulting bytecode on any Java Virtual Machine. It's a very seamless and smooth integration with Java. From Python you can access all Java libraries, build applets, integrate with Java beans and subclass Java classes in Python and vice versa. Like Python, and unlike Java, JPython can be used interactively; just type some JPython code at the prompt and see the results immediately.
In more simple terms, JPython lets you script any Java code you want, which translates into two to ten times fewer lines of code. Because Python is a dynamically typed language, you can develop applications much faster, with many fewer bugs, and end up with a much more flexible program.
Mertz: How did development on JPython get started?
Warsaw: Well, JPython was invented by Jim Hugunin, who's now working for Xerox PARC's Aspect Oriented Programming project. Knowing Jim, I think he was probably primarily interested in the challenge. A lot of people in the Python world just didn't think it could be done. Guido himself was one of the skeptics. And Jim proved them wrong!
So why continue to develop JPython now that the challenge has been met? Because it's the most valuable Java tool that most Java programmers don't know about. Yet!
Mertz: What do you think spurred the need for JPython?
Warsaw: First you have to understand that JPython isn't a competitor to Java; it's the perfect complement to it. Java is a statically typed, compiled language. This ensures type-safety of libraries and faster execution speeds. It's interesting that although it's bytecode-interpreted, most people see Java as a traditional write-compile-run-edit program. And of course, Java leverages a huge segment of the software world, so there are a lot of resources available to the Java programmer.
But the same static typing and traditional programming cycle increase the cost of Java application development in terms of human resources. And here, Python absolutely excels. Because Python is such a simple and small language, it's very easy to learn. Most experienced programers can learn enough Python to be productive in about a day. And Python was designed with the idea that code is read a lot more than it's written. So Python source code is easy to share in a large team project.
But more importantly, Python is a very high-level, dynamically typed language. This translates to a huge savings in the amount of code needed to perform a task. Because I can write fewer lines of code using Python, I can write them faster and have fewer bugs. This is wonderful for rapid application development.
Python also provides an interactive interpreter, which means that you can sit at the interpreter prompt, import Java code, create Java class instances, make method calls, etc. all interactively. This is a wonderful tool for training programers how to use corporate Java libraries, or for experimenting with new Java APIs.
But in my humble opinion, all programers should have both CPython and JPython in their arsenals.
Mertz: What would you say JPython's advantages are over CPython?
Bock: JPython gives complete access to its underlying implementation language. In most (probably all) C-based script languages, a C function has to be wrapped in a thin layer of code that serves to expose the C function to the script language, and fine tools like SWIG exists to automate the creation of this wrapper code. But JPython doesn't require the wrapper in the first place. All Java code ever written is directly available for use from JPython, and the integration goes both ways. Classes and instances defined in JPython can be passed to Java code as if they were ordinary Java classes and instances (which is exactly what they are).
The embedding/extending API makes access to JPython objects from within an application or module quite elegant. This beauty comes in part from the fact that JPython and Java are both object oriented languages. Jim took great advantage of this fact.
Warsaw: What CPython lacks is access to the vast quantity of Java code out in the world. If there are Java libraries you need to use, JPython is the answer. Conversely, of course, JPython doesn't have easy access to any existing C libraries out in the world. Finn has done work integrating things like Tkinter and a POSIX module through JNI, but those will always be non-standard in JPython because we want to retain the 100% Pure Java certification.
Mertz: So how would you calculate JPython's weak points?
Finn Bock: JPython only gives access to Java code, not to any of the existing C modules. So every Python module implemented in C must be re-implemented in Java. And CPython has quite a lot of modules.
Also, there is almost no documentation for the embedding/extending API other than the source code.
Mertz: Are you finding any advantages with JPython over pure Java?
Warsaw: I think we've covered most of that already. But let's talk about performance issues with JPython. Since JPython implements Python's dynamic semantics, there's a fairly extensive runtime that comes with JPython. This can have a performance impact on some applications. Standard Java optimizations such as just-in-time compilers and Hotspot technology can mitigate those considerably (benchmarks eight months ago showed that with a JIT-enabled JVM, JPython 1.1 could approach and sometimes surpass CPython 1.5.2 speeds). We'll be updating those benchmark results, and concentrating on performance issues after we roll out JPython's successor (more on that below).
But in analogy with CPython, you can always rewrite performance critical sections of your application in Java.
Mertz: How widely used do you think JPython is?
Warsaw: I think it's becoming more and more widely used. People are finding that it's critical to technological success. JPython is valuable for all kinds of tasks, from providing an approachable scripting environment for end users to making it easy to create testing frameworks for Java libraries and applications. JPython's biggest disadvantage at the moment is that it needs more publicity. I hope this article will help in that department!
Mertz: Do you think JPython is an attempt to keep up with CPython?
Bock: Yes. Right now, JPython is trying to catch up. Almost all new features are added to CPython first. (Well, JPython did have string method before CPython did). JPython has a disadvantage here because CPython has fifteen times as many core developers as JPython does. But even so, a JPython version exists for almost all the new features in CPython 2.0.
But I think actually they're pretty equal, even though in the real world, one is a little more equal that the other.
Warsaw: I firmly believe that at the language level, JPython and CPython ought to be completely compatible. In cases where this is impossible, Guido decides whether the differences are implementation dependent, or whether one or the other implementations are "buggy". I would like to see CPython and JPython become coequal eventually, with JPython pushing CPython development in certain directions as much as CPython pushes JPython.
One current example of this is in Unicode support. JPython is already all-unicode. Another example is the types/class dichotomy. In CPython, you have built-in types like strings, dictionaries, lists, and numbers. You also have classes and instances. The built-in types can't be inherited from. And to add more confusion, an instance has both a type and a class. It may be easier to fix this rift in JPython first because of its object-oriented implementation.
Mertz: What do you think the incompatibilities are between JPython and CPython?
Warsaw: Well, there are a lot of small differences in the way things work. They're all outlined in the JPython documentation. Again, some are classified as acceptable differences given the language definition, and some point out places where one or the other implementation should be fixed. Most are quite minor.
Bock: Some modules haven't or can't be implemented in JPython. Some modules can only be implemented as JNI modules and as such may not be useful in the deployment environment.
Bock: Actually, when I ported my own script and programs (along with IDLE, PySol, and the PMW toolkit), the problems I encountered were not the random reclaiming of the garbage collection or the missing _del_ method. They were minor things that no one else had encountered before, such as in CPython behavior.
Warsaw: The next version of JPython will be compatible with the Python 2.0 language definition, so the biggest gotchas will be in the libraries. Any of the standard library modules from the CPython distribution that are written in pure Python should be portable. C extension modules won't be, unless they are specifically integrated through a JNI bridge or re-implemented in Java. And any JPython application that extensively uses Java APIs will have a hard time porting back to CPython.
On the other hand, there is a lot of common functionality in the libraries of the two systems. With sufficient foresight, you can build compatibility layers into your application.
Mertz: Any thoughts on future directions for JPython?
Warsaw: Well, we've created JPython's successor, "Jython", based on the public JPython 1.1 release. We did this to ensure the longevity and stability of the project. It was all done in accordance with CNRI's JPython 1.1.x license. We've moved the entire development process to SourceForge, and we'll manage it using the same kind of open process that's working so well for CPython. Finn and I are obviously intimately involved in Jython's future development, which will be released with the OSI-approved CPython 2.0 license. This is as close to an "official" fork as you're going to get, so the current JPython community should be confident that Jython won't ever go away. We're hoping they'll all eventually migrate over to Jython.
Right now, the code is still in alpha phase, but Finn and I will be working on several technical milestones for the Jython 2.0 release, which already contains Finn's errata. CPython 2.0 has features such as augmented assignments and extended print (with list comprehensions coming soon). We've integrated the free Apache Jakarta OROMatcher code, eliminating the need for dual licensing, and we've fixed lots of bugs. I don't know when the first alpha release of Jython 2.0 will happen, but all the code is currently available in the SourceForge CVS tree.
- Read the previous installments of Charming Python.
- Go to the Python home page.
- Or better yet, visit the Jython home page.
- Check out Visual Python, the Python plug-in for Visual Studio .NET.
- Read comments by Mercury developers on just what the .NET platform is and on Microsoft's financial and legal relationship to developers of "little languages".
- Read extra comments on this interview.
- Find out about the Ninth International Python Conference next March.
- Check out the Java Lobby and their Java Foundation Applications Project.
- Go to the Blackdown Organization, home of the Linux-Java porting project.
- Read JPython: the Felicitous Union of Python and Java, an excerpt from Learning Python (O'Reilly, 1999).
David Mertz writes many apocopetic articles. He may be reached at mertz@gnosis.cx; his life pored over at his Web site. Suggestions and recommendations on this, past, or future columns are welcome.
