JVM as platform for dynamic languages?

Martin Fowler recently blogged about how to choose between the emerging JRuby and Groovy scripting languages on the JVM. He pretty much ignores both Rhino, and Jython, which is strange since Jython is the grand-daddy of dynamic languages on the JVM, and Rhino seems to have the most internal support from Sun, and has been getting some serious press in the blogosphere. And I think his failure to address Jython, and Rhino reduces the value of the article.

But he does, finally, at the end of his article, he get around to a quick mention of Jython in the context of asking a bigger question:

Will either matter to Java? After all Jython’s been around for a long time without making a huge impact on the JVM. Tool support is frankly pathetic for any of these languages when you compare it to what you have for Java at the moment.

This question is skating right around the critical issue, without ever touching it directly.

Why is there so little uptake for dynamic languages on the JVM? And what will it take to change that?

Another way to ask the same question would be: We have a lot of deep experience with C integration in the Perl/Python/Ruby communities, but there is no such deep experience with the JVS as a platform for scripting languages. Why?

To sharpen the question a bit further, we should note that the Jython/java integration story is in many waysbetter than the Python/C story. In Python 2.5, c-types makes using C libraries from python easier than ever — but you don’t have to do anything special at all to use Java libraries from jython. In spite of this, Jython and all the other new “dynamic languages” on the JVM have not become a popular way to work with Java libraries.

Why?

I’m going to address this from the perspective of Jython user, because that’s the JVM/dynamic language experience that I have. But, I’ve played with Jruby enough to know that the issues are similar.

One problem that seems to stop python programmers from flowing over into Jython easily is the impedance mismatch between the way things are done in Python and the way they would be done in Java. As a python developer you find yourself constantly switching mental gears to write anything complex in Jython. You’re thinking about the python code you’d write, and things sort of flow until you run up to something in a java library which requires you to look at the world in a fundamentally different way.

Of course, the same is could be said to be true of Python and C. Sure, you could argue that C programmers and python have more similar pragmatic approach to library development, but I don’t think that’s the main difference. The main difference is that developers tend to “wrap” C libraries and provide “pythonic” bindings which reduce the impedance mismatch. Nobody seems to do this in jython, partly because it’s just too easy to ignore the need for good pythonic API’s and just get something done. It seems that easy is sometimes the enemy of good — which is definitely not news to any experienced programmer.

But, perhaps there is another possible reason, is that the JVM was written explicitly for java, and is not particularly friendly to dynamic languages. Reflection is slow, and in general features used by dynamic languages haven’t been a priority for the JVM. Fortunately there’s some work happening already to make the JVM a much, much friendlier place for dynamic languages. But I don’t expect that Jython is going to be faster than C Python any time soon. So that, means that the JVM as a platform needs to offer something more than just a good way to implement python. I’ve heard some rumblings from the Jython team that this may not be a significant factor in the future, but it’s certainly colored people’s perceptions in the past.

This brings us back to the fundamental nature of the JVM as a viable platform. It’s one thing to have languages that run on the JVM, and call that the platform, but the language and the managed runtime environment are only part of the picture — in my experience the platform is defined as much by it’s libraries as anything else.

I think this is the main problem that the JVM will face as it evolves into a multi-language platform over the next few years — the java standard library is complex in unexpected ways (how do you open a file and process it line by line — I always have to look it up?) and that makes it difficult to use as the foundation for a vibrant scripting platform.

If however people aren’t working with java libraries, you’re just running Jython on the JVM, and all you get is a slower slightly more out of date version of Python, with fewer libraries, and that’s not a very compelling case. Perhaps you need to run on the JVM for political reasons, or you need access to one of the java libraries that don’t have an obvious python equivalent, like i-text or lucene. But except for those relatively narrow use cases, why wouldn’t you stick with CPython?

The Microsoft threat

Microsoft intentionally has worked to create libraries that make sense for C# users, but which are accessible from languages like VB.net, and IronPython. Thus they have created an alternative Python platform which is much more interesting than Jython. And to be successful in turning the JVM into a multi-language platform I think something similar needs to happen to Java.

At least right now, the situation the way that Groovy, Jython, Scala and JRuby have been implemented makes it unclear how well you’ll be able to integrate libraries written in one of these languages with libraries from another. Effectively this places Java libraries at the center, and relegates dynamic languages to “second class” status. At this moment, writing libraries in Jython that would be in an attempt to make them usable to Jruby and Groovy folks seems like a fools errand.

So, having a common core of reusable libraries written in Java seems like a key ingredient in making the JVM a better platform. These libraries would have to be built:

* with multi-language reuse in mind
* in ways that don’t require lots of boilerplate code
* to work well with dynamically typed languages

That kind of standard library would benefit allof these new players on the JVM. And that’s what I think Sun ought to invest in if they want the JVM to become a platform that JRuby, Jython, Scala, Groovy, and whatever else comes next, can thrive.

Another Microsoft Threat

At the same time, Microsoft has been widening the gap in another way, by introducing the DLR — a new set of tools for dynamic language implementors.

Much of the thinking behind the DLR comes from Jim Hugunin (the original implementor of Jython), and if it works the DLR will help to make dynamic languages first class citizens on the Microsoft CLR since you will be able to use them to write libraries that are cross-compatible, making IronPython libraries usable in IronRuby, etc.

The aforementioned project to make the JVM more friendly to dynamic languages is a good place for language implementors to work together. And it would be very interesting to see the jython, jruby, and other language implementors get together and provide a good solid answer to the DLR. I’ve been talking to some of the Jython guys this week, and they tell me that this is happening, and that a lot of the work is already done.

Given the politics of language communities, and the necessities of the Open Source development model, I’m sure this has been more difficult than it could have been, so I’m very excited to hear that the people involved have taken the time to make that happen. I haven’t seen a lot of talk about this on the internets, but it looks like people are quietly doing the right thing without running the hype machine in overdrive which is refreshing.

After talking to the Jython guys this week, I’m actually pretty optimistic about the JVM. It’s growing in a direction that provides a solid memory-managed platform for the dynamic languages of the present and future, and it could become a platform for collaboration between language developers.

The CLR could do the same thing, but I want to see it happen on the JVM, because that would make it free in all the important senses of the word.

22 Responses to “JVM as platform for dynamic languages?”


  1. “One problem that seems to stop python programmers from flowing over into Jython easily is the impedance mismatch [...] You’re thinking about the python code you’d write, and things sort of flow until you run up to something in a java library which requires you to look at the world in a fundamentally different way.”

    Interestingly I *haven’t* found the same thing to be true when using .NET libraries from IronPython. Although there are some areas where .NET makes you jump through a few extra hoops, *largely* the .NET libraries seem sane and clean (static typing aside).

    I wonder if the Java standard library is worse or if the integration with Jython isn’t so good?

    Michael

  2. I think the problem with Jython is that the JVM is a very unappealing platform. If I want to serve up some web application and I could choose a self-hosting server, something like mod_python or mod_wsgi, or a JVM container… well, the JVM container is so far out that it would never enter my mind. Places where they use the JVM seem modestly interested in this (not interested enough to actually make it work well!), but no one else would ever go that direction.

    I haven’t seen any compelling arguments for the JVM platform. The configuration and setup is awful. Package management doesn’t seem particularly good (not that Python alone is great, but the JVM doesn’t seem *better*). Its performance looks good in microbenchmarks, but I don’t trust its macro performance — the benchmarks all conveniently leave out stuff like startup time, and worse-case scenarios. CPython isn’t fast, but it’s fairly consistent, and if I wanted to improve the platform I’d want something *more consistent*, not faster. I want less warm-up in a platform, the JVM just gives me more warm-up.

    Is the JVM an easier environment to debug in? I find that unlikely. Is it easier to navigate the system? Is it less prone to heisenbugs? Does it use less memory?

    I think the JVM is just a bad platform for most tasks (especially tasks where Python is appealing), and entirely the wrong direction for Python, and for programming in general. We need a new platform, written with open source sensibilities instead of corporate sensibilities.

  3. “We need a new platform, written with open source sensibilities instead of corporate sensibilities.”

    LLVM, Tamarin, Parrot, Strongtalk ? (Dare I suggest Mono…)

  4. Interestingly JRuby is proving a *great* success. Ruby itself has performance and deployment issues that the JVM platform alleviates and many Ruby programmers are refugees from Java anyway…

    Perhaps the lack of success for Jython is more historical/cultural – most Python programmers *chose* Python *instead* of Java. IronPython is seeing refugees from C# and (in a limited way) is proving popular…

  5. You essentially ask, “Why aren’t CPython programmers interested in Jython?” I think this is the wrong question. Jython doesn’t offer much to a CPython programmer. Instead you should ask why Java programmers aren’t interested in Jython? IMO Jython offers a huge boost in productivity and convenience while leveraging a Java programmer’s knowledge of standard, third-party and proprietary Java libraries. In spite of this, I have found it nearly impossible to get Java programmers to look seriously at Jython.

  6. The original Martin Fowler article:http://martinfowler.com/bliki/GroovyOrJRuby.html

  7. Kent, good catch. I’d thought I’d included that link in the article, but I guess I somehow missed it.

    I also agree that Jython ought to be a more attractive way to work on the JVM than Java. But I think it has to also offer a compelling case over abandoning the JVM for Cpython altogether — and most of the java expatriots that I know didn’t leave java for jython, they left for cpython or ruby.

  8. I have a colleague )a Ruby fan) who uses JRuby over CRuby because of a library for parsing chess files that was better than any Ruby libraries available.

    Similarly, the ‘killer’ feature for IronPython is access to .NET libraries (many of which are very good – better multi-threading support because of the lack of the GIL doesn’t hurt either mind).

    Java programmers who like Python but want to be able to use a familiar set of libraries should find Jython appealing. Again I wonder if the Java/Jython integration is less appealing than the IronPython/.NET integration?

    Jython *is* widely used don’t forget – I know of several companies who use it for working with a Java infrastructure – perhaps not as successful as it could be though.

  9. Ian,

    You make a lot of good points. I think there are some other advantages of the Jython that are important. The lack of a Global Interpreter Lock is one. The ability to drop down to java even more easily than you can drop down to C in cPython is another. And direct access to things like Lucene is the big one.

    Still, most of the time, for most purposes I think Cpython is easier, faster, and more stable. At least at the moment.

    And, I wonder if the benefits you get from totally owning your data representation in C doesn’t make dropping to C for performance stuff a lot more of a win than dropping to Java would be.

    But I do want the JVM to be successful, because if it doesn’t I think the default VM will be the CLR, and that’s much slower on non Windows platoforms, and is Microsoft propritary, which is a significant drawback in my mind.

  10. Michael,

    I do think, from my limited experience with the .net libraries, that they are easier in general to work with than many of the java equivalents. However, it does seem that there are some issues with the web dev libraries in particular which I find frustrating. I also find the .net toolchain a bit restrictive.

    That said, the GUI stuff in .net is way better than anything the java people have come up with yet. Perhaps JavaFX will turn it around, but I don’t hold out much hope on that front.

    Heck, I wish there were really good gui development tools in Python. I find myself building webapps rather than GUI’s even for local administration stuff, just because it’s easier.

  11. For working with .NET we use Python and ‘normal’ Python tools (PyLint, Wing IDE, Vim etc).

    We use Visual Studio for designing our dialogs though – with the visual designer. We generate C# so that no-one is tempted to edit the generated code…

  12. And as for webdev libraries in .NET! Ha! I’m afraid I have little respect for ASP.NET. The new MVC ASP.NET framework looks pretty good though (as rthese things go).

    Although people have got Django running with IronPython I can’t see why you would want to. (Deployment in a Microsoft only corporate environment perhaps?)

  13. Michael,

    it seems to me that if the ASP.NET libraries are a mess, the presence of Django would on .NET would be a significant reason to use IronPython when you want to expose .NET stuff over the web.

    Sure, if you’re doing something from scratch it makes sense to do it in TurboGears on Cpython ;) But if you’ve got a large .NET codebase and you need a web UI it seems like Django on IronPyton makes sense.

  14. Right – Django on .NET might make sense for .NET programmers moving to Python – but not for Python programmers interested in .NET.

    As getting Django working *smoothly* on IronPython is likely to take a decent amount from some experienced Python programmers it probably won’t happen unless someone ponies up some cash.

    OTOH – if it can be made smooth, maybe it makes Python all the more compelling for the .NETTIES…

    (ASP.NET’s problem is not so much the libraries – although webforms both sucks and blows big rocks apparently – but the fundamental premise of the framework is old and useless, so shoehorning decent architecture into it is painful… It does have some advocates though, but not many.)

  15. 15Paul Boddie

    With regard to Lucene, I find PyLucene to be more than sufficient for my purposes. In fact, I’d rather use the version of PyLucene which uses SWIG and gcj than the version which uses JNI and a JVM, mostly because I then don’t need to consider all the stuff that can go wrong with a JVM.

    That said, I think that people trying to integrate CPython with JVMs should take a look at what the PyLucene people have been doing – I never got things like JPype to work, but given that recent PyLucene releases are routinely built for JVM integration, it would suggest that they’ve perfected the art.

    As for Jython, I think it was a remarkable achievement when it was done, especially since people did wonder whether it would make CPython obsolete, and it’s very useful if you have to use the Java platform. Really, the Java platform is the problem: one always gets the impression that it demands centre stage (reinventing a lot of what the operating system does for you already), has baroque libraries and frameworks, and encourages sluggish software.

    Until last year when someone asked about it, I wasn’t really that bothered in giving WebStack much exercise on the Java platform (beyond the initial confirmation that it worked) precisely because it’s such a chore to test anything in a servlet container: start the thing slowly, wait, re-deploy the application, hope that nothing got cached anywhere, test, find that something did get cached, stop, clean out, fix, start, wait, re-deploy, and so on. Only something like Zope 2 would give a similar experience, and even then it’s nowhere near stuff like Tomcat in terms of time and memory consumption (let alone other more bloated offerings).

  16. Azureus has to be the slickest Java application I’ve ever used. A desktop application with a *great* user interface and cross-platform to boot. Creating good applications with Java is certainly possible…

  17. Hmmm… final thought.

    JRuby is backed by Sun and is [seen as...] successful
    IronPython is backed by Microsoft and is [seen as...] successful

    Probably as simple as that.

  18. Michael,

    I suppose that’s true, but the Jython team’s been doing some remarkable work in the last year, with a new release, and 2.5 compatibility coming “real soon now.” So, I at least think 2007 should be seen as one of the most successful years ever for Jython, and it seems like 2008 will be even better.

  19. Right – I think I was wrong to wonder about the Java/Python integration in Jython, I’m sure it’s fine. With Rails on JRuby working so well (apparently) I’m glad that there is an active push for Django on Jython.

    It would be nice to see Django on IronPython as well – even though I don’t do much web development at the moment. (We do have a Resolver One web server, which currently uses our own ‘hand-rolled’ server framework and it would be nice to use Django with this even though our server isn’t a traditional database backed application.)

  20. One question that I have is why does the average java programmer need “tools” to do anything. Are they all stupid and crutch bound to eclipse or other editors that write things for them? I have been using jython for a while and love it. I can move from java to jython without any issues. When jrunscript appeared with the jdk I was then able to use javascript like I had wanted to for a long time.

  21. tonetheman,

    Here’s an interesting post on that subject:

    http://osteele.com/archives/2004/11/ides

    I think some people are cripled by the idea that there’s one true tool for the job. Grep, sed, awk, vi, and bash make me think that even language oriented people are kind of tool oriented, but they are just into small tools that can be re-used many creative ways.

  1. [...] Ramm-Christensen posed some questions about using the JVM as a platform for dynamic languages. Many people do, in fact, use dynamic languages on the JVM (Groovy, Beanshell, Rhino, Jython, JRuby [...]

Comments are currently closed.