Method Handles & invokedynamic – on by default

No Comments

Quite a significant milestone today – The change request to switch on Java 7′s MethodHandle and invokedynamic features by default is in. Once this propagates into the OpenJDK repositories, we will see one of the largest features of Java 7 finally land ready for use.

The details of the CR are here.

MethodHandle and invokedynamic are covered in a brand-new chapter of the book “Classfiles & Bytecode” – which will be coming to a MEAP near you very soon.

Did you like this? Share it:

Project Coin: the try-with-resources lock support debate

6 Comments

If you have recently been following Project Coin’s mailing list , you will have spotted an interesting discussion regarding the inclusion of lock management inside the scope of a try-with-resources (TWR) code block.

The Initial Idea

The idea was initiated by Gernot Neppert who proposed a class java.util.concurrent.AutoLockable which would implement AutoCloseable, and therefore be eligible for usage with TWR:

package java.util.concurrent.locks;

public abstract class AutoLockable implements AutoCloseable
{
  public static AutoLockable locked(final Lock lock)
  {
    lock.lock();
    return new AutoLockable()
    {
      @Override
      public void close() {
        lock.unlock();
      }
    };
  }

  public abstract void close();
}

Its usage would be something like this:

try(AutoLockable locked = AutoLockable.locked(lock))
{
  // Do something in locked scope
}

This proposed idea certainly has merit, however the sharp eyed amongst you will notice a performance issue in creating a new object each time you perform a lock. Additionally, you would also have to have an extra method to acquire the lock (the locked(...) method in the above example). So what else could be tried?

An Alternative Idea

Dr Heinz M. Kabutz posted a link to a newsletter where he describes an interesting way of using JDK 7′s TWR to automatically unlock Java 5 locks. Long story short, the main idea is to build a wrapper class around a Lock, which implements AutoCloseable and manually unlocks in the close() method. The wrapper along with usage of static imports would bring verbose code like:

lock.lock();
try
{
  printLockStatus();
}
finally
{
  lock.unlock();
}

… to a more readable form:

try (lock(lock))
{
  printLockStatus();
}

However, as the author himself mentioned, the problem of creating extra objects every time you perform a lock still remains. Moreover, as David Holmes signaled in the mailing list, the spec has been updated to prohibit usage of resources as general expressions in the TWR argument and only allow passing them with an accompanying explicit variable declaration.

Why TWR (as it stands) can’t support this

Among the reasons for this update were difficulties encountered by the parser which was not always able to distinguish between the start of an expression and the start of a type:

try(i < j // Ternary operator on variables i and j
                 ? new Resource1() :
                 new Resource2()) {...}

… compared to code like

try(Box < Resource // Simple generic wrapper around a resource
 > resourceBox = Box<>(new Resource1())) {...}

Another reason for change was the use case of managing an existing variable which changes its value inside the try block. The example presented on the update site is the following:

public class TwrExamples implements AutoCloseable
{
  public static void main(String... args)
  {
    TwrExamples twrEg1 = new TwrExamples();
    System.out.println(twrEg1.hashCode());

    try(twrEg1)
    {
      twrEg1 = new TwrExamples();  // Mutating the variable!
      System.out.println(twrEg1.hashCode());
    }
  }

  @Override
  public void close()
  {
    System.out.println(hashCode());
  }
}

.. where close() will be called on the first twrEg1 instance, not on the one pointed to at the time the try block finishes. Thus, output results after running a TWR having a TwrExamples resource as argument, may look like this:

1607576787
1051296202
1607576787

The documentation of the Project Coin features posted on March 1st 2011 by Joe Darcy further strengthens the rule of requiring variables to declare a resource in detriment of using general expressions:

“A resource specification declares one or more local variables; the type of each variable must be a subtype of AutoCloseable or a compile-time error occurs.”

However, assigning a new variable to an already existing one referring to the lock just because TWR prohibits identifiers seems troublesome. There is a section in the JSR 334 spec changelog stating that:

“One potential future extension is to allow a resource to be specified as an expression that is a final or effectively final variable. Such a restricted expression form would remove the need to declare a resource variable for the sole purpose of aliasing an existing variable while avoiding pathologies stemming from the resource variable being modified within the body of the |try|-with-resources statement.”

Joe Darcy states in the mailing list that even though this change is too late to be applied to JDK 7, it would be a nice addition to JDK 8.

Some Extra Analysis

Another heads-up in this direction is related to constructs such as try (Resource r = getResource()), where methods such as getResource() are not expected to fail. Some workarounds are proposed by Reinier Zwitserloot in coin-dev, for example not considering the expression Resource r = getResource() as part of the try, and do something like this try (Resource r = ...) { try { ..... }} . Another solution would be setting r to null if the exception that triggers the catch block is resulted from getResource() method. The example from the mailing list is :

try (Resource r = getResource())
{
  /*.... */
}
catch (Exception e)
{
  icon = Icons.DEFAULT_ICON;
  logger.log("App icon resource " + r.getURL() + " is not available", e);
}

In this case we would prefer obtaining a NullPointerException in the log, rather than a core dump.

Your Thoughts?

We’re curious to find out any opinions the readers of this blog might have regarding this ongoing topic discussion on coin-dev. Do you think automatic TWR lock support might be useful to you?

Did you like this? Share it:

Conservativism in Language Design

11 Comments

One of the most interesting things about the development of Java 7 has been the open debate around language features.

However, one of the side-effects of that discussion has been a certain amount of visible frustration from some quarters about the pace of change. The process has been criticized for being too timid, too slow, too conservative by some people. There has also been rather a lot of vitriol over the scope or non-inclusion of specific features.

Language design is hard. It’s hard to think about all possible use cases for a feature up front. It’s even worse to try to think about how your language features will interact. Very smart people get this wrong. A lot.

Now stop and think for a minute about how successful a language Java is. It’s everywhere. There are at least 1 billion Java runtimes in the world today. Millions of companies deploying Java technology and 7-8 million Java developers worldwide. Stability and backwards compatibility is a huge deal.

Consider a largeish company with, say 100-1000 developers, which is introducing a new technology platform. Empirical / anecdotal data suggests this platform will probably be in use for 7+ years. Once enterprises start using a technology, it can be very difficult to get it out again. That same company will probably also spend around $2-3 million on tooling, support, etc over the lifetime of the platform within the company. Multiplying that up by the tens of thousands of such companies that exist, gives some idea of the dollar value of the corporate Java ecosystem.

Putting all of this together, and we can easily see how bad it would be if a misfeature in the language design was allowed to escape into the wild. Once a language feature is out there, the Java community basically has to live with it forever.

This is why the caretakers of the Java language tend to take the view that Java is not the right place for language feature experimentation. Every new feature interacts with the existing “feature surface” of the language, and often in surprising ways. The place we see this the clearest in Java is with the type system.

Looking at the changes in Project Coin, the new multicatch and diamond syntax features are the places where the type system is most deeply involved, and in the formal spec for each contains a large screed of type theory.

Once those features are out in the wild, any further change will now have to deal with the larger amount of type theory that now forms part of the language standard. This acts as a constraint on further language innovation.

All of this leads to a situation whereby change has to proceed slowly and calmly. Features aren’t added lightly – and the engineers who create them aren’t idiots.

Finally – if a new feature isn’t exactly as you expect, then stop, think, take a deep breath. Think about the reasons why the feature may not be precisely tuned for your use case – and about the inevitable compromises that the new feature may represent.

Language design, like legislation, can be more about tradeoffs, compromises and the art of the possible than is sometimes recognised. When combined with the engineering and project management constraints common to all software projects, the fact that we manage to get important new features out at all starts to look like a triumph.

There’s always scope for improvement, and the JCP could use more individual end user and JUG members. If you have strong opinions about new features, or want to be more involved – then this is a great time. With Java 7 just around the corner, and the work already underway for SE 8 and EE 7, this is your chance to make your views known. Early feedback from end users on proposed features is highly prized – you just need to get involved and start contributing.

Did you like this? Share it:

At TSSJS 2011

No Comments

Martijn and I are at TheServerSide Java Symposium in Caesar’s Palace this week.

We’re at this afternoon’s Meet The Authors session, and we have talks tomorrow (Back to the Future with Java 7) and Martijn’s Diabolical Developer talk on Friday.

If you’re around, do come and say Hi.

Did you like this? Share it:

A glimpse at MethodHandle and its usage

3 Comments

MethodHandles are covered in full in our The Well-Grounded Java Developer title

Due to Java’s Reflection API we have been able to inspect and alter program execution at runtime. In particular, we can observe interfaces/classes/methods and fields at runtime without knowing their names at compile time.

JDK 7 introduces a new player to this dynamic/runtime inspection, the method handle (i.e. a subclass of the abstract class java.dyn.MethodHandle). Method handles gives us unrestricted capabilities for calling non-public methods, e.g. it  can be formed on a non-public method by a class that can access it. Compared to using the Reflection API, access checking is performed when the method handle is created as opposed to every time the method is called.

Suppose we have a class which needs to allow controlled access to one of its private methods. Below is the class defining this method and describing two ways (Reflection / MethodHandle) to access it.

public class MethodAccessExampleWithArgs {
    private final int i;

    public MethodAccessExampleWithArgs(int i_) {
        i = i_;
    }

    private void bar(int j, String msg) {
        System.out.println("Private Method \'bar\' successfully accessed : "
                                          + i +", "+ j +" : "+ msg +"!");
    }

    // Using Reflection
    public static Method makeMethod() {
        Method meth = null;

        try {
            Class<?>[] argTypes =
                new Class[] { int.class, String.class };

            meth = MethodAccessExampleWithArgs.class.
                            getDeclaredMethod("bar", argTypes);

            meth.setAccessible(true);
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        } catch (SecurityException e) {
            e.printStackTrace();
        }

        return meth;
    }

    // Using method handles
    public static MethodHandle makeMh() {
        MethodHandle mh;

        MethodType desc = MethodType.methodType(void.class, int.class, String.class);

        try {
            mh = MethodHandles.lookup().findVirtual(MethodAccessExampleWithArgs.class, "bar", desc);
            System.out.println("mh="+mh);
        } catch (NoAccessException e) {
            throw (AssertionError)new AssertionError().initCause(e);
        }
        return mh;
      }
  }

Following, is a class meant to test the two approaches of accessing the private method “bar” :

public class MethodAccessMain {

    private static void withReflectionArgs() {
        Method meth = MethodAccessExampleWithArgs.makeMethod();

        MethodAccessExampleWithArgs mh0 =
            new MethodAccessExampleWithArgs(0);
        MethodAccessExampleWithArgs mh1 =
            new MethodAccessExampleWithArgs(1);

        try {
            System.out.println("Invocation using Reflection");
            meth.invoke(mh0, 5, "Jabba the Hutt");
            meth.invoke(mh1, 7, "Boba Fett");
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (InvocationTargetException e) {
            e.printStackTrace();
        }
    }

    private static void withMhArgs() {
        MethodHandle mh = MethodAccessExampleWithArgs.makeMh();

        MethodAccessExampleWithArgs mh0 =
            new MethodAccessExampleWithArgs(0);
        MethodAccessExampleWithArgs mh1 =
            new MethodAccessExampleWithArgs(1);

        try {
            System.out.println("Invocation using Method Handle");
            mh.invokeExact(mh0, 42, "R2D2");
            mh.invokeExact(mh1, 43, "C3PO");
        } catch (Throwable e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        withReflectionArgs();
        withMhArgs();
    }
}

How to run the code – JDK7 b129 and Netbeans 7.0 Beta 2

The above code has been tested with build 129 of JDK 7, under Netbeans IDE 7.0 Beta 2 and for it to work, it required the following extra VMOptions: -XX:+UnlockExperimentalVMOptions -XX:+EnableMethodHandles -XX:+EnableInvokeDynamic under Run >> Set Project Configuration >> Customize >> VMoptions in order to make use of InvokeDynamic and MethodHandle without receiving a runtime exception.

Issues with JDK7 b131 and Netbeans 7.0 Beta 2

If you’ve upgraded to the latest build – 131 – the outcome starts to be dependent of the circumstances under which you compile and run your code. More specifically, the Netbeans IDE 7.0 Beta 2 outputs the following:

run:
Invocation using Reflection
Private Method 'bar' successfully accessed : 0, 5 : Jabba the Hutt!
Private Method 'bar' successfully accessed : 1, 7 : Boba Fett!

java.dyn.WrongMethodTypeException: (ILjava/lang/String;)V cannot be called as ([Ljava/lang/Object;)Ljava/lang/Object;
mh=bar(MethodAccessExampleWithArgs,int,String)void
Invocation using Method Handle
    at ben.example.MethodAccessMain.withMhArgs(MethodAccessMain.java:46)
    at ben.example.MethodAccessMain.main(MethodAccessMain.java:55)

BUILD SUCCESSFUL (total time: 0 seconds)

Issues with JDK7 b131 and Eclipse 3.6.2

The example has been further tested on Eclipse 3.6.2 Helios with the same parameters -XX:+UnlockExperimentalVMOptions -XX:+EnableMethodHandles -XX:+EnableInvokeDynamic mentioned under Run>>Run Configurations>>Arguments>>VM arguments, but we have been provided with an identical output.

From the exception we have been prompted with, we can see that our method which was expected to be called with an int (I), a string (Ljava/lang/String) and return a void (V), is instead called with an array of Object, and returns an Object. This shows that there are some issues with the compiler used from within the IDE (which may be different from the command-line compiler).

Running the code JDK7 b131 on the command line

If you happen to have a post-129 JDK7 build on your computer and want to run the afore-mentioned examples without any problems, you should probably stick to compiling (javac) and running ( java -XX:+UnlockExperimentalVMOptions -XX:+EnableMethodHandles -XX:+EnableInvokeDynamic com.wgjd.MethodHandleExample.MethodAccessMain) your MethodHandle code from the command line.

Fix for JDK7 b131 and IDEs

Currently there is no set date of when we will be able to run method handles without the extra VMOptions overhead, but should you want to keep up with the latest developments of the Da Vinci Machine Project which concerns itself with the implementation of dynamic invocation, make sure you subscribe to its mailing list.

It is also worth mentioning that from the previously mentioned mailing list, we find out that there is a complex change which consolidates the multi-package code into a single package and consists of preparing for a clean rename from java.dyn to java.lang.invoke. This is required to fix some defects in the API which arise from dependencies between multiple packages.

MethodHandles are covered in full in our The Well-Grounded Java Developer title

Did you like this? Share it:

Modularity is Hard. Let’s do a Jigsaw.

12 Comments

Don’t forget: The 50% discount on the Early Access edition of our book expires tomorrow!

(Before we get underway, I want to make the point that all of this is based on my current understanding of the positions of both the OSGi folk and the current plans for Jigsaw. I’m not an expert in either, so there may well be egregious lies in this post. Informed comment from people who are an expert in either technology would be very welcome.)

Technologists don’t possess crystal balls. Early releases of technology platforms often have problems in them that are not ironed out (or even discovered) until much later.

One example of this is the Java platform’s lack of generalised modularisation and dependency management capability in the core. Sure, there are jar files, and as a basic means of shipping around units of functionality which are larger than a class, they’re OK.

However, when dealing with larger systems, the model of “just ship around a bunch of jars and manually manage the dependencies” starts to break down. What’s needed is a way to automatically manage the dependencies between functional units, via some metadata which is shipped within each unit.

This is where systems like Apache Ivy and Maven come from – they run outside of the core platform and automatically manage a repository of jar files. Build and runtime dependencies are then much easier to manage – all the jars are in one place. However, note that the platform doesn’t provide any support for these platforms out of the box – a classpath still has to be managed for start scripts, etc.

What would be better is if the platform provided for automatic management of dependencies in the core. Both of the mostly widely used systems broadly agree about which relationships between functional units need to be expressible, so there’s no major disagreement about what needs to be expressed.

The simplest cases are that a unit needs to be able to say that it requires:

  • a particular version of a certain other unit
  • a particular version or anything higher
  • any version within a given range.

Dependencies should also be able to be marked as mandatory or optional, and different isolated contexts within the same JVM should be able to have different, incompatible versions of the same unit.

This much is basically agreed upon. Where the different viewpoints diverge is over what constitutes a functional unit, and at what stage of platform startup the dependency management and modularity functionality should kick in. The two big schools of thought are the OSGi view and the Project Jigsaw view.

OSGi

  1. Unit of sharing should be the package. This reduces conceptual burden on developers to learn a new model and is simpler.
  2. Dependency management should start once the platform is fully bootstrapped. This allows the benefits to be brought to any compliant installation, and doesn’t require changes to the core platform.

Project Jigsaw

  1. Unit of sharing should be a new concept – the module. This is “larger” than a package, and packages can also be split between modules.
  2. Changes should be made to the core, and the platform itself should be aware of module dependencies as early as possible. This is more work, and requires more co-ordination between implementations, but has potentially has big wins (quicker startup time, smaller footprint).

Which should you choose for your application? Well, Jigsaw is still being developed, and won’t land as part of Java 7 – it’s targeting Java 8. OSGi is a production technology, used in a surprising number of places (if you’ve ever used Eclipse, you’ve used OSGi technology, you just probably weren’t aware of it). However, it is still maturing – there are gaps around tooling, and there are gaps in the standard which has led to different approaches between vendors, with regard to e.g. bundle deployment.

If you want to learn a new technology, then Jigsaw is still very new, and a moving target, whereas OSGi is here today. On the other hand, unless you have a clear benefit from deploying something like this today, it may be preferable to live with the pain of Ivy/Maven and wait for the space to mature.

Finally, as a modest proposal, let’s talk about deprecation.

Java has always been bad at deprecation. We’ve known that Thread#stop and friends were the wrong thing to do for a very long time, but they’re still there, hanging on as deprecated methods. In the current setup it seems impossible to do anything to remove them. With a wide-ranging modularity solution, such as Jigsaw, in place – could we produce a release of a core module which removed the likes of Thread#stop and Object#finalize ?

Did you like this? Share it: