Tom's Blog

Part 2 of my IBM developerWorks article, Invoke dynamic languages dynamically, creates a simple rules engine using the Java scripting API. Business rules, written in a combination of Ruby, Groovy, and JavaScript, determine whether a borrower qualifies for a variety of home loans. I used a rules engine as a sample application because it seemed more compelling than another hello-world application, and it also seemed like an interesting use of the scripting API.

The Java scripting API, also known as JSR-223, works as a viable basis for a rules engine when a full-blown business rules engine isn't needed because it offers several of the benefits you get from using a regular rules engine. For instance, when business rules are stored as external scripts, the scripting API:

• Allows you to work easily with large sets of rapidly changing rules
• Allows frequent and flexible additions and changes to rules
• Separates rules from processing logic
• Centralizes rules and makes them easier to manage

The Java scripting API fulfills those design goals because scripting code can be kept external to the main Java application, and can be discovered, read and invoked at run time. These same advantages are provided by rules-engine products such as Drools, Jess, or JRules. However, you derive additional advantages by using scripting languages to hold your rules and the Java scripting API to invoke them:

• Easy to program: Use a scripting language -- or several -- of your choice
• Free and easy to set up (partially built into Java SE 6)
• Small number of required external dependencies
• No need to learn a complex declarative business-rules language. For example, here's a sample from Drools:

  rule "Approve if not rejected"
    salience -100
    agenda-group "approval"
      when
          not Rejection()
          p : Policy(approved == false, policyState:status)
          exists Driver(age > 25)
          Process(status == policyState)
      then
          log("APPROVED: due to no objections.");
          p.setApproved(true);
  end
  

What would the design of a rules engine based on the Java scripting API look like? The ScriptMortgageQualifier class in part 2 of my article shows one such design. It stores business objects that the external rules will use in decision-making in the ScriptEngine's context, and receives rule execution results in a separate shared Java object stored in the ScriptEngine context. Rules (scripts) are responsible for storing results of their decisions in the shared Java object, which the main Java code inspects after the rules are run to determine what action to take.

In my sample application, I use individual files to store the rules. The application scans the rules directory on each pass and executes whatever rule scripts it finds there. An advantage of using the Java scripting API to find the rule scripts is the rules can be written in any of dozens of languages supported by script-engine implementations. The rules engine doesn't care what language the rules are written in as long as the applicable script engine and interpreter can be loaded at runtime, such as being supplied by JARs in the classpath. In my sample, I coded rules in Groovy, JavaScript, and Ruby.

Another possible way of structuring rule logic would be to have the rules themselves set additional attributes that other rules could then use (that is, learn from). For instance, say one set of rules runs and determines that the prospective home purchaser has a bank balance of $10 million. The rule could set a property (a global script variable) called VIP (very important person) to true. As a global variable, the property would be available in the ScriptEngine context and passed along to the next rule to be run. That next rule could use different logic based on the fact that this borrower is a VIP.

The above example begins to reveal the shortcomings of designing a rules engine around the scripting API. Most formal rules engines have the notion that all rules are considered to be in effect at all time. Setting a fact such as "customer has VIP status" in one rule should be taken into consideration by all rules to determine if that new fact changes other facts. But satisfying that feature by invoking external rules stored as scripts would require script writers to order the rules in the proper sequence. Trying to sequence your business rules correctly to account for fact-dependencies is error prone -- and impossible when the rules have mutual dependencies. This limitation of requiring rules to be run in a proper sequence is certainly where you would want to consider using a better rules engine.

Rule sequencing isn't the only disadvantage to executing rules stored as external scripts. Writing business rules in Groovy, Ruby or another scripting language has the disadvantage of:

• Rules in scripting languages are written imperatively rather than declaratively
• Complex business logic written imperatively might require deeply nested conditional statements, which makes the rules hard to read and prone to error
• To avoid the above problem of coding deeply nested if-then statements in your script, you might be tempted to write code that processes a decision table -- reinventing the wheel built by better rules engines
• The temptation to write your business rules in multiple scripting languages could become a maintenance headache

In other words, the Java scripting API will not always work as the best solution when your application needs a rules engine. However, the Java scripting API allows business rules to be stored externally, to be written in a language that probably is easier to read than Java, and lets the rules change regularly and fairly easily without having to rebuild your application. If you don't mind writing your business rules in a procedural language instead of a dedicated, declarative rules language, the scripting API could be a good solution. It fills the gap between those times when writing business rules as Java code inside your application has gotten out of hand and when graduating to a fully fledged rules engine isn't yet necessary.

If you're trying to decide whether your application calls for a dedicated rules engine, the Jess website has a good article, Some Guidelines For Deciding Whether To Use A Rules Engine.
Wednesday September 12, 2007 Permalink Comments [3] Since the Java Scripting API makes it easy to execute external scripts written in a variety of dynamic languages, I tried to find a consistent way to return early from top-level code written in JavaScript and Ruby. My goal was to be able to structure short Ruby and JavaScript scripts by coding everything at the "top level," that is, outside of any defined function, method, or class. That way, the Ruby or JavaScript scriptlets would be easier to write and I could eval them from Java without having to call a specific function or method by name.

After hunting around, I found no simple or easy way a JavaScript or Ruby script could return early from being evaluated when the scripting code is outside of a function or method. A return statement is not allowed outside a function in JavaScript, nor is it allowed outside a method in Ruby. The only consistent language feature I found that guaranteed early script exit was for the code to throw an exception.

If you're unfamiliar with the Java Scripting API (JSR-223, Scripting for the Java Platform), it was added in Java Standard Edition 6 to provide a consistent way to embed scripting-language interpreters into a Java application. The API's javax.script package contains classes and interfaces that let you call and share data with an external script written in dozens of scripting languages, including powerful dynamic languages like Ruby and Groovy. The Java Scripting API is based primarily on the Apache Jakarta Bean Scripting Framework project, but provides extra features and is now built into the Java language. You can use the Scripting API in Java 1.5 by adding the new packages, available by downloading the JSR-223 reference implementation.

Here is what I set out to accomplish.

I wanted to be able to pass Java objects to scripts written in Ruby and JavaScript and let those scripts process the shared Java objects. The goal was to take advantage of the cleaner, more concise syntax these languages offer and allow end-users the ability to supply the Ruby and JavaScript code. That was why I didn't want to require script providers to code their logic inside a method or function. But by placing all code at the top level, the script writer would have no language feature available to return early from script processing.

For example, the Java code that called the script would look something like:

    // Java objects to share with the scripts:
    String textToProcess = ... // Text for scripts to process
    int myStatus = ...         // Some type of status indicator
    // etc.

    ScriptEngineManager scriptEngineMgr = new ScriptEngineManager();
    ScriptEngine rubyEngine = scriptEngineMgr.getEngineByName("ruby");
    rubyEngine.put("textToProcess", textToProcess);
    rubyEngine.put("status", Integer.valueOf(myStatus));
    // ...

    // Put a shared object the script will use to return results.
    ResultsObject result = new ResultsObject();
    rubyEngine.put("result", result);

    // Read Ruby script from external source and execute it
    String rubyScript = ...
    rubyEngine.eval(rubyScript);

    // Read results set by the script.
    Long resultCode = result.getResultCode();

    // etc...

The Ruby script would look something like:

# Don't process the text if the status is greater than 200
if $status > 200
    return   # <-- This is illegal Ruby!
end

# Process the $textToProcess text...
...

although the conditions in which the script writer would want to exit could be a lot more complicated and couldn't be structured around an if-else statement.

The problem here is the Ruby script has no simple, clear way to prevent the entire script from being run, short of raising an exception. It is possible to work around the problem by requiring the script to be coded inside of a method. You also could require script writers to code around the problem by wrapping all code inside a needless outer loop and using a break statement to serve the purpose of a return statement.

The above code could thus be replaced by:

1.times do
    # Don't process the text if the status is greater than 200
    if $status > 200
        break # This does work.
    end

    # Process the $textToProcess text...
    ...
end

An extra outer loop should work for JavaScript, too.

The problem with using an outer loop to provide a script return is that it requires the script writer to code the loop. That solution violates my goal of making the scripts as easy as possible to write -- and read.

My eventual solution, which I'm not satisfied with, was to allow the script to perform the equivalent of a top-level return statement by throwing an exception. To make the solution more palatable and cleaner for the script writer, I created a Java class that would throw the actual exception. The Java class also permits the script to return an optional reason message when exiting.

Here is the revised Java code that would call the scripts:

    // Java objects to share with the scripts:
    String textToProcess = ... // Text for scripts to process
    int myStatus = ...         // Some type of status indicator
    // etc.

    ScriptEngineManager scriptEngineMgr = new ScriptEngineManager();
    ScriptEngine rubyEngine = scriptEngineMgr.getEngineByName("ruby");
    rubyEngine.put("textToProcess", textToProcess);
    rubyEngine.put("status", Integer.valueOf(myStatus));
    // ...

    // Put a shared object the script will use to return results.
    ResultsObject result = new ResultsObject();
    rubyEngine.put("result", result);

    // Add an object scripts can call to exit early from processing.
    rubyEngine.put("scriptExit", new ScriptEarlyExit());

    // Read Ruby script from external source and execute it
    String rubyScript = ...
    rubyEngine.eval(rubyScript);

    // Read results of the script.
    Long resultCode = result.getResultCode();

    // etc...

The Java code now supplies all scripts with a ScriptEarlyExit object they can use to invoke the equivalent of a return statement. Here is the ScriptEarlyExit class:

/** Object passed to all scripts so they can indicate an early exit. */
public class ScriptEarlyExit {
    public void withMessage(String msg) throws ScriptEarlyExitException {
        throw new ScriptEarlyExitException(msg);
    }
    public void noMessage() throws ScriptEarlyExitException {
        throw new ScriptEarlyExitException(null);
    }
}

The ScriptEarlyExitException class is a simple Exception subclass:

/** Internal exception so ScriptEarlyExit methods can exit scripts early */
public class ScriptEarlyExitException extends Exception {
    public ScriptEarlyExitException(String msg) {
        super(msg);
    }
}

With the ScriptEarlyExit object made available to scripts by the call to rubyEngine.put("scriptExit", new ScriptEarlyExit()), any script in any language should now be able to exit early. The Ruby script revised to use the new object would be coded like:

# Don't process the text if the status is greater than 200
if $status > 200
    $scriptExit.with_message 'Not processing because of invalid status'
end

# Continue processing
...

The Java method call from the script provides a consistent, fairly clean way to return early from script processing. I tested calling this ScriptEarlyExit object from Ruby using JRuby 1.0, from JavaScript using the Rhino interpreter built into Sun's Java 1.6, and from Groovy 1.0. It worked well with them all.

This solution did require solving another problem. Using a Java exception to end script processing means the script engine is going to bubble up a javax.script.ScriptException back to Java. I needed a way to determine whether that exception was a real ScriptException or my fake ScriptEarlyExitException.

The solution was to check the script exception message to see if my special exception was embedded in the string. The coded ended up looking like:

    try {
        rubyEngine.eval(rubyScript);
    } catch (ScriptException se) {
        // Re-throw exception unless it's our early-exit exception.
        if (se.getMessage() == null ||
            !se.getMessage().contains("ScriptEarlyExitException")
        ) {
            throw se; // a real ScriptException
        }
        // Set script result message if early-exit exception embedded.
        // Will not work with Java 6's included JavaScript engine.
        Throwable t = se.getCause();
        while (t != null) {
            if (t instanceof ScriptEarlyExitException) {
                result.setExitMessage(t.getMessage());
                break;
            }
            t = t.getCause();
        }
    }

The catch block examines the exception's message for the "ScriptEarlyExitException" string, and ignores the ScriptException if found. The code in the catch block then looks to see if one of the causes of the ScriptException was the ScriptEarlyExitException. If so, the ScriptEarlyExitException exception's message string will hold the value set when the script called the withMessage method on the shared ScriptEarlyExit object. That is, when Ruby calls:

    $scriptExit.with_message 'Not processing because of invalid status'

the

ScriptEarlyExitException.getMessage()

will contain the string "Not processing because of invalid status". The catch clause sets that string to the ResultsObject object's exitMessage property using the code:

    result.setExitMessage(t.getMessage());

As the comment in the above code indicates, retrieving the "exit" message from the Rhino JavaScript engine doesn't work. Or at least finding and parsing the exit string out of the resulting ScriptException is more tedious. That's because the Rhino script engine does not wrap caught Java exceptions into the resulting stack trace. With Rhino, the loop:

    Throwable t = se.getCause();
    while (t != null) {
        if (t instanceof ScriptEarlyExitException) {
            result.setExitMessage(t.getMessage());
            break;
        }
        t = t.getCause();
    }

never finds a ScriptEarlyExitException.

As I mentioned, this solution of having scripts call a method on a shared Java object in order to exit script processing early by throwing an exception isn't elegant. But it does work to let scripts execute the equivalent of a top-level "return" statement. This solution likely will work with other JSR-223 scripting engines besides the ones I tested. It seems, though, that there must be a better way. Groovy, by the way, permits a return statement in top-level code. That's pretty nice.

Are you a Ruby or JavaScript pro with a better solution? Is there an easier way for Ruby or JavaScript to return from a script even when the script code is outside a method/function? If you would like to share better techniques, please post a comment here or email me at the address shown in the right-hand column under the "Feedback" heading. If you post a comment on this blog, I ask your forgiveness in that comments are moderated before appearing, but there is no indication of that when you click the "Post" button.


Thursday August 02, 2007 Permalink Comments [5] Pop quiz: Hashtable is to HashMap as StringBuffer is to ... <fill in the blank>

Answer: StringBuilder.

I recently worked on a Java project where the target environment was Java 1.5. Although Java 1.5 has been out for almost three years, the client was just upgrading to it to take advantage of its language features and APIs.

While working on the project, I noticed most developers continued to use the StringBuffer class when StringBuilder would have been the better choice. In asking around, most developers said they were unaware of StringBuilder.

In case you're using Java 1.5 or 1.6 but not yet using StringBuilder, StringBuilder is an unsynchronized version of the tried-and-true StringBuffer class. Most of StringBuffer's public methods are synchronized to allow multiple threads to read and modify the string simultaneously. But since StringBuffer is almost always used to build up a string within a method, or to build a string over several method calls within a single-threaded environment, the synchronized nature of StringBuffer is overkill. An article in Dr. Dobb's Journal in June 2006 estimated switching from StringBuffer to StringBuilder could speed string building by 38%.

That's why Sun added StringBuilder to the language in JDK 5. None of StringBuilder's methods is synchronized, so the class is not meant to be used when multiple threads need to access the string. In multi-threaded contexts, you will want to use StringBuffer. But consider your own code. How many times have you needed to share a StringBuffer between multiple threads? You'll probably find that StringBuilder is often the better choice.


Tuesday July 17, 2007 Permalink Comments [6] Yesterday saw another great celebration on the National Mall in Washington of our nation's declared independence. Two hundred thirty-one years ago, the Continental Congress adopted Thomas Jefferson's draft of the Declaration of Independence.

We began the morning at the National Archives, where the original Declaration of Independence is stored, for the annual dramatic reading of the document by men portraying three of the original signers: John Adams, Thomas Jefferson and Benjamin Franklin. Last year, the last couple of paragraphs were read by two men of our armed forces who were wounded in Iraq or Afghanistan. One of the men suffered head injuries, and his reading was stilted and slurred, yet he bravely read through the document. It brought tears to many in the crowd assembled on the steps outside the archives and spilling out onto Pennsylvania Avenue.

This year, they brought a veteran of World War II to read the last part of the Declaration, and filmmaker Ken Burns talked about his upcoming World War II documentary, The War, which recounts the war from soldiers who fought it. I heard no mention of any active war going on, or of any of the men and women fighting in it. Iraq already seems like a war we're fighting to forget.

We watched a little of the Independence Day parade down Pennsylvania Avenue, walked through the exhibits and listened to music at the Smithsonian Folklife Festival on the Mall, then returned home in the afternoon to watch the fireworks from our balcony.

At around 5 p.m., a lightning storm prompted police to evacuate the open areas of the Mall and the Marine Corps Memorial. Officers asked picnickers and others staking out seats for the concert and fireworks to seek shelter in the various museums and memorials. The storm passed through after about an hour, and the 8 p.m. concert at the Capitol began on time, as did the fireworks an hour later. Last year we watched the fireworks from the Lincoln Memorial. This year, we were able to enjoy the view from our home in Rosslyn.

The fireworks show was great, as usual, but this year I thought it was marred a bit by two orbiting police helicopters, one to the east of the Mall and one to the west. Security was visibly tighter this year, the terror tenor of our times.

And to put another damper on an otherwise perfect evening, three men who put on the fireworks display were hurt and burned, one seriously, when unexploded fireworks went off about 15 minutes after the finale. I was still looking toward the Lincoln Memorial and saw two or three fireworks explode at ground level. May the injured fireworkers recover fully.


Thursday July 05, 2007 Permalink I opened an Eclipse project today, ran a unit test, and got a socket exception I'd never seen before. The project was one I had set aside a few weeks ago after playing with the NetBeans 6 preview release.

After opening the project in Eclipse, I went straight to one of the JUnit test classes, made a small tweak to one of the test methods, then hit my usual Alt-Shift-X + T keyboard shortcut to run the test case with JUnit. Instead of seeing a green or red bar, Eclipse just sat there staring at me, saying it was running the test class with JUnit. The console view showed the red "terminate" button in bright red, indicating the run was proceeding, albeit at an exceedingly slow pace. After about 30 seconds, the console displayed:

Could not connect to:  : 3393
	java.net.ConnectException: Connection refused: connect
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:519)
	at java.net.Socket.connect(Socket.java:469)
	at java.net.Socket.(Socket.java:366)
	at java.net.Socket.(Socket.java:179)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.connect(RemoteTestRunner.java:560)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:377)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)

A socket connection error? I was just trying to run a local JUnit test, not connect with any remote server.

My first theory was I must have been playing with remote debugging for this application a few weeks ago and configured Eclipse to connect with a remote JVM. I spent a minute going through the Eclipse configuration for the JUnit test to check out its settings. I saw nothing set for any remote JUnit connection. (I'm not even sure Eclipse's JUnit runner can do that.) Everything looked right, so I ran the test again and got the same connection refused exception.

My second theory was that I hadn't rebuilt the application since upgrading to JSE 1.6.0_01 from 1.6.0, and that Eclipse was doing its best to find a running 1.6.0 JVM to connect with. (This seemed far-fetched, but a rebuild only took a couple of seconds.) A rebuild didn't solve the problem.

My third theory was I had been using NetBeans for so long I must have forgotten how to run the JUnit test in Eclipse. Perhaps I was telling Eclipse to debug a remote application instead of running JUnit. I ran the test again, this time through the menu option. No luck.

That sent me searching the web for the solution. I found it pretty quickly, but not the underlying reason behind the problem.

The solution was to restart Eclipse. Why this worked I don't know, since I had just launched Eclipse minutes before. Apparently the JUnit runner thread in Eclipse attaches to an Eclipse server thread to run the tests. It would seem the client thread was trying to connect to the wrong port (3393) or that the server thread that had been listening on port 3393 for runtime requests failed. Either way, I would have expected Eclipse to log the error. Strangely, the only item in the Eclipse error log said:

Warnings while parsing the commands from the 'org.eclipse.ui.commands'
and 'org.eclipse.ui.actionDefinitions' extension points.

with a sub-message saying:

Commands should really have a category: plug-in='org.codehaus.groovy.eclipse',
id='org.codehaus.groovy.eclipse.debug.ui.testShortcut.debug',
categoryId='org.eclipse.debug.ui.category.debug'

Well, I did recently install the Groovy plugin. Did that cause the problem? If so, Eclipse thinks not being able to connect with the JUnit runtime is just a warning?

Anyone have the real answer as to what caused Eclipse to get so confused while trying to launch the JUnit runner? None of the web pages I viewed talking about the problem mentioned the cause for the failure.

Tuesday June 26, 2007 Permalink