This manual is mostly examples-based. It starts with two simple step-by-step examples (showing how to deal with static and dynamic libraries), which should give enough information for basic JEL usage (but don't forget to read the rest of this manual to learn how to get the top performance from JEL). The additional information can be found in API documentation.
The main design goal was to create light weight expression compiler generating extremely fast code. The main emphasis is the code execution time and not the compilation time (it is nevertheless small). The other goal was to make JEL language to be very close to Java language with direct access to all built-in Java data types and functions.
Support for all Java data types (boolean, byte, char, short, long, int, float, double, arrays, references)
Octal (0456) and hexadecimal (0x1FFF) literals.
Support for all Java arithmetic operators: + (add),- (subtract), * (multiply), / (divide), % (remainder), & (bitwise and),| (bitwise or), ^ (bitwise xor),~ (bitwise complement), << (left shift), >> (right signed shift), >>> (right unsigned shift); on most of supported data types according to Java Language Specification (JLS)
Comparison operators (==,!=,<,>=,>,<=) as defined by JLS.
dot (".") operator on objects ("abc".length()==3).
dot (".") operator on objects ("abc".length()==3).
Boolean logical operators (&&,||,!) with lazy evaluation (i.e. in the expression false&&complexBooleanFunction() the function is never called).
Conditionals (true?2:3 = 2)
Direct access to methods and fields of Java objects.
Method overloading according to JLS.
Dynamic variables interface allowing to add variables to JEL namespace without supplying the class file defining them.
Automatic unwrapping of designated objects to Java primitive types.
Support for strings. Objects of class java.lang.String can be directly entered into expressions using double quotes, also the standard Java escape codes are parsed. Example : "a string\n\015".
String concatenation ("a"+2+(2>3)+object = "a2false"+object.toString()).
User definable string comparison using usual relational operators "<", "<=", ">", ">=", "==", "!=", which employs locale by default.
User-controllable object down-casting using "(class.name)var" syntax. It is possible to assign names to classes in JEL expressions to be different from their real Java class names.
Constants folding, extended (by default, but can be controlled) to static methods (which are automatically called at compile time) and static fields (which are considered constants).
High performance of generated code.
In this section a simple example of a program using JEL is given, and explained with references to more detailed sections of this manual. The example program evaluates the expression given on its command line (similar program exists in the distribution under the name ./samples/Calculator.java), let's follow it step by step.
public static void main(String[] args) { // Assemble the expression StringBuffer expr_sb=new StringBuffer(); for(int i=0;i<args.length;i++) { expr_sb.append(args[i]); expr_sb.append(' '); }; String expr=expr_sb.toString();
This first part of the program is not related to JEL. It's purpose is to assemble the expression, possibly, containing spaces into the single line. This has to be done, because shells tend to tokenize parameters but we don't need it here.
// Set up the library Class[] staticLib=new Class[1]; try { staticLib[0]=Class.forName("java.lang.Math"); } catch(ClassNotFoundException e) { // Can't be ;)) ...... in java ... ;) }; Library lib=new Library(staticLib,null,null,null,null); try { lib.markStateDependent("random",null); } catch (NoSuchMethodException e) { // Can't be also };This piece of code establishes the namespace for use in JEL compiled expressions. The
gnu.jel.Library
object
maintains this namespace.There can be two types of names in the Library : static and virtual (dynamic).
Methods and variables of the first class are assumed (by
default) to be dependent only on their arguments i.e. not to save
any information from call to call (they are
"stateless")... Examples are mathematical functions like
sin
, cos
,
log
, constants E
,
PI
in java.lang.Math
. For
such methods (fields) it does not matter how many times (when) they
will be called (their value will be taken) the result will always be
the same provided arguments (if they are present) are the
same. Stateless methods will be evaluated by JEL at compile time if
their arguments are constants (known at compile time). To define set
of static functions(fields) it is needed to pass the array of Class
objects, defining those functions, as the first parameter of the
library constructor (see example above). Note ONLY STATIC functions
of the Classes, passed in the first argument of the
gnu.jel.Library
constructor will be defined
in the namespace. By default all static functions are considered
"stateless" by JEL.
However, some static functions still save their state (in
static variables) in between calls. Thus they return different
results, depending on when (how many times) they are is called even
if their arguments are the same. If such function is evaluated at
compile time, we have troubles, because it will be evaluated only
once during expression lifetime and it's state dependence will be
lost. Typical example of the static function, having a state is
java.lang.Math.random
. JEL has special
mechanism, provided by gnu.jel.Library
class
to mark static functions as state dependent. (see the above example
to find out how it was done for the
java.lang.Math.random
)
The virtual functions, which are explicitly state dependent, will be discussed later in this document. The example we currently consider does not use them. However, virtual functions are, actually, most important to JEL because expression, containing all stateless functions, is a constant, it will be completely evaluated at compile time, there is absolutely no sense to evaluate such expression repeatedly (this is what JEL was designed for). Still we shall continue with this simple example as the following code is mostly independent of whether we use virtual functions or not...
// Compile CompiledExpression expr_c=null; try { expr_c=Evaluator.compile(expr,lib); } catch (CompilationException ce) { System.err.print("–––COMPILATION ERROR :"); System.err.println(ce.getMessage()); System.err.print(" "); System.err.println(expr); int column=ce.getColumn(); // Column, where error was found for(int i=0;i<column+23-1;i++) System.err.print(' '); System.err.println('^'); };This chunk of code is for the expression compilation. The crucial line is the call to
Evaluator.compile
, it is the
point, where expression gets transformed into Java bytecode,
loaded into the Java Virtual Machine using JEL ClassLoader and
returned to caller as an instance of the subclass of
gnu.jel.CompiledExpression
.
Typical user of JEL is not
required to know what magic is going on inside of
Evaluator.compile(...)
.
Other code in this chunk is for
the error reporting and will be discussed in the specialized
section Error detection and reporting below.
if (expr_c !=null) { // Evaluate (Can do it now any number of times FAST !!!) Number result=null; try { result=(Number)expr_c.evaluate(null); } catch (Throwable e) { System.err.println("Exception emerged from JEL compiled"+ " code (IT'S OK) :"); System.err.print(e); };This code does the evaluation of the expression. It is done by calling the
evaluate
method of the JEL
compiled class, it is defined abstract in
gnu.jel.CompiledExpression
but is
redefined in the class compiled by JEL. The argument of this method
is discussed in the section on virtual functions below. If only
static functions are present in the library it is safe to pass the
null
pointer as the argument to
evaluate
.
Result of the evaluate
method is always
an object. JEL converts primitive numeric types into instances of
corresponding Java reflection classes (read the section
Making things faster to find out how to avoid
this conversion). For example, a value of primitive type
long will be returned as an instance of
java.lang.Long
class (int maps to
java.lang.Integer
, float to
java.lang.Float
, etc.). If result is an arbitrary Java
object it is returned as the reference to that object.
The
try ... catchclause around the call to
evaluate
will be enforced by the Java
compiler. It is required as errors can appear during evaluation. The
general rule is: syntax, types incompatibility and function
resolution errors will be reported at compile time (as thrown
instance of gnu.jel.CompilationException
),
while the errors in the
values of numbers will be reported at the execution
time. For example expression "1/0" will generate no error
at compile time (nevertheless it is the constant expression and its
evaluation is attempted), but at the time of calling
execute
you will get a java.lang.ArithmeticError
(division
by zero) as it should be.// Print result if (result==null) System.out.println("void"); else System.out.println(result.toString()); }; };This last piece of code will print the result. And is concluding our brief tour of the JEL usage.
The namespace of JEL expressions is represented by
gnu.jel.Library
class. Its constructor:
Library(Class[] staticLib, Class[] dynamicLib, Class[] dotClasses, DVMap resolver, Hashtable cnmap)has five arguments. Their purposes are following:
enumerates classes whose static
methods are exported to JEL namespace and become usable from within
expressions. Such methods do not require this
pointer
supplied to them at execution time.
More details
enumerates classes whose virtual methods
are exported. These methods require the references to the
corresponding classes (this
pointers) supplied to the
expression at run-time. This is done using the Class[]>
argument of CompiledExpression
's
evaluate
method.
More details
controls access for the dot (".") operator on classes. More details
Dynamic variables interface. Allows to add new variables to the expressions names without supplying the class files defining them. More details
Maps the class names usable inside JEL expressions for non-primitive type casts into the Java classes More details
The details on usage of each of these arguments are given in a separate sections below.
The working example using all current functionality of JEL namespace is given in the examples/YourTestBed directory in the distribution. You'll want to check it after reading this section.
The array of references to classes
(java.lang.Class
) whose public
static methods and fields are to be exported should
be passed as the first argument of the library constructor
(staticLib
). The public static fields and
methods of all these classes are merged together into the JEL namespace. The
non-public or non-static members of staticLib
classes
are ignored.
Methods overloading is supported and works also across classes
(because the JEL namespace works similarly to the namespace defined
in a single Java class). For example, if a class C1
contains the method public static C1.func(int)
and a class C2
contains the method
public static C2.func(double)
and both these
classes are passed as elements of the staticLib
array. Then, the JEL expression "func(1)" calls
C1.func(int)
and the expression
"func(1.0)" calls
C2.func(double)
. It also means
that methods and fields of all classes supplied to the
Library
are subject to the same constraints
as members of a single Java class.
Moreover, because JEL allows to call methods with no arguments
omitting the empty brackets (that is "func()"
and "func" are equivalent) there should be no
fields and methods with no arguments having the same names in all
classes presented to the Library
constructor.
To check whether the set of classes you gave to the library constructor satisfies all required constraints run your program against the debug version of JEL library (jel_g.jar). Then, potential problems will be reported to you on the standard output.
The second argument of the library constructor
(dynamicLib
) works similarly to the first one.
Except that only public virtual members are taken from
the listed classes. These members are merged into the namespace created from
classes from the staticLib
. The rules for methods
overloading are the same as for classes listed in the first argument of library
constructor. Also, the overloading is working across the classes
listed in both first and second arguments of the Library constructor.
The crucial difference in the handling of classes listed in the
dynamicLib
and the staticLib
comes from the fact that virtual members of dynamicLib
require this
reference to the instance of the object of
their defining class be supplied at run-time. Thus, if
C1
contains the virtual method
public func(double x)
its invocation actually requires
two arguments, one is x
and the
other is the reference to the instance of class
C1
.
References to the instances of classes of the
dynamicLib
array are supplied at the
execution time to the argument of the
evaluate(Object[] context)
method of
gnu.jel.CompiledExpression
.
The elements of the context
array
should be instances of classes listed in dynamicLib
array at compile time and there should be one-to-one correspondence between
them. For example, if
dynamicLib[0]=com.mycompany.MyClass.class), the corresponding entry in the context array,
context[0]
,
must be a reference to
the instance of
com.mycompany.MyClass
.Formally, for every i
, it should be possible to cast
the object in the context[i]
into the class, supplied in the dynamicLib[i]
array
of the Library
constructor,
otherwise ClassCastException
will be thrown from
evaluate
.
Let's walk through the example, which calculates function of the single variable many times and uses virtual method calls. This example will consist of two classes : a user written class (providing access to the variable) and the main class compiling and evaluating expressions. First start with the variable provider:
public class VariableProvider { public double xvar; public double x() {return xvar;}; };This class is trivial, it just defines the function, returning the value of the variable
x
.In the main class (see the first JEL example for headers) the code, constructing the library will be replaced with:
// Set up library Class[] staticLib=new Class[1]; try { staticLib[0]=Class.forName("java.lang.Math"); } catch(ClassNotFoundException e) { // Can't be ;)) ...... in java ... ;) }; Class[] dynamicLib=new Class[1]; VariableProvider variables=new VariableProvider(); Object[] context=new Object[1]; context[0]=variables; dynamicLib[0]=variables.getClass(); Library lib=new Library(staticLib,dynamicLib,null,null,null); try { lib.markStateDependent("random",null); } catch (NoSuchMethodException e) { // Can't be also };Absent in the static example, the additional code creates the
VariableProvider
and assigns its
reference to an element of context
array (to be
passed to the evaluate
method
of the compiled expression). Also, now the
dynamicLib
array as not null and contains
the reference to the VariableProvider
class.The code for compilation is exactly the same as in the example for
static functions, except we have additional function x
and the variable xvar
defined for use inside the
compiled expressions. JEL has the special notation for the functions,
having no arguments, namely, brackets in "x()"
can be omitted to be "x". This allows to compile now ( with the above
defined library) the expressions like "sin(x)",
"exp(x*x)",
"pow(sin(x),2)+pow(cos(x),2)"...
The code for evaluation of an expression having virtual functions is replaced with:
if (expr_c !=null) { try { for(int i=0;i<100;i++) { variables.xvar=i; // <- Value of the variable System.out.println(expr_c.evaluate(context)); //^^^^^^^^^^^^^^^ evaluating 100 times }; } catch (Throwable e) { System.err.println("Exception emerged from JEL compiled"+ " code (IT'S OK) :"); System.err.print(e); }; };Note the two major differences: 1. we have explicitly assigned the value to the variable; 2. the array of object references (consisting of one element in this example) is passed to the
evaluate
method. This piece of code will evaluate
expressions for x=0..99
with
step 1
.This concludes our dynamic library example. Try to modify the ./Calculator.java sample yourself to allow compilation of virtual functions as described above.
The third argument of gnu.jel.Library
constructor enumerates classes which are available for dot operator
within the expression. If this parameter is null
JEL would not allow to use the dot operator at all. If it is an array
of the length zero (e.g. new Class[0]
)
JEL will open access to public methods
of ALL objects encountered in the expression. From the
security point of view allowing access to all objects can be
dangerous, that is why there is a third case of non-zero length
array explicitly enumerating classes allowing the dot operator on
them.
Once the dot operator is allowed on a class, it is possible to call all its public methods using the syntax ".method(arg1,arg2,...)" in any context where this class appears in an expression.
All methods of exporting names into JEL namespace described up to this point relied on the Java class files for actual description of methods names and parameters. However, sometimes it is required to add a new variable to JEL namespace at run-time.
One of the solutions would be to generate a new class file (e.g. using JEL) and supply it as a first or second argument of the library constructor. Unfortunately this can be quite cumbersome and time consuming.
The other solution can be to define a family of methods in JEL namespace
YYY getXXXProperty(String name)for each possible variable types, where
YYY
is the class
representing the property type and XXX
is the name
of the type. Then, supposing we have methods
double getDoubleProperty(String name); // YYY=double XXX=Double String getStringProperty(String name); // YYY=java.lang.String XXX=Stringin the JEL namespace (either static or dynamic), the variables with arbitrary names can be entered into expression using the syntax
getStringProperty("x") + (getDoubleProperty("y")+1.0)
This way has two drawbacks: 1) user has to remember the type of the
variable (to call the appropriate getXXX()
method);
2) a lot to type.
Since the version 0.9.3 JEL provides the way to solve both
these problems. To do that the fourth argument
(resolver
) of the library constructor is
used. This argument supplies the reference to the subclass of
gnu.jel.DVMap
, and is used by JEL to resolve
the dynamic variable names. The gnu.jel.DVMap
has an abstract method
public String getTypeName(String name)which returns XXX (see above) for a given variable name, or null if no such variable is defined. Note that for resolver to work the family of methods
YYY getXXXProperty(String name)must still be present in JEL namespace (e.g. as members of one of
dynamicLib[]
classes).Then, supposing
resolver.getTypeName("x")=="String" && resolver.getTypeName("y")=="Double"the expression "x+(y+1.0)" will be automatically converted by JEL into
getStringProperty("x")+(getDoubleProperty("y")+1.0)and compiled. Thus, user does not have to remember the variable types, typing is reduced and the existence of variables can be checked at the compile time.
JEL also supports a hierarchical structure of variables. This means the dot (".") symbol can be present in the dynamic variable names. For example if
resolver.getTypeName("x")!=null && resolver.getTypeName("x.f1")=="String" && resolver.getTypeName("x.f2")=="Double"the expression "x.f1+(x.f2+1.0)" will be compiled by JEL as
getStringProperty("x.f1")+(getDoubleProperty("x.f2")+1.0)and (combined with dot operator) the expression "x.f1.length()" will result in the length of the string
getString("x1.f1")
.Notice in the last example that if one wants to have defined
the dynamic variable "x.y" the variable
"x" must
also be the dynamic variable
(resolver.getTypeName("x")!=null
).
If there is conflict between the dynamic variable name and other name in JEL namespace the dynamic variable has a priority.
Since JEL 0.9.9 it is possible to translate the names of dynamic
variables from strings into the constants of Java primitive
types. This is done using non-identity DVMap.translate
method. The translation helps to improve performance in some cases.
Consider the following example. Suppose the underlying storage for
dynamic variables is an array (or Vector
), so that
the value of the variable can be obtained by an integer index into that
array (like numbered columns in a spreadsheet). Next, assume you still
want to refer to the variables by names (e.g. you allowed user to assign
names to the columns). Now, if the first column is named
"x" and
is of Double type, an expression "x",
using dynamic variables interface with identity translation will be
compiled into getDoubleProperty("x").
It means the translation of
the string "x" into the column number
1
will have to be
performed at run-time each time the expression is
evaluated. Considering that Java strings are immutable, this may incur
a substantial performance penalty.
The performance can be improved if the translate
method of DVMap
is overridden by the following:
public Object translate(String name) { if (name.equals("x")) return new Integer(1); return name; };This is already a non-identity translation. With such
DVMap
the expression "x" will be
compiled by JEL into getDoubleProperty(1),
note that it is
getDoubleProperty(int)
method, which is called.
This way the mapping of the variable name into the variable index is
performed at compile-time, while at run-time the index is readily available.
By defining the appropriate translations the dynamic variable lookup can
be split in a user-controlled way between the expression compilation
and execution stages to achieve the best performance.The translate
method is allowed to return
only instances of Java reflection classes wrapping the primitive types
(java.lang.Integer
,
java.lang.Double
, etc), or
strings (otherwise an exception will emerge at compile-time). This is
because only these types of objects can be stored in the Java class
files directly. Also, it is responsibility of the caller to ensure
that JEL namespace contains getXXXProperty
methods
with all the necessary argument types, corresponding to the translations
defined in DVMap
. For identity translations only
getXXXProperty
methods accepting strings are
necessary.
The cnmap
argument of
gnu.jel.Library
constructor,
allows to enable the non-primitive type casts in JEL compiled
expressions. If cnmap!=null
it must be
java.util.Hashtable
with
java.lang.Class
objects as
elements and java.lang.String
objects as keys.
When the object cast
"(non_primitive_type_name) var" is
encountered in the expression, "the non_primitive_type_name"
string is looked in the cnmap
hashtable and the
cast to the corresponding class is generated by JEL. The absence of the
name in the hashtable produces the compile-time error. It is possible for
keys in cnmap
to contain "." (dot) symbols
in them.
This problem appears mostly when one uses dynamic variables, but may
also arise in other cases. Suppose a reference to the object of the
class Weight
(representing a weight of a certain item)
appeared in the expression. It is clear that
Weight
is always represented by a floating point
number (although it may have other properties, like units). If the
class Weight
has the method
public double getValue()the value of weight can be accessed in expressions using syntax
w.getValue()
, supposing the variable
w
has type Weight
.To save typing (since version 0.9.3 of JEL) one may have the class
Weight
implement
gnu.jel.reflect.Double
interface. Then,
the aforementioned getValue method will be called automatically by JEL
(or object w
will be "unwrapped" to
primitive type). This
unwrapping will be performed automatically when needed: one can have
expressions "w+1.0" meaning
"w.getValue()+1" and
"w.getUnits()" both
valid (in the second case w
is not "unwrapped").
There are gnu.jel.reflect.*
interfaces
for all Java primitive types. To use the automatic unwrapping one
just needs to make his classes to implement one of these interfaces.
There is a similar mechanism for strings (since version 0.9.6)
and a corresponding empty interface
gnu.jel.reflect.String
to denote objects automatically convertible to
java.lang.String
by means of their
.toString()
method. For
example, if x
is of a class implementing
gnu.jel.reflect.String
interface the expression
x+"a" will be compiled into
x.toString()+"a" (otherwise this expression
produces a error message). The objects automatically convertible to
strings can also be supplied as arguments of methods requiring
java.lang.String
(usual method overloading rules
apply). Still, in the current version of JEL it is impossible to
cast methods of java.lang.String
on such objects.
That is x.substring(1) is a syntax error
(unless x
> itself
has the .substring(int)
method). This deficiency can be
addressed in future.
Expressions are made by human, and making errors is the natural property of humans, consequently, JEL has to be aware of that.
There are two places, where errors can appear. First are the
compilation errors, which are thrown in the form of
gnu.jel.CompilationException
by the
gnu.jel.Evaluator.compile
. These errors signal about
syntax problems in the entered expressions, wrong function names,
illegal types combinations, but NOT about illegal
values of arguments of functions. The second source of errors is the
compiled code itself, Throwables, thrown out of
gnu.jel.CompiledExpression.evaluate
are primarily due to
the invalid values of function arguments.
Compilation errors are easy to process. Normally, you should surround compilation by the
try { // ... compilation catch (CompilationException e) { // ... process and report the error }block. Caught
gnu.jel.CompilationException
can be
interrogated, then, on the subject of WHERE error has occurred
(getCol
) and WHAT was the
error (getMessage
). This
information should then be presented to user. It is wise to use
information about error column to position the cursor automatically
to the erroneous place in the expression.Errors of the second type are appearing during the function
evaluation and can not be so nicely dealt with by JEL. They depend
on the actual library, supplied to the compiler. For example
methods of java.lang.Math
do not generate any checked
exceptions at all (still, Errors are possible), but you may connect
library, of functions throwing exceptions. As a general rule :
exceptions thrown by functions from the library are thrown from
evaluate
method
In the above text the result of the computation, returned by
evaluate
was always an object. While this is
very flexible it is not very fast. Objects have to be allocated on
heap and garbage collected. When the result of computation is the
Java primitive type it can be desirable to retrieve it without
creation of the object. This can be done (since the version 0.2 of
JEL) with evaluateXX()
family of calls (see
gnu.jel.CompiledExpression
. There is an
evaluateXX()
method for each Java primitive
type, if you know what type expression has you can just call the
corresponding method.
If you do not know the type of the compiled expression you can
query it using getType
. Be warned, that the
call to wrong evaluateXX
method will result in
exception. Another tricky point is that JEL always selects smallest
data type for constant representation. Namely, expression
"1" has type
byte and not int, thus in
most cases you will have to query the type, and only then, call the
proper evaluateXX
method.
It is anyway possible to eliminate type checks at evaluation
time completely. There is a version of
compile
method in
gnu.jel.Evaluator
, which allows to fix
the type of the result. It directs the compiler to perform the
widening conversion to the given type, before returning the
result. For example: if you fix the type to be int
(passing java.lang.Integer.TYPE
as an
argument to compile) all expressions (such as
"1", "2+5",
"2*2") will be evaluated by
evaluate_int
method of
the compiled expression. Also, the attempt to evaluate
"1+2L" will be rejected by compiler,
asking to insert the explicit narrowing conversion (such as
"(int)(1+2L)").
There used to be a specialized serialization interface in JEL up to version 0.8.3. The need for such interface was dictated by the fact that JEL allowed to use constants of arbitrary reference types in expressions, which is not supported directly by the Java class file format. Starting with version 0.9 this feature was removed and now JEL generates ordinary Java class files.
To store compiled expressions into a file just grab their code with
gnu.jel.Evaluator.compileBits
. The code is returned as a
byte array which is easy to save/restore. Then, the expression can be
instantiated using gnu.jel.ImageLoader
with the code
byte[] image; // ... code to read the JEL-generated class file into the "image" ... CompiledExpression expression=(CompiledExpression)(ImageLoader.load(image)).newInstance();or, alternatively, by compiling your source against generated class file. Note that in this version of JEL all generated classes have the name "dump" and are in the root package. If there will be such need in future the Evaluator interface can be extended to assign user-supplied names for new expressions.
There is one serious limitation, which should be mentioned. Actually it is not a JEL limitation but rather a limitation of the typical Java run-time
To load compiled expressions into the Java virtual machine memory
JEL uses a custom java.lang.ClassLoader
. While there
is nothing wrong with that, setting up a classLoader is a privileged
operation in Java. This means either JEL should run in a Java
application (there are no security restrictions on Java
applications), or , if JEL is distributed in some custom
applet the applet should be
signed.
I hope you found JEL useful. Don't hesitate to contact me if there are any problems with JEL, please, report BUGS, suggest tests, send me your patches,... There are still many improvements to be done.
Most current information about JEL should be available at http://galaxy.fzu.cz/JEL/.
JEL is the "free software" and is distributed to you under terms of GNU General Public License. Find the precise terms of the license in the file ./COPYING in the root of this distribution.
Please, contact the author directly if you'd like JEL to be commercially licensed to you on a different terms.