Editor's Note: This post is a free chapter from the book from Manning Publications "In the OCA Java SE 7 programmer certification guide" by Mala Gupta In this article, I'll cover if and if-else constructs. We'll examine what happens when these constructs are used with and without curly braces {}. We'll also cover nested if and if-else constructs. The if construct and its flavors An if construct enables you to execute a set of statements in your code based on the result of a condition. This condition must always evaluate to a boolean or a Boolean value. You can specify a set of statements to execute when this condition evaluates to true or false. (In many Java books, you'll notice that the terms constructs and statements are used interchangeably.) Figure 1 shows multiple flavors of the if statement with their corresponding representations. if if-else if-else-if-else Figure 1 Multiple flavors of if statement: if, if-else, and if-else-if In figure 1, condition1 and condition2 refer to a variable or an expression that must evaluate to boolean or Boolean value. statement1, statement2, and statement3 refer to a single line of code or a code block. Because the Boolean wrapper class isn't covered in the OCA Java SE 7 Programmer I exam, we won't cover it here. We'll work with only the boolean data type. Exam Tip: then isn't a keyword in Java and isn't supposed to be used with the if statement. Let's look at the use of some flavors by first defining a set of variables: score, result, name, and file, as follows: int score = 100; String result = ""; String name = "Lion"; java.io.File file = new java.io.File("F"); Figure 2 shows the use of if, if-else, and if-else-if-else constructs and compares them by showing the code side by side. Figure 2 Multiple flavors of if statements implemented using code Let's quickly go through the code used in above if, if-else, and if-else-if-else statements. In the following example code, if condition name.equals("Lion") evaluates to true, a value of 200 is assigned to the variable score: if (name.equals("Lion")) #A score = 200; #A #A Example of if construct In the following example, if condition name.equals("Lion") evaluates to true, a value of 200 is assigned to the variable score. If this condition were to evaluate to false, a value of 300 is assigned to the variable score: if (name.equals("Lion")) #A score = 200; #A else #A score = 300; #A #A Example of if else construct In the following example, if score is equal to 100, the variable result is assigned a value of A. If score is equal to 50, the variable result is assigned a value of B. If the score is equal to 10, the variable result is assigned a value of C. If score doesn't match either of 100, 50, or 10, a value of F is assigned to the variable result. An if-else-if-else construct may use different conditions for all its if constructs: if (score == 100) #A result = "A"; else if (score == 50) #B result = "B"; else if (score == 10) #C result = "C"; else #D result = "F"; #A Condition 1 -> score == 100 #B Condition 2 -> score == 50 #C Condition 3 -> score == 10 #D If none of previous conditions evaluate to true, execute this else Figure 3 shows the previous code. Figure 3 The execution of the if-else-if-else code Figure 3 makes clear multiple points: The last else statement is part of the last if construct and not any of the if constructs before it. The if-else-if-else is an if-else construct, where its else part defines another if construct. A few other programming languages, such as VB and C#, use if-elsif and if-elseif (without space) constructs to define if-else-if constructs. If you've programmed with any of these languages, note the difference is with respect to Java. The following code is equal to the previous code: if (score == 100) result = "A"; else if (score == 50) result = "B"; else if (score == 10) result = "C"; else result="F"; Again, note that none of the previous if constructs use then to define the code to execute if a condition evaluates to true. As mentioned previously, unlike other programming languages, then isn't a keyword in Java and isn't used with the if construct. Exam Tip The if-else-if-else is an if-else construct, where else part defines another if construct. The boolean expression used as a condition for if construct can also include assignment operation. Missing else blocks What happens if you don't define the else statements for an if construct? It's acceptable to define one course of action for an if construct, as follows (omitting the else part): boolean testValue = false; if (testValue == true) System.out.println("value is true"); But you can't define the else part for an if construct, skipping the if code block. The following code won't compile: boolean testValue = false; if (testValue == true) else #A System.out.println("value is false"); #A This won't compile What follows is another interesting and bizarre piece of code: int score = 100; if((score=score+10) > 110); #1 #1 Missing then or else part Line #1 is a valid line of code, even if it doesn't define both the then and else part of the if statement. In this case, if condition evaluates and that's it. The if construct doesn't define any code that should execute based on the result of this condition. Note if(testValue==true) is same as using if(testValue). Similarly, if(testValue==false) is same as using if(!testValue). Implications of presence and absence of {} in if-else constructs You can execute a single statement or a block of statements, when if condition evaluates to true or false values. A block of statement is marked by enclosing single or multiple statements within a pair of curly braces ({}). Examine the following code: String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; What happens if you want to execute another line of code, if value of variable name is equal to Lion? Is the following code correct? String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; name = "Larry"; #1 #1 Set name to Larry Exam Tip In the exam, watch out for code similar to the above mentioned if construct that uses misleading indentation. In the absence of a code block definition (marked with a pair of {}), only the statement following the if construct forms its part. What happens to the same code if you define an else part for your if construct as follows: String name = "Lion"; int score = 100; if (name.equals("Lion")) score = 200; name = "Larry"; #A else score = 129; #A This statement isn't part of the if construct In this case, the previous code won't compile. The compiler will report that the else part is defined without an if statement. If this leaves you confused, examine the following code, which is indented in order to emphasize the fact that line name = "Larry" isn't part of the else construct: String name = "Lion"; int score = 100; if (name.equals ("Lion")) score = 200; name = "Larry"; #A else #B score = 129; #A Right indentation to emphasize that this statement isn't part of the if construct #B else seems to be defined without a preceding if construct If you want to execute multiple statements for if construct, you should define them within a block of code. You can do so by defining all this code within curly braces ({}). To follow is an example: String name = "Lion"; int score = 100; if (name.equals("Lion")) { #A score = 200; #B name = "Larry"; #B } #C else score = 129; #A Start of code block #B Statements to execute if (name.equals("Lion")) evaluates to true #C End of code block Similarly, you may define multiple lines of code for the else part (incorrectly) as follows: String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 #1 Not part of else construct. Will execute irrespective of the value of variable name The output of the above code is as follows: Lion Again, not a Lion Though code on line #1 seems to execute only if value of variable name matches with value Lion, this is not the case. It is indented incorrectly to trick you into believing that it is a part of the else block. The above code is same as the following code (with correct indentation): String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 #1 Not part of else construct. Will execute irrespective of the value of variable name If you wish to execute the last two statements in the previous code, only if the if condition evaluates to false, you can do so by using {}: String name = "Lion"; if (name.equals("Lion")) System.out.println("Lion"); else { System.out.println("Not a Lion"); System.out.println("Again, not a Lion"); #1 } #1 Now part of else construct. Will execute only when if condition evaluates to false You can define another statement, construct or loop, to execute for an if condition, without using {}, as follows: String name = "Lion"; if (name.equals("Lion")) #A for (int i = 0; i < 3; ++i) #B System.out.println(i); #C #A if condition #B for loop is a single construct that will execute if name.equals("Lion") evaluates to true #C This code is part of the for loop defined at previous line System.out.println(i) is part of the for loop, and not an unrelated statement that follows the for loop. So this code is correct and gives the following output: 0 1 2 Appropriate vs. inappropriate expressions passed as arguments to an if statement The result of an expression used in an if construct must evaluate to a boolean or Boolean value. Given the following definition of variables: int score = 100; boolean allow = false; String name = "Lion"; Up next are examples of some of the valid expressions that can be passed on to an if construct. Note that using == is not a good practice to compare two String objects for equality. The correct way to compare two String objects is to use equals method from the String class. However, comparing two String values using == is a valid expression that returns a boolean value and may also be used in the exam: (score == 100) #A (name == "Lio") #B (score <= 100 || allow) #C (allow) #D #A Evaluates to true #B Evaluates to false #C Evaluates to true #D Evaluates to false Now comes the tricky part of passing an assignment operation to an if construct. What do you think is the output of the following code? boolean allow = false; if (allow = true) #A System.out.println("value is true"); else System.out.println("value is false"); #A This is assignment, not comparison You may think that because the value of the boolean variable allow is set to false, the previous code output's value is false. Revisit the code and notice that assignment operation allow = true assigns the value true to the boolean variable allow. Further, its result is also a boolean value, which makes it eligible to be passed on as an argument to the if construct, Although the previous code has no syntactical errors, it's a logical error-an error in the program logic. The correct code to compare a boolean variable with a boolean literal value should be defined as follows: boolean allow = false; if (allow == true) #A System.out.println("value is true"); else System.out.println("value is false"); #A This is comparison Exam Tip Watch out for the code in the exam that uses the assignment operator (=) to compare a boolean value in the if condition. It won't compare the boolean value; it'll assign a value to it. The correct operator to compare a boolean value is equality operator (==). Nested if constructs A nested if construct is an if construct defined within another if construct. Theoretically, you don't have a limit on the levels of nested if and if-else constructs. Whenever you come across nested if and if-else constructs, you need to be careful about determining the else part of an if statement. If this statement doesn't make a lot of sense, take a look at the following code and determine its output: int score = 110; if (score > 200) #1 if (score <400) #2 if (score > 300) System.out.println(1); else System.out.println(2); else #3 System.out.println(3); #3 #1 if (score>200) #2 if (score<400) #3 To which if does this else belongs? Based on the way the code is indented, you may believe that else at #3 belongs to the if defined at #1. But it belongs to the if defined at #2. To follow is the code with the correct indentation: int score = 110; if (score > 200) if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); else #A System.out.println(3); #A #A This else belongs to the if with condition (score<400) Next, you need to understand how to do the following: How to define an else for an outer if, other than the one that it'll be assigned to by default How to determine to which if does an else belong in nested if constructs Both of these tasks are simple. Let's start with the first one. How to define an else for an outer if other than the one that it'll be assigned to by default The key point is to use curly braces, as follows: int score = 110; if (score > 200) { #1 if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); } #2 else #3 System.out.println(3); #3 #1 Start if construct for score > 200 #2 End if construct for score > 200 #3 else for score > 200 The curly braces at #1 and #2 mark the start and the end of the if condition (score>200) defined at #1. Hence, the else at #3 that follows #2 belongs to the if defined at #1. How to determine to which if an else belongs in nested if constructs If code uses curly braces to mark the start and end of the territory of an if or else construct, it can be simple, as mentioned in the previous section, "How to define an else for an outer if than the one that it'll be assigned to by default." When the if constructs don't use curly braces, don't get confused by the code indentation. Try to match all if with their corresponding else in the following poorly indented code: if (score > 200) if (score <400) if (score > 300) System.out.println(1); else System.out.println(2); else System.out.println(3); Start working inside out, with the innermost if-else statement, matching else with its nearest unmatched if statement. Figure 4 shows how to match the if-else pairs for the previous code, marked with 1, 2, and 3. Figure 4 Matching if-else pairs for poorly indented code Summary We covered the different flavors of the if construct. You saw what happens when these constructs are used with and without curly braces {}. We also covered nested if and if-else constructs. The humble if-else construct can virtually define any set of simple or complicated conditions. OCA Java SE 7 Programmer I Certification Guide By Mala Gupta In the OCA Java SE 7 programmer exam, you'll be asked you'll be asked how to define and control the flow in your code. In this article, based on chapter 4 of OCA Java SE 7 Programmer I Certification Guide, author Mala Gupta How show you to use if, if-else, if-else-if-else and nested if constructs and the difference when these if constructs are used with and without curly braces {}. Here are some other Manning titles you might be interested in: Unit Testing in Java Lasse Koskela Making Java Groovy Kenneth Kousen Play for Java Nicolas Leroux and Sietse de Kaper

these days i’m playing with backbone and using public api as a source. the web browser has one horrible feature: it don’t allow you to fetch any external resource to our host due to the cross-origin restriction. for example if we have a server at localhost we cannot perform one ajax request to another host different than localhost. nowadays there is a header to allow it: access-control-allow-origin . the problem is that the remote server must set up this header. for example i was playing with github’s api and github doesn’t have this header. if the server is my server, is pretty straightforward to put this header but obviously i’m not the sysadmin of github, so i cannot do it. what the solution? one possible solution is, for example, create a proxy server at localhost with php. with php we can use any remote api with curl (i wrote about it here and here for example). it’s not difficult, but i asked myself: can we create a dummy proxy server with php to handle any request to localhost and redirects to the real server, instead of create one proxy for each request?. let’s start. problably there is one open source solution (tell me if you know it) but i’m on holidays and i want to code a little bit (i now, it looks insane but that’s me ). the idea is: ... $proxy->register('github', 'https://api.github.com'); ... and when i type: http://localhost/github/users/gonzalo123 and create a proxy to : https://api.github.com/users/gonzalo123 the request method is also important. if we create a post request to localhost we want a post request to github too. this time we’re not going to reinvent the wheel, so we will use symfony componets so we will use composer to start our project: we create a conposer.json file with the dependencies: { "require": { "symfony/class-loader":"dev-master", "symfony/http-foundation":"dev-master" } } now php composer.phar install and we can start coding. the script will look like this: register('github', 'https://api.github.com'); $proxy->run(); foreach($proxy->getheaders() as $header) { header($header); } echo $proxy->getcontent(); as we can see we can register as many servers as we want. in this example we only register github. the application only has two classes: restproxy , who extracts the information from the request object and calls to the real server through curlwrapper . request = $request; $this->curl = $curl; } public function register($name, $url) { $this->map[$name] = $url; } public function run() { foreach ($this->map as $name => $mapurl) { return $this->dispatch($name, $mapurl); } } private function dispatch($name, $mapurl) { $url = $this->request->getpathinfo(); if (strpos($url, $name) == 1) { $url = $mapurl . str_replace("/{$name}", null, $url); $querystring = $this->request->getquerystring(); switch ($this->request->getmethod()) { case 'get': $this->content = $this->curl->doget($url, $querystring); break; case 'post': $this->content = $this->curl->dopost($url, $querystring); break; case 'delete': $this->content = $this->curl->dodelete($url, $querystring); break; case 'put': $this->content = $this->curl->doput($url, $querystring); break; } $this->headers = $this->curl->getheaders(); } } public function getheaders() { return $this->headers; } public function getcontent() { return $this->content; } } the restproxy receive two instances in the constructor via dependency injection (curlwrapper and request). this architecture helps a lot in the tests , because we can mock both instances. very helpfully when building restproxy. the restproxy is registerd within packaist so we can install it using composer installer: first install componser curl -s https://getcomposer.org/installer | php and create a new project: php composer.phar create-project gonzalo123/rest-proxy proxy if we are using php5.4 (if not, what are you waiting for?) we can run the build-in server cd proxy php -s localhost:8888 -t www/ now we only need to open a web browser and type: http://localhost:8888/github/users/gonzalo123 the library is very minimal (it’s enough for my experiment) and it does’t allow authorization. of course full code is available in github .

My last blog introduced Spring 3.1’s profiles and explained both the business case for using them and demonstrated their use with Spring XML configuration files. It seems, however, that a good number of developers prefer using Spring’s Java based application configuration, so Spring have designed a way of using profiles with their existing @Configuration annotation. I’m going to demonstrate profiles and the @Configuration annotation using the Person class from my previous blog. This is a simple bean class whose properties vary depending upon which profile is active. public class Person { private final String firstName; private final String lastName; private final int age; public Person(String firstName, String lastName, int age) { this.firstName = firstName; this.lastName = lastName; this.age = age; } public String getFirstName() { return firstName; } public String getLastName() { return lastName; } public int getAge() { return age; } } Remember that the Guys at Spring recommend that Spring profiles should only be used when you need to load different types or sets of classes and that for setting properties you should continue using the PropertyPlaceholderConfigurer. The reason I’m breaking the rules is that I want to try to write the simplest code possible to demonstrate profiles and Java configuration. At the heart of using Spring profiles with Java configuration is Spring’s new @Profile annotation. The @Profile annotation is used attach a profile name to an @Configuration annotation. It takes a single parameter that can be used in two ways. Firstly to attach a single profile to an @Configuration annotation: @Profile("test1") and secondly, to attach multiple profiles: @Profile({ "test1", "test2" }) Again, I’m going to define two profiles “test1” and “test2” and associate each with a configuration file. Firstly “test1”: @Configuration @Profile("test1") public class Test1ProfileConfig { @Bean public Person employee() { return new Person("John", "Smith", 55); } } ...and then “test2”: @Configuration @Profile("test2") public class Test2ProfileConfig { @Bean public Person employee() { return new Person("Fred", "Williams", 22); } } In the code above, you can see that I'm creating a Person bean with an effective id of employee (this is from the method name) that returns differing property values in each profile. Also note that the @Profile is marked as: @Target(value=TYPE) ...which means that is can only be placed next to the @Configuration annotation. Having attached an @Profile to an @Configuration, the next thing to do is to activate your selected @Profile. This uses exactly the same principles and techniques that I described in my last blog and again, to my mind, the most useful activation technique is to use the "spring.profiles.active" system property. @Test public void testProfileActiveUsingSystemProperties() { System.setProperty("spring.profiles.active", "test1"); ApplicationContext ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } Obviously, you wouldn’t want to hard code things as I’ve done above and best practice usually means keeping the system properties configuration separate from your application. This gives you the option of using either a simple command line argument such as: -Dspring.profiles.active="test1" ...or by adding # Setting a property value spring.profiles.active=test1 to Tomcat’s catalina.properties So, that’s all there is to it: you create your Spring profiles by annotating an @Configuration with an @Profile annotation and then switching on the profile you want to use by setting the spring.profiles.active system property to your profile’s name. As usual, the Guys at Spring don’t just confine you to using system properties to activate profiles, you can do things programatically. For example, the following code creates an AnnotationConfigApplicationContext and then uses an Environment object to activate the “test1” profile, before registering our @Configuration classes. @Test public void testAnnotationConfigApplicationContextThatWorks() { // Can register a list of config classes AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext(); ctx.getEnvironment().setActiveProfiles("test1"); ctx.register(Test1ProfileConfig.class, Test2ProfileConfig.class); ctx.refresh(); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } This is all fine and good, but beware, you need to call AnnotationConfigApplicationContext’s methods in the right order. For example, if you register your @Configuration classes before you specify your profile, then you’ll get an IllegalStateException. @Test(expected = IllegalStateException.class) public void testAnnotationConfigApplicationContextThatFails() { // Can register a list of config classes AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext( Test1ProfileConfig.class, Test2ProfileConfig.class); ctx.getEnvironment().setActiveProfiles("test1"); ctx.refresh(); Person person = ctx.getBean("employee", Person.class); String firstName = person.getFirstName(); assertEquals("John", firstName); } Before closing today’s blog, the code below demonstrates the ability to attach multiple @Profiles to an @Configuration annotation. @Configuration @Profile({ "test1", "test2" }) public class MulitpleProfileConfig { @Bean public Person tourDeFranceWinner() { return new Person("Bradley", "Wiggins", 32); } } @Test public void testMulipleAssignedProfilesUsingSystemProperties() { System.setProperty("spring.profiles.active", "test1"); ApplicationContext ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); Person person = ctx.getBean("tourDeFranceWinner", Person.class); String firstName = person.getFirstName(); assertEquals("Bradley", firstName); System.setProperty("spring.profiles.active", "test2"); ctx = new ClassPathXmlApplicationContext("profiles-config.xml"); person = ctx.getBean("tourDeFranceWinner", Person.class); firstName = person.getFirstName(); assertEquals("Bradley", firstName); } In the code above, 2012 Tour De France winner Bradley Wiggins appears in both the “test1” and “test2” profiles.

At the end of July 2012, Groovy 2.0 was released with support for static type checking and some performance improvements through the use of JDK7 invokedynamic and type inference as a result of type information now available through static typing. I was interested in seeing some estimate as to how significant the performance improvements in Groovy 2.0 have turned out and how Groovy 2.0 would now compare to Java in terms of performance. In case the performance gap had become minor, or at least acceptable, in the meantime, it would certainly be time to take a serious look at Groovy. Groovy has been ready for production for a long time. So, let's see whether it can compare with Java in terms of performance. The only performance measurement I could find on the Internet was this little benchmark measurment on jlabgroovy. The measurement only consists of calculating Fibonacci numbers with and without the @CompileStatic annotation. That's it; i.e., it's certainly not very meaningful in striving to get an overall impression. I was only interested in obtaining some rough estimate of how Groovy compares to Java as far as performance is concerned. Java performance measurement included Alas, no measurement was included in this little benchmark as to how much time Java takes to calculate Fibonacci numbers. So I "ported" the Groovy code to Java (here it is) and repeated the measurements. All measurements were done on an Intel Core2 Duo CPU E8400 3.00 GHz using JDK7u6 running on Windows 7 with Service Pack 1. I used Eclipse Juno with the Groovy plugin using the Groovy compiler version 2.0.0.xx-20120703-1400-e42-RELEASE. These are the figures I obtained without having a warm-up phase: Groovy 2.0 without @CompileStatic Groovy/Java performance factor Groovy 2.0 with @CompileStatic Groovy/Java performance factor Kotlin 0.1.2580 Java static ternary 4352ms 4.7 926ms 1.0 1005ms 924ms static if 4267ms 4.7 911ms 0.9 1828ms 917ms instance ternary 4577ms 2.7 1681ms 1.8 994ms 917ms instance if 4592ms 2.9 1604ms 1.7 1611ms 969ms I also did measurements with a warm-up phase of various length with the conclusion that there is no benefit for either language with or without the @CompileStatic. Since the Fibonacci algorithm is that recursive the warm-up phase seems to be "included" for any Fibonacci number that is not very small. We can see that the performance improvements due to static typing have made quite a difference. This little comparison does little justice, though. To me, the impression that static typing in Groovy has had in conjunction with type inference has led to significant performance improvements—and in the same way it has led to Groovy++ becoming very strong. With the @CompileStatic, the performance of Groovy is about 1-2 times slower than Java, and without Groovy, it's about 3-5 times slower. Unhappily, the measurements of "instance ternary" and "instance if" are the slowest. Unless we want to create masterpieces in programming with static functions, the measurements for "static ternary" and "static if" are not that relevant for most of the code with the ambition to be object-oriented (based on instances). Conclusion When Groovy was about 10-20 times slower than Java (see benchmark table almost at the end of this article) it is questionable whether the @CompileStatic was used or not. This means to me that Groovy is ready for applications where performance has to be somewhat comparable to Java. Earlier, Groovy (or Ruby, Closure, etc.) could only serve as a plus on your CV because of the performance impediment (at least here in Europe). New JVM kid on the block: Kotlin I added the figures for Kotlin as well (here is the code). Kotlin is a relatively new statically typed JVM-based Java-compatible programming language. Kotlin is more concise than Java by supporting variable type inferences, higher-order functions (closures), extension functions, mixins and first-class delegation, etc. Contrary to Groovy, it is more geared towards Scala, but also integrates well with Java. Kotlin is still under development and has yet to be officially released. So the figures have to be taken with caution as the guys at JetBrains are still working on the code optimization. Ideally, Kotlin should be as fast as Java. The measurements were done with the current "official" release 0.1.2580. And what about future performance improvements? At the time when JDK1.3 was the most recent JDK, I still earned my pay with Smalltalk development. At that time the performance of VisualWorks Smalltalk (now Cincom Smalltalk) and IBM VA for Smalltalk (now owned by Instantiations) was very good comparable to Java. And Smalltalk is a dynamically typed language, like pre-Goovy 2.0 and Ruby, where the compiler cannot make use of type inference to do optimizations. Because of this, it always appeared strange to me that Groovy, Ruby and other JVM-based dynamic languages had such a big performance penalty compared to Java when Smalltalk had not. From that point of view I think there's still room for Groovy performance improvements beyond @CompileStatic.

Even before talking about the technicalities of converting an image to a canvas element, check out the demo! DEMO input any image URL there and hit the convert button! P.S : It also accepts data-uri! The code : function draw() { // Get the canvas element and set the dimensions. var canvas = document.getElementById('canvas'); canvas.height = window.innerHeight; canvas.width = window.innerWidth; // Get a 2D context. var ctx = canvas.getContext('2d'); // create new image object to use as pattern var img = new Image(); img.src = document.getElementById('url').value; img.onload = function(){ // Create pattern and don't repeat! var ptrn = ctx.createPattern(img,'no-repeat'); ctx.fillStyle = ptrn; ctx.fillRect(0,0,canvas.width,canvas.height); } } The Magic Behind : All the credits goes to createPattern() nsIDOMCanvasPattern createPattern(in nsIDOMHTMLElement image, in DOMString repetition); Elobrating : context.createPattern(image,"repeat|repeat-x|repeat-y|no-repeat"); Hope this was useful, anyway there are loads of fun with the canvas element, happy hacking! Edit 0 After the interesting question by @pinkham in the comment section, from the page of MDN : Although you can use images without CORS approval in your canvas, doing so taints the canvas. Provided that you have a server hosting images along with appropriate Access-Control-Allow-Origin header, you will be able to save those images to localStorage as if they were served from your domain. var img = new Image, canvas = document.createElement("canvas"), ctx = canvas.getContext("2d"), src = "http://example.com/image"; // insert image url here img.crossOrigin = "Anonymous"; img.onload = function() { canvas.width = img.width; canvas.height = img.height; ctx.drawImage( img, 0, 0 ); localStorage.setItem( "savedImageData", canvas.toDataURL("image/png") ); } img.src = src; // make sure the load event fires for cached images too if ( img.complete || img.complete === undefined ) { img.src = "data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw=="; img.src = src; }

Recently I have come across a lot of people asking how to read data from, or write data to the middle of an XML document. In this post I will demonstrate how this can be done using JAXB with StAX. Note: JAXB (JSR-222) and StAX (JSR-173) implementations are included in the JDK/JRE since Java SE 6. XML (input.xml) We will be using a SOAP message as our sample XML. The outer portions of the XML document represent information relevant to the Web Service and the inner portions (lines 5-8) represent the data we want to convert to our domain model. Jane Doe Java Model Our Java model consists of a single domain class. The concepts in this example also apply to larger domain models. package blog.stax.middle; import javax.xml.bind.annotation.*; @XmlAccessorType(XmlAccessType.FIELD) public class Customer { @XmlAttribute int id; String firstName; String lastName; } Unmarshal Demo To unmarshal from the middle of an XML document all we need to do is the following: Create an XMLStreamReader from the XML input (line 12). Advance the XMLStreamReader to the return element (lines 13-16). Unmarshal an instance of Customer from the XMLStreamReader (line 20) package blog.stax.middle; import javax.xml.bind.*; import javax.xml.stream.*; import javax.xml.transform.stream.StreamSource; public class UnmarshalDemo { public static void main(String[] args) throws Exception { XMLInputFactory xif = XMLInputFactory.newFactory(); StreamSource xml = new StreamSource("src/blog/stax/middle/input.xml"); XMLStreamReader xsr = xif.createXMLStreamReader(xml); xsr.nextTag(); while(!xsr.getLocalName().equals("return")) { xsr.nextTag(); } JAXBContext jc = JAXBContext.newInstance(Customer.class); Unmarshaller unmarshaller = jc.createUnmarshaller(); JAXBElement jb = unmarshaller.unmarshal(xsr, Customer.class); xsr.close(); Customer customer = jb.getValue(); System.out.println(customer.id); System.out.println(customer.firstName); System.out.println(customer.lastName); } } Output Below is the output from running the unmarshal demo. 123 Jane Doe Marshal Demo To marshal to the middle of an XML document all we need to do is the following: Create an XMLStreamWriter for the XML output (line 18). Start the document and write the outer elements (lines 19-22). Set the Marshaller.JAXB_FRAGMENT property on the Marshaller (line 26) to prevent the XML declaration from being written. Marshal an instance of Customer to the XMLStreamWriter (line 27). End the document, this will close any elements that have been opened (line 29). package blog.stax.middle; import javax.xml.bind.*; import javax.xml.namespace.QName; import javax.xml.stream.*; public class MarshalDemo { public static void main(String[] args) throws Exception { Customer customer = new Customer(); customer.id = 123; customer.firstName = "Jane"; customer.lastName = "Doe"; QName root = new QName("response"); JAXBElement je = new JAXBElement(root, Customer.class, customer); XMLOutputFactory xof = XMLOutputFactory.newFactory(); XMLStreamWriter xsw = xof.createXMLStreamWriter(System.out); xsw.writeStartDocument(); xsw.writeStartElement("S", "Envelope", "http://schemas.xmlsoap.org/soap/envelope/"); xsw.writeStartElement("S", "Body", "http://schemas.xmlsoap.org/soap/envelope/"); xsw.writeStartElement("ns0", "findCustomerResponse", "http://service.jaxws.blog/"); JAXBContext jc = JAXBContext.newInstance(Customer.class); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FRAGMENT, true); marshaller.marshal(je, xsw); xsw.writeEndDocument(); xsw.close(); } } Output Below is the output from running the marshal demo. Note that the output from running the demo code will appear on a single line, I have formatted the output here to make it easier to read. Jane Doe

Recently I came upon a groovy oddity. (At least it is perceived by me to be an oddity). Closures in a groovy class do not have access to a private method if that method is defined in the superclass. This seems odd paired against the fact that regular methods in a super class can access private method defined in the super class Background Groovy closures have the same scope access to class member variables and methods as a regular groovy method. In other words, closures are bound to variables in the scope they are defined. See the codehaus link for the official documentation: http://groovy.codehaus.org/Closures This implies that a closure will play by the rules of Object Orientation in the Java language. However, I found that closures do not have access to private methods that are defined in a super class. The best way to demonstrate is through a short example from Grails. I have used TDD for the example. Note this is a dummy case with no business purpose. Later on I will offer a more reasonable scenario in the business context where I encountered this scenario. Simple Groovy Example Take a class that has a closure, a public method, and a private method. Then extend that class. Try and invoke the closure. We get an error. package closure.access class SendCheckService { def calendarService /** * Closure that invokes a private method */ def closureToSendCheck = { sendPersonalCheck() } def regularMethod() { sendPersonalCheck() } /* * Private method that we want to see executed */ private sendPersonalCheck() { println "Sending Personal Check" } } class FooService extends SendCheckService { } Here are some tests the demonstrate the error. package closure.access import grails.test.* class SendCheckServiceTests extends GrailsUnitTestCase { def sendCheckService protected void setUp() { super.setUp() sendCheckService = new SendCheckService() } /** * Invocation of the closure from the super class prints out the message: * "Sending Personal Check" */ void testClosureToSendCheck() { sendCheckService.closureToSendCheck() } /** * Invocation of the method from the super class prints out the message: * "Sending Personal Check" */ void testRegularMethod() { sendCheckService.regularMethod() } /** * Invocation of the the closure from the subclass yields an error: * groovy.lang.MissingMethodException: No signature of method: closure.access.FooService.sendPersonalCheck() * is applicable for argument types: () values: [] * This essentially means it does not exists. */ void testFoo_ClosureToSendCheck() { def fooService = new FooService() fooService.closureToSendCheck() } /** * Invocation of the method does not yield and error! */ void testFoo_RegularMethod() { def fooService = new FooService() fooService.regularMethod() } } All is Well testClosureToSendCheck() executes fine where the closure can access the private method. This is as expected. It is all well and good because we have not extended class yet. testRegularMethod() just demonstrates regular OO principles. A method is able to invoke private methods. Not as Expected testFoo_ClosureToSendCheck() invokes a subclass of SendCheckService. It calls the same closure, yet we get served up a MissingMethodException. testFoo_RegularMethod() just contrasts testFoo_ClosureToSendCheck(). I invoked this test to show that we should be able to have closures access private methods because other regular methods can! Why Even Try to Have a Closure Call A Private Method This may be a double loop learning question any intelligent developer might ask. It questions why we need to even get into this mess. This is a valid point and should be explained. It is optimal to use closures in an attempt to reuse existing template logic. Let me give a simple business problem. Imagine we need to code a system that sends out various types of checks: personal checks and business checks. We have the stipulation that these two events must NEVER be done together. Only send a personal check at one instance, and send a business check at another time. However, they both need to follow the same logic. They must be sent on a business day (no holidays or weekends). Thus, we have a scenario where they need the same calendar logic, but it is needed separately. Duplicate Logic We could just code the calendar logic twice. (Remember sending business checks and personal checks together in one request cannot occur!) package closure.access class SendCheckService2 { def calendarService def triggerPersonalCheck () { if (calendarService.todayIsBusinessDay()) { sendPersonalCheck() } else { println "DO NOTHING" } } def triggerBusinessCheck() { if (calendarService.todayIsBusinessDay()) { sendBusinessCheck() } else { println "DO NOTHING" } } private sendPersonalCheck() { println "Sending Personal Check" } private sendBusinessCheck() { println "Sending Business Check" } } Use the DRY Principle In order to avoid this and follow DRY, we can use closures. Create a method that accepts a closure, and pass it the code snippets to execute in a closure. As a result we have the calendar logic defined once, but executed separately upon a different code snippet. package closure.access class SendCheckService3 { def calendarService def triggerPersonalCheck() { checkIfBusinessDayAndExecute(sendBusinessCheck) } def triggerBusinessCheck() { checkIfBusinessDayAndExecute(sendBusinessCheck) } def checkIfBusinessDayAndExecute(Closure closure) { if (calendarService.todayIsBusinessDay()) { closure() } else { println "DO NOTHING" } } /** * This is now a closure we can pass around */ def sendPersonalCheck = { println "Sending Personal Check" } /** * This is now a closure we can pass around */ def sendBusinessCheck = { println "Sending Business Check" } } In my real world scenario where I encountered the closure issue, it happened that my closure was trying to execute a private method in an abstract class. This is where I observed the problem. JVM Thoughts I honestly do not know the gory details behind why closures in super classes cannot access private methods, but I have an idea. Groovy creates closures by compiling them as inner classes. Since the subclass extends the superclass and then contains a closure, the inner class does not have access to the super class' methods. The reason why a method has access, is because it is compiled as one instance of the class. The closure implementation is not that way (being a inner class), and thus this is why we see a violation of Object Oriented behavior. If you have a better explanation or futher knowledge of the details behind this issue, please describe them in the comments. Thanks for taking the time to delve in this area. Thanks to Scott Risk for helping with examples.

Original Article: http://borislam.blogspot.hk/2012/07/adding-hibernate-entity-level-filter.html Those who have used data filtering features of hibernate should know that it is very powerful. You could define a set of filtering criteria to an entity class or a collection. Spring data JPA is a very handy library but it does not have fitering features. In this post, I will demonstarte how to add the hibernate filter features at entity level. You can use this features when you are using Hibernate Entity Manager. We can just define annotation in your repositoy interface to enable this features. Step 1. Define filter at entity level as usual. Just use hibernate @FilterDef annotation @Entity @Table(name = "STUDENT") @FilterDef(name="filterBySchoolAndClass", parameters={@ParamDef(name="school", type="string"),@ParamDef(name="class", type="integer")}) public class Student extends GenericEntity implements Serializable { // add your properties ... } Step2. Define two custom annotations. These two annotations are to be used in your repository interfaces. You could apply the hibernate filter defined in step 1 to specific query through these annotations. @Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME) public @interface EntityFilter { FilterQuery[] filterQueries() default {}; } @Retention(RetentionPolicy.RUNTIME) public @interface FilterQuery { String name() default ""; String jpql() default ""; } Step3. Add a method to your Spring data JPA base repository. This method will read the annotation you defined (i.e. @FilterQuery) and apply hibernate filter to the query by just simply unwrap the EntityManager. You could specify the parameter in your hibernate filter and also the parameter in you query in this method. If you do not know how to add custom method to your Spring data JPA base repository, please see my previous article for how to customize your Spring data JPA base repository for detail. You can see in previous article that I intentionally expose the repository interface (i.e. the springDataRepositoryInterface property) in the GenericRepositoryImpl. This small tricks enable me to access the annotation in the repository interface easily. public List doQueryWithFilter( String filterName, String filterQueryName, Map inFilterParams, Map inQueryParams){ if (GenericRepository.class.isAssignableFrom(getSpringDataRepositoryInterface())) { Annotation entityFilterAnn = getSpringDataRepositoryInterface().getAnnotation(EntityFilter.class); if(entityFilterAnn != null){ EntityFilter entityFilter = (EntityFilter)entityFilterAnn; FilterQuery[] filterQuerys = entityFilter.filterQueries() ; for (FilterQuery fQuery : filterQuerys) { if (StringUtils.equals(filterQueryName, fQuery.name())) { String jpql = fQuery.jpql(); Filter filter = em.unwrap(Session.class).enableFilter(filterName); //set filter parameter for (Object key: inFilterParams.keySet()) { String filterParamName = key.toString(); Object filterParamValue = inFilterParams.get(key); filter.setParameter(filterParamName, filterParamValue); } //set query parameter Query query= em.createQuery(jpql); for (Object key: inQueryParams.keySet()) { String queryParamName = key.toString(); Object queryParamValue = inQueryParams.get(key); query.setParameter(queryParamName, queryParamValue); } return query.getResultList(); } } } } } return null; } Last Step: example usage In your repositry, define which query you would like to apply hibernate filter through your @EntityFilter and @FilterQuery annotation. @EntityFilter ( filterQueries = { @FilterQuery(name="query1", jpql="SELECT s FROM Student LEFT JOIN FETCH s.Subject where s.subject = :subject" ), @FilterQuery(name="query2", jpql="SELECT s FROM Student LEFT JOIN s.TeacherSubject where s.teacher = :teacher") } ) public interface StudentRepository extends GenericRepository { } In your service or business class that inject your repository, you could just simply call the doQueryWithFilter() method to enable the filtering function. @Service public class StudentService { @Inject private StudentRepository studentRepository; public List searchStudent( String subject, String school, String class) { List studentList; // Prepare parameters for query filter HashMap inFilterParams = new HashMap(); inFilterParams.put("school", "Hong Kong Secondary School"); inFilterParams.put("class", "S5"); // Prepare parameters for query HashMap inParams = new HashMap(); inParams.put("subject", "Physics"); studentList = studentRepository.doQueryWithFilter( "filterBySchoolAndClass", "query1", inFilterParams, inParams); return studentList; } }

The more I use dependency injection (DI) in my code, the more it alters the way I see both my design and implementation. Injection is so convenient and powerful that you end up wanting to make sure you use it as often as you can. And as it turns out, you can use it in many, many places. Let’s cover briefly the most obvious scenarios where DI, and more specifically, Guice, are a good fit: objects created either at class loading time or very early in your application. These two aspects are covered by either direct injection or by providers, which allow you to start building some of your object graph before you can inject more objects. I won’t go too much in details about these two use cases since they are explained in pretty much any Guice tutorial you can find on the net. Once the injector has created your graph of objects, you are pretty much back to normal and instantiating your “runtime objects” (the objects you create during the life time of your application) the normal way, most likely with “new” or factories. However, you will quickly start noticing that you need some runtime information to create these objects, other parts of them could be injected. Let’s take the following example: we have a GeoService interface that provides various geolocation functions, such as telling you if two addresses are close to each other: public interface GeoService { /** * @return true if the two addresses are within @param{miles} * miles of each other. */ boolean isNear(Address address1, Address address2, int miles); } Then you have a Person class which uses this service and also needs a name and an address to be instantiated: public class Person { // Fields omitted public Person(String name, Address address, GeoService gs) { this.name = name; this.address = address; this.geoService = gs; } public boolean livesNear(Person otherPerson) { return geoService.isNear(address, otherPerson.getAddress(), 2 /* miles */); } } Something odd should jump at you right away with this class: while name and address are part of the identity of a Person object, the presence of the GeoService instance in it feels wrong. The service is a singleton that is created on start up, so a perfect candidate to be injected, but how can I achieve the creation of a Person object when some of its information is supplied by Guice and the other part by myself? Guice gives you a very elegant and flexible way to implement this scenario with “assisted injection”. The first step is to define a factory for our objects that represents exactly how we want to create them: public interface PersonFactory { Person create(String name, Address address); } Since only name and address participate in the identity of our Person objects, these are the only parameters we need to construct our objects. The other parameters should be supplied by Guice so we modify our Person constructor to let Guice know: @Inject public Person(@Assisted String name, @Assisted Address address, GeoService geoService) { this.name = name; this.address = address; this.geoService = geoService; } In this code, I have added an @Inject annotation on the constructor and an @Assisted annotation on each parameter that I will be providing. Guice will take care of injecting the rest. Finally, we connect the factory to its objects when creating the module: Module module1 = new FactoryModuleBuilder() .build(PersonFactory.class); The important part here is to realize that we will never instantiate PersonFactory: Guice will. From now on, all we need to do whenever we want to instantiate a Person object is to ask Guice to hand us a factory: @Inject private PersonFactory personFactory; // ... Person p = personFactory.create("Bob", new Address("1 Ocean st")); If you want to find out more, take a look at the main documentation for assisted injection, which explains how to support overloaded constructors and also how to create different kinds of objects within the same factory. Wrapping up Let’s take a look at what we did. First, we started with a suspicious looking constructor: public Person(String name, Address address, GeoService s) { This constructor is suspicious because it accepts parameters that do not participate in the identity of the object (you won’t use the GeoService parameter when calculating the hash code of a Person object). Instead, we replaced this constructor with a factory that only accepts identity fields: public interface PersonFactory { Person create(String name, Address address); } and we let Guice’s assisted injection take care of creating a fully formed object for us. This observation leads us to the Identity Constructor rule: If a constructor accepts parameters that are not used to define the identity of the objects, consider injecting these parameters. Once you start looking at your objects with this rule in mind, you will be surprised to find out how many of them can benefit from assisted injection.

DrupalCon Munich is next week, and I am lucky enough to be going. As part of preparing for the conference, I thought it would be worthwhile to see just how easy (or difficult) it would be to migrate an existing Drupal site to Windows Azure Web Sites. So, in this post, I’ll do just that. Fortunately, because Windows Azure Web Sites supports both PHP and MySQL, the migration process is relatively straightforward. And, because Drupal and PHP run on any platform, the process I’ll describe should work for moving Drupal to Windows Azure Web Sites regardless of what platform you are moving from. Of course, Drupal installations can vary widely, so YMMV. I tested the instructions below on relatively small (and simple) Drupal installation running on CentOS 5. (Unfortunately, I won’t be using Drush since it isn’t supported on Windows Azure Websites.) If you are considering moving a large and complex Drupal application, may want to consider moving to Windows Azure Cloud Services (more information about that here: Migrating a Drupal Site from LAMP to Windows Azure). Before getting started, it’s worth noting that Windows Azure Websites lets you run up to 10 Web Sites for free in a multitenant environment. And, you can seamlessly upgrade to private, reserved VM instances as your traffic grows. To sign up, try the Windows Azure 90-day free trial. 1. Create a Windows Azure Web Site and MySQL database There is a step-by-step tutorial on http://www.windowsazure.com that walks you through creating a new website and a MySQL database, so I’ll refer you there to get started: Create a PHP-MySQL Windows Azure web site and deploy using Git. If you intend to use Git to publish your Drupal site, then go ahead and follow the instructions for setting up a Git repository. Make sure to follow the instructions in the Get remote MySQL connection information section as you will need that information later. You can ignore the remainder of the tutorial for the purposes of deploying your Drupal site, but if you are new to Windows Azure Web Sites (and to Git), you might find the additional reading informative. Ok, now you have a new website with a MySQL database, your have your MySQL database connection information, and you have (optionally) created a remote Git repository and made note of the Git deployment instructions. Now you are ready to copy your database to MySQL in Windows Azure Web Sites. 2. Copy database to MySQL in Windows Azure Web Sites I’m sure there is more than one way to copy your Drupal database, but I found the mysqldump tool to be effective and easy to use. To copy from a local machine to Windows Azure Web Sites, here’s the command I used: mysqldump -u local_username --password=local_password drupal | mysql -h remote_host -u remote_username --password=remote_password remote_db_name You will, of course, have to provide the username and password for your existing Drupal database, and you will have to provide the hostname, username, password, and database name for the MySQL database you created in step 1. This information is available in the connection string information that you should have noted in step 1. i.e. You should have a connection string that looks something like this: Database=remote_db_name;Data Source=remote_host;User Id=remote_username;Password=remote_password Depending on the size of your database, the copying process could take several minutes. Now your Drupal database is live in Windows Azure Websites. Before you deploy your Drupal code, you need to modify it so it can connect to the new database. 3. Modify database connection info in settings.php Here, you will again need your new database connection information. Open the /drupal/sites/default/setting.php file in your favorite text editor, and replace the values of ‘database’, ‘username’, ‘password’, and ‘host’ in the $databases array with the correct values for your new database. When you are finished, you should have something similar to this: $databases = array ( 'default' => array ( 'default' => array ( 'database' => 'remote_db_name', 'username' => 'remote_username', 'password' => 'remote_password', 'host' => 'remote_host', 'port' => '', 'driver' => 'mysql', 'prefix' => '', ), ), ); Be sure to save the settings.phpfile, then you are ready to deploy. 4. Deploy Drupal code using Git or FTP The last step is to deploy your code to Windows Azure Web Sites using Git or FTP. If you are using FTP, you can get the FTP hostname and username from you website’s dashboard. Then, use your favorite FTP client to upload your Drupal files to the /site/wwwroot folder of the remote site. If you are using Git, you need to set up a Git repository in Windows Azure Web Sites (steps for this are in the tutorial mentioned earlier). And, you will need Git installed on your local machine. Then, just follow the instructions provided after you created the repository: One note about using Git here: depending on your Git settings, your .gitignore file (a hidden file and a sibling to the .git folder created in your local root directory after you executed git commit), some files in your Drupal application may be ignored. In my case, all the files in the sites directory were ignored. If this happens, you will want to edit the .gitignore file so that these files aren’t ignored and redeploy. After you have deployed Drupal to Windows Azure Web Sites, you can continue to deploy updates via Git or FTP. Related information If you are looking for more information about Windows Azure Web Sites, these posts might be helpful: Windows Azure Websites- A PHP Perspective Windows Azure Websites, Web Roles, and VMs- When to use which- Configuring PHP in Windows Azure Websites with .user.ini Files One last thing you might consider, depending on your site, is using the Windows Azure Integration Module to store and serve your site’s media files.

This article will provide you with 8 ways to improve your production support skills which may help you better enjoy your IT support job.

Learn how to use spring profiles in an XML config with this great tutorial.

If you have already installed Ubuntu 12.04 you probably have realized that Sun java(oracle java) does not come prepacked with Ubuntu like it used to be , instead OpenJDK comes with it. Here is how you can install Oracle java on Ubuntu 12.04 manually. Download jdk-6u32-linux-x64.bin from this link. If you have used 32-bit Ubuntu installation, download jdk-6u32-linux-x32.bin instead. To make the downloaded bin file executable use the following command chmod +x jdk-6u32-linux-x64.bin To extract the bin file use the following command ./jdk-6u32-linux-x64.bin Using the following command create a folder called "jvm" inside /usr/lib if it is not already existing sudo mkdir /usr/lib/jvm Move the extracted folder into the newly created jvm folder sudo mv jdk1.6.0_32 /usr/lib/jvm/ To install the Java source use following commands sudo update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jdk1.6.0_32/bin/javac 1 sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.6.0_32/bin/java 1 sudo update-alternatives --install /usr/bin/javaws javaws /usr/lib/jvm/jdk1.6.0_32/bin/javaws 1 To make this default java sudo update-alternatives --config javac sudo update-alternatives --config java sudo update-alternatives --config javaws To make symlinks point to the new Java location use the following command ls -la /etc/alternatives/java* To verify Java has installed correctly use this command java -version

@XmlRootElement is an annotation that people are used to using with JAXB (JSR-222). It's purpose is to uniquely associate a root element with a class. Since JAXB classes map to complex types, it is possible for a class to correspond to multiple root elements. In this case @XmlRootElement can not be used and people start getting a bit confused. In this post I'll demonstrate how @XmlElementDecl can be used to map this use case. XML Schema The XML schema below contains three root elements: customer, billing-address, and shipping-address. The customer element has an anonymous complex type, while billing-address and shipping-address are of the same named type (address-type). createBillingAddress(AddressType value) { return new JAXBElement(_BillingAddress_QNAME, AddressType.class, null, value); } @XmlElementDecl(namespace = "http://www.example.org/customer", name = "shipping-address") public JAXBElement createShippingAddress(AddressType value) { return new JAXBElement(_ShippingAddress_QNAME, AddressType.class, null, value); } } package-info The package-info class is used to specify the namespace mapping (see JAXB & Namespaces). @XmlSchema(namespace = "http://www.example.org/customer", elementFormDefault = XmlNsForm.QUALIFIED) package org.example.customer; import javax.xml.bind.annotation.*; Unmarshal Operation Now we look at the impact of the type of root element when unmarshalling XML. customer.xml Below is a sample XML document with customer as the root element. Remember the customer element had an anonymous complex type. 1 Any Street 2 Another Road shipping.xml Here is a sample XML document with shipping-address as the root element. The shipping-address element had a named complex type. 2 Another Road Unmarshal Demo When unmarshalling XML that corresponds to a class annotated with @XmlRootElement you get an instance of the domain object. But when unmarshalling XML that corresponds to a class annotated with @XmlElementDecl you get the domain object wrapped in an instance of JAXBElement. In this example you may need to use the QName from the JAXBElement to determine if you unmarshalled a billing or shipping address. package org.example.customer; import java.io.File; import javax.xml.bind.*; public class UnmarshalDemo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance("org.example.customer"); Unmarshaller unmarshaller = jc.createUnmarshaller(); // Unmarshal Customer File customerXML = new File("src/org/example/customer/customer.xml"); Customer customer = (Customer) unmarshaller.unmarshal(customerXML); // Unmarshal Shipping Address File shippingXML = new File("src/org/example/customer/shipping.xml"); JAXBElement je = (JAXBElement) unmarshaller.unmarshal(shippingXML); AddressType shipping = je.getValue(); } } Unmarshal Demo - JAXBIntrospector If you don't want to deal with remembering whether the result of the unmarshal operation will be a domain object or JAXBElement, then you can use the JAXBIntrospector.getValue(Object) method to always get the domain object. package org.example.customer; import java.io.File; import javax.xml.bind.*; public class JAXBIntrospectorDemo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance("org.example.customer"); Unmarshaller unmarshaller = jc.createUnmarshaller(); // Unmarshal Customer File customerXML = new File("src/org/example/customer/customer.xml"); Customer customer = (Customer) JAXBIntrospector.getValue(unmarshaller .unmarshal(customerXML)); // Unmarshal Shipping Address File shippingXML = new File("src/org/example/customer/shipping.xml"); AddressType shipping = (AddressType) JAXBIntrospector .getValue(unmarshaller.unmarshal(shippingXML)); } } Marshal Operation You can directly marshal an object annotated with @XmlRootElement to XML. Classes corresponding to @XmlElementDecl annotations must first be wrapped in an instance of JAXBElement. The factory method you you annotated with @XmlElementDecl is the easiest way to do this. The factory method is in the ObjectFactory class if you generated your model from an XML schema. package org.example.customer; import javax.xml.bind.*; public class MarshalDemo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance("org.example.customer"); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); // Create Domain Objects AddressType billingAddress = new AddressType(); billingAddress.setStreet("1 Any Street"); Customer customer = new Customer(); customer.setBillingAddress(billingAddress); // Marshal Customer marshaller.marshal(customer, System.out); // Marshal Billing Address ObjectFactory objectFactory = new ObjectFactory(); JAXBElement je = objectFactory.createBillingAddress(billingAddress); marshaller.marshal(je, System.out); } } Output Below is the output from running the demo code. 1 Any Street 1 Any Street

The ExecutorService feature came with Java 5. It extends the Executor interface and provides a thread pool feature to execute asynchronous short tasks. There are five ways to execute the tasks asyncronously by using the ExecutorService interface provided Java 6. ExecutorService execService = Executors.newCachedThreadPool(); This approach creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for 60 seconds are terminated and removed from the cache. ExecutorService execService = Executors.newFixedThreadPool(10); This approach creates a thread pool that reuses a fixed number of threads. Created nThreads will be active at the runtime. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available. ExecutorService execService = Executors.newSingleThreadExecutor(); This approach creates an Executor that uses a single worker thread operating off an unbounded queue. Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Methods of the ExecutorService : execute(Runnable) : Executes the given command at some time in the future. submit(Runnable) : Submit method returns a Future Object which represents executed task. Future Object returns null if the task has finished correctly. shutdown() : Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down. shutdownNow() : Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution. There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt, so any task that fails to respond to interrupts may never terminate. A sample application is below : STEP 1 : CREATE MAVEN PROJECT A maven project is created as below. (It can be created by using Maven or IDE Plug-in). STEP 2 : CREATE A NEW TASK A new task is created by implementing the Runnable interface(creating Thread) as below. TestTask Class specifies business logic which will be executed. package com.otv.task; import org.apache.log4j.Logger; /** * @author onlinetechvision.com * @since 24 Sept 2011 * @version 1.0.0 * */ public class TestTask implements Runnable { private static Logger log = Logger.getLogger(TestTask.class); private String taskName; public TestTask(String taskName) { this.taskName = taskName; } public void run() { try { log.debug(this.taskName + " is sleeping..."); Thread.sleep(3000); log.debug(this.taskName + " is running..."); } catch (InterruptedException e) { e.printStackTrace(); } } STEP 3 : CREATE TestExecutorService by using newCachedThreadPool TestExecutorService is created by using the method newCachedThreadPool. In this case, created thread count is specified at the runtime. package com.otv; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import com.otv.task.TestTask; /** * @author onlinetechvision.com * @since 24 Sept 2011 * @version 1.0.0 * */ public class TestExecutorService { public static void main(String[] args) { ExecutorService execService = Executors.newCachedThreadPool(); execService.execute(new TestTask("FirstTestTask")); execService.execute(new TestTask("SecondTestTask")); execService.execute(new TestTask("ThirdTestTask")); execService.shutdown(); } } When TestExecutorService is run, the output will be seen as below : 24.09.2011 17:30:47 DEBUG (TestTask.java:21) - SecondTestTask is sleeping... 24.09.2011 17:30:47 DEBUG (TestTask.java:21) - ThirdTestTask is sleeping... 24.09.2011 17:30:47 DEBUG (TestTask.java:21) - FirstTestTask is sleeping... 24.09.2011 17:30:50 DEBUG (TestTask.java:23) - ThirdTestTask is running... 24.09.2011 17:30:50 DEBUG (TestTask.java:23) - FirstTestTask is running... 24.09.2011 17:30:50 DEBUG (TestTask.java:23) - SecondTestTask is running... STEP 4 : CREATE TestExecutorService by using newFixedThreadPool TestExecutorService is created by using the method newFixedThreadPool. In this case, required thread count has to be set as the following : package com.otv; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import com.otv.task.TestTask; /** * @author onlinetechvision.com * @since 24 Sept 2011 * @version 1.0.0 * */ public class TestExecutorService { public static void main(String[] args) { ExecutorService execService = Executors.newFixedThreadPool(2); execService.execute(new TestTask("FirstTestTask")); execService.execute(new TestTask("SecondTestTask")); execService.execute(new TestTask("ThirdTestTask")); execService.shutdown(); } } When TestExecutorService is run, ThirdTestTask is executed after FirstTestTask and SecondTestTask’ s executions are completed. The output will be seen as below: 24.09.2011 17:33:38 DEBUG (TestTask.java:21) - FirstTestTask is sleeping... 24.09.2011 17:33:38 DEBUG (TestTask.java:21) - SecondTestTask is sleeping... 24.09.2011 17:33:41 DEBUG (TestTask.java:23) - FirstTestTask is running... 24.09.2011 17:33:41 DEBUG (TestTask.java:23) - SecondTestTask is running... 24.09.2011 17:33:41 DEBUG (TestTask.java:21) - ThirdTestTask is sleeping... 24.09.2011 17:33:44 DEBUG (TestTask.java:23) - ThirdTestTask is running... STEP 5 : CREATE TestExecutorService by using newSingleThreadExecutor TestExecutorService is created by using the method newSingleThreadExecutor. In this case, only one thread is created and tasks are executed sequentially. package com.otv; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import com.otv.task.TestTask; /** * @author onlinetechvision.com * @since 24 Sept 2011 * @version 1.0.0 * */ public class TestExecutorService { public static void main(String[] args) { ExecutorService execService = Executors.newSingleThreadExecutor(); execService.execute(new TestTask("FirstTestTask")); execService.execute(new TestTask("SecondTestTask")); execService.execute(new TestTask("ThirdTestTask")); execService.shutdown(); } } When TestExecutorService is run, SecondTestTask and ThirdTestTask is executed after FirstTestTask’ s execution is completed. The output will be seen as below : 24.09.2011 17:38:21 DEBUG (TestTask.java:21) - FirstTestTask is sleeping... 24.09.2011 17:38:24 DEBUG (TestTask.java:23) - FirstTestTask is running... 24.09.2011 17:38:24 DEBUG (TestTask.java:21) - SecondTestTask is sleeping... 24.09.2011 17:38:27 DEBUG (TestTask.java:23) - SecondTestTask is running... 24.09.2011 17:38:27 DEBUG (TestTask.java:21) - ThirdTestTask is sleeping... 24.09.2011 17:38:30 DEBUG (TestTask.java:23) - ThirdTestTask is running... STEP 6 : REFERENCES http://download.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html http://tutorials.jenkov.com/java-util-concurrent/executorservice.html

In my office laptop, I have installed two versions of JDK. For the office work, I need JDK6 because the internal framework needs it. I’m using JDK7 for my personal projects and exploring the latest and greatest in Java. I have two versions of Eclipse too (one for office work and one is the latest Juno). But, the tricky thing is to manage these multiple JDKs and IDEs. It’s a piece of cake if I just use Eclipse for compiling my code, because the IDE allows me to configure multiple versions of Java runtime. Unfortunately (or fortunately), I have to use the command line/shell to build my code. So, it is important that I have the right version of JDK present in the PATH and other related environment variables (such as JAVA_HOME). Manually modifying the environment variables every time I want to switch between JDKs, isn’t a happy task. But, thanks to Windows Powershell, I’m able to write a scriplet that can do the heavy-lifting for me. Basically, what I want to achieve is to set PATH variable to add Java bin folder and set the JAVA_HOME environment variable and then launch the correct Eclipse IDE. And, I want to do this with a single command. Let’s do it. Open a Windows Powershell. I prefer writing custom Windows scripts in my profile file so that it is available to run when ever I open the shell. To edit the profile, run this command: notepad.exe $profile - the $profile is a special variable that points to your profile file. Write the below script in the profile file and save it. function myIDE{ $env:Path += "C:\vraa\java\jdk7\bin;" $env:JAVA_HOME = "C:\vraa\java\jdk7" C:\vraa\ide\eclipse\eclipse set-location C:\vraa\workspace\myproject play } function officeIDE{ $env:Path += "C:\vraa\java\jdk6\bin;" $env:JAVA_HOME = "C:\vraa\java\jdk6" C:\office\eclipse\eclipse } Close and restart the Powershell. Now you can issue the command myIDE which will set the proper PATH and environment variables and then launch the eclipse IDE. As you can see, there are two functions with different configurations. Just call the function name that you want to launch from the Powershell command line (myIDE or officeIDE).

There appears to be a misconception that annotations are required on the model in order to use a JAXB (JSR-222) implementation. The truth is that JAXB is configuration by exception, so annotations are only required when you want to override default behaviour. In this example I'll demonstrate how to use JAXB without providing any metadata. Domain Model I will use the following domain model for this example. Note how there are no annotations of any kind. Customer Customer is the root object in this example. Normally we would annotate it with @XmlRootElement. Later in the demo code you will see how we can use an instance of JAXBElement instead. package blog.defaults; import java.util.List; public class Customer { private String firstName; private String lastName; private List phoneNumbers; public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } public List getPhoneNumbers() { return phoneNumbers; } public void setPhoneNumbers(List phoneNumbers) { this.phoneNumbers = phoneNumbers; } } PhoneNumber I have purposefully given the fields in this class nonsense names, so that later when we look at the XML you will be able to see that by default the element names are derived from the properties and not the fields. package blog.defaults; public class PhoneNumber { private String foo; private String bar; public String getType() { return foo; } public void setType(String type) { this.foo = type; } public String getNumber() { return bar; } public void setNumber(String number) { this.bar = number; } } Demo Code Since we haven't used @XmlRootElement (or @XmlElementDecl) to associate a root element with our Customer class we will need to tell JAXB what class we want to unmarshal the XML document to. This is done by using one of the unmarshal methods that take a Class parameter (line 14). This will return a JAXBElement, the Customer object is then accessed by calling getValue on it (line 15). To marshal the object back to XML we need to ensure that it is wrapped in a JAXBElement to supply the root element information (line 17). package blog.defaults; import javax.xml.bind.*; import javax.xml.namespace.QName; import javax.xml.transform.stream.StreamSource; public class Demo { public static void main(String[] args) throws Exception { JAXBContext jc = JAXBContext.newInstance(Customer.class); StreamSource xml = new StreamSource("src/blog/defaults/input.xml"); Unmarshaller unmarshaller = jc.createUnmarshaller(); JAXBElement je1 = unmarshaller.unmarshal(xml, Customer.class); Customer customer = je1.getValue(); JAXBElement je2 = new JAXBElement(new QName("customer"), Customer.class, customer); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); marshaller.marshal(je2, System.out); } } input.xml/Output The following is the input to and output from running the demo code. The first thing we see is that it is a very reasonable XML representation of the data, there aren't any JAXB artifacts. By default JAXB will marshal everything as XML elements, and based on our PhoneNumber class we see that the element names were derived from the property names. Jane Doe 555-1111 work 555-2222 home Further Reading If you enjoyed this post then you may also be interested in: The majority of the articles on this blog describe how to leverage the power of JAXB's metadata to support different use cases, I invite you to check them out: http://blog.bdoughan.com/search/label/JAXB If you are interested in specifying metadata without using annotations, you may be interested in EclipseLink JAXB (MOXy)'s external mapping document: Extending JAXB - Representing Metadata as XML Extending JAXB - Representing Metadata as JSON

Mustache is templating system with implementation in many languages including Java and JavaScript . The templates are also supported by various web frameworks and client side JS libraries. Mustache has simple idea of "logic-less" system because it lacks any explicit control statements, like if, else or goto and also it does not have for statement however looping and conditional calculation can be achieved using custom tags that work with lists and lambdas. The name unfortunately has less to do with Tom Selleck but more with the heavy use of curly braces that look like mustache. The similarity is more than comparable. Mustache has implementation for most of the widely used languages like: Java, Javascript, Ruby,Net and many more. The client side template's in JavaScript Let say that you have some REST service and you have created a book view object that has an additional function that appends amazon associates id to the book url: var book = { id : 12, title : "A Game of Thrones", url : "http://www.amazon.com/gp/product/0553573403/", amazonId : "myAwesomeness", associateUrl : function() { return this.url + '?tag=' + this.amazonId; }, author : { name : 'George R. R. Martin', imdbUrl : 'http://www.imdb.com/name/nm0552333/', wikiUrl : 'https://en.wikipedia.org/wiki/George_R._R._Martin' }, haveInStock : true, similarBooks : [{ id : 13, title : "Decision Points" }, { id : 13, title : "Spoken from the Heart" }], comments : [] }; The standard way of rendering data without using templates would be create an output variable and just append everything inside and at the end just place the data where it should be. jQuery(document).ready(function() { var out = '' + book.title + ' is awesome book get it on Amazon'; jQuery('#content-jquery').html(out); }); This is fairly simple but if you for example want to change the span element with div it takes a little bit of time to figure where it should be closed and often you can miss if the element should be in single quotes or double quotes. The bigger issue here is that the content is peaces of strings that need to be easy to styled via CSS and JavaScript. As the code gets bigger this becomes unmanageable and changes to anything become slower especially if you add on top of this jQuery's manipulation functions like appendTo() or prependTo(). This direct use of out+= type of creating the content reminds me a lot of HttpServlet style of using print writer and doing out.print() and for the same reason why this was almost abandoned we should not do this in JavaScript. To simplify work we can add template engine like Mustache that is one of many client side tempting engines. So how does a template in mustache looks like, well for the example above with the book it would look like :

Don’t be afraid – Some clarification to common misunderstandings Since version 3.1, Apache Lucene and Solr use MMapDirectory by default on 64bit Windows and Solaris systems; since version 3.3 also for 64bit Linux systems. This change lead to some confusion among Lucene and Solr users, because suddenly their systems started to behave differently than in previous versions. On the Lucene and Solr mailing lists a lot of posts arrived from users asking why their Java installation is suddenly consuming three times their physical memory or system administrators complaining about heavy resource usage. Also consultants were starting to tell people that they should not use MMapDirectory and change their solrconfig.xml to work instead with slow SimpleFSDirectory or NIOFSDirectory (which is much slower on Windows, caused by a JVM bug #6265734). From the point of view of the Lucene committers, who carefully decided that using MMapDirectory is the best for those platforms, this is rather annoying, because they know, that Lucene/Solr can work with much better performance than before. Common misinformation about the background of this change causes suboptimal installations of this great search engine everywhere. In this blog post, I will try to explain the basic operating system facts regarding virtual memory handling in the kernel and how this can be used to largely improve performance of Lucene (“VIRTUAL MEMORY for DUMMIES”). It will also clarify why the blog and mailing list posts done by various people are wrong and contradict the purpose of MMapDirectory. In the second part I will show you some configuration details and settings you should take care of to prevent errors like “mmap failed” and suboptimal performance because of stupid Java heap allocation. Virtual Memory[1] Let’s start with your operating system’s kernel: The naive approach to do I/O in software is the way, you have done this since the 1970s – the pattern is simple: whenever you have to work with data on disk, you execute a syscall to your operating system kernel, passing a pointer to some buffer (e.g. a byte[] array in Java) and transfer some bytes from/to disk. After that you parse the buffer contents and do your program logic. If you don’t want to do too many syscalls (because those may cost a lot processing power), you generally use large buffers in your software, so synchronizing the data in the buffer with your disk needs to be done less often. This is one reason, why some people suggest to load the whole Lucene index into Java heap memory (e.g., by using RAMDirectory). But all modern operating systems like Linux, Windows (NT+), MacOS X, or Solaris provide a much better approach to do this 1970s style of code by using their sophisticated file system caches and memory management features. A feature called “virtual memory” is a good alternative to handle very large and space intensive data structures like a Lucene index. Virtual memory is an integral part of a computer architecture; implementations require hardware support, typically in the form of a memory management unit (MMU) built into the CPU. The way how it works is very simple: Every process gets his own virtual address space where all libraries, heap and stack space is mapped into. This address space in most cases also start at offset zero, which simplifies loading the program code because no relocation of address pointers needs to be done. Every process sees a large unfragmented linear address space it can work on. It is called “virtual memory” because this address space has nothing to do with physical memory, it just looks like so to the process. Software can then access this large address space as if it were real memory without knowing that there are other processes also consuming memory and having their own virtual address space. The underlying operating system works together with the MMU (memory management unit) in the CPU to map those virtual addresses to real memory once they are accessed for the first time. This is done using so called page tables, which are backed by TLBs located in the MMU hardware (translation lookaside buffers, they cache frequently accessed pages). By this, the operating system is able to distribute all running processes’ memory requirements to the real available memory, completely transparent to the running programs. Schematic drawing of virtual memory (image from Wikipedia [1], http://en.wikipedia.org/wiki/File:Virtual_memory.svg, licensed by CC BY-SA 3.0) By using this virtualization, there is one more thing, the operating system can do: If there is not enough physical memory, it can decide to “swap out” pages no longer used by the processes, freeing physical memory for other processes or caching more important file system operations. Once a process tries to access a virtual address, which was paged out, it is reloaded to main memory and made available to the process. The process does not have to do anything, it is completely transparent. This is a good thing to applications because they don’t need to know anything about the amount of memory available; but also leads to problems for very memory intensive applications like Lucene. Lucene & Virtual Memory Let’s take the example of loading the whole index or large parts of it into “memory” (we already know, it is only virtual memory). If we allocate a RAMDirectory and load all index files into it, we are working against the operating system: The operating system tries to optimize disk accesses, so it caches already all disk I/O in physical memory. We copy all these cache contents into our own virtual address space, consuming horrible amounts of physical memory (and we must wait for the copy operation to take place!). As physical memory is limited, the operating system may, of course, decide to swap out our large RAMDirectory and where does it land? – On disk again (in the OS swap file)! In fact, we are fighting against our O/S kernel who pages out all stuff we loaded from disk [2]. So RAMDirectory is not a good idea to optimize index loading times! Additionally, RAMDirectory has also more problems related to garbage collection and concurrency. Because the data residing in swap space, Java’s garbage collector has a hard job to free the memory in its own heap management. This leads to high disk I/O, slow index access times, and minute-long latency in your searching code caused by the garbage collector driving crazy. On the other hand, if we don’t use RAMDirectory to buffer our index and use NIOFSDirectory or SimpleFSDirectory, we have to pay another price: Our code has to do a lot of syscalls to the O/S kernel to copy blocks of data between the disk or filesystem cache and our buffers residing in Java heap. This needs to be done on every search request, over and over again. Memory Mapping Files The solution to the above issues is MMapDirectory, which uses virtual memory and a kernel feature called “mmap” [3] to access the disk files. In our previous approaches, we were relying on using a syscall to copy the data between the file system cache and our local Java heap. How about directly accessing the file system cache? This is what mmap does! Basically mmap does the same like handling the Lucene index as a swap file. The mmap() syscall tells the O/S kernel to virtually map our whole index files into the previously described virtual address space, and make them look like RAM available to our Lucene process. We can then access our index file on disk just like it would be a large byte[] array (in Java this is encapsulated by a ByteBuffer interface to make it safe for use by Java code). If we access this virtual address space from the Lucene code we don’t need to do any syscalls, the processor’s MMU and TLB handles all the mapping for us. If the data is only on disk, the MMU will cause an interrupt and the O/S kernel will load the data into file system cache. If it is already in cache, MMU/TLB map it directly to the physical memory in file system cache. It is now just a native memory access, nothing more! We don’t have to take care of paging in/out of buffers, all this is managed by the O/S kernel. Furthermore, we have no concurrency issue, the only overhead over a standard byte[] array is some wrapping caused by Java’s ByteBuffer interface (it is still slower than a real byte[] array, but that is the only way to use mmap from Java and is much faster than all other directory implementations shipped with Lucene). We also waste no physical memory, as we operate directly on the O/S cache, avoiding all Java GC issues described before. What does this all mean to our Lucene/Solr application? We should not work against the operating system anymore, so allocate as less as possible heap space (-Xmx Java option). Remember, our index accesses rely on passed directly to O/S cache! This is also very friendly to the Java garbage collector. Free as much as possible physical memory to be available for the O/S kernel as file system cache. Remember, our Lucene code works directly on it, so reducing the number of paging/swapping between disk and memory. Allocating too much heap to our Lucene application hurts performance! Lucene does not require it with MMapDirectory. Why does this only work as expected on operating systems and Java virtual machines with 64bit? One limitation of 32bit platforms is the size of pointers, they can refer to any address within 0 and 232-1, which is 4 Gigabytes. Most operating systems limit that address space to 3 Gigabytes because the remaining address space is reserved for use by device hardware and similar things. This means the overall linear address space provided to any process is limited to 3 Gigabytes, so you cannot map any file larger than that into this “small” address space to be available as big byte[] array. And when you mapped that one large file, there is no virtual space (address like “house number”) available anymore. As physical memory sizes in current systems already have gone beyond that size, there is no address space available to make use for mapping files without wasting resources (in our case “address space”, not physical memory!). On 64bit platforms this is different: 264-1 is a very large number, a number in excess of 18 quintillion bytes, so there is no real limit in address space. Unfortunately, most hardware (the MMU, CPU’s bus system) and operating systems are limiting this address space to 47 bits for user mode applications (Windows: 43 bits) [4]. But there is still much of addressing space available to map terabytes of data. Common misunderstandings If you have read carefully what I have told you about virtual memory, you can easily verify that the following is true: MMapDirectory does not consume additional memory and the size of mapped index files is not limited by the physical memory available on your server. By mmap() files, we only reserve address space not memory! Remember, address space on 64bit platforms is for free! MMapDirectory will not load the whole index into physical memory. Why should it do this? We just ask the operating system to map the file into address space for easy access, by no means we are requesting more. Java and the O/S optionally provide the option to try loading the whole file into RAM (if enough is available), but Lucene does not use that option (we may add this possibility in a later version). MMapDirectory does not overload the server when “top” reports horrible amounts of memory. “top” (on Linux) has three columns related to memory: “VIRT”, “RES”, and “SHR”. The first one (VIRT, virtual) is reporting allocated virtual address space (and that one is for free on 64 bit platforms!). This number can be multiple times of your index size or physical memory when merges are running in IndexWriter. If you have only one IndexReader open it should be approximately equal to allocated heap space (-Xmx) plus index size. It does not show physical memory used by the process. The second column (RES, resident) memory shows how much (physical) memory the process allocated for operating and should be in the size of your Java heap space. The last column (SHR, shared) shows how much of the allocated virtual address space is shared with other processes. If you have several Java applications using MMapDirectory to access the same index, you will see this number going up. Generally, you will see the space needed by shared system libraries, JAR files, and the process executable itself (which are also mmapped). How to configure my operating system and Java VM to make optimal use of MMapDirectory? First of all, default settings in Linux distributions and Solaris/Windows are perfectly fine. But there are some paranoid system administrators around, that want to control everything (with lack of understanding). Those limit the maximum amount of virtual address space that can be allocated by applications. So please check that “ulimit -v” and “ulimit -m” both report “unlimited”, otherwise it may happen that MMapDirectory reports “mmap failed” while opening your index. If this error still happens on systems with lot’s of very large indexes, each of those with many segments, you may need to tune your kernel parameters in /etc/sysctl.conf: The default value of vm.max_map_count is 65530, you may need to raise it. I think, for Windows and Solaris systems there are similar settings available, but it is up to the reader to find out how to use them. For configuring your Java VM, you should rethink your memory requirements: Give only the really needed amount of heap space and leave as much as possible to the O/S. As a rule of thumb: Don’t use more than ¼ of your physical memory as heap space for Java running Lucene/Solr, keep the remaining memory free for the operating system cache. If you have more applications running on your server, adjust accordingly. As usual the more physical memory the better, but you don’t need as much physical memory as your index size. The kernel does a good job in paging in frequently used pages from your index. A good possibility to check that you have configured your system optimally is by looking at both "top" (and correctly interpreting it, see above) and the similar command "iotop" (can be installed, e.g., on Ubuntu Linux by "apt-get install iotop"). If your system does lots of swap in/swap out for the Lucene process, reduce heap size, you possibly used too much. If you see lot's of disk I/O, buy more RUM (Simon Willnauer) so mmapped files don't need to be paged in/out all the time, and finally: buy SSDs. Happy mmapping! Bibliography [1] http://en.wikipedia.org/wiki/Virtual_memory [2] https://www.varnish-cache.org/trac/wiki/ArchitectNotes [3] http://en.wikipedia.org/wiki/Memory-mapped_file [4] http://en.wikipedia.org/wiki/X86-64#Virtual_address_space_details

This blog post explores JavaScript’s eval function by implementing the foundation for an interactive command line. As a bonus, you’ll get to work with ECMAScript.next’s generators (which can already be tried out on current Firefox versions). Writing an evaluator Let’s say you want to implement an interactive command line for JavaScript (such as [1]). On one hand, you would need to get the graphical user interface right: The user inputs JavaScript code, the command line evaluates the code and displays the result. On the other hand, you would have to implement the evaluation. That’s what we will take on here. It is more complex that it initially seems and teaches us a lot about eval. For starters, let’s write a constructor Evaluator: function Evaluator() { } Evaluator.prototype.evaluate = function (str) { return JSON.stringify(eval(str)); }; To use the evaluator, we create an instance and send JavaScript code to it: > var e = new Evaluator(); > e.evaluate("Math.pow(2, 53)") '9007199254740992' > e.evaluate("3 * 7") '21' > e.evaluate("'foo'+'bar'") '"foobar"' JSON.stringify is used so that the evaluation results can be shown to the user and look like the input. Without stringify, things look as follows: > console.log(123) // OK 123 > console.log("abc") // not OK abc With stringify, everything looks OK: > console.log(JSON.stringify(123)) 123 > console.log(JSON.stringify("abc")) "abc" Note that undefined is not valid JSON, but stringify converts it to undefined (the value, not the string), which is fine for our purposes. What we have implemented so far works for basic things, but still has several problems. Let’s tackle them one at a time. Problem: declarations You can evaluate variable and function declarations, but they are forgotten immediately afterwards: > e.evaluate("var x = 12;") undefined > e.evaluate("x") ReferenceError: x is not defined How do we fix this? The following code is a solution: function Evaluator() { this.env = {}; } Evaluator.prototype.evaluate = function (str) { str = rewriteDeclarations(str); var __environment__ = this.env; // (1) with (__environment__) { // (2) return JSON.stringify(eval(str)); } }; function rewriteDeclarations(str) { // Prefix a newline so that search and replace is simpler str = "\n" + str; str = str.replace(/\nvar\s+(\w+)\s*=/g, "\n__environment__.$1 ="); // (3) str = str.replace(/\nfunction\s+(\w+)/g, "\n__environment__.$1 = function"); return str.slice(1); // remove prefixed newline } this.env holds all variable declarations and function declarations in its properties. We make it accessible to the input in two steps. Step 1 – declare: We assign this.env to __environment__ (1) and rewrite the input so that, among other things, each var declaration assigns to __environment__ (3). That demonstrates one important aspect of eval: it sees all variables in surrounding scopes. That is, if you invoke eval inside your function, you expose all of its internals. The only way to keep those internals secret is to put the eval call in a separate function and call that function. Step 2 – access: Use a with statement so that the properties of __environment__ appear as variables to the eval-ed code. This is not an ideal solution, more of a compromise: with should be avoided [2] and can’t be used in the advantageous strict mode [3]. But it is a quick solution for us now. A work-around is quite complex [4]. > var e = new Evaluator(); > e.evaluate("var x = 123;") '123' > e.evaluate("x") '123' Minor drawback: Normal var declarations have the result undefined; due to our rewriting we now get the value that is assigned to the variable. Problem: exceptions Right now, throwing an exception in evaluate’s input means that the method will throw: > e.evaluate("* 3") SyntaxError: Unexpected token * That is obviously unacceptable: In a graphical user interface, we want to report errors back to the user, not (invisibly) throw an exception. Here is one simple way of doing so: Evaluator.prototype.evaluate = function (str) { try { str = rewriteDeclarations(str); var __environment__ = this.env; with (__environment__) { return JSON.stringify(eval(str)); } } catch (e) { return e.toString(); } }; There is nothing surprising in this code, we simply use try-catch and report back what happened. More sophisticated solutions will want to do more, e.g. display the exception’s stack trace. The new evaluator in action: > var e = new Evaluator(); > e.evaluate("* 3") 'SyntaxError: Unexpected token *' Problem: console.log How do we handle calls to console.log in the input? Logged messages should be shown to the user, not be sent to the browser’s console. The solution is surprisingly easy: function Evaluator(cons) { this.env = {}; this.cons = cons; } Evaluator.prototype.evaluate = function (str) { try { str = rewriteDeclarations(str); var __environment__ = this.env; var console = this.cons; with (__environment__) { return JSON.stringify(eval(str)); } } catch (e) { return e.toString(); } }; The constructor now receives a custom implementation of console and assigns it to this.cons. By assigning that object to a local variable named console (1), we temporarily shadow the global console for eval, there is no need to replace it. Beware that that shadowing affects all of the function, you won’t be able to use the browser’s console anywhere in evaluate. The new evaluator in action: > var cons = { log: function (m) { console.log("### "+m) } }; > var e = new Evaluator(cons); > e.evaluate("console.log('hello')") ### hello undefined Problem: eval creates bindings inside the function One scary feature of eval is that it creates variable bindings inside the function that invokes it: > (function () { eval("var x=3"); return x }()) 3 Fortunately, the fix is easy: use strict mode. > (function () { "use strict"; eval("var x=3"); return x }()) ReferenceError: x is not defined You can’t use with in strict mode, so you’ll have to replace it with a work-around [4]. Keeping declarations in an environment An environment is where JavaScript keeps the parameters and variables of a function. It maps variable names to values and is thus similar to an object. We might be able to avoid rewriting the input and manage declarations via environments. The idea is as follows. eval puts declarations in some environment: Non-strict mode: the environment of the surrounding function. Strict mode: a newly created environment. What if we could reuse that environment for the next invocation of eval, instead of throwing it away? Then eval would properly remember prior declarations. Strict mode gives us no way to access the temporary environment it creates for each invocation. However, in non-strict mode, we might be able to keep the environment of the surrounding function around. The following subsections explore two ways of doing so. Declarations via nested scopes If you create a function g inside another function f, then g permanently retains a reference to f’s current environment envf. Whenever g is called, a new g-specific environment envg is created. But envg points to its parent environment envf. Variables that can’t be found in g’s scope (as managed via envg), are looked up in f’s scope (via envf). Thus, envf is not lost, as long as g exists. That gives us a strategy for keeping the environment of the function that calls eval around. In the following code that function is called evalHelper and creates a new function that has to be used for the next call of eval. Hence, declarations made in the former function are accessible in the later function. function Evaluator() { var that = this; that.evalHelper = function (str) { that.evalHelper = function (str) { return eval(str); }; return eval(str); }; } Evaluator.prototype.evaluate = function (str) { return this.evalHelper(str); }; The fatal problem of this implementation is that you cannot nest to arbitrary depth. But, for the above depth of 2, it works perfectly: > var e = new Evaluator(); > e.evaluate("var x = 7;"); undefined > e.evaluate("x * 3") 21 Declarations via a generator It would be great if we could “restart” the function that calls eval, re-enter it with its previous environment still in place. ECMAScript.next’s generators [5] let you do that. Current versions of Firefox already support generators. Here is a demonstration of how they work in these versions (in ECMAScript.next, you will have to write function*, but apart from that, the code is the same): function mygen() { console.log((yield 0) + " @ 0"); console.log((yield 1) + " @ 1"); console.log((yield 2) + " @ 2"); } The above is a generator function. Invoke it and it will create a generator object. On that object, you first need to invoke the next() method to start execution. A yield x inside the code pauses execution and returns x to the the previously called generator object method. After the first next(), you can either call next() or send(y). The latter means that the currently paused yield will continue and produce the value y. The former is equivalent to send(undefined). The following interaction shows mygen in use: > var g = mygen(); > g.next() // can’t use send() the first time 0 > g.send("a") // continue after yield 0, pause again a @ 0 1 > g.send("b") b @ 1 2 The following is an implementation of Evaluator that calls eval via the generator evalGenerator. Because of that, eval always sees the same environment and remembers declarations. function evalGenerator(console) { var str = yield; while(true) { try { var result = JSON.stringify(eval(str)); str = yield result; } catch (e) { str = yield e.toString(); } } } function Evaluator(cons) { this.evalGen = evalGenerator(cons); this.evalGen.next(); // start } Evaluator.prototype.evaluate = function (str) { return this.evalGen.send(str); }; The new evaluator works as expected. > var e = new Evaluator(); > e.evaluate("var x = 7;") undefined > e.evaluate("x * 2") "14" > e.evaluate("* syntax_error") "SyntaxError: missing ; before statement" The biggest problem with this solution is that it uses the deprecated features non-strict eval together with the new feature generators. There will probably be a way in ECMAScript.next to make this combination work, but it will be a hack and should thus be avoided. Conclusion We have used eval to implement a helper type for a command line. While doing so, we learned a few interesting things about eval: Letting it remember declarations between invocations is complicated; it can access all variables in the scopes surrounding its invocation; and in non-strict mode, it can even create new variables inside the invoking function. The best solution for remembering declarations would be for eval to have an optional parameter for an environment (to be reused), but that is not in the cards. Therefore, the only truly safe solution in pure JavaScript is to use a full-featured JavaScript parser such as esprima to rewrite critical parts of the input code. That is left as an exercise to the reader. References Combining code editing with a command line JavaScript’s with statement and why it’s deprecated JavaScript’s strict mode: a summary Handing variables to eval Asynchronous programming and continuation-passing style in JavaScript