Shutdown Hooks are a special construct that allow developers to plug in a piece of code to be executed when the JVM is shutting down.

It seems that lots of projects/products/services want to expose a REST API these days. But I have found very few that actually follow the REST constraints, and in a lot of the cases it doesn't even make sense for them to follow REST constraints in the first place. One of the main constraints that is commonly violated is the hypertext constraint. Basically, all state changes have to be done by following links, starting from a bookmarked URL. But almost noone does that. However, should they? This article will outline various layers that REST API's can be implemented in, and when it makes sense, and when not. To begin with, in a typical enterprise app there are three options for layers that you might want to expose using a REST API. These are the infrastructure layer, the domain layer, and the application layer. Infrastructure layer If we start with the infrastructure layer, we are typically talking about a database vendor that wants to allow developers to access it using "REST". The API would allow you to create/remove databases, and then insert/update/delete data. Typically it's pretty normal stuff, and the API doesn't change all that much between versions. Accessing this over HTTP maybe makes sense, but is it RESTful? I'll give you an example. I installed CouchDB, and given the hypermedia constraint I should then be able to go to "http://localhost:5984/", and it will tell me what I can do next (like create a database). But when I do a GET on that URL I get this: {"couchdb":"Welcome","version":"1.0.1"} So now what? The hypermedia doesn't tell me what I can do, so therefore as a REST client I will assume there's nothing I can do. This very simple test shows that the HTTP API for CouchDB isn't really RESTful at all. The question is: should it be? That is obviously up to the developers to decide. But if I were the architect I would maybe say, no, it shouldn't be RESTful. Why? Because I want to allow URL templates to be used, so that the client, given the server URL and a document id, is allowed to construct a URL on its own and GET the document. If this was truly RESTful the client would have to do a query in a form first, with the id, in order to get the URL of the document to be retrieved. That might be inefficient for a database, so I might opt not to do this. Which is, in effect, what they already have done. The only problem is that they call it RESTful, when it isn't, so it gives me as a developer the wrong impression of what I can expect from it. This line of reasoning could be done for pretty much most infrastructure layer API's. They're not RESTful, though many say they are, and most likely they shouldn't try to be! IT'S OK! Just say "Accessible over HTTP, see docs for URL templates and whatnot", and be done with it. Domain layer The next potential layer to be exposed over REST is the domain layer. This typically means that you take your domain entities and expose their data straight on the web, through CRUD operations. Very straightforward. There are tons of articles and blogs that show how to do this. But is it RESTful? Or is it even a good idea in the first place? The first test, again, would be to see if the app follows the hypermedia constraint. In this option it is technically possible to allow queries that will list the various URL's to entities in your domain, which you then can update/delete. So on the surface it might seem like you are following the hypermedia constraint. The problem usually comes with the fact that you are exposing domain state rather than application state. Let me explain through a simple example. Let's say you are building an issue tracker. You can access individual issues through links like: /issue/123 which on GET gives you documents such as: {"status":"OPEN","description":"Some issue"} Awesome. Now a client can change the status to "CLOSED" and PUT that. Tada! Case closed. Or is it? What if a client then decides to reopen it, by simply posting a new status of "OPEN" to it. Ok, that worked. But should it? Maybe your domain model really would have wanted it to only go to "REOPENED" from the "CLOSED" state. But how do you express that? How is the client to know that this is the only valid transition? And what happens when we have many versions of clients, each of which has a slightly different set of rules for what you are allowed to do when? Basically, chaos is ensured. And this is the problem with exposing your domain model using a REST API. The client has to own the application logic, and there's no way the server can be sure that it has the "right" logic. And the client, even if it *wants* to play nice (if code ever wants anything is debatable), will have a hard time knowing whether it is playing by the rules or not. It might even get a bit neurotic, trying to do the right thing, whatever that means. In summary, exposing your domain model does not help the client know what the valid state transitions are, and makes it very hard to do other things like role-based security authorization (maybe only an admin is allowed to REOPEN a CLOSED case?). I would therefore recommend that noone exposes their domain models using a REST API. Application layer Finally we come to the application layer. The application layer is designed to implement usecases of the domain model, and has all the context and logic needed to ensure that only valid state transitions are made. In short, it seems like it is especially appropriate to being exposed through a REST API, as it can at all points tell the client what it can do (either based on state or authorization rules or any other type of rules it might have). If we go back to the issue tracker, what would this mean in practice? It could mean that when you do a GET on /issue/123 you get something like this back: {"data":{"status":"OPEN","description":"Some issue"},"links":[{"close":"/issue/123/close.json"}]} This now instead of referring to viewing the domain state of an issue refers to the usecase of viewing an issue with the intent of working on it. There might be other URL's and other queries that only return the data, or maybe a table of the data, or somesuch. But this one, specifically, refers to the usecase of working with the issue. So, as a REST client I can now inspect the data, and then look at what links are available. If the client has a UI it can enable a button that says "Close issue" based on the available link, since it detected a link relation "close" that it understands. The client can then do GET on that link, find out whether the server expects any form to be filled in, and then submit it using POST, thereby letting the server application layer logic transition the issue to the "CLOSED" state. We are no longer relying on the client to contain the logic of knowing when to allow what, and the client also does not have to know how to construct the URL. As long as it can parse the hypertext (and we might use a custom JSON mediatype to indicate what "data" and "links" mean) and do something with it, we're fine. If we in the future change the domain model to also allow the "resolve" link relation for "OPEN" issues, old clients can ignore it, and new clients can enable new actions in the UI that uses it. In summary, the application layer is a very good candidate to be exposed through a REST API. It encapsulates the application rules for when the various state transitions are allowed, and can make use of user authorization to further enable/disable actions. This takes away a lot of responsibilities from the client, which now also can be "dynamic" in the sense that it can easily react to what state changes are available when by simply checking link availability in the hypermedia returned from the server. The main issue with exposing the application layer through a REST API is that there are pretty much no available frameworks that help you do all this in an easy way. But this is not REST's "fault", obviously, but rather that the "REST" community hasn't yet matured to understand what it should and what it should not do. In the Streamflow project we rolled our own simple framework for doing the above, and I'm very happy with that, but unfortunately most other frameworks seems to be in the "expose your domain model" camp, which means that a lot of this link management is non-trivial to do. This is a fixable situation though. I hope that this post has somewhat clarified what the issues are with exposing infrastructure and domain models through REST API's, and why it's not really a good idea in the first place, and why exposing the application layer really is the logical and simpler option. From http://www.jroller.com/rickard/entry/rest_api_for_infrastructure_domain

Originally Barcodes were 1D representation of data using width and spacing of bars. Common bar code types are UPC barcodes which are seen on product packages. There are 2D barcodes as well (they are still called Barcodes even though they don’t use bars). A common example of 2D bar code is QR code (shown on right) which is commonly used by mobile phone apps. You can read history and more info about Barcodes on Wikipedia. There is an open source Java library called ‘zxing’ (Zebra Crossing) which can read and write many differently types of bar codes formats. I tested zxing and it was able to read a barcode embedded in the middle of a 100 dpi grayscale busy text document! This article demonstrates how to use zxing to read and write bar codes from a Java program. Getting the library It would be nice if the jars where hosted in a maven repo somewhere, but there is no plan to do that (see Issue 88). Since I could not find the binaries available for download, I decided to download the source code and build the binaries, which was actually quite easy. The source code of the library is available on Google Code. At the time of writing, 1.6 is the latest version of zxing. 1. Download the release file ZXing-1.6.zip (which contains of mostly source files) from here. 2. Unzip the file in a local directory 3. You will need to build 2 jar files from the downloaded source: core.jar, javase.jar Building core.jar cd zxing-1.6/core mvn install cd zxing-1.6/core mvn install This will install the jar in your local maven repo. Though not required, you can also deploy it to your company’s private repo by using mvn:deploy or by manually uploading it to your maven repository. There is an ant script to build the jar as well. Building javase.jar Repeat the same procedure to get javase.jar cd zxing-1.6/javase mvn install cd zxing-1.6/javase mvn install Including the libraries in your project If you are using ant, add the core.jar and javase.jar to your project’s classpath. If you are using maven, add the following to your pom.xml. com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT Once you have the jars included in your project’s classpath, you are now ready to read and write barcodes from java! Reading a Bar Code from Java You can read the bar code by first loading the image as an input stream and then calling this utility method. InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); Writing a Bar Code from Java You can encode a small text string as follows: String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); In the above example, the bar code for “97802017507991″ is written to the file “qrcode_97802017507991.png” (click to see the output). JavaDocs and Documentation The Javadocs are part of the downloaded zip file. You can find a list of supported bar code formats in the Javadocs. Open the following file to see the javadocs. zxing-1.6/docs/javadoc/index.html zxing-1.6/docs/javadoc/index.html From http://www.vineetmanohar.com/2010/09/java-barcode-api/

In the first and second parts of this series I talked about some of the new features like enum and concurrency support that have been added in commons-lang 3. In this article, I am going to talk about a new package 'org.apache.commons.lang3.text.translate' which has been added in commons-lang 3. This package is added to fix problems in the design and implementation of the StringEscapeUtils class which exists in versions prior to 3.0. To make it clearer, let's first talk about the purpose of StringEscapeUtils class and the problems it had prior to version 3. Purpose of StringEscapeUtils StringEscapeUtils is a utility class which escapes and unescapes String for Java, JavaScript, HTML, XML, and SQL. For example, @Test public void test_StringEscapeUtils() { assertEquals("\\\\\\n\\t\\r", StringEscapeUtils.escapeJava("\\\n\t\r")); // escapes the Java String assertEquals("\\\n\t\r",StringEscapeUtils.unescapeJava("\\\\\\n\\t\\r")); //unescapes the Java String assertEquals("I didn\\'t say \\\"you to run!\\\"",StringEscapeUtils.escapeJavaScript("I didn't say \"you to run!\""));//escapes the Javascript assertEquals("<xml>", StringEscapeUtils.escapeXml(""));//escapes the xml } Problems with StringEscapeUtils There were a lot of problems in the StringEscapeUtils implementation prior to version3. Some of these were: The implementation was not extensible. Let's take an example of escapeJava, suppose we want to add support in the escapeJava method that it should start escaping single quotes. To add such support we would have to change the existing class code and another if condition which if satisfied will escape single quotes. So, the API was breaking the open-closed principle i.e. a class should be open for extension and closed for modification. It was not symmetric i.e. original should be equal to unescape(escape(original)) but it was not the case. StringEscapeUtils.escapeHtml() escapes multibyte characters like Chinese, Japanese etc. Issue 339 @Test public void testEscapeHiragana() { // Some random Japanese unicode characters String original = "\u304B\u304C\u3068"; String escaped = StringEscapeUtils.escapeHtml(original); assertEquals(original, escaped); } StringEscapeUtils.escapeHtml incorrectly converts unicode characters above U+00FFFF into 2 characters. Issue 480 @Test public void testEscapeHtmlHighUnicode() throws java.io.UnsupportedEncodingException { byte[] data = new byte[] { (byte) 0xF0, (byte) 0x9D, (byte) 0x8D,(byte) 0xA2 }; String original = new String(data, "UTF8"); String escaped = StringEscapeUtils.escapeHtml(original); assertEquals(original, escaped); } StringEscaper.escapeXml() escapes characters > 0x7f . Issue 66 @Test public void shouldNotEscapeValuesGreaterThan0x7f() { assertEquals("XML should not escape >0x7f values", "\u00A1",StringEscapeUtils.escapeXml("\u00A1")); } Solution -- Rewritten StringEscapeUtils In version 3.0, StringEscapeUtils is completely rewritten to fix all the bugs associated with this class and to provide a way for the user to customize the behavior of its methods. They have moved all the logic present in the StringEscapeUtils to the classes in the package 'org.apache.commons.lang3.text.translate'. Let's take an example of escapeJava function in StringEscapeUtils, escapeJava function does not contain any business logic, it just calls the translate method on CharSequenceTranslator reference. What they did can be best understood by looking at the code below public static final CharSequenceTranslator ESCAPE_JAVA = new AggregateTranslator(new LookupTranslator( new String[][] { {"\"", "\\\""}, {"\\", "\\\\"}, }),new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()),UnicodeEscaper.outsideOf(32, 0x7f)); and in the escapeJava method public static final String escapeJava(String input) { return ESCAPE_JAVA.translate(input); } A constant of type CharSequenceTranslator was assigned an AggregateTranslator object. AggregateTranslator can take an array of translators, and it iterates over each of them. The LookupTranslator replaces the string at zeroth index with the string at the first index. UnicodeEscaper translates values outside the given range to unicode values. As you can see, you can very easily write your own escape methods. For example, if you want to add the support of escaping &, you can do it like this public static final CharSequenceTranslator ESCAPE_JAVA = new LookupTranslator( new String[][] { {"\"", "\\\""}, {"\\", "\\\\"}, }).with(new LookupTranslator( new String[][]{ {"&", "&"}, {"<", "<"} )).with( new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()) ).with( UnicodeEscaper.outsideOf(32, 0x7f) ); StringEscapeUtils.escapeSql has been removed from the API as it was misleading developers to not use PreparedStatement.This method was not of much use as it was only escaping single quotes.

The new Apple Magic Mouse is a controversial piece of hardware. Most people either really hate it or adore it. Personally, I think it is probably the best mouse I've ever used. There's a lot of criticism regarding the low profile of the mouse, suggesting it is not ergonomic. From my experience, I don't have any more wrist pains since I started using it. Ergonomics aside, the highlight of the mouse is the upper multi-touch surface with the enabled gestures. I use the back and forward gestures a lot. Especially when browsing the web. Swipe two fingers to the left and go back. Swipe right to go forward. It is very easy to get used to it. It works in web browsers, it works in Finder windows and native applications are adding support as well. However, it doesn't work in Eclipse. I want to swipe back and forward when browsing code. Back, go to previous location. Forward, return to the next location. I incidentally found a solution for that. There are many programs on the market that augment the Magic Mouse behavior. The reason for their existence is because Apple provides very limited gesture functionality. Other than back/forward and scroll, there's simply no support for other functions, not even Exposé or Spaces which were supported in the previous Mighty Mouse and are supported on the multi-touch trackpads. The most popular tools are MagicPrefs and BetterTouchTool (both free) but there are many others, free and commercial. Personally, I use MagicDriver, which is commercial (free while in beta). The reason I prefer it is because it has much lower CPU utilization, which was an issue for me in MagicPrefs. MagicDriver replaces the back/forward gestures with their keyboard equivalent: ⌘+[ and ⌘+]. These shortcuts are commonly used in OS X. Eclipse, by default, also uses these keyboard shortcuts to navigate back and forward. It just works. MagicPrefs and BetterTouchTool will require some customization: you can define the two finger swipe left and right to fire these keyboard shortcuts rather than use the default back/forward functionality. If you use a newer MacBook with a multi-touch trackpad or a Magic Trackpad, you can achieve the same functionality by using BetterTouchTool. AFAIK MagicPrefs does not support it and the current version of MagicDriver doesn't support it either. BetterTouchTool also has the ability to define gestures per application, so you can customize the behavior specifically for Eclipse and leave it as is for the rest of the applications. If you are new to these tools, I should warn you: defining too many gestures doesn't work very well. There are tons of options and it is very easy to get carried away and use as much as you can. However, there's probably a reason why Apple did not include support for all those gestures in the first place. It is very easy to "miss-fire" and perform gestures by accident. You don't always pay close attention to the number of fingers you have on the surface, so mistakes are very common. I just use a 3-finger click for expose. Don't be greedy and it will work just fine. Finally, if you want proper native support for back/forward gestures in Eclipse, you can vote for this bug.

whenever you hover over any piece of code in eclipse, it pops up a tooltip that displays more information about the item, such as its declaration, variable values or javadoc information, as in the example below. although useful at times , this becomes extremely annoying after a while, especially when you’re using your mouse to browse some code. popup after popup of unwanted information keeps obscuring your view of the code, leading to some lengthy expletives and big productivity loss. it’s useful information, but not every time all the time , almost like your car’s gps giving you directions to 10 different places at once while you’re still parked in the driveway. luckily there is a way to alleviate the problem and all it takes is changing some preferences in eclipse. we don’t want to completely disable tooltips (they can be useful), so i’ll show you how to tell eclipse to bring up the tooltips only when you request them. show tooltips only on request to tell eclipse to only show tooltips on request, do the following: go to window > preferences > java > editor > hovers . select the source item in the list. make sure the entire row is highlighted and that the checkbox is selected. move focus to the pressed key modifier while hovering field, positioning the cursor at the end of the default value shift . press and release ctrl . the value should now read shift+ctrl . if you made a mistake, just clear the field by pressing backspace or delete and try again. select the combined hover item in the list. move focus to the pressed key modifier while hovering field and press shift . the value should now read shift . click ok to accept the changes. the preferences should look something like this: you should now notice the following: if you want to show the javadoc or the declaration of some element hold down shift while hovering over the element. variable values won’t popup automatically anymore. to show a variable’s value in debug mode, press shift and hover over the variable. this is probably the only time when it’s a bit of a hassle to press shift, but the pros totally outweigh the cons here as i view variable values a lot less than the number of times annoying tooltips appear. if you want to show a quick preview of a method’s code, press ctrl+shift and hover over the method. this is a nice feature combined with the tooltip enrichment eclipse provides, so you can view a method’s code in a scrollable tooltip without having to navigate to the method and back again. other options – configure, disable or use the keyboard you can, of course, choose your own modifier keys in preferences, but these are the ones i found that work the best, especially given that ctrl and a click is a way to go to an element’s declaration or implementation in eclipse. i’ve found shift to conflict the least with existing features and easy enough to use on a frequent basis. keep in mind that 2 hovers can’t share the same modifier key (which is why we reassigned the source hover to use shift+ctrl ). you can also disable tooltips completely by deselecting the combined hover checkbox in the preference above, but i’ve found the shift key to be a nice middle ground so keep it enabled. to get similar information using the keyboard, press f2 while the cursor’s positioned on the code. to show variable values in debug mode, press ctrl+shift+d . a nice tip if you’ve disabled tooltips completely. from http://eclipseone.wordpress.com/2010/08/24/configure-tooltips-in-eclipse-to-only-popup-on-request/

In this series we are now entering the realm of concurrency, an option which adds complexity to an application as many different threads of execution are accessing the state storage at the same time. There is no native multithreading support in PHP (every script gets its own isolated process), but still concurrency can easily become an issue: multiple clients from all over the world continuosly make requests to PHP applications, and they can easily mutually overwrite their changesets. A classic example of race condition in the PHP world is two different clients filling an editing form referred to the same entity. They both will submit the form once it is complete, and the first one will get his changes overwritten by the second request. There are many other common situations were concurrent user can provoke errors in the system. Just think of different people choosing the same username and being told via Ajax that it is available; the slower one of them will be surprised when he submits his registration form. Background Before entering the explanation of patterns emerged to solve the concurrency problems, we need some definitions. First of all the notion of transaction is necessary: we define a transaction as a change of state of the application. Editing an entity is a transaction; adding or deleting another is still a transaction. The first kind of transaction we are interested in is the database transactions, which is totally accomplished in one PHP script. This is usually automatically enforced by mechanisms supported at the database level. The other kind of transaction is the business transaction, which spans over multiple HTTP requests and makes uses of from one to N database transactions. It comprehends checking out data, populating a form or other kind of rich user interface, modifying or adding data (a human-based action), and sending it back to the server. There is no automatic enforcement for business transaction, since they are defined by the business rules of the domain. This is not a problem that originates because of PHP nature, but because of the separation between client and server which the web is based on. Optimistic lock The Optimistic Offline Lock pattern is a way of ensuring integrity of data, avoiding the option that different clients submit conflicting changes. As the name suggests, it assumes that the chances of conflict are low. Indeed, when this is the case the optimistic lock does not slow down the user interaction a bit. The goal of this pattern is detecting a conflicting change and instead of applying it, rollback the business transaction and present an error to the user. It accomplishes this goal by validating that no one else has tampered with a record in the data source prior to allowing the modification to be committed. All open source version control systems such as Subversion and Git implement optimistic lock: anyone may check out a source file and work on it, to end his little fork later with a commit or push. The pain comes while merging, so you are supposed to integrate often. We also borrowed terminology from the source control systems, so in this article you'll encounter terms like commit, checkout, and merge. Implementation The most common implementation of Optimistic Offline Lock is a numerical version field on the record to protect from condurrency issues; to aid rollback notifications, other additional fields are useful for signaling the conflict, like the id of the last user that modified the record. The pattern inner working is not complex: when the data is submitted along with the version field value kept on the client, the version field in it must be the same currently present in the database. Only then it is incremented and the changeset committed. Encountering a different version field value in the database record means someone else has modified the data in between our checkout and commit, and so it must be preserved. For example we can show a diff to the user, like VCS does; in any case, we should interrupt the transaction. RDBMS and ORMs can simply use an additional column on the table where the root object is stored to support this pattern. Alternatives An alternative implementation consists in using all fields in the WHERE clause of the UPDATE (or only the sensible ones, or only the modified ones to let transactions that affect different field succeed when the business logic allows it. See below). This solution is handy when you can't add a version field, but it may have performance impact. Another alternative is to check conditions instead of version fields, which is practical in different use cases. For example, we can check the existence of a record before deleting it. This is already indirectly done to a cartain extent by ORMs and other abstraction layers when they provide you with an object abstraction you can calla delete() method on. An extension to the functionality of this pattern is checking that a current editing will (probably) commit, as a feature available at any time during editing of the data. This feature should check that the checkout data is still current. The domain An important is that it is part of the job of business domain logic to decide when a conflict occurs: some concurent changesets may be acceptable, while others may not be allowed even if they modify different fields. In his book, Fowler makes the example of adding elements to a collection concurrently. We can't know if this is right by seeing that the object is a collection, because it is the abstraction that it represent that must be maintained valid: sometimes it is right to add elements, sometimes the transaction should be stopped. Also merging strategies, which solve the conflicts, are subsceptible to domain considerations. Some are valuable and should be pursued, while some are costly and a rollback with manual user editing of the data is fine. Advantages The greatest advantage of this pattern is that it can support real time concurrency, like the check out of multiple items by multiple users simultaneously, as long as there is a merging strategy in place. It can also easily prevent race conditions. This pattern is also easy to implement, and thus it is the default choice to solve concurrency issues. Disadvantages When the conflict probability is high, since there are many concurrent transactions, this pattern produces too many rollbacks. It is not adequate for use cases where a pessimistic pattern should be adopted. Examples Doctrine 2 uses natively database transactions: it only commits the changes made in a PHP script when the EntityManager::flush() method is called. It automatically rolls back if an error is detected. Besides that, Doctrine 2 has also automatic Optimistic Offline Locking support, via the addition of a version field to the entity to lock.

Yesterday I wondered if the copyFile method in JTheque Utils was the best method or if I need to change. So I decided to do a benchmark. So I searched all the methods to copy a File in Java, even the bad ones and found the following methods : Naive Streams Copy : Open two streams, one to read, one to write and transfer the content byte by byte. Naive Readers Copy : Open two readers, one to read, one to write and transfer the content character by character. Buffered Streams Copy : Same as the first but using buffered streams instead of simple streams. Buffered Readers Copy : Same as the second but using buffered readers instead of simple readers. Custom Buffer Stream Copy : Same as the first but reading the file not byte by byte but using a simple byte array as buffer. Custom Buffer Reader Copy : Same as the fifth but using a Reader instead of a stream. Custom Buffer Buffered Stream Copy : Same as the fifth but using buffered streams. Custom Buffer Buffered Reader Copy : Same as the sixth but using buffered readers. NIO Buffer Copy : Using NIO Channel and using a ByteBuffer to make the transfer. NIO Transfer Copy : Using NIO Channel and direct transfer from one channel to other. I think, this is the ten principal methods to copy a file to another file. The different methods are available at the end of the post. Pay attention that the methods with Readers only works with text files because Readers are using character by character reading so it doesn’t work on a binary file like an image. Here I used a buffer size of 4096 bytes. Of course, use a higher value improve the performances of custom buffer strategies. For the benchmark, I made the tests using different files. Little file (5 KB) Medium file (50 KB) Big file (5 MB) Fat file (50 MB) And I made the tests first using text files and then using binary files. The source file is not on the same hard disk as the target file. I used a benchmark framework, described here, to make the tests of all the methods. The tests have been made on my personal computer (Ubuntu 10.04 64 bits, Intel Core 2 Duo 3.16 GHz, 6 Go DDR2, SATA Hard Disks). And after a long time of bench, here are the results : Little Text File - All results We see that the method with a simple stream (Naive Streams) is from far the slowest followed by the simple readers methods (Naive Readers). The readers method is a lot faster than the simple stream because FileReader use a buffer internally. To see what happens to the other, here are the same graph but without the first two methods : Little Text File - Best results The best two versions are the Buffered Streams and Buffered Readers. Here this is because the buffered streams and readers can write the file in only one operation. Here the times are in microseconds, so there is really little differences between the methods. So the results are not really relevant. Now, let’s test with a bigger file. Medium Text File We can see that the versions with the Readers are a little slower than the version with the streams. This is because Readers works on character and for every read() operation, a char conversion must be made, and the same conversion must be made on the other side. Another observation is that the custom buffer strategy is faster than the buffering of the streams and than using custom buffer with a buffered stream or a single stream doesn’t change anything. The same observation can be made using the custom buffer using readers, it’s the same with buffered readers or not. This is logical, because with custom buffer we made 4096 (size of the buffer) times less invocations to the read method and because we ask for a complete buffer we have not a lot of I/O operations. So the buffer of the streams (or the readers) is not useful here. The NIO buffer strategy is almost equivalent to custom buffer. And the direct transfer using NIO is here slower than the custom buffer methods. I think this is because here the cost of invoking native methods in the operating system level is higher than simply the cost of making the file copy. Big Text File - All results Here we see that the Naive Readers shows its limit when the file size if growing. So let’s concentrate us on the best methods only, namely, remove the Naive Readers : Big Text File - Best results Here, it’s now clear that the custom buffer strategy is a better than the simple buffered streams or readers and that using custom buffer and buffered streams is really useful for bigger files. The Custom Buffer Readers method is better than Custom Buffer Streams because FileReader use a buffer internally. And now, continue with a bigger file : Fat Text File Results You can see that it doesn’t take 500 ms to copy a 50 MB file using the custom buffer strategy and that it even doesn’t take 400 ms with the NIO Transfer method. Really quick isn’t it ? We can see that for a big file, the NIO Transfer start to show an advantage, we’ll better see that in the binary file benchmarks. We will directly start with a big file (5 MB) for this benchmark : Big Binary File Results So we can make the same conclusion as for the text files, of course, the buffered streams methods is not fast. The other methods are really close. Fat Binary File Results We see here again that the NIO Transfer is gaining advantages more the files is bigger. And just for the pleasure, a great file (1.3 GB) : Enormous Binary File Results We see that all the methods are really close, but the NIO Transfer method has an advantage of 500 ms. It’s not negligible. Conclusion In conclusion, the NIO Transfer method is the best one for big files but it’s not the fastest for little files (< 5 MB). But the custom buffer strategy (and the NIO Buffer too) are also really fast methods to take files. So perhaps, the best method is a method that make a custom buffer strategy on the little files and a NIO Transfer on the big ones. But it will be interesting to also make the tests on an other computer and operating system. We can take several rules from this benchmark : Never made a copy of file byte by byte (or char by char) Prefer a buffer in your side more than in the stream to make less invocations of the read method, but don’t forget the buffer in the side of the streams Pay attention to the size of the buffers Don’t use char conversion if you only need to tranfer the content of a file Don’t hesitate to use channels to make file transfer, it’s the fastest way to make a file transfer I’ve also made some tests, but not complete, for files in the same hard disk and here the NIO Transfer method is a lot faster than the other. I think this is because on the same disk this method can make better use of the filesystem cache. I hope this benchmark (and its results) interested you. Here are the sources of the benchmark : Java Benchmark of File Copy methods From http://www.baptiste-wicht.com/2010/08/file-copy-in-java-benchmark

Recently I attended a local rheinJUG meeting in Düsseldorf. While the topic of the session was Eclipse e4, the night’s sponsor itemis provided some handouts on Xtext which got me very interested. The reason is that currently at work we are developing a mobile Java application (J9, CDC/Foundation 1.1 on Windows CE6) for which we needed an easy to use and reliable way for configuring navigation through the application. In a previous iteration we had – mostly because of time constraints – hard coded most of the navigational paths, but this time the app is more complex and doing that again was not really an option. First we thought about an XML based configuration, but this seemed to be a hassle to write (and read) and also would mean we would have to pay the price of parsing it on every application startup. Enter Xtext: An Eclipse based framework/library for building text based DSLs. In short, you just provide a grammar description of a new DSL to suit your needs and with – literally – just a few mouse clicks you are provided with a content-assist, syntax-highlight, outline-view-enabled Eclipse editor and optionally a code generator based on that language. Getting started: Sample Grammar There is a nice tutorial provided as part of the Xtext documentation, but I believe it might be beneficial to provide another example of how to put a DSL to good use. I will not go into every step in great detail, because setting up Xtext is Eclipse 3.6 Helios is just a matter of putting an Update Site URL in, and the New Project wizard provided makes the initial setup a snap. I assume, you have already set up Eclipse and Xtext and created a new Xtext project including a generator project (activate the corresponding checkbox when going through the wizard). In this post I am assuming a project name of com.danielschneller.navi.dsl and a file extension of .navi. When finished we will have the infrastructure ready for editing, parsing and generating code based on files like these: navigation rules for MyApplication mappings { map permission AdminPermission to "privAdmin" map permission DataAccessPermission to "privData" map coordinate Login to "com.danielschneller.myapp.gui.login.LoginController" in "com.danielschneller.myapp.login" map coordinate LoginFailed to "com.danielschneller.myapp.gui.login.LoginFailedController" in "com.danielschneller.myapp.login" map coordinate MainMenu to "com.danielschneller.myapp.gui.menu.MainMenuController" in "com.danielschneller.myapp.menu" map coordinate UserAdministration to "com.danielschneller.myapp.gui.admin.UserAdminController" in "com.danielschneller.myapp.admin" map coordinate DataLookup to "com.danielschneller.myapp.gui.lookup.LookupController" in "com.danielschneller.myapp.lookup" } navigations { define navigation USER_LOGON_FAILED define navigation USER_LOGON_SUCCESS define navigation OK define navigation BACK define navigation ADMIN define navigation DATA_LOOKUP } navrules { from Login on navigation USER_LOGON_FAILED go to LoginFailed on navigation USER_LOGON_SUCCESS go to MainMenu from LoginFailed on navigation OK go to Login from MainMenu on navigation ADMIN go to UserAdministration with AdminPermission on navigation DATA_LOOKUP go to DataLookup with DataAccessPermission from UserAdministration on navigation BACK go to MainMenu from DataLookup on navigation BACK go to MainMenu } As you can see it is a nice little language for defining coordinates in an application, meaning a specific GUI for a certain task and the possible navigation paths between them. Optionally a navigation path can be tagged to require one or more permissions to work. So for example one possible navigation path shown in the above sample is from the applications main menu, identified by the identifier MainMenu and represented in code by the com.danielschneller.myapp.gui.menu.MainMenuController class in the com.danielschneller.myapp.menu OSGi bundle to a GUI identified as DataLookup, implemented by com.danielschneller.myapp.gui.lookup.LookupController in the com.danielschneller.myapp.lookup bundle. For this path to be taken, the application must request the DataLookup navigation path and the currently logged in user be assigned the DataAccessPermission. What exactly that means is not the focus of this tutorial, suffice it to say that we somehow need to get the information contained in this specialized language into our Java application in some shape or form that can be evaluated at runtime. In the following example all information will be transformed into a HashMap based data structure. For our little mobile application this has several advantages over the XML option mentioned earlier: No XML parsing necessary on application startup, saving some performance Validation of the navigation rules ahead of time, preventing parse errors at runtime No libraries needed to access the information – by putting everything in a simple HashMap we do not have to rely on any non-standard classes whatsoever First thing I did when I started with Xtext was define a sample input file such as the one above. Then – following its general structure – I began to extract a formal grammar for it. Of course, the first draft of the sample data was not perfect, over the course of a few iterations I refined some of the syntax, but in the end this is the grammar definition I came up with. It is heavily commented to allow you to copy it out and still leave the documentation intact: grammar com.danielschneller.navi.NavigationRules with org.eclipse.xtext.common.Terminals generate navigationRules "http://com.danielschneller/fw/funkmde/navi/NavigationRules" /* * The top level entry point for the file. * "Root" is just a name as good as any, but * makes the meaning quite clear. */ Root: // first thing in the file is a "keyword", // followed by an attribute that will be // accessible as "name" later and allow // definition of an ID type of thing. 'navigation rules for' name=ID // after the keyword and "name" attribute // three sections follow, each assigned // to an attribute for later reference // (called "mappingdefs", "transitiondefs" // and "ruledefs"). // Their types are defined later in the file. mappingsdefs=Mappings transitiondefs=TransitionDefinitions ruledefs=NavigationRules // semicolon ends the definition of "Root" ; // mappings section >>>>>>>>>>>>>>>>>>>>>>>>>>>>> /* * Definition of the "Mappings" type used in * the "Root" type. */ Mappings: // first the keyword "mappings" is expected, // then an open curly 'mappings' '{' // after that a collection of "Mapping"s is // expected. The "+=" means that they will // all be collected in a collection type element // called "mappings" for future reference. // The "+" at the end means "at least one, but // more is just fine". (mappings+=Mapping)+ // finally the "Mappings" type requires a closing // curly brace. '}' // semicolon ends the definition of "Mappings" ; /* * Definition of a single "Mapping", those we are * collecting in the "mappings" attribute of the * "Mappings" type. */ Mapping: // each mapping starts with the keyword "map" // and is followed by an element of type "MappingSpec" 'map' MappingSpec ; /* * Definition of a "MappingSpec" element. This is * actually just a "parent type" for two more specific * kinds of "MappingSpec": */ MappingSpec: // no keywords are defined here, a "MappingSpec" // can be either a "PermissionMappingSpec" or a // "CoordinateMappingSpec". Any of these will be // fine where a "MappingSpec" is asked for. PermissionMappingSpec | CoordinateMappingSpec ; /* * Definition of a "PermissionMappingSpec" element. */ PermissionMappingSpec: // first the keyword "permission" is required. // then a "name" attribute is expected of type ID. // Following the name the "to" keyword is expected, // followed by a string that is stored in the "value" // attribute 'permission' name=ID 'to' value=STRING ; /* * Definition of a "CoordinateMappingSpec" element. * The definition is very similar to the "PermissionMappingSpec" * but has more attributes. */ CoordinateMappingSpec: // first the keyword "coordinate", then an ID stored as "name", // the keyword "to", followed by a string stored as "controllername", // next the keyword "in" and finally another string, memorized as // "bundleid" 'coordinate' name=ID 'to' controllername=STRING 'in' bundleid=STRING ; // <<<<<<<<<<<<<<<<<<<<<<<<<<<<< mappings section // >>>>>>>>>>>>>>>>>>>>>>>>>>>>> navigations section /* * Definition of the "TransitionDefinitions" type used in * the "Root" type. */ TransitionDefinitions: // first, this element is introduced with the "navigations" // keyword, followed by an open curly brace. 'navigations' '{' // after that a collection of "TransitionDefinition"s is // expected. The "+=" means that they will // all be collected in a collection type element // called "transitions" for future reference. // The "+" at the end means "at least one, but // more is just fine". (transitions+=TransitionDefinition)+ // the element ends with a closing curly brace '}' ; /* * Definition of a "TransitionDefinition" element. This * one is very simple. */ TransitionDefinition: // the keyword "define navigation" is required first, // then a "name" attribute of type ID is expected. 'define navigation' name=ID ; // <<<<<<<<<<<<<<<<<<<<<<<<<<<<< navigations section // >>>>>>>>>>>>>>>>>>>>>>>>>>>>> navrules section /* * Definition of the "NavigationRules" element. */ NavigationRules: // Element starts with the keywords "navrules" and // open curly. 'navrules' '{' // collection attribute called "rules", consisting // of one or more occurrences of a "Rule" element. (rules+=Rule)+ // element finishes with a closing curly keyword '}' ; /* * Definition of a "Rule" element as used in the "NavigationRules" * element. */ Rule: // first the "from" keyword, then a reference to one of the // coordinate mappings defined earlier. This time no new // definition of a coordinate is required, but one of those // that have been listed before. So the type here is put in // square brackets 'from' source=[CoordinateMappingSpec] // following the source specification, one or more "Destination" // type elements are expected, collected in a collection attribute // named "destinations" (destinations+=Destination)+ ; /* * Definition of a "Destination" type. These are collected * in a "Rule". */ Destination: // first comes an "on navigation" keyword. After that a // reference to one of the Transition elements defined // in the "navigations" section is required and stored // in the "transition" attribute. // after that follows a "go to" keyword and a reference // to a coordinate mapping, stored in the "target" attribute. // finally - as with the "destinations" collection attribute // in the "Rule" element - a "permissions" collection is // defined to store none or more (*) "PermissionReference" // elements. 'on navigation' transition=[TransitionDefinition] 'go to' target=[CoordinateMappingSpec] (permissions+=PermissionReference)* ; /* * Definition of a "PermissionReference" type. This is used * in the "permissions" collection of a "Destination". */ PermissionReference: // first, a "with" keyword is expected. After that a // "permission" attribute stores a reference to one of // the previously defined permission mappings from the // "mappings" section. 'with' permission=[PermissionMappingSpec] ; // <<<<<<<<<<<<<<<<<<<<<<<<<<<<< navrules section This is what XText can digest and create an editor plugin and outline view for. Just save this as navigationRules.xtext – when you created the XText project in Eclipse using the wizard it should have been prepared for you. Copying and pasting this into a .xtext file in Eclipse will provide you with syntax highlighting, code completion and syntax checking, making it easy to play around with grammar files. Once done, right click the .mwe2 file lying next to the grammar file in the Package Explorer view and select Run As MWE2 Workflow from the context menu. This will take a moment and generate several classes, both in the current (XText) project and the accompanying ...ui project. Next, right click the Xtext project and select Run As Eclipse Application from the context menu. This will bring up another Eclipse instance with the newly created support for navigation rules files (with a .navi suffix) installed. To try it out, just create a new project and in that a new file. Make sure its name ends in .navi. When asked, make sure to accept adding the Xtext nature to the project. You will be presented with a new, empty editor that already has an error marker in it. This is because according to our grammar definition, an empty file does not comply to all the rules we specified. Try hitting the code-completion shortcut (Ctrl-Space) twice and see what happens: The first code-completion fills in the navigation rules for part. According to the grammar this is the only valid text at the beginning of a file, so it is automatically inserted. Hitting Ctrl-Space again will tell you that now you need a Name of type ID. Just go ahead and try out the completion. It will help you create a syntactically sound navigation rules file. Notice that the Problems View tells you what is currently wrong. Also notice, that one you reach a part where references are expected by the grammar (e. g. when defining source and destination coordinates in a navigation rule) you will get suggestions based on what you entered earlier. This is what the whole sample from above looks like in the editor: While you are still fleshing out and fine tuning your grammar definitions, you will probably close this Eclipse instance and reopen it, once you repeated the Run As MWE2 Workflow steps in the main instance. In the long run I suggest you create a Feature and an Update Site project to allow easier distribution and updates of the intermediate iterations. Generating Code Now, as we have a complete Xtext DSL defined and in place let’s have a look at the Code Generation side of things. This part is completely optional: You are free to include the necessary Xtext libraries into your applications runtime (although they seem to be numerous) and just use them to dynamically load and parse .navi files on-the-fly. This would probably be a good idea if you were writing an Eclipse based application anyway. However, when targeting a very limited platform like JavaME this option is not viable. Instead we will now create a code generator that provides a transformation from the DSL syntax into more classic Java terms – specifically we will create a HashMap based data structure that carries all the same information, but in Java terms. This is a sample of what the generated output is going to look like: public class NaviRules { private Map navigationRules = new Hashtable(); // ... public NaviRules() { NaviDestination naviDest; // ========== From Login (com.danielschneller.myapp.gui.login.LoginController) // ========== On USER_LOGON_FAILED // ========== To LoginFailed (com.danielschneller.myapp.gui.login.LoginFailedController in com.danielschneller.myapp.login) naviDest = new NaviDestination(); naviDest.action = "USER_LOGON_FAILED"; naviDest.targetClassname = "com.danielschneller.myapp.gui.login.LoginFailedController"; naviDest.targetBundleId = "com.danielschneller.myapp.login"; store("com.danielschneller.myapp.gui.login.LoginController", naviDest); // ========== On USER_LOGON_SUCCESS // ========== To MainMenu (com.danielschneller.myapp.gui.menu.MainMenuController in com.danielschneller.myapp.menu) naviDest = new NaviDestination(); naviDest.action = "USER_LOGON_SUCCESS"; naviDest.targetClassname = "com.danielschneller.myapp.gui.menu.MainMenuController"; naviDest.targetBundleId = "com.danielschneller.myapp.menu"; store("com.danielschneller.myapp.gui.login.LoginController", naviDest); // ============================================================================= // ========== From LoginFailed (com.danielschneller.myapp.gui.login.LoginFailedController) // ========== On OK // ========== To Login (com.danielschneller.myapp.gui.login.LoginController in com.danielschneller.myapp.login) naviDest = new NaviDestination(); naviDest.action = "OK"; naviDest.targetClassname = "com.danielschneller.myapp.gui.login.LoginController"; naviDest.targetBundleId = "com.danielschneller.myapp.login"; store("com.danielschneller.myapp.gui.login.LoginFailedController", naviDest); // .... and so on ... } } The support class NaviDestination is omitted but is generally just a value holder struct type class. When creating the Xtext project using the wizard earlier we created a third Eclipse project, ending in ...generator. Its src folder contains three subdirectories called model, templates and workflow. Put the sample .navi file into the model directory. It will serve as the input for the generator. Create the first template Code generation is based on templates. Xtext leverages the Xpand template engine. In the templates directory create a new Xpand template using the context menu. Call it NaviRules.xpt, open it and insert the following: «REM» import the namespace defined in our DSL model «ENDREM» «IMPORT navigationRules» «REM» Define a template called "main" for elements of type "Root". The minus sign at the end takes care of not adding a newline at the end of it. «ENDREM» «DEFINE main FOR Root-» «ENDDEFINE» As there is only one instance of a Root element in a navigation rules file, this will be the main entry point - hence the name. There is no need to call it main, but it seems fitting. Now between the DEFINE and ENDDEFINE insert what is to be generated: As shown above, we need a new Java source file called NaviRules.java: ... «DEFINE main FOR Root-» «FILE "NaviRules.java"-» «ENDFILE-» «ENDDEFINE» ... Again, the contents to be generated is put in between the FILE and ENDFILE brackets. Anything not enclosed in «» will be used verbatim in the output file. So first of all, put in the static parts of the Java file. What I did was first write the source for a single navigation rule by hand, made sure it compiled and then copied over the relevant parts into the template piece by piece: ... «FILE "NaviRules.java"-» import java.util.*; public class NaviRules { public static class NaviDestination { String action; List requiredPermissions = new ArrayList(); String targetClassname; String targetBundleId; NaviDestination() {}; public final List getRequiredPermissions() { return new ArrayList(requiredPermissions); } // let Eclipse generate getters, setters, // equals and hashCode methods for this } private Map navigationRules = new Hashtable(); «ENDFILE-» ... Now, this is nothing special so far. To fill in the elements from the navigation rules DSL file put in the following: ... private Map navigationRules = new Hashtable(); public NaviRules() { NaviDestination naviDest; «REM» Iterate all elements in the "rules" collection attribute of the "ruledefs" attribute of the "Root" element. Call each iterated element (which is of type "Rule") "rule" and expand the "ruletmpl" template for it here. «ENDREM» «FOREACH ruledefs.rules AS r»«EXPAND ruletmpl FOR r»«ENDFOREACH» } ... In the class constructor we first define a local variable naviDest of the previously declared type. Then - as the comment states - the FOREACH instruction will iterate over all Rule type elements. This might not seem to be completely obvious at first. Remember at this point in the template the current scope is the "Root" element from the navigation rules file. It has an attribute called ruledefs as per the grammer definition. This attribute is of type NavigationRules which in turn has a collection attribute called rules, containing of Rule type objects. Inside the loop the current element can then be adressed by the template variable name r. The loop body (between FOREACH and ENDFOREACH) contains another Xpand instruction to expand a template called ruletmpl which will be declared next. Don't worry, even though this is a little difficult at first - switching contexts between the Java and the template scopes is made significantly easier in Eclipse, because the Xpand template editor will syntax color (static parts are blue) and also assist you with code completion inside the Xpand template parts. Ctrl-Spacing your way through it will make things more obvious than they are when reading an example. Now for the ruletmpl template. Place it below the ENDDEFINE statement belonging to the main template: ... «ENDFILE-» «ENDDEFINE» «DEFINE ruletmpl FOR Rule-» // ========== From «source.name» («source.controllername») «FOREACH destinations AS d»«EXPAND destTmpl(source) FOR d»«ENDFOREACH» // ============================================================================= «ENDDEFINE» You see the same idea used again: Static parts that get transferred into the output file 1:1 and Xpand statements that fill in data from the navigation rules definition file. In this case you see references to the attributes of the Rule element. As per the FOREACH instruction in the previous template, the one at hand will be repeated for every instance of Rule in our source file. Inside this definition the current scope is that of Rule, so with «source.name» the name attribute of the CoordinateMappingSpec object referenced as source in a Rule is taken first, then the controllername attribute likewise. Next up another FOREACH loop iterates the one or more possible Destinations of each Rule. Instead of just applying a template (destTmpl) for every Destination we also pass in the corresponding CoordinateMappingSpec stored in the source attribute of the Rule. This is then used in the following template: ... «DEFINE destTmpl(CoordinateMappingSpec source) FOR Destination-» // ========== On «transition.name» // ========== To «target.name» («target.controllername» in «target.bundleid») naviDest = new NaviDestination(); naviDest.action = "«transition.name»"; naviDest.targetClassname = "«target.controllername»"; naviDest.targetBundleId = "«target.bundleid»"; «FOREACH permissions AS p»«EXPAND permTmpl FOR p»«ENDFOREACH» store("«source.controllername»", naviDest); «ENDDEFINE» «DEFINE permTmpl FOR PermissionReference-» naviDest.requiredPermissions.add("«permission.value»"); «ENDDEFINE» In this innermost templates the attributes of the CoordinateMappingSpec objects source and target are accessed and put into place to be assigned to the members a NaviDestination Java object instance per Destination. There is only one more (very simple) template for the PermissionReference elements. With this, the Xpand file is complete. Set Up The Generator Workflow The wizard initially created a NavigationRulesGenerator.mwe2 file in the workflow folder. Open it and replace its contents with the following: module workflow.NavigationRulesGenerator import org.eclipse.emf.mwe.utils.* var targetDir = "src-gen" var fileEncoding = "Cp1252" var modelPath = "src/model" Workflow { component = org.eclipse.xtext.mwe.Reader { path = modelPath // this class has been generated by the xtext generator register = com.danielschneller.navi.NavigationRulesStandaloneSetup {} load = { slot = "root" type = "Root" } } component = org.eclipse.xpand2.Generator { metaModel = org.eclipse.xtend.typesystem.emf.EmfRegistryMetaModel {} expand = "templates::NaviRules::main FOREACH root" outlet = { path = targetDir } fileEncoding = fileEncoding } } The most interesting parts of this workflow file are the load section in the Reader component and the expand and outlet sections in the Generator component: The first one will connect a so-called slot with the Root element from our navigation rules. The second one will trigger the evaluation of the main template in the NaviRules.xpt file in the templates folder and feed any Root instances it finds in the *.navi files from the src/model (modelPath) into it. Now it is time for some actual generation. Run the generator workflow Right click the MWE2 file you just edited and select the Run As MWE2 Workflow command from the context menu. The Eclipse console will show this output: 0 [main] DEBUG org.eclipse.xtext.mwe.Reader - Resource Pathes : [src/model] 431 [main] DEBUG xt.validation.ResourceValidatorImpl - Syntax check OK! Resource: file:/Users/ds/ws/ws36_xtext/com.danielschneller.navi.dsl.generator/src/model/MyApp.navi 1013 [main] INFO org.eclipse.xpand2.Generator - Written 1 files to outlet [default](src-gen) 1014 [main] INFO .emf.mwe2.runtime.workflow.Workflow - Done. Then have a look at the newly generated contents of the src-gen source folder. If everything went alright, you should find a fresh NaviRules.java file placed there, based on the contents of your navigation rules file and the Xpand templates. Try and make some changes to the template, then re-run the workflow. You will see the changes reflected in the generated source file. Generate a second source File In the templates directory add another Xpand template file Navigation.xpt with the following content: «IMPORT navigationRules»; «DEFINE main FOR Root-» «FILE "Navigation.java"-» public final class Navigation { «FOREACH ruledefs.rules.destinations.transition.collect(e|e.name).toSet().sortBy(e|e) AS t»«EXPAND actionTmpl FOR t»«ENDFOREACH» private final String name; private Navigation(String aName) { name = aName; } public String getName() { return name; } } «ENDFILE-» «ENDDEFINE» «DEFINE actionTmpl FOR String-» /** Constant for Navigation «this» */ public static final Navigation «this» = new Navigation("«this»"); «ENDDEFINE» This is a template for a type-safe enumeration that can be used in Java 1.4 - remember I had to do this for JavaME. Notice the FOREACH loop in this case. It demonstrates that not only simple iterations are possible, but that Xpand allows more complex operations as well. In this case it will collect the names of all the navigation transitions from all the Destinations in the navigation rules. These are of type String. They are made unique by converting them to a Set datastructure and then finally sorted in their natural order. The resulting list of sorted strings is then iterated, each one - called t - is passed to the actionTmpl template. It is very simple, just placing the string itself («this») into a single line of Java source code. Of course, strictly speaking this is a rather complicated procedure to get the same information we could also have taken from the TransitionDefinitions element in the rules definition. However I think it serves as a nice example for additional Xpand capabilities. For a full description of its possibilities, have a look at the Xpand Reference in the Eclipse documentation. To use the new template, add another section to the MWE2 workflow definition: component = org.eclipse.xpand2.Generator { metaModel = org.eclipse.xtend.typesystem.emf.EmfRegistryMetaModel {} expand = "templates::Navigation::main FOREACH root" outlet = { path = targetDir } fileEncoding = fileEncoding } Running it again will produce a slightly different output, making clear that two files have been generated. This is what comes out in the src-gen folder as Navigation.java: public final class Navigation { /** Constant for Navigation ADMIN */ public static final Navigation ADMIN = new Navigation("ADMIN"); /** Constant for Navigation BACK */ public static final Navigation BACK = new Navigation("BACK"); /** Constant for Navigation DATA_LOOKUP */ public static final Navigation DATA_LOOKUP = new Navigation("DATA_LOOKUP"); /** Constant for Navigation OK */ public static final Navigation OK = new Navigation("OK"); ... More... This was just about my first experiments with Xtext. I am sure there is plenty more to be done with it. For more reading, please have a look at this very nice Getting started with Xtext tutorial by Peter Friese of Itemis. From http://www.danielschneller.com/2010/08/code-generation-with-xtext.html

Last week you may have noticed that Eclipse SDK 4.0 got an early adopter release, allowing developers to play around with the updated SDK to create their own rich client applications. It's a bit different from the "traditional" Eclipse, introducing a model based user interface and CSS for application styling, as well as a services-oriented programming model. While the main focus of the release is to allow Eclipse projects and plugins to prepare for future releases, the following tutorial shows how to write applications in Eclipse 4.0. Tom Schindl has also produced a useful introduction. I will be kicking off my own tutorial into e4 in the next few weeks. I asked Mike Wilson, Eclipse Project PMC Lead a bit more about the e4 release: DZone: What is the difference between e4 and the core Eclipse stream? Mike Wilson: e4 is the name of an incubator, not a development stream. Ignoring source bundles, the difference between the Eclipse 3.6 SDK and 4.0 SDK is (only) a new version of the Workbench bundle, some new branding, and some new bundles to support the new workbench's implementation. The other 184 (assuming I counted correctly) bundles are common between the two versions. Internally, the workbench code has been completely re-architected to provide a new CSS-based look and feel, on top of a fully modelled user-interface. The changes involved in building this have been so significant that we have labelled 4.0 as an "Early Adopter Release". The intent is for it to be used by those early adopters that want to test backwards compatibility and migrate their plug-ins and RCP applications. I expect Eclipse end users will generally adopt the next release, Eclipse 4.1. DZone: Can I use any Eclipse based framework like EMF or GMF in e4? Mike Wilson: Yes. If the framework makes direct use of internals from the 3.x workbench implementation, it will need to be updated to be API clean first, anything else should work fine. The new workbench is actually built using EMF core. DZone: How long will you keep parallel streams going? Mike Wilson: As you can see from the previous answer, the differences between the streams are currently quite small. Depending on where innovation is happening, the delta could get larger but, in any case, the incremental cost of maintaining the existing 3.x stream is low. Really, the constraints that 3.x has (i.e. stability and backwards compatibility above anything else) mean that that we will be able to maintain it as long as the community needs it. DZone: e4 seems ready for early adopters. What is the plan to mature it past incubation status? Mike Wilson: At the risk of sounding like a broken record, e4 is the name of an incubator, of the form which I believe the foundation is calling a "perpetual incubator". It is a sandbox to allow new innovations in the Eclipse platform to be created. It will exist as long as the community believes innovation at the platform level is important. If you mean the Eclipse SDK 4.0 Early Adopter Release, that is *not* in incubation. It differs from any other SDK release only in that sweeping changes in the internals of the workbench may mean that those who consume it will see more visible bugs than in other recent releases. We fully expect to resolve those bugs to bring the quality up to the expected level by next year's Indigo release. Interim milestone builds should provide evidence of that. [Aside: Because we are aligning our 3.7 and 4.1 milestones, "M1" happens one week after 4.0 ships, so you likely won't see much difference for this first one.] DZone: What is your favourite e4 feature? Mike Wilson: My favourite feature is that we have found a way to make innovation possible at the platform level. The new community that has grown around the e4 Incubator is strong evidence that we all care that Eclipse has a future DZone: What is the rationale behind the name? Mike Wilson: "e4" just comes from "e" for Eclipse, and "4" to indicate that the goal is to build the "next major version" of Eclipse after "3.x". The Eclipse SDK 4.0 Early Adopter Release is built using technology from e4, and is the first release of the Eclipse SDK that is part of that new 4.x development stream.

i really don’t like it when eclipse shows errors/warnings annotations in a class. it’s sometimes nice to jump from one to the next and clean out a class one line at a time, but most of the time they’re just distractions, so i want to be able to find and fix them fast. so there must be a better way to jump between the errors/warnings than to use the mouse or page down to the next error. these methods are not only slow but often frustrating because you tend to miss the annotation, especially if it’s a big class. and navigating to the problems view using the keyboard is ok, but sometimes overkill for just clearing out errors/warnings in one class. a good thing then that eclipse offers keyboard shortcuts that take you to the next/previous annotation in the class. and it does so in a way that selects the annotation immediately, allowing you to use quick fix (ctrl+1) to fix it fast. so here’s how to use these shortcuts to navigate between the error/warning annotations and fix some of the errors easily. how to jump between errors and warnings below are the keys you can use to go to the next/previous annotation and to initiate quick fix. shortcut description ctrl+. next annotation. moves to the next warning/error in the class. ctrl+, previous annotation. moves to the previous warning/error in the class. ctrl+1 quick fix. a fast way to resolve certain warnings/errors automatically, but also useful for automating common editing tasks . to see how fast the next annotation/quick fix combo works, i’ve set up an example in the following video that shows a class with multiple errors/warnings and how using the next/previous annotation and quick fix combo can make you work faster. there’s 1 warning (an unused variable, message , that should die) and 2 errors (not wrapping a fileoutputstream call in a try-catch and not initialising a local variable output ). obviously not all errors are going to be solvable by quick fix, but some are: adding a missing cast, filling in types for generics and adding the @suppresswarning or @override annotations. and for the rest, at least you’ll be able to get to them easily. bonus tip : also see how to cleanup some of these warnings automatically every time you save . you can, of course, remap any of these keyboard shortcuts if they’re inconvenient. have a look at how to manage your keyboard shortcuts , specifically looking out for the commands next , previous (yes, for some reason they’re just called next and previous) and quick fix . only jumping between errors you might sometimes want to tell eclipse to only move between errors (and not warnings) when you press the next/previous annotation command. well, this is controlled by a next/previous annotation dropdown menu in the toolbar (as shown in the image below). as you press the next/previous annotation key (either the keyboard shortcut or toolbar button), eclipse will move to whatever annotations are checked in the dropdown. eclipse’s default only has errors and warnings selected (which is a reasonably good default, for once). you could disable warnings if you only want to move between errors, or vice versa. bonus tip: the other selection you might want to enable is occurrences . an occurrence is when you stand on a variable/method and eclipse highlights it and its declaration and usages within the class. if you select occurrences, pressing the next/previous annotation keys will also jump to the variable’s/method’s declaration and usages. this is nice when you want to quickly move to the variable’s usage in a long class. it’s optional if you want this on, so play around and see if it works for you. from http://eclipseone.wordpress.com

Bit.ly is a quite popular URL shortening service. On Tiwtter, almost all of the links I see provided by the people I follow are posted as bit.ly shortcuts. If you’ve used a Twitter client, you probably already know that some of them (if not almost every single of them) offers URL shortening as a built-in capability. Now, you can implement the same functionality, thanks to the fact that bit.ly offers a public API to do this. But let’s start with coding. First of all, all data that is passed to the service is transferred via HTTP requests. The response generated by the service is by default formatted as a JSON document, however the developer can explicitly specify that XML data should be returned. A request to the bit.ly shortening service requires authentication, and the username and API key are required to be passed as parameters. This means that in order to use the service, a bit.ly account is needed (it is free). The API key can be found here (http://bit.ly/account/your_api_key) once the user registered. Shortening The first (and probably the most important method) is the one that actually shortens the URL and it is called /v3/shorten. V3 at the beginning stands for the API version (that is 3.0 at the moment, so don’t worry about that). This method accepts 5 parameters: • format – determines the output format for the request • longUrl –determines the long URL that needs to be shortened • domain – [optional] the domain used for shortening – either bit.ly or j.mp (default: bit.ly) • x_login – the user ID (although in the documentation it is indicated as optional, it is not) • x_apiKey – the user API key (although in the documentation it is indicated as optional, it is not) Let’s look at the method I’ve written to shorten the URL: enum Format { XML, JSON, TXT } enum Domain { BITLY, JMP } string ShortenUrl(string longURL, string username, string apiKey, Format format = Format.XML, Domain domain = Domain.BITLY) { string _domain; string output; // Build the domain string depending on the selected domain type if (domain == Domain.BITLY) _domain = "bit.ly"; else _domain = "j.mp"; HttpWebRequest request = (HttpWebRequest)WebRequest.Create( string.Format(@"http://api.bit.ly/v3/shorten?login={0}&apiKey={1}&longUrl={2}&format={3}&domain={4}", username, apiKey, HttpUtility.UrlEncode(longURL), format.ToString().ToLower(), _domain)); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader (response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } There are two enums that hold the possible data formats as well as the domain names. This is made for safety reasons – if I would pass these as string parameters, there is a higher chance the end-user will pass the wrong string, and then the function will fail. The code is based on a single HttpWebRequest that creates a HTTP request to the URL that is built according to the data passed to it. Then, I am getting the response stream and passing the string representation to the returned string variable. Notice the fact that I am explicitly returning a string value for this method. In fact, I could either return a JsonDocument instance (requires a third-party library to use this class) or XmlDocument. But since there are two possible formats for the request to handle, it is better to return this as a simple string and then let the developer decide what he wants to do next. Once called, the function will return data similar to this: 200 OK http://j.mp/crnexS crnexS msft http://www.microsoft.com 0 Or this (for JSON): { "status_code": 200, "status_txt": "OK", "data": { "long_url": "http:\/\/www.microsoft.com", "url": "http:\/\/j.mp\/crnexS", "hash": "crnexS", "global_hash": "msft", "new_hash": 0 } } Or this (for TXT): http://j.mp/crnexS I am using a custom domain here, but as you see – the format and domain are optional parameters. I can leave them with default values without actually passing them to the function, and then the only values that need to be indicated are the user ID, API key and the long URL. Decoding There is also a way to decode the URL to its initial state from what was the shortened one. The method is called /v3/expand and is used in a similar manner as the shortening one. In this method, I am also using the Format enum to specify the output format: string DecodeUrl(string[] urlSet, string[] hashSet, string username, string apiKey, Format format = Format.XML) { string output; string URL = string.Format(@"http://api.bit.ly/v3/expand?login={0}&apiKey={1}&format={2}", username, apiKey, format.ToString().ToLower()); if (urlSet != null) { foreach (string url in urlSet) URL += "&shortUrl=" + HttpUtility.UrlEncode(url); } if (hashSet != null) { foreach (string hash in hashSet) URL += "&hash=" + hash; } HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } It works a bit different though. As you can see, I am requesting the user to pass two arrays – one with URLs and one with hashes. One of them can be null, therefore the URL can be decoded either by the hash or by the shortened URL. The user can pass both arrays, and get a result similar to this: 200 OK http://j.mp/crnexS http://www.microsoft.com crnexS msft crnexS http://www.microsoft.com crnexS msft The URLs are sanitized inside the function – I am not assuming that the user will pass the encoded URL. In fact, the developer should never assume that the user will pass the correct value – the code should be as foolproof as possible. User validation If you work on an application that depends on the URL shortening service, it would be a good idea to validate the user before making the API calls. Bit.ly provides a method for this as well and it is called /v3/validate. It only requires three parameters – the username, the API key and the output format (that is in fact optional). The C# implementation for this method looks like this: string ValidateUser(string username, string apiKey, string userToCheck, string keyToCheck, Format format = Format.XML) { string output; string URL = string.Format(@"http://api.bit.ly/v3/validate?x_login={0}&x_apiKey={1}&login={2}&apiKey={3}&format={4}", userToCheck, keyToCheck, username,apiKey, format.ToString().ToLower()); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } A bit of confusion can be caused by the fact that there are x_ -prefixed copies of login and API key. You need to pass your ID and API key to verify someone else’s account validity. X_ -prefixed parameters represent the end user. The output should look similar to this: 200 OK 1 Count clicks Bit.ly provides click statistics, so once you shorten a URL, you can track its basic usage. Statistics are available through the /v3/clicks method. It doesn’t have a TXT output format, so you will have to avoid using that (or create a separate enum, that is the best choice). The implementation for it looks like this: string GetClicks(string[] urlSet, string[] hashSet, string username, string apiKey, Format format = Format.XML) { string output; string URL = string.Format(@"http://api.bit.ly/v3/clicks?login={0}&apiKey={1}&format={2}", username, apiKey, format.ToString().ToLower()); if (urlSet != null) { foreach (string url in urlSet) URL += "&shortUrl=" + HttpUtility.UrlEncode(url); } if (hashSet != null) { foreach (string hash in hashSet) URL += "&hash=" + hash; } HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } Same as for the Expand method, an array of URLs and hash codes can be passed and statistics will be generated for multiple entries at once. The response looks like this (in XML format): 200 http://j.mp/crnexS msft 0 crnexS 2230 0 msft crnexS crnexS 2230 OK Note that the XML won’t be indented by default. Check for PRO domain Bit.ly offers pro, customizable domains. That means, that not only bit.ly and j.mp can be used for shortening, but user-defined domains as well. The /v3/bitly_pro_domain method allows to check whether a domain is bit.ly PRO-powered or not. It is very similar to the user validation method, but it accepts the domain name instead of the user credentials. The C# implementation looks like this: string CheckPro(string username, string apiKey, string domain, Format format = Format.XML) { string output; string URL = string.Format(@"http://api.bit.ly/v3/bitly_pro_domain?login={0}&apiKey={1}&domain={2}&format={3}", username, apiKey, domain, format.ToString().ToLower()); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } Once called, the output looks like this: 200 nyti.ms 1 OK URL lookup Bit.ly also allows the lookup of long URLs. For example, you might want to find if there is an existing short URL for the existing long URL. To do this, there is the /v3/lookup method. The implementation is quite simple and as other methods, it has the same base structure: string Lookup(string username, string apiKey, string[] url, Format format = Format.XML) { string output; string URL = string.Format(@"http://api.bit.ly/v3/lookup?login={0}&apiKey={1}&format={2}", username, apiKey, format.ToString().ToLower()); foreach (string _url in url) URL += "&url=" + HttpUtility.UrlEncode(_url); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL); using (WebResponse response = request.GetResponse()) { using (StreamReader reader = new StreamReader(response.GetResponseStream())) { output = reader.ReadToEnd(); } } return output; } The XML response looks similar to this for a positive result (there is a URL found): 200 http://www.dreamincode.net http://bit.ly/mviGY mviGY OK Notice that I can pass an array of URLs to be checked. Mind, though, that the maximum number of URLs that can be passed to the method is 15. With the methods described above, you can harness the power of bit.ly and bring it to your .NET application (the code can easily be ported to any .NET-compatible programming language). For official documentation, you can take a look here.

What exactly is an exception? Exceptions are irregular or unusual events that happen in a method the program is calling which usually occurs at runtime. The method throws the Exception back to the caller to show that it is having a problem. If the programmer runs into this case, then they will need to extend an Exception from the Exception class that is already in the Java class library. In Eclipse, on the class declaration panel, the coder and request “constructors from superclass” and it will give the programmer constructors in a child Exception class that will accept error messages or the address of another Exception as a constructor parameter. When creating an Exception class, the programmer has to designate a kind of exception that must be caught or optionally caught. If you declare the Exception class to extend Exception as shown below, the compiler will insist that the method that is being thrown should also be in a caught in catch block. public class CodeName extends Exception { ……. } The compiler gives the programmer two choices when they call a method that throws an Exception that must be caught: 1. Add a try/catch in the code that is being call to catch the Exception 2. Pass the Exception back on to the caller If the programmer chooses option two then they can do this by adding a "throws" clause to end of the method declaration line. The compiler will generate the code to pass the Exception back to the caller at run-time. In the code below, the AnApplcation program is calling a Java Bean object's openFile() method, passing it the file name. The compiler will say to the openFile() method at compile time: "How do you want to handle the Exception?" The AJavaBean should realize there is nothing they can do in the openFile() method to fix the problem so the programmer should throw the Exception back to AnApplication. Eclipse can see the myProgram() in AnApplication is calling the openFile() method and when openFile() adds "throws FileNotFoundException" to its method declaration line, the compiler gives myProgram() method an error message asking the application method how it would like to handle the Exception. public class AnApplication { public void myProgram() { bean.openFile(filename); …… } } public class AJavaBean { public void openFile(String filename) throws FileNotFoundException { file.open(filename); //open() may throw FileNotFoundException …. } } If the programmer choose the first method then they can see below that all the code to process, open, and read are put into a try block. Once a method is called that might throw and Exception, the call has to be from within a try block because there is a chance of failure. If an Exception has been thrown by a method that is called in the try block shown below, the execution jumps out of the try block and into one of the catch blocks. The code that is left below that point in the try block is skipped as you can tell from the structure below. Execution will continue out the bottom of the catch block which branches to the bottom of the catch group if the catch block doesn’t stop the method processing by doing a return. Try/Catch example in Java: public void myProgram() { try { bean.openFile(fileName); // throws FileNotFoundException help.readFileContents(); // throws ReadException do.processFileData(); // throws ProcessException } catch(FileNotFoundException ex) { System.out.println(ex); // calls toString() on ex } catch(ReadException ex) { System.out.println(ex); // calls toString() on ex } catch(ProcessException ex) { System.out.println(ex); // calls toString() on ex } } } What happen if it was really vital that we close the file we open at the top of the try block? If you close it at the bottom of the try block, an exception is thrown and the execution will never get to the bottom of the try block. To guarantee the file gets closed, you would have to repeat the close() action in every one of the catch blocks which becomes repetitive coding. So you could remove all the close() and include a finally block at the bottom like shown below. After the try block has been entered, the finally code will be executed. The finally code will also be executed also if a catch block is entered, even if the catch does a return. public void myProgram() { try { bean.openFile(fileName); // throws FileNotFoundException help.readFileContents(); // throws ReadException do.processFileData(); // throws ProcessException } catch(FileNotFoundException ex) { System.out.println(ex); // calls toString() on ex } catch(ReadException ex) { System.out.println(ea); // calls toString() on ex } catch(ProcessException ex) { System.out.println(ex); // calls toString() on ex } finally { bean.close(filename); } } All the catch blocks print the Exception object’s toString() on the console as an error message. If you are not going to differentiate processing for different kinds of Exceptions, then you could use a “catch-all” block. Since all exception objects are type Exception then they will be directed into this catch block as shown below. Any unanticipated types of exception will be caught also. public void myProgram() { try { bean.openFile(fileName); // throws FileNotFoundException helper.readFileContents(); // throws ReadException do.processFileData(); // throws ProcessException } catch(Exception ex) { System.out.println(ex); // calls toString() on ex } finally { bean.close(filename); } }

Steven Yi (pictured right) is a programmer and composer living in Rochester, NY. He studied music composition in college and became a programmer afterwards. He started off as a Flash and server side developer (which he did for about 7 years), and has spent the past few years at his current company doing mobile development with J2ME, Android, and iPhone, as well as server-side development with Spring, Hibernate, etc. He started learning and using Java and Swing for personal work in 2000 and has been using it since then for the development of blue, the focus of the interview that follows below. In the interview, Steven talks about the "blue" music composer, how it works, and how the NetBeans Platform and Python form the basis of this cool open-sourced Java music composer. What's "blue"? blue is a music composition environment I started in the fall of 2000. It was actually my very first Java program! At the time, I had started using the music software Csound (http://www.csounds.com) to compose, but found it slow to work with when it came to accomplishing what I was interested in musically. I had the idea to create a simple program that would have a timeline and the ability to scale musical score material in time. Fast forward many years later: in trying to solve other musical problems, and responding to feedback from the community of users, I've expanded blue's features a great deal. It now includes things like a mixer and effects system, a GUI builder tool for creating synthesizer interfaces, embedded Jython processing of musical scripts, and more. It's been quite satisfying to create a tool that can express my musical interests and to find a community of users who have found value in this program for their own musical work. Some screenshots: The Orchestra manager shows a BlueSynthBuilder instrument being edited. The "Reson 6" instrument is shown in edit mode. The BSB Object Properties panel shows the properties for the selected knob: The Score timeline shows a project using multiple parameter automations. The values automated include things like volume, panning, and a time pointer for a phase-vocoder instrument. All BlueSynthBuilder instruments, Effects, and the Mixer volume sliders can be automated: The Score timeline showing the author's composition "Reminiscences". The timeline shows multiple Python SoundObjects used. The SoundObject Editor shows the editor for the selected SoundObject in the timeline. The SoundObject Properties panel shows different properties for the selected SoundObject: The Score timeline showing a Tracker SoundObject being used. The timeline is configured to snap at every 4 beats and the time bar has been configured to show in numbers rather than time: The Score timeline showing a PianoRoll SoundObject being used. The PianoRoll is unique in that it is microtonal, meaning it can adapt the number of steps per "octave", depending on the values configured from a tuning file. In this screenshot, the scale loaded was a Bohlen-Pierce scale, which has 13-tones per tritave (octave and a half): The blue Mixer is shown docked into the bottom bar and in an open state. The interfaces for user-created Chorus and Reverb effects are shown. The interfaces were created using the same GUI builder tool that is found in the BlueSynthBuilder instrument: It's got a very special appearance. How did that come about? blue's custom look and feel started off one day when I was using my Palm PDA. I remember thinking that I enjoyed the look of the device with the backlight on, and so I wanted to recreate that kind of look for my program. Later, I modified the color scheme to tone it down in some ways, but I also introduced more colors than white and cyan to highlight secondary and tertiary features. Maybe now it is now more like Tron than it is like Palm. :) Overall, I enjoy the darker look of the application when I'm working on music. I tend to work on music when I have free time, and that is usually only late at night—I've found having a darker screen has been easier on my eyes. Also, if anyone was wondering, yes, blue is my favorite color. The blue look and feel is encapsulated in a module named "blue-plaf" and is available in the "blue" Mercurial repository (http://bluemusic.hg.sourceforge.net/hgweb/bluemusic/blue). The look and feel is quite hacked up (redoing it properly has been another item on my todo list), but it can be dropped into another application and it should work, as shown below with the CRUD Sample (which can be created from a tutorial found here): Can you explain how blue's timeline works? blue has a concept of SoundLayers and SoundObjects. SoundObjects are objects that primarily produce notes and have a start and duration. There are many different types of SoundObjects in blue and each has an editor (viewed in the SoundObject Editor TopComponent when a SoundObject is selected), and a BarRenderer, which is used to draw the content area of the bar on the timeline. A PolyObject is a special SoundObject. It consists of SoundLayers, which contain SoundObjects. The root timeline is itself just a PolyObject that you can add as many layers to as you like. You can also group individual SoundObjects into their own PolyObjects, and then use the resulting PolyObjects just like any other SoundObject on the timeline. If you double-click a PolyObject, the timeline is then reset with the timeline of the PolyObject you selected. As a result, PolyObjects allow timelines to be embedded within other timelines. If you think about how music is grouped into motives, phrases, sections, and even larger groupings, you can see how PolyObjects might represent these kinds of musical abstractions. For the component design, the ScoreTopComponent starts off with a JSplitPane to split between a SoundLayerListPanel on the left and a JScrollPane on the right. The JScrollPane has a ScoreTimeCanvas (the main timeline) in the main viewPort's view, a panel with the the time bar and tempo editor in the column header, and the corner is used to open up an extra panel to modify properties for the timeline. The JScrollPane has customized JScrollBars used to add the ± buttons that perform zooming on the timeline. There are a number of other features involved that are implemented amongst a number of classes, but the details of how viewPorts are synchronized (among other things) may be a bit too technical to discuss here. For those who are interested, the code can be viewed within the blue.ui.core.score package within the blue-ui-core module. How did blue come to find itself atop the NetBeans Platform? I first started to be interested in NetBeans IDE around the time 4.1 came out, but didn't really get into using it until the release of 5.0. At that time, I had hand-written Swing components for about 4-5 years (I don't really remember when 5.0 was released), and I found Matisse to be quite nice and began using it here and there. I had looked at the NetBeans Platform as an RCP at that time, but found it to be quite a bit to understand. However, I still kept it on my radar. Around the time 6.0 or 6.5 came out, I started to reconsider migrating to the NetBeans Platform once again. By this time, I had moved over to using NetBeans IDE full time for blue development and had been using NetBeans IDE more in general—particularly Java Web development and Ruby on Rails. One of the biggest things I found attractive about NetBeans IDE is its windowing system... and the things I read about in the platform development articles I'd seen online made me curious once again to see what the NetBeans Platform offered. I still felt that there was going to be a big learning curve to learn the NetBeans Platform, but the NetBeans Platform tutorials online were really quite helpful, as were the members of the NetBeans Platform mailing list, and there were also many more books available to help me get started. I think I ultimately spent about 6-8 months migrating blue to using the NetBeans Platform. Granted, it was a busy time in my life and I was working on this only in my spare time, so I think in the end it was a reasonable amount of time. Users have been very positive about the new blue interface and application as a whole, and I think it has been worth spending the time to use the NetBeans Platform. blue's window layout is quite unexpected for a NetBeans Platform application. By the time I had started migrating blue to the NetBeans Platform, the application was already some 7-8 years old. The interface I designed for blue in pure Swing was influenced by my experiences in using Flash, looking at other music composition environments (Digital Audio Workstations and Sequencing Programs), and evaluating the different aspects of working with Csound. Mapping the components from the Swing-based application to the NetBeans Platform was a little tricky in that I couldn't quite get the exact same design of panels as I had in pure Swing. In the end, I tried to think about where most of the components resided physically, and created TopComponents and placed them in the center, left, right, or bottom parts of the main window. I kept some of the dialogs from the old codebase as-is, but I migrated others to be TopComponents so that they could be docked, opened, or dragged out into a dialog as the user wished. In the end, the GUI is different and took a little getting used to after years of using and building the old interface, but I quickly adjusted to the changes and I think there is much greater consistency and usability now. The users have responded very positively to the general polish of the application and to being able to customize their environment. I myself have very much enjoyed being able to dock all of the windows as well as using full-screen mode, especially when I am on my netbook and composing. Excellent! What features of the NetBeans Platform are you using and what do you find to be most useful? Currently, I am using only a very small part of the NetBeans Platform. By the time I started to move my code to the NetBeans Platform, the codebase was already some 7 or 8 years old. I took the approach recommended to me on the mailing list and started off small, focusing primarily on migrating my project to using the Windows System API, the Options API, and a few other utility API's like IO and Dialogs. Having an old codebase, I found that I spent most of my time during migration just reorganizing my UI into TopComponents and working out communications between the components. I also spent time looking at API's that I had developed myself and seeing which ones could be replaced with API's provided by the NetBeans Platform. At this time, the application is still using a number of API's I wrote from the old codebase, but over time I would like to migrate more of the appplication to use the Nodes and Visual Library API's. I think migrating a codebase of this size in phases really worked out well. In the first phase, I was able to take advantage of the Window System API and have a very visible result on the application and gained a lot for usability. Also, a big part of the migration involved moving the codebase from a monolithic source tree and partitioning it into logical modules. I think there really is a great deal of benefit to working with a codebase with modular design, and that too is a very positive result of working with the NetBeans Platform. Please say something about how Jython relates to this application, how you are using it (what the benefits are), and your general opinions on Jython. I have had a Python SoundObject in blue for quite some time—I think since 2002. For me, it is one of the most important tools in blue when it comes to accomplishing what I want musically. With computer music, we have a lot of tools for what I call Common Practice computer music: PianoRolls, Pattern Editors, and Notation Objects. For computer musicians who are interested in Uncommon Practice music, the ability to use a scripting language opens up a number of ways to express musical ideas that cannot be easily conveyed using those other tools. In blue, Jython is primarily used to allow users to write scripts that will generate notes. For myself, I use Python scripts to model orchestral composition, creating Performer and PerformerGroup objects that I write in Python. I also write performance functions, usually per-project, to perform different musical material in different ways. Other users have used Python scripts in exploring things like algorithmic composition and genetic algorithms in their work. A blue project can contain any number of Python objects. The score generated by each Python object is translated and scaled in time by moving and resizing the SoundObject in the timeline. This allows a user who may want to use scripting to create musical material to also take advantage of blue's timeline to organize how the different musical objects will work together. One of the things I most appreciate about Jython (and scripting languages on the JVM in general) is that it is embeddable within a Java application. By packaging and embedding the Jython interpreter within the blue application, users can rest assured that the Python scripts they write can be interpreted anywhere that blue is installed. It's an extra assurance that their musical projects will be long-lasting, but they can still take advantage of a full programming language like Python in their work. Overall, I think that Jython is a fine piece of software and I hope that it will continue to grow and develop for years to come. Is the application open source and are you looking for code contributions and, if so, in which areas? Yes, the application is available under the GPL v2 license, and the source code can be viewed from the Mercurial repository on SourceForge at http://bluemusic.hg.sourceforge.net/hgweb/bluemusic/blue/. I am a strong proponent of open source, especially for creative work. In the same way that we can today look at and study musical works by composers of the past (like Josquin and Bach), I would like to imagine that the work composers and other artists are now creating with computers will also be open and available for study in the future. I believe that using open source software for creative work greatly helps in making musical projects available for the years ahead. I have done most of the development of blue myself, and over the years I've certainly built up a long list of things that I would like to implement. Users have also made wonderful feature requests that I would love to see in the program—but unfortunately, there are only so many hours in the day. It would certainly be nice to have others contributing code! Beyond new features, there are a number of infrastructural things that would be nice to address. The codebase is many years old, and while the application has been refactored multiple times over its lifetime, there are still some areas of the application that could be much more cleanly implemented. Also, in moving over to the NetBeans Platforms, I only really took the first steps. There are a number of components within the application that could probably be better served by migrating to using more of the NetBeans API's. For internal work, things like modifying the timeline to implement zooming to use Graphics2d and transforms, implementing a better waveform renderer for audio files, and further enhancing the instrument GUI builder are all things I'd like to see. I'd also love to get help in migrating all of the tables and trees to using the Nodes API, something that I have not yet had the time to do. It would also be nice to get the manual (currently in HTML and PDF, generated from DocBook) integrated into the application as JavaHelp, but this is another thing that I have had to postpone due to lack of time. For features, some interesting things I'd love to see are a Notation SoundObject, a separate graphical instrument builder using the Visual Library API, and a Sampler instrument. There's also a sound drawing SoundObject, enhancements to existing SoundObjects, and more I'd love to see moving forward. Maybe someone will find these kinds of things interesting and will take a look at blue's code sometime! Thanks Steven and happy music making with blue!

Cucumber is a Behavior-Driven Development tool that lets developers describe their software's behavior in plain text using a business-readable DSL (Domain-Specific Language). Project developers have added a useful adapter for Cucumber which allows users to write step definitions in JavaScript instead of Ruby (described in Joseph Wilk's blog). To use Cucumber, you previously needed to know a slight amount of Ruby, now you can completely forgo using Ruby if you know a little JavaScript. Cucumber supports testing for Java, Ruby, .Net, Flex, Python, web languages, and more. Here are the home page's seven steps for using Cucumber: Describe behaviour in plain text Write a step definition in Ruby (Now you can do this in pure JS!) Run and watch it fail Write code to make the step pass Run again and see the step pass Repeat 2-5 until green like a cuke Repeat 1-6 until the money runs out The new adapter in Cucumber is able to provide JS support for step definitions through TheRubyRacer. This tool allowed Cucumber developers to build the JS adapter by embedding Google's V8 JavaScript interpreter into Ruby. Here is an example of the feature: Feature: Fibonacci In order to calculate super fast fibonacci series As a Javascriptist I want to use Javascript for that @fibonacci Scenario Outline: Series When I ask Javascript to calculate fibonacci up to Then it should give me Examples: | n | series | | 1 | [] | | 2 | [1, 1] | | 3 | [1, 1, 2] | | 4 | [1, 1, 2, 3] | | 6 | [1, 1, 2, 3, 5] | | 9 | [1, 1, 2, 3, 5, 8] | | 100 | [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89] | And the step definitions in JS: Before(['@fibonacci'], function(){ fibResult = 0; }); When(/^I ask Javascript to calculate fibonacci up to (\d+)$/, function(n){ assertEqual(0, fibResult) fibResult = fibonacciSeries(n); }); Then(/^it should give me (\[.*\])$/, function(expectedResult){ assertEqual(expectedResult, fibResult) }); Cucumber developers have tried to make the JS API and the Ruby API as similar as possible, but the JS API currently doesn't have support for calling step definitions within step definitions with multi-line arguments. It also doesn't support line reporting on step definitions. The JS API also has a different way for loading code into the 'World' to make sure it is in scope within the step definitions. For this kind of folder structure: my_js_project/lib/code_lives_here.js my_js_project/features/support/env.js my_js_project/features/my_feature.feature There would be this code within the features/support/env.js setup file: //Cucumber resolves the files relative to the folder that contains the features folder. World(['lib/code_lives_here.js']) Code inside the code_lives_here.js file would be available in the step definitions.

I just installed TPTP into my Eclipse 3.5 under Ubuntu 9.04 and tried to profile a class. The Profile Configuration opened with a red warning reading “the launch requires at least one data collector to be selected“. Clicking the configuration’s Monitor tab reveals a more detailed error (and nothing to select): IWATO435E An error occured when connecting to the host. A quick check of the error log (Window – Show View – Other… – General – Error Log) reveals the cause: RAServer generated the following output: [Error Stream]:ACServer: error while loading shared libraries: /home/jholy/development/tools/eclipse-ide/pulse2-2.4.2/Common/plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/bin/../lib/libtptpUtils.so.4: file too short Checking the content of the lib/ folder revealed an interesting thing: -rw-r–r– 1 jholy jholy 17 2010-02-16 23:16 libtptpUtils.so -rw-r–r– 1 jholy jholy 21 2010-02-16 23:16 libtptpUtils.so.4 -rwxr-xr-x 1 jholy jholy 100K 2010-02-16 23:16 libtptpUtils.so.4.5.0 As also the content of the two small files suggests (they contain a name of the corresponding file with a longer name), the *.so and *.so.4 files should have been links but the installer failed to create them. Solution List all files in the lib/ folder, you will see that there are many real files like libtptpUtils.so.4.5.0 and libxerces-c.so.26.0 and many should-be-links files. The solution is, of course, to replace all those files that shoud be links with actual links. For me the solution was: $ cd .../plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/lib # Move out the files that are OK lib$ mkdir tmp lib$ mv libswt-* libcbe.so tmp/ # Fix the links lib$ for FILE in `ls *.so`; do ln -sf "${FILE}.4.5.0" $FILE; ln -sf "${FILE}.4.5.0" "${FILE}.4"; done # Move the correct files back lib$ mv tmp/* . lib$ rmdir tmp # Fix links for files with *.26 instead of *.4.5.0 lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so.26 lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so.26 lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so lib$ rm libxerces-depdom.so.4 libxerces-c.so.4 # Done! Try to open the profile configuration now, the IWATO435E should have disappeared and you should be able to select a data collector. If not, restart Eclipse, try again, check the error log. My environment Ubuntu 9.04 Eclipse 3.5 TPTP – see above From http://theholyjava.wordpress.com/2010/05/13/eclipse-profile-configuration-the-launch-requires-at-least-one-data-collector/

When you want to work with comments in Eclipse, you could use the slow way of moving to the start of the line, pressing // and then repeating this for all the lines you have. Or you could use the quick way of adding a comment with a single keystroke no matter where the cursor’s positioned in the statement. The same goes for Javadocs – there are just too many things to type before you can start commenting the good stuff. That’s why Eclipse also has a shortcut that let’s you add Javadoc to a field, method or class. Keyboard shortcuts for comments and JavaDocs Here are the keyboard shortcuts for manipulating comments. Shortcut Command Description Ctrl+/ Toggle Comment Add/remove line comments (//…) from the current line. The position of the cursor can be anywhere on the line. Works with multiple selected lines as well. Ctrl+Shift+/ Add Block Comment Wrap the selected lines in a block comment (/*… */). Ctrl+Shift+\ Remove Block Comment Remove a block comment (/*… */) surrounding the selected lines. Alt+Shift+J Add Javadoc Comment Add a Javadoc comment to the active field/method/class. See the notes below for more details on where to position the cursor. Bear the following in mind when using Add Javadoc comment (Alt+Shift+J): To add a comment to a field, position the cursor on the field declaration. To add a comment to a method, position the cursor anywhere in the method or on its declaration. To add a comment to a class, the easiest is to position the cursor on the class declaration. Also works if you’re in the class, but not in a method, field or nested type. The Javadoc comment inserted is based on the Code Templates defined under Window > Preferences > Java > Code Style > Code Templates. If you expand the Comments section, you can change the default for Fields, Methods, Types (eg. classes), etc. Here’s a video to give you an idea of how fast and easy it is to add/remove comments using these shortcuts. The video shows toggling of single line comments, block comments and also adding a Javadoc comment to the method and class. Once I’ve commented out lines, I often find myself copying them and moving them around (eg. to try different variations of the code). You can do this faster by moving and copying lines using with a single keystroke. You can also have Eclipse format the comments whenever you save, saving you formatting time. From http://eclipseone.wordpress.com/2010/05/05/add-comments-and-javadocs-in-eclipse-with-a-single-keystroke/

After a long hiatus, I guess it is time for another SOA anti-pattern to see the light. It is probably also a good time to remind you that I am looking for your insights on this project. In any event I hope you’d find this anti-pattern useful and as always comments are more than welcomed (do keep in mind this is an unedited draft :) ) ------------------------------------- There are many unsolved mysteries, you’ve probably heard about some of them like the Loch Ness monster, Bigfoot etc. However, the greatest mystery, or so I’ve heard, is getting the granularity of services right… Kidding aside, getting right-sized services is indeed one of the toughest tasks designing services – there’s a lot to balance here e.g. the communications overhead, the flexibility of the system, reuse potential etc. I don’t have the service granularity codex and deciding the best granularity depends on the specific context and decisions (e.g. the examples in the Knot anti-pattern above). It is an easier task to define what shouldn’t be a service for instance, calling all of your existing ERP system a single service should definitely be shunned. The Nanoservices anti-pattern talks about the other extreme… the smaller services Consider, for instance, the “calculator service” which appears in samples web-wide (I’ve personally seen examples in .NET, Java, PHP, C++ and a few more). A basic desk calculator, as we all know, supports several simple operations like add, subtract, multiply and divide and sometimes a few more. Implementing a calculator service isn’t very complicated - Listing 10.1 below, for example, shows part of WSDL for a java calculator service that, lo and behold, accepts two numbers and adds them. Listing 10.1 excerpt from a WSDL of a stateless calculator service example. The sample only includes the data needed for the “Add” operation. The add operation accepts two numbers and returns a result (http://cwiki.apache.org/GMOxDOC21/jaxws-calculator-simple-web-service-with-jax-ws.html) Calculator services can be even more advanced and have memory - consider listing 10.2 below, which shows an interface definition for a .NET (WCF) sample that uses workflow services and accepts a single value at a time Listing 10.2 a Service contract definition for a statufil calculator service (http://msdn.microsoft.com/en-us/library/bb410782.aspx). The service accepts a single number at a time and remembers the former state from operation to operation. [ServiceContract(Namespace = "http://Microsoft.WorkflowServices.Samples")] public interface ICalculator { [OperationContract()] int PowerOn(); [OperationContract()] int Add(int value); [OperationContract()] int Subtract(int value); [OperationContract()] int Multiply(int value); [OperationContract()] int Divide(int value); [OperationContract()] void PowerOff(); } The calculator service (both versions of it) is a very fine grained service. Naturally, or hopefully anyway, the calculator examples are just over simplified services used to demonstrate SOA related technologies (JAX-WS in the first excerpt and WCF and WF in the second one). The problem is when we see this level of granularity in real life services 1.1.1Consequences Problem? Why is “fine granularity” a problem anyway? Isn’t SOA all about breaking down monolith “silos” into small reusable services? More so, the finer grained a service is, the less context it carries. The less context a service carries the more reuse potential it has – and reuse is one of the holy grails of SOA isn’t it? The calculator service above seems like the epitome of a reusable service. There’s no doubt we can reuse it over and over and over. Reuse is indeed a noble goal (I’ll leave discussing how real it is for another occasion), the culprit of fine grained services, however, is the network. Services are consumed over networks – both local (LANs) and remote (extranets, WANs etc.). The result is that services are bound by the limitations and costs incurred by those network. Trying to disregard these costs is exactly what ailed most, if not all, RPC distributed system approaches that predated SOA (Corba, DCOM etc.) - The calculator service and other similarly sized services are nanoserivces. Nonoservice is an Anti-pattern where a service is too fine grained. Nanoservice is a service whose overhead (communications, maintenance etc.) out-weights its utility. So how can nanoservices harm your SOA? Nanoservices cause many problems, the major ones being poor performance, fragmented logic and overhead. Let’s look at them one by one Every time we send a request to a service we incur a few costs such as serialization on caller, moving caller process to the OS network service, translation to the underlying network protocol, traveling on the network, moving from the OS network service to the called process, deserialization on the called process – and that’s before adding security (encryption, firewalls etc), routing , retries etc. Modern networks and servers can make all this happen rather fast but if we have a lot of nano-services running around these numbers add-up to a significant performance nightmare, Nano-services cause fragmented logic - almost by definition. As we break what should have been a meaningful cohesive service, into miniscule steps our logic is scattered between the bits that are needed to complete the business service. The fact that you need to haul over several services to accomplish something meaningful also spell increased chances of the Knot anti-pattern, mentioned above. Proliferation of Nanoservices also causes development and management overhead. Just look at the amount of WSDL needed to define the calculator services in listing 10.1 above and for what? A service that adds a couple of numbers… There is a relatively fixed overhead associated with managing a service. This include things like keeping track of a service in a service registry, making sure it adheres to policy, writing the cruft (things we have to write around the business logic) for configuring it etc. Having nano-services around means we have to do this a whole-lot more times (i.e. per service) compared with having fewer coarser grained services. The point of overhead out-weighing utility that appears in the Nano-services definition above is subtle but important. The fact that a contract does not have a lot of operations means we want to make sure we don't have a nano-service, but it doesn't automatically mean that it is. For instance, a fraud detection service contract might only accept transaction details and decide whether to authorize the transaction, deny it or move to further investigation. However the innards of this service involve a complex process like running the details in a rule engine checking for fraudulent behavior patterns, matching to black lists etc. In fact Fraud detection is such a complicated issue that these are actually systems and a SOA based one would be comprised of several services in itself. The other side of the equation is also true a comprehensive contract does not guarantee a service is not a nano-service. For instance, in a system I designed on the initial iterations we developed a resource management service. It supported some very nice operations like getting status of all the services in the system, running sagas and of course allocating services. Allocating services meant that whenever an event went out that needed a (new) service instance to handle it, we had to make a call to the resource manager to get one. This provides for a neat centralized management and also for a performance bottleneck that slows the whole system. To solve this we went with distributed resource management but that't beyond the scope of this discussion. The point, however that is that the utility of the resource management (e.g. easy management of running sagas ) vs. overhead associated with the service (the number of calls and performance hit on the system) was not worth it – Hench a nano-service. 1.1.2Causes From a more technical point of view, we get to nanoservices from not paying attention to at least a couple of the fallacies of distributed computing. Mentioned in chapter 1, the fallacies of distributed computing are a few false assumptions that are easy to make and prove to be wrong and costly down the road. Specifically, we are talking here about assuming that § Bandwidth is infinite – Even though bandwidth gets better and better, it is still not infinite within a specific setup. For instance in one project we were sending images over the wire and distribute them to computational services (a la map/reduce – see also Gridable Service in chapter 3). Things were working ok when we sent small images, but when we sent larger images we understood we were sending them as bitmaps and not as much more compact jpegs which caused a burden on the backbone of our switches which wasn’t ready for that load. § Transport cost is zero – As explained in the previous section every over-the-wire call incurs a lot of costs vs. a local call (also see figure 10.5 below).The costs of the transport can be considered both from the time it take to make each of these calls but even the real dollar value attached to making sure you have enough bandwidth (connection/routers, firewalls) to handle the traffic incurred Figure 10.5 Local objects can “afford” to have intricate interactions with their surroundings. A similar functionality delivered over a network is more likely than not to cause poor performance because of the network related overhead. Another reason to get Nano-Services, at least for beginners are poor examples – as, noted the calculator services above are taken from real examples provided by various vendors. SOA newcomers and/or people without a lot of distributed systems development experience can be easily take these samples at face value, and go about implementing services with similar granularity. The fact that that web-service framework mostly map service calls to object method calls makes this even more tempting. Nano-services is also an inherent risk when applying the orchestrated choreography pattern. Adding an orchestration engine, capable of controlling flow and external to services tempts us to think that we can use it to drive all flow as little as it may seem. Couple this with the fact that the smaller the services are the more “Reuseable” they are (less context) and, again, you may end up with a lot of nano-services on your hands. Lastly, since the nano-services boundary is soft (remember utility vs. overhead weight) behaviors that can look promising at design time can prove to be nano-services moving along (like the resource manager example above). This can be an acceptable if your SOA is developed iteratively (see 10.2.4 exceptions below) but it still mean that we have to come up with ways to refactor nano-services. 1.1.3Refactoring There are basically two main ways to solve the nano-services problem. One, which is relatively easy, is to group related nano-services into a larger service. The second option, which is more complicated, is to redistribute its functionality among other services. Let's take a look at them one by one. On one project I was working on we needed to send out notifications to users and admins via SMS messages. Since the software component that did the actual SMS dissemination was a 3rd party app we’ve decided to create a simple service (not unlike OO adapter) that accepts requests for SMS and talks to the 3rd party software. A nano-service was born, it even got a nice little name Post Office Service (ok, ok the original name was Spam Server but I thought it would look bad in presentations J). Why is this a nano service? Well, it really doesn’t do much and it would be even simpler to package this as a library that other services can use and it does have all the management overhead of maintain as another system service. What we did about it was to add similar functionality to the service so it also learned to send emails, tweets and MMSs. A serendipitous effect of this was that now instead of sending a request like TweetMessage or SendSMS to this service we could now raise more meaningful events such as SystemFailureEvent and have the service make decisions on how to alert administrators based on the severity of the problem etc. So combining the related functionality helped make the overall service even more meaningful. Unfortunately it isn’t always possible to take the functionality of Nano-services and find suitable “other services” (nano-or right-sized) that can assimilate them. In those cases getting rid of a nano-service is more of an exercise in redesign than is a refactoring. For instance, in a project we’ve built we had a services allocation service (SAS). The SAS role was to know about other services location and health status and utilization and upon a request, such as beginning a Saga (see chapter 5 for the saga pattern) decide what service instances should be used. The service also provided “reporting” capabilities for active sagas, services utilization etc. This might not sound like a nano-service, and at first we thought so too, but as the project progressed we found that being a central hub, as seen in figure 10.6 below, made the SAS a performance bottleneck, incurring additional costs (in latency) on a lot of the calls and interactions made by other services. The utility of the SAS, of finding what service instance to talk to, was being diminished by the cost – yep it is a nano-service after all. Figure 10.6 An example for a nano-service. The SAS service is a performance bottleneck as a lot of calls go through it. It provides an important service but the costs of its are too dear. To solve the SAS problem we had to put in quite a lot of work. The solution, essentially was to move to distributed resource management, so that each service had some knowledge of what the world looks like so that it could decide what service instant to talk to by itself. To sum this section, sometimes it is easy to notice that something is a nano-services, chances are that in these cases it would also be easy to take the functionality and group it with related functionality in other services. However on other occasions the fact that a service provides too little benefit is not as apparent and only becomes clear as we move along. In those cases it is also harder to fix the problem. One question we still need to cover is are there any situations where we would go with a nano-service even if we know it is one on the onset. 1.1.4Known Exceptions When is it ok to have Nano-services? When you are starting out. When your approach to SOA is evolutionary and you don’t plan everything in advance (something that rarely work anyway, but that’s another story), there’s a good chance that first versions of services you build will not show a lot of business benefit, but they will already need the full overhead of a service. The post office service in the example above is a good example for that as starting out it only dealt with a single type of message and it didn’t do a whole lot with it either. The post office service is also a good example for another reason to have a nano-service which is when you want to build an adapter or bridge to other systems be that legacy systems or 3rd party ones. In these cases you need to weight the advantage of using a service vs. building the same functionality as a library that can be used within services, but in many cases keeping the flexibility and composability of SOA can triumph over the overhead associated with having an additional service to manage. Lastly, one point to keep in mind is that NanoServices is a rather soft pattern and the value of a small service can radically change from system to system or even in a certain system as time and requirements progress. It is worthwhile questioning our assumptions and looking at the services that we grow from time to time to validate the usefulness of what we’re building.

unused variables and methods should alway be unwelcome. removing them keeps the code cleaner and easier to read. now, by default eclipse warns you about unused private variables and methods, but it doesn’t warn you (by default) about unused method arguments. but there is a compiler setting in eclipse that can warn you when you don’t use an argument in a method. you can even handle arguments on inherited methods, especially useful when using 3rd party libraries. setup the compiler preferences to get warnings of unused arguments: go to window > preferences > java > compiler > errors/warnings . open up the section unnecessary code . change the setting for parameter is never read from ignore to warning. (recommended, but optional) deselect ignore overriding and implementing methods . i recommend deselecting this option. you can leave it selected if you want to, but the feature loses a bit of its usefulness. i discuss this option a bit more in the next section. here’s what the preference should look like: and here’s an example of such a warning. in the example, the argument capacity isn’t used so is annotated as a warning. what’s the easiest way to get rid of the warning? well, just remove the argument using the eclipse change method signature refactoring . this will remove the argument from the method declaration and any method callers in one go. however, if you can’t remove the argument from the method then read the next section. on an old codebase you may get lots of warnings initially. i’d normally handle these on a class-by-class basis only for the classes i’m currently working on, unless the team has a specific cleanup project/task. just something to bear in mind, especially if you share workspace settings across the team via svn or similar. what about method signatures that can’t be changed? when you deselected ignore overriding and implementing methods, you told eclipse to warn you about unused arguments in implemented/overridden methods. it’s a good indicator that your superclass/interface method is passing in more than it needs to (ie. remove the argument from the method signature) or that you’re ignoring something important about the interface contract (ie. use it somewhere in your method). but sometimes you can’t remove the argument because you don’t have control over inherited methods, especially if you implement/override a 3rd party library’s method or if the method is part of a bigger framework that’s difficult to change. other times you won’t have any need for the argument in very isolated cases. so you don’t want a warning to appear for these. that’s why you can use the @suppresswarning compiler annotation to stop eclipse from reporting the warning. here’s an example: public string process(@suppresswarnings("unused") int capacity, int max) {...} you can apply suppresswarning to an individual argument or the whole method. i’d recommend annotating only the argument that you want to ignore. eclipse makes it easy to add the compiler annotation. just navigate to the warning , press ctrl+1 and choose add @suppresswarning ‘unused’ to ‘variable’ (quick fix does its job again). from http://eclipseone.wordpress.com/2010/04/27/get-warnings-about-unused-method-arguments-from-eclipse/

if you’ve been hanging onto one eclipse workspace for the last couple of years, you’ll probably have dozens of projects cluttering up your workspace. if you’re an eclipse rcp developer, you may be sitting with around 50+ projects easily. the thing is that you’ll often only work with 1 or 2 at a time, not the whole lot. and sometimes you want a convenient way of only browsing projects belonging to a specific product/feature/layer/any-other-grouping-that-makes-sense. having all projects in a long list makes it difficult to manage and more difficult to browse. that’s why the package explorer and project explorer have a nice feature that reduces the clutter and allows you to organise your project into categories that make sense. so instead of the package explorer looking unwieldy and flat like this, it could look like this: how to categorise projects this feature works closely with another eclipse feature called working sets. a working set is just a grouping of resources (ie. files, folders and projects). each category is just a working set. to categorise projects, we’ll first create some working sets and then enable the feature in the package explorer. to create a working set do the following: open the package explorer (or project explorer) and in the view menu (triangle in the upper, right corner), click select working set… click new… in the select working set dialog. select resource working set type and click next. on the next page select all the projects you want to include in the working set and pick a descriptive name for your working set. repeat the process for any other working sets you want. click finish. here’s an example of what the working set dialog should look like: now to enable the grouping feature. open the package explorer’s view menu again and enable top level elements > working sets . if it’s the first time you do this, eclipse will prompt you to configure working sets – a way to ask you which working sets to display in the package explorer. select all the working sets you created and click ok. you should now see your working sets in the package explorer and if you expand it, all the projects you selected will be shown. here’s a quick video to give you an example of how this works and looks. in the example, there are 8 projects, 3 belonging to the ui and 3 to the model. the working sets will group them according to ui and model. there are also 2 projects that are scratchpad projects, so for now they won’t be categorised. notes: a project can belong to as many working sets as you want , so it can appear in a number of categories. you can choose which working sets display in the package explorer by choosing configure working sets … from the view menu and then selecting/deselecting the relevant working sets. nb! this menu item is not available if you’re not in the working set layout. to get back to the normal view, select top level elements > projects from the view menu. any project not assigned to a working set goes to a category called other projects . this isn’t really a working set, but a default category that eclipse assigns to group unassigned projects. you can reorder working sets by dragging and dropping them anywhere. new projects can quickly be assigned to working sets by either dragging and dropping them onto the working set or right-clicking on the project and selecting assign working sets . limitations: although working sets help you organise resources, any new resources won’t automatically be added to a working set, even if their parent already belongs to that working set . in our case, this means that any new project needs to be explicitly added to a working set, otherwise it will appear in the other projects category. normally you can do this on new project wizards, so watch out for these. some good reasons why you should categorise your projects now that you know how to group projects, you might be wondering why you should. well, apart from just looking a lot cleaner and less cluttered, it also means that you can act on a group of projects at a time. for example, you can right-click on a working set in package explorer and open, close or refresh all projects in that working set. the same applies to team commands (nice for selective updates). another reason is that you can now selectively search only those projects by telling eclipse to only search certain working sets. now you don’t have to pick up occurrences in test or scratchpad projects if you don’t want to. there are a host of other eclipse commands that act only on individual working sets (eg. build, problems, etc). and the last reason is that now it’s a lot easier to navigate the package explorer with the keyboard. another reason to start learning eclipse keyboard shortcuts and also impress your colleagues.