Earlier this month, I was delivering a proof of concept around Mule that required the use of the schema validation filter. The use case was pretty simple. XML messages comes in through a SOAP request, and some fields in the incoming message had to be validated against a bunch of XML schemas. Well this is the perfect use case scenario for the Schema Validator that is provided out of the box with Mule. Its an element you can drop from Mule Studio palette straight into your flow. schema-validator Configuration of this element is a bit tricky. This accepts a comma separated list of schemas to validate the payload against. Fair enough, but you immediately face the first problem. How to come up with a URL that works both in Mule Studio and in Mule standalone if your schemas are part of the classpath? Usually in Spring, you do this by pre-pending your URL with “classpath:”. Unfortunately this does not work with this filter. We solved the issue by placing the schemas under the src/main/resources folder in Mule Studio. From the schema validation filter, we referred to them using the ${app.home} variable: The second issue that immediately comes up is that any xs:imports or xs:includes in your schemas are happily ignored by the schema validator. The schema validator will fail on startup with an exception that looks similar to the following (exception detail removed for simplicity). Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'schemaValidator': Invocation of init method failed; nested exception is org.mule.api.lifecycle.InitialisationException: src-resolve: Cannot resolve the name 'xxxyyy' to a(n) 'type definition' component. ... Caused by: org.mule.api.lifecycle.InitialisationException: src-resolve: Cannot resolve the name 'xxxyyy' to a(n) 'type definition' component. ... Caused by: org.xml.sax.SAXParseException: src-resolve: Cannot resolve the name 'xxxyyy' to a(n) 'type definition' component. ... This is due to the class; javax.xml.validation.SchemaFactory. Mule’s schema validation filter uses this class to build the schemas to be able to validate against. This factory class ignores any XSD ‘includes’ and ‘imports’. (See here for more information.) Luckily there is a solution. Mule’s schema validation filter allows you to configure a custom resource resolver. This is the class that is used to load external resources, like XSD ‘includes’ and ‘imports’. We have written a very simple resource resolver that loads the external resources from the classpath: package com.ricston.xml; import java.io.IOException; import java.io.InputStream; import org.apache.xerces.dom.DOMInputImpl; import org.mule.util.IOUtils; import org.w3c.dom.ls.LSInput; import org.w3c.dom.ls.LSResourceResolver; public class ClasspathResourceResolver implements LSResourceResolver { @Override public LSInput resolveResource(String type, String namespaceURI, String publicId, String systemId, String baseURI) { try { InputStream resource = IOUtils.getResourceAsStream(systemId, getClass()); return new DOMInputImpl(publicId, systemId, baseURI, resource, null); } catch (IOException e) { e.printStackTrace(); return null; } } } As you can see, it is extremely simple. Just 2 lines of Java code. The idea here is that we are loading the external resource passed in by the SchemaFactory by using the Mule IOUtils library. Once loaded, we have to pass back an instance of org.w3c.dom.ls.LSInput. We used org.apache.xerces.dom.DOMInputImpl to help us with that. The last step is now to configure the schema validation filter to use our custom resource resolver: Of course you can code your custom resource loader to load resources from absolute paths, relative paths, or any other method that you would like to implement. It’s simple enough to take our ClasspathResourceResolver and convert it to meet your needs. Another issue that I found with this filter is the way it handles the errors. If you supply it with an XML that does not match any schema, it silently blocks your message. There won’t be anything displayed in the logs by default. To see the errors, you need to configure the SchemaValidationFilter class to log to debug level through log4j: log4j.logger.org.mule.module.xml.filters.SchemaValidationFilter=DEBUG We are going to conclude this blog post by saying that the out of the box schema validation filter of Mule is quite useful for obvious reasons, but to exploit its full potential does require a bit of configuration work.

This article presents the usage of sripted data set in the eclipse's BIRT.

Using Apache Commons Configuration to configure long-running applications.

This morning an interesting topic was posted to the Apache ServiceMix user forum, asking the question: To ServiceMix or not ServiceMix. In my mind the short answer is: NO Guillaume Nodet one of the key architects and long time committer on Apache ServiceMix already had his mind set 3 years ago when he wrong this blog post - Thoughts about ServiceMix. What has happened on the ServiceMix project was that the ServiceMix kernel was pulled out of ServiceMix into its own project - Apache Karaf. That happened in spring 2009, which Guillaume also blogged about. So is all that bad? No its IMHO all great. In fact having the kernel as a separate project, and Camel and CXF as the integration and WS/RS frameworks, would allow the ServiceMix team to focus on building the ESB that truly had value-add. But that did not happen. ServiceMix did not create a cross product security model, web console, audit and trace tooling, clustering, governance, service registry, and much more that people were looking for in an ESB (or related to a SOA suite). There were only small pieces of it, but never really baked well into the project. That said its not too late. I think the ServiceMix project is dying, but if a lot of people in the community step up, and contribute and work on these things, then it can bring value to some users. But I seriously doubt this will happen. PS: 6 years ago I was working as a consultant and looked at the next integration platform for a major Danish organization, and we looked at ServiceMix back then and dismissed it due its JBI nature, and the new OSGi based architecture was only just started. And frankly it has taken a long long time to mature Apache Karaf / Felix / Aries and the other pieces in OSGi to what they are today to offer a stable and sound platform for users to build their integration applications. That was not the case 4-6 years ago. Okay No to ServiceMix - what are my options then? So what should use you instead of ServiceMix? Well in my mind you have at least these two options. 1) Use Apache Karaf and add the pieces you need, such as Camel, CXF, ActiveMQ and build your own ESB. These individual projects have regular releases, and you can upgrade as you need. The ServiceMix project only has the JBI components in additional, that you should NOT use. Only legacy users that got on the old ServiceMix 3.x wagon may need to use this in a graceful upgrade from JBI to Karaf based containers. 2) Take a look at fabric8. IMHO fabric8 is all that value-add the ServiceMix project did not create, and a lot more. James Strachan, just blogged today about some of his thoughts on fabric8, JBoss Fuse, and Karaf. I encourage you to take a read. For example he talks about how fabric becomes poly container, so you have a much wider choice of which containers/JVM to run your integration applications. OSGi is no longer a requirement. (IMHO that is very very existing and potentially a changer). I encourage you to check out fabric8 web-site, and also read the overview and motivation sections of the documentation. And then check out some of the videos. After the upcoming JBoss Fuse 6.1 release, the Fuse team at Red Hat will have more time and focus to bring the documentation at fabric8 up to date covering all the functionality we have (there is a lot more), and as well bring out a 1.0 community released using pure community releases. This gives end users a 100% free to use out of the box release. And users looking for a commercial release can then use JBoss Fuse. Best of both worlds. Summary Okay back to the question - to ServiceMix or not. Then NO. Innovation happens outside ServiceMix, and also more and more outside Apache. If you have thoughts then you can share those in comments to this blog, or better yet, get involved in the discussion forum at the ServiceMix user forum. PPS: The thoughts on this blog is mine alone, and are not any official words from my employer.

Renaming domain names is not supported by Glassfish application server. There is no toolkit or any command available by Glassfish to perform domain name rename. You can choose your desired domain name when you install glassfish 3.x, but Glassfish 4 doesn’t let you choose your desired domain name (auto selecting domain1 for domain name). Without diving into domain configuration files or messing with domain XML files, the rename operation can be done quickly and cleanly. In case another name is required for currently operating Glassfish domain, follow these simple steps: Steps are so easy and simple: 1. Execute asadmin utility of your Glassfish distribution. 2. Stop your domain if it’s running by issuing stop-domain command. 3. Run backup-domain . 4. After executing the backup domain command, your backed up domain is located under glassfish_ home/domains//backups directory a .zip file. 5. Copy the generated .zip backup file to other location. 6. Now delete your existing domain by issuing delete-domain command. Don’t worry you have already backed up your domain settings, remember? 7. Open your backup file, normally packaged as a zip file. Open the backup.properties file and change the domain.name property to your desired name. 8. Save the file and make sure that the modification is applied to your zip file too. 9. Bring up asadmin utility up and issue the following command restore-domain domain1 --filename --force 10. Execute list-domains command to see your domain with its new name. That's it. Your domain name is changed and your settings are restored as well.

In this post I would like to describe how to build a fat JAR using NetBeans IDE.

gaurav gupta is a senior software engineer working on the bpmn workflow engine. he's created several plugins for netbeans ide, notably the js css minify compress plugin ( here ) and the jbpmn plugin, which is the topic of this interview. the plugin provides netbeans ide tools for working with business process modeling notation (bpmn). the plugin can be accessed here, including for netbeans ide 7.4 and 8.0 beta, together with screencasts and tutorials: http://plugins.netbeans.org/plugin/50735/jbpmn hi, gaurav, why did you create the bpmn plugin? business process modeling notation is an increasingly important standard for process modelling and has enjoyed high levels of adoption, so the specific intent of the jbpmn plugin is to create a bpmn modeler that supports the complete bpmn specification and can be integrated by multiple bpmn engine vendors into netbeans ide. as a result, different bpmn engine vendors will not need to create separate bpmn modelers for netbeans ide. another reason i created this plugin was to bring the netbeans community closer to the bpm community because bpmn has become the de-facto standard for business process modeling. also, when we talk about "netbeans vs eclipse", the basic conclusion is that netbeans is much more intuitive and easy to use, while eclipse has a wider range of third-party plugins support from more companies. in that context, i have taken this initiative to create the bpm plugin to help the netbeans community. what would you consider to be the best features of the bpmn plugin? the jbpmn nb modeler is a graphical modelling tool which allows the creation and editing of bpmn process diagrams. it provides debugging supports for jbpm 5.0. it is bpmn engine vendor neutral, can be used by any vendor or generated xml, and can run on any bpmn engine which adopts the bpmn 2.0 standard. here's another screenshot: what are the future development plans for this plugin? to cover the complete bpmn ssecification with user friendly gui and properties support. jbpmn currently supports only the bpmn process model, it will also support the bpmn conversation model. it will also provide support to extends the modeller with your own palette elements, properties, and generated xml tags. it will provides debugging supports for jbpm 6.0, activities, and also extensions so that any bpmn engine vendor can integrate debugging functionality within jbpmn. it will also provide a netbeans modeler platform api, which any business modeller can use to easily build solutions such as a bpmn conversation model.

Companies which use Checkstyle usually extend current set of checks by their own or modify existing ones to satisfy their needs. And there are lots of ready-to-use solutions which help to use Checkstyle in a number of ways: Maven Checkstyle Plugin, Intellij IDEA Checkstyle Plugin and Eclipse Checkstyle Plugin. There is a specific IDE environment which is different between the same company departments or even between team members. Integration of custom checks to all of them is not that simple. There is Sonar Checkstyle Plugin which could help integrate checks and let to show validation results to all of its users, no matter what IDE they use. In this article I'll provide an example about Checkstyle usage in Sonar which is a cross IDE solution for different platforms and environment. The example will be shown on sevntu.checkstyle project which contains a number of additional (non-standard) checks for Checkstyle. Here are some of the valuable checks to my opinion (7 out of 32): AvoidNotShortCircuitOperatorsForBooleanCheck – forces user not to use ShortCircuit operators ("|", "&" for boolean calculations). CustomDeclarationOrderCheck – adjusts class structure to make it more predictable. VariableDeclarationUsageDistanceCheck – checks distance between declaration of variable and its first usage of it. EitherLogOrThrowException – notifies about either log the exception, or throw it, but never do both. AvoidHidingCauseExceptionCheck – checks for hiding the cause of exception by throwing a new exception. ConfusingConditionCheck – prevents negation within an "if" expression if "else" is present. ReturnNullInsteadOfBoolean – notifies about returning null instead of boolean. There is an extension for Sonar's Checkstyle plugin which allows to use non-standard checks within Sonar. Let's dive a bit into the process of integration. Each check is represented as a separate rule in Sonar. After creating a new check we have to add a new rule in order so Sonar could understand and use this new check. To accomplish this we use checkstyle-extensions.xml configuration file in sevntu-checkstyle-sonar-plugin project. For instance, here is a rule for ReturnNullInsteadOfBoolean: com.github.sevntu.checkstyle.checks.coding.ReturnNullInsteadOfBoolean Returning Null Instead of Boolean Method declares to return Boolean, but returns null. Checker/TreeWalker/com.github.sevntu.checkstyle.checks.coding.ReturnNullInsteadOfBoolean To make Sonar know about a new check we have to complete the following steps: # build the project $ cd sevntu-checkstyle-sonar-plugin $ mvn clean install # copy the resulted jar file into Sonar $ cp target/sevntu-checkstyle-sonar-plugin-x.x.x.jar [SONAR_HOME]/extensions/plugins/ # restart Sonar $ [SONAR_HOME]/bin/linux-x86-64/sonar.sh restart The only thing is left is that we have to create a new profile in Sonar's “Quality Profiles” tab. We have already created a default Checkstyle configuration which contains all the non-standard checks from “sevntu.checkstyle” project. So, we can just import this configuration when creating a new profile and that's it: Now we can configure and use non-standard Checkstyle checks in addition to the standard ones within Sonar: This project is a good example of how you can integrate your custom checks into a static stage of code analysis, and make it user friendly, accessible for all members in your team and not get involved in a war of “which IDE is the best and more functional for static code analysis”. Useful links: Install Sonar and analyze a project How to integrate sevntu checks into SonarQubeTM (developer's guide) How to integrate sevntu checks into SonarQubeTM (user's guide) Mail-list for QnA

Get started with Spring IDE and the Spring Tool Suite – a set of plugins to simplify the development of Spring-based applications in Eclipse.

This Document tries to help you install Sonar, analyze your project with your Sonar installation, integrate with Eclipse, clean up violations dynamically, and practice better coding. Table of Contents Sonar Installation Download Sonar Unzip and Install Run Sonar Sonar Console Access your Sonar installation Generate Sonar Report Update your POM with SONAR configurations Example Access your project in Sonar Integrate SONAR with Eclipse Eclipse Sonar Plug-In Installation Eclipse Integration (To install this plugin in the Eclipse IDE) - With Eclipse Market Place Eclipse Integration (To install this plugin in the Eclipse IDE) - With Eclipse Software Update Configure Sonar in your Eclipse Link your project for the first time Analyze and clean up the code violations Run Sonar Analysis in Local Sonar Installation Download Sonar Download the sonar here http://dist.sonar.codehaus.org/sonar-3.5.1.zip and unzip the download to your favorite folder Unzip and Install After Unzip you will see folder structure would look something like as follows.. Figure 1 – Sonar Dir Structure Run Sonar Depends on your OS, you need to run the executable , for an instance if you are running linux-x86 and 64 bit, then you need to run start.sh Figure 2 – Run Sonar Sonar Console After you start the sonar you will see some info as follows after you run the sonar Figure 3 - Sonar Console Access your Sonar installation Now you can browse your sonar installation http:localhost:9000 Generate Sonar Report Update your POM with SONAR configurations After we have the sonar installed, we can generate the reports for any maven project, by adding the following lines in your project pom.xml (sonar hosts in your properties section) Figure 4 - POM XML for Sonar Generation Example Let’s take an example of project-common; do the following steps · Checkout the latest code from repository to your work space · Do mvn clean install · Modify your pom.xml (pom.xml) to have the following under properties section · http://localhost:9000/ · Save the file · Do mvn sonar:sonar in your command / terminal · You will see some messages as following. Figure 5 - Sonar report Generation - I Note: And after few minutes (depends on the size of the modules the sonar report would even take longer) Figure 6 - Sonar Report Generation - II Finally you would see the following that indicates the sonar reporting is completed.. Figure 7 - Sonar Report Generation Successful Access your project in Sonar Now go to you http://localhost:9000 you would see the project report that you ran for Figure 8 - Sonar Project Report at your Local Integrate SONAR with Eclipse Eclipse Sonar Plug-In Installation Eclipse Integration (To install this plugin in the Eclipse IDE) - With Eclipse Market Place Figure 9 - Sonar Eclipse Plug-in Install (Market Place) Figure 10 - Sonar Eclipse Plug-in Install (Market Place) II Eclipse Integration (To install this plugin in the Eclipse IDE) - With Eclipse Software Update Go to Help > Install New Software... This should display the Install dialog box. Paste the Update Site URL (http://dist.sonar-ide.codehaus.org/eclipse/) into the field Work with and press Enter. This should display the list of available plugins and components: Figure 11- Sonar Eclipse Plug-in Install (With Install New Software Menu) Choose Sonar Java, follow the steps and install the plugin Note: Please make sure the project that you want to associate with sonar has already analyzed in your sonar installation Configure Sonar in your Eclipse Configure your local/remote sonar in your Eclipse Go to Window > Preferences > Sonar > Servers. Sonar Eclipse is pre-configured to access a local Sonar server listening on http://localhost:9000/. You can edit this server, delete it or add a new one. Figure 12 - Configure Sonar Server in Eclipse Link your project for the first time Once the Sonar server is defined, the next step is to link your Eclipse projects with projects defined and analyzed on this Sonar server. To do so, right-click on the project in the Project Explorer, and then Configure > Associate with Sonar...: Figure 13 - Configure / Associate your Eclipse Project with Sonar In the Sonar project text field, start typing the name of the project and select it in the list box: Figure 14 - Associate your Eclipse Project with Sonar II Click Finish. Your project is now associated to one analyzed on your Sonar server. Analyze and clean up the code violations Do local analysis and clean the violations Figure 15 - Configure Modules Figure 16 - configure sonar modules from Eclipse Note Please make sure you have started your local sonar server (as described in Run sonar section) otherwise you would not able to see the right sonar project that you intend to configure Run Sonar Analysis in Local Figure 17.a – Set Sonar Analysis on Local Mode Figure 17:b - Run Sonar Analysis on Local Figure 18 - sonar violation analysis console Figure 19 - Sonar violation analysis console II Figure 20 - Sonar violations Markers

This is the first in a series of posts on the new “Red Deer” (https://github.com/jboss-reddeer/reddeer) open source testing framework for Eclipse. In this post, we’ll introduce Red Deer, and take a look at the some of the advantages that it offers by building a sample test program from scratch. Some of the features that Red Deer automated offers are: An easy to use, high-level API for testing standard Eclipse components Support for creating custom extensions for your own applications A requirements validation mechanism to assist you in configuring complex tests Eclipse Tooling to Assist in Creating new Projects A record and playback tool to enable you to quickly create automated tests An integration with Selenium for testing web based applications Support for running tests in a Jenkins CI environment Note that as of this writing, Red Deer is in an incubation stage. The current release is at level 0.5. The target date for the 1.0 release of Red Deer is late 2014. But, as a community-based, open source project, now is a great time to try Red Deer and make suggestions or even contribute code! A Look at Red Deer’s Architecture The Red Deer project itself is comprised of utilities and the API that supports the development and execution of automated tests. The API (the parts of the above diagram that are enclosed in dashed line boxes) can be thought of as having three layers: The top layer consists of extensions to Red Deer’s abstract classes or implementations for Eclipse components such as Views, Editors, Wizards, or Shells. For example, if you are writing tests for a feature that uses a custom Eclipse View, you can extend Red Deer’s View class by adding support for the specific functions of the feature. The advantage that this API layer gives you is that your test programs do not have to focus on manipulating the individual UI elements directly to perform operations. Your programs can instead instantiate an instance of an Eclipse component such as a View, and then use that instance’s methods to perform operations on the View. This layer of abstraction makes your test programs easier to write, understand, and maintain. The middle layer consists of the Red Deer implementations for SWT UI elements such as: Button, Combo, Label, Menu, Shell, TabItem, Table, ToolBar, Tree. This API layer supports the API’s higher level by providing the building blocks for the API’s Views, Editors, Shells, and WIzards. This middle layer of the API also provides Red Deer packages that enable your tests to enforce requirements, so that necessary setup tasks are performed before a test is run. The bottom layer consists of Red Deer packages that support the execution of tests such as: Conditions, Matchers, Widgets, Workbench, and Red Deer extensions to JUnit. What Makes Red Deer different from other Tools? A Layer of Abstraction The top-most layer of the API enables you to instantiate Eclipse UI elements as objects, and then manipulate them through their methods. The resulting code is easier to read and maintain, instead of being brittle and subject to failures when the UI changes. For example, for a test that has to open a view and press a button, without Red Deer, the test would have to navigate the top level menu, find the view menu, then the view type in that menu, then find the view open dialog, then locate the “OK” button, etc. Your test would have to spend a lot of time navigating through the UI elements before it could even begin to perform the test’s steps. With Red Deer, the code to open a view (in this case, the servers view) is simply: ServersView view = new ServersView(); view.open(); Furthermore, within that ServersView, your test program can perform operations on the View through methods which are defined in the view (and are incidentally also well debugged by the Red Deer team), instead of having to explicitly locate and manipulate the UI elements directly. For example, to obtain a list of all the servers, instead of locating the UI tree that contains the server list, and extracting that list of servers into an array, your Red Deer program can simply call the “getServers()” method. Likewise, the code to open a PackageExplorer, and then select a project within that PackageExplorer is as follows: PackageExplorer packageExplorer = new PackageExplorer(); packageExplorer.open(); packageExplorer.getProject("myTestProject").select(); And, the code to retrieve all the projects within that PackageExplorer is simply: packageExplorer.getProjects(); The result are that your tests are easier to write and maintain and you can focus on testing your application’s logic instead of writing brittle code to navigate through the application. Installing Red Deer The only prerequisites to using Red Deer are Eclipse and Java. In this post, we’ll use Eclipse Kepler and OpenJDK 1.7, running on Red Hat Enterprise Linux (RHEL) 6. To install Red Deer 0.4 (this is the latest stable milestone version as of this writing) follow these steps: Open up Eclipse Navigate to: Help->Install New Software Define a new download site using the Red Deer update site URL: http://download.jboss.org/jbosstools/updates/stable/kepler/core/reddeer/0.4.0/ Select Red Deer, click on the Finish button and Red Deer will install Now that you have Red Deer installed, let’s move onto building a new Red Deer test. Building your First Red Deer Test To create a new Red Deer test project, you make use of the Red Deer UI tooling and select New->Project->Other->Red Deer Test: Before we move on, let’s take a look at the WEB-INF/MANIFEST.MF file that is created in the project: Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: com.example.reddeer.sample Bundle-SymbolicName: com.example.reddeer.sample;singleton:=true Bundle-Version: 1.0.0.qualifier Bundle-ActivationPolicy: lazy Bundle-Vendor: Sample Co Bundle-RequiredExecutionEnvironment: JavaSE-1.6 Require-Bundle: org.junit, org.jboss.reddeer.junit, org.jboss.reddeer.swt, org.jboss.reddeer.eclipse The line we’re interested in is the final line in the file. These are the bundles that are required by Red Deer. After the empty project is created by the wizard, you can define a package and create a test class. Here's the code for a minimal functional test. The test will verify that the eclipse configuration is not empty. package com.example.reddeer.sample; import static org.junit.Assert.assertFalse; import java.util.List; import org.jboss.reddeer.swt.api.TreeItem; import org.jboss.reddeer.swt.impl.button.PushButton; import org.jboss.reddeer.swt.impl.menu.ShellMenu; import org.jboss.reddeer.swt.impl.tree.DefaultTree; import org.junit.Test; import org.junit.runner.RunWith; import org.jboss.reddeer.junit.runner.RedDeerSuite; @RunWith(RedDeerSuite.class) public class SimpleTest { @Test public void TestIt() { new ShellMenu("Help", "About Eclipse Platform").select(); new PushButton("Installation Details").click(); DefaultTree ConfigTree = new DefaultTree(); List ConfigItems = ConfigTree.getAllItems(); assertFalse ("The list is empty!", ConfigItems.isEmpty()); for (TreeItem item : ConfigItems) { System.out.println ("Found: " + item.getText()); } } } After you save the test's source file, you can run the test. To run the test, select the Run As->Red Deer Test option: And - there's the green bar! Simplifying Tests with Requirements Red Deer requirements enable you to define actions that you want happen before a test is executed. The advantage to using requirements is that you define the actions with annotations instead of using a @BeforeClass method. The result is that your test code is easier to read and maintain. The biggest difference between a Red Deer requirement and the the @BeforeClass annotation from the JUnit framework is that if a requirement cannot be fulfilled the test is not executed. Like everything else in Red Deer, you can make use of predefined requirements, or you can extend the feature by adding your own custom requirements. These custom requirements can be made complex and for convenience can be stored in external properties files. (We’ll take a look at defining custom requirements in a later post in this series when we examine how to create and contribute extensions to Red Deer.) The current milestone release of Red Deer provides predefined requirements that enable you to clean out your current workspace and open a perspective. Let’s add these to our example. To do this, we need to add these import statements: import org.jboss.reddeer.eclipse.ui.perspectives.JavaBrowsingPerspective; import org.jboss.reddeer.requirements.cleanworkspace.CleanWorkspaceRequirement.CleanWorkspace; import org.jboss.reddeer.requirements.openperspective.OpenPerspectiveRequirement.OpenPerspective; And these annotations: @CleanWorkspace @OpenPerspective(JavaBrowsingPerspective.class) And, we also have to a reference to org.jboss.reddeer.requirements to the required bundle list in our example’s MANIFEST.MF file: Require-Bundle: org.junit, org.jboss.reddeer.junit, org.jboss.reddeer.swt, org.jboss.reddeer.eclipse, org.jboss.reddeer.requirements When we’re done, our example looks like this: package com.example.reddeer.sample; import static org.junit.Assert.assertFalse; import java.util.List; import org.jboss.reddeer.swt.api.TreeItem; import org.jboss.reddeer.swt.impl.button.PushButton; import org.jboss.reddeer.swt.impl.menu.ShellMenu; import org.jboss.reddeer.swt.impl.tree.DefaultTree; import org.junit.Test; import org.junit.runner.RunWith; import org.jboss.reddeer.junit.runner.RedDeerSuite; import org.jboss.reddeer.eclipse.ui.perspectives.JavaBrowsingPerspective; import org.jboss.reddeer.requirements.cleanworkspace.CleanWorkspaceRequirement.CleanWorkspace; import org.jboss.reddeer.requirements.openperspective.OpenPerspectiveRequirement.OpenPerspective; @RunWith(RedDeerSuite.class) @CleanWorkspace @OpenPerspective(JavaBrowsingPerspective.class) public class SimpleTest { @Test public void TestIt() { new ShellMenu("Help", "About Eclipse Platform").select(); new PushButton("Installation Details").click(); DefaultTree ConfigTree = new DefaultTree(); List ConfigItems = ConfigTree.getAllItems(); assertFalse ("The list is empty!", ConfigItems.isEmpty()); for (TreeItem item : ConfigItems) { System.out.println ("Found: " + item.getText()); } } } Notice how we were able to add those functions to the test code, while only adding a very small amount of actual new code? Yes, it can pay to be a lazy programmer. ;-) What’s Next? What’s next for Red Deer is its continued development as it progresses through its incubation stage until its 1.0 release. What’s next for this series of posts will be discussions about: The Red Deer Recorder - To enable you to capture manual actions and convert them into test programs How you can Extend Red Deer - To provide test coverage for your plugins’ specific functions. And How you can Contribute these extensions to the Red Deer project. How you can Define Complex Requirements - To enable you to perform setup tasks for your tests. Red Deer’s Integration with Selenium - To enable you to test web interfaces provided by your plugins. Running Red Deer tests with Jenkins - To enable you to take advantage of Jenkins’ Continuous Integration (CI) test framework. Author’s Acknowledgements I’d like to thank all the contributors to Red Deer for their vision and contributions. It’s a new project, but it is growing fast! The contributors (in alphabetic order) are: Stefan Bunciak, Radim Hopp, Jaroslav Jankovic, Lucia Jelinkova, Marian Labuda, Martin Malina, Jan Niederman, Vlado Pakan, Jiri Peterka, Andrej Podhradsky, Milos Prchlik, Radoslav Rabara, Petr Suchy, and Rastislav Wagner.

this post is not about variables in my application code (which i debug). it is about using variables in eclipse for building projects. eclipse variables allow me to make my projects ‘position independent’ whenever i cannot use a path relative to my projects or workspace. eclipse variables which variables are used where in eclipse might be sometimes not very clear. depending in which context variables are used, not everything might be available. this link for example gives a list of variables which can be used to invoke an external tool. build variables eclipse comes with many built-in variables, especially for the build system. if i want to see what variables are already defined, i can show them in the project properties, under c/c++ build > build variables with enabled option ‘show system variables’: system build variables with the ‘add…’ button i can define and add my own variables, available for that project: define a new build variable if above operation is done on a project, then the setting is for the project only. if i want to add a variable for the workspace, i can do this using the menu window > preferences : workspace build variables global system variables eclipse automatically includes the system (e.g. windows) environment variables. many dialogs have the ‘variables…’ button where i can use my variables, including the variables defined on system level: system variables system variables: one way or the other so if i want to have a variable for every workspace, one way is to define it at the system level. however, this is not a good way as this clutter the variables for every application. batch file a solution to this to create my custom batch file where i define my variables, and at the end of this batch file i launch eclipse. that way the extra variables are only for this eclipse session. cwide-env file another very nice way codewarrior eclipse offers is using the cwide-env file located in the eclipse sub-folder of the installation: cwide-env file i can define variables here, or extend existing ones: -add : add string to the variable at the end -prepend : add string to the variable at the beginning that way i can easily manipulate existing system variables or create new ones which then are used by eclipse. summary variables in eclipse help me to define paths to source files and folders outside of a project or workspace. with variables i avoid using absolute paths which would make porting projects from one machine to another difficult. i can define variables for projects, for the workspace or use system variables. with codewarrior i have a cwide-env file which is used to extend the system variables. happy variabling

Since Entity Framework now has a very nice support of performing all its actions asynchronously, the methods in the repositories in our projects will turn into asynchronous methods soon and so will be the code depending on it. Tom Fitzmacken did a nice job by putting together a tutorial on unit testing Web API 2 Controllers on official ASP.NET site. The tutorial discusses on testing synchronous action methods. The same techniques can be applied to test asynchronous action actions as well. In this post, we will see how easy it is to test asynchronous Web API action methods using MS Test. I created a simple repository interface with just one method in it. The implementation class uses Entity Framework to get a list of contacts from the database. public interface IRepository { Task> GetAllContactsAsync(); } public class Repository : IRepository { ContactsContext context = new ContactsContext(); public async Task> GetAllContactsAsync() { return await context.Contacts.ToArrayAsync(); } } Following is the ASP.NET Web API controller that uses the above repository: public class ContactsController : ApiController { IRepository repository; public ContactsController() : this(new Repository()) { } public ContactsController(IRepository _repository) { repository = _repository; } [Route("api/contacts/plain")] public async Task> GetContactsListAsync() { IEnumerable contacts; try { contacts = await repository.GetAllContactsAsync(); } catch (Exception) { throw; } return contacts; } [Route("api/contacts/httpresult")] public async Task GetContactsHttpActionResultAsync() { IEnumerable contacts; try { contacts = await repository.GetAllContactsAsync(); } catch (Exception ex) { return InternalServerError(ex); } return Ok(contacts); } } As we see, the controller has two action methods performing the same task, but the way they return the results is different. Since both of the action methods respond to HTTP GET method, I used attribute routing to distinguish them. I used poor man’s dependency injection to instantiate the repository; it can be easily replaced using an IoC container. Before writing unit tests for the above action methods, we need to create a mock repository. public class MockRepository:IRepository { List contacts; public bool FailGet { get; set; } public MockRepository() { contacts = new List() { new Contact(){Id=1, Title="Title1", PhoneNumber="1992637281", CustomerId=1}, new Contact(){Id=2, Title="Title2", PhoneNumber="9172735171", SupplierId=2}, new Contact(){Id=3, Title="Title3", PhoneNumber="8361910353", CustomerId=2}, new Contact(){Id=4, Title="Title4", PhoneNumber="7801274518", SupplierId=3} }; } public async Task> GetAllContactsAsync() { if (FailGet) { throw new InvalidOperationException(); } await Task.Delay(1000); return contacts; } } The property FailGet in the above class is used to force the mock to throw an exception. This is done just to cover more test cases. In the test class, we need a TestInitialize method to arrange the objects needed for unit testing. [TestClass] public class ContactsControllerTests { MockRepository repository; ContactsController contactsApi; [TestInitialize] public void InitializeForTests() { repository = new MockRepository(); contactsApi = new ContactsController(repository); } } Let us test the GetContactsListAsync method first. Testing this method seems to be straight forward, as it returns either a plain generic list or throws an exception. But the test method can’t just return void like other tests, as the method is asynchronous. To test an asynchronous method, the test method should also be made asynchronous and return a Task. Following test checks if the controller action returns a collection of length 4: [TestMethod] public async Task GetContacts_Should_Return_List_Of_Contacts() { var contacts = await contactsApi.GetContactsListAsync(); Assert.AreEqual(contacts.Count(), 4); } If the repository encounters an exception, the exception is re-thrown from the GetContactsListAsync method as well. This case can be checked using the ExpectedException attribute. [TestMethod] [ExpectedException(typeof(InvalidOperationException))] public async Task GetContacts_Should_Throw_Exception() { repository.FailGet = true; var contacts = await contactsApi.GetContactsListAsync(); } Now let’s test the GetContactsHttpActionResultAsync method. Though this method does the same thing as the previous method, it doesn’t return the plain .NET objects. To test this method, we need to extract the result from the IHttpActionResult object obtained from the action method. Following test checks if the action result contains a collection when the repository is able to fetch results. Return type of Ok() method used above is OkNegotiatedContentResult. IHttpActionresult has to be converted to this type to check for the result obtained: [TestMethod] public async Task GetContactsHttpActionResult_Should_Return_HttpResult_With_Contacts() { var contactsResult = await contactsApi.GetContactsHttpActionResultAsync() as OkNegotiatedContentResult>; Assert.AreEqual(contactsResult.Content.Count(), 4); } Similarly, in case of error, we are calling InternalServerError() method to return the exception for us. We need to convert the result to ExceptionResult type to be able to check the type of exception thrown. It is shown below: [TestMethod] public async Task GetContactsHttpActionResult_Should_Return_HttpResult_With_Exception() { repository.FailGet = true; var contactsResult = await contactsApi.GetContactsHttpActionResultAsync() as ExceptionResult; Assert.IsInstanceOfType(contactsResult.Exception,typeof(InvalidOperationException)); } Happy coding!

Editor's note: Be sure to check out part 2 as well. This is the fourth of an introductory series of blogs on Apache HBase. In the third part, we saw a high level view of HBase architecture . In this part, we'll use the HBase Java API to create tables, insert new data, and retrieve data by row key. We'll also see how to setup a basic table scan which restricts the columns retrieved and also uses a filter to page the results. Having just learned about HBase high-level architecture, now let's look at the Java client API since it is the way your applications interact with HBase. As mentioned earlier you can also interact with HBase via several flavors of RPC technologies like Apache Thrift plus a REST gateway, but we're going to concentrate on the native Java API. The client APIs provide both DDL (data definition language) and DML (data manipulation language) semantics very much like what you find in SQL for relational databases. Suppose we are going to store information about people in HBase, and we want to start by creating a new table. The following listing shows how to create a new table using the HBaseAdmin class. Configuration conf = HBaseConfiguration.create(); HBaseAdmin admin = new HBaseAdmin(conf); HTableDescriptor tableDescriptor = new HTableDescriptor(TableName.valueOf("people")); tableDescriptor.addFamily(new HColumnDescriptor("personal")); tableDescriptor.addFamily(new HColumnDescriptor("contactinfo")); tableDescriptor.addFamily(new HColumnDescriptor("creditcard")); admin.createTable(tableDescriptor); The people table defined in preceding listing contains three column families: personal, contactinfo, and creditcard. To create a table you create an HTableDescriptor and add one or more column families by adding HColumnDescriptor objects. You then call createTable to create the table. Now we have a table, so let's add some data. The next listing shows how to use the Put class to insert data on John Doe, specifically his name and email address (omitting proper error handling for brevity). Configuration conf = HBaseConfiguration.create(); HTable table = new HTable(conf, "people"); Put put = new Put(Bytes.toBytes("doe-john-m-12345")); put.add(Bytes.toBytes("personal"), Bytes.toBytes("givenName"), Bytes.toBytes("John")); put.add(Bytes.toBytes("personal"), Bytes.toBytes("mi"), Bytes.toBytes("M")); put.add(Bytes.toBytes("personal"), Bytes.toBytes("surame"), Bytes.toBytes("Doe")); put.add(Bytes.toBytes("contactinfo"), Bytes.toBytes("email"), Bytes.toBytes("john.m.doe@gmail.com")); table.put(put); table.flushCommits(); table.close(); In the above listing we instantiate a Put providing the unique row key to the constructor. We then add values, which must include the column family, column qualifier, and the value all as byte arrays. As you probably noticed, the HBase API's utility Bytes class is used a lot; it provides methods to convert to and from byte[] for primitive types and strings. (Adding a static import for the toBytes() method would cut out a lot of boilerplate code.) We then put the data into the table, flush the commits to ensure locally buffered changes take effect, and finally close the table. Updating data is also done via the Put class in exactly the same manner as just shown in the prior listing. Unlike relational databases in which updates must update entire rows even if only one column changed, if you only need to update a single column then that's all you specify in the Put and HBase will only update that column. There is also a checkAndPut operation which is essentially a form of optimistic concurrency control - the operation will only put the new data if the current values are what the client says they should be. Retrieving the row we just created is accomplished using the Get class, as shown in the next listing. (From this point forward, listings will omit the boilerplate code to create a configuration, instantiate the HTable, and the flush and close calls.) Get get = new Get(Bytes.toBytes("doe-john-m-12345")); get.addFamily(Bytes.toBytes("personal")); get.setMaxVersions(3); Result result = table.get(get); The code in the previous listing instantiates a Get instance supplying the row key we want to find. Next we use addFamily to instruct HBase that we only need data from the personal column family, which also cuts down the amount of work HBase must do when reading information from disk. We also specify that we'd like up to three versions of each column in our result, perhaps so we can list historical values of each column. Finally, calling get returns a Result instance which can then be used to inspect all the column values returned. In many cases you need to find more than one row. HBase lets you do this by scanning rows, as shown in the second part which showed using a scan in the HBase shell session. The corresponding class is the Scan class. You can specify various options, such as the start and ending row key to scan, which columns and column families to include and the maximum versions to retrieve. You can also add filters, which allow you to implement custom filtering logic to further restrict which rows and columns are returned. A common use case for filters is pagination. For example, we might want to scan through all people whose last name is Smith one page (e.g. 25 people) at a time. The next listing shows how to perform a basic scan. Scan scan = new Scan(Bytes.toBytes("smith-")); scan.addColumn(Bytes.toBytes("personal"), Bytes.toBytes("givenName")); scan.addColumn(Bytes.toBytes("contactinfo"), Bytes.toBytes("email")); scan.setFilter(new PageFilter(25)); ResultScanner scanner = table.getScanner(scan); for (Result result : scanner) { // ... } In the above listing we create a new Scan that starts from the row key smith- and we then use addColumn to restrict the columns returned (thus reducing the amount of disk transfer HBase must perform) to personal:givenName and contactinfo:email. A PageFilter is set on the scan to limit the number of rows scanned to 25. (An alternative to using the page filter would be to specify a stop row key when constructing the Scan.) We then get a ResultScanner for the Scan just created, and loop through the results performing whatever actions are necessary. Since the only method in HBase to retrieve multiple rows of data is scanning by sorted row keys, how you design the row key values is very important. We'll come back to this topic later. You can also delete data in HBase using the Delete class, analogous to the Put class to delete all columns in a row (thus deleting the row itself), delete column families, delete columns, or some combination of those. Connection Handling In the above examples not much attention was paid to connection handling and RPCs (remote procedure calls). HBase provides the HConnection class which provides functionality similar to connection pool classes to share connections, for example you use the getTable() method to get a reference to an HTable instance. There is also an HConnectionManager class which is how you get instances of HConnection. Similar to avoiding network round trips in web applications, effectively managing the number of RPCs and amount of data returned when using HBase is important, and something to consider when writing HBase applications. Conclusion to Part 4 In this part we used the HBase Java API to create a people table, insert a new person, and find the newly inserted person information. We also used the Scan class to scan the people table for people with last name "Smith" and showed how to restrict the data retrieved and finally how to use a filter to limit the number of results. In the next part, we'll learn how to deal with the absence of SQL and relations when modeling schemas in HBase. References HBase web site, http://hbase.apache.org/ HBase wiki, http://wiki.apache.org/hadoop/Hbase HBase Reference Guide http://hbase.apache.org/book/book.html HBase: The Definitive Guide, http://bit.ly/hbase-definitive-guide Google Bigtable Paper, http://labs.google.com/papers/bigtable.html Hadoop web site, http://hadoop.apache.org/ Hadoop: The Definitive Guide, http://bit.ly/hadoop-definitive-guide Fallacies of Distributed Computing, http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing HBase lightning talk slides, http://www.slideshare.net/scottleber/hbase-lightningtalk Sample code, https://github.com/sleberknight/basic-hbase-examples

Sometimes, you need your jobs to be persisted to a database. Existing solutions such as Gearman only used relational or file-based persistence, so they were a no-go for us and we went with MongoDB. Fast-forward a few months, and we have some problems with the database load. However, it's not that workers are pestering it too much: the problem was related to locks. MongoDB locking model As of 2.4, MongoDB holds write locks on an entire database for each write operation. Since atomicity is guaranteed only on a single document, this isn't usually a problem because even if you are inserting thousands of documents you are doing so in thousands of different operations that can be interleaved with queries and other inserts with a fair policy. This sometimes results in count() queries being inconsistent as documents are moved and indexes are asynchronously updated. However, write corruption is inexistent as documents are a very cohesive entity. However, atomic operations over a single document still lock the whole database, as in the case of findAndModify(), which looks for a document matching a certain query and updates it with a $set operation before returning it; all in a single shot and with the guarantee no other process will be able to perform the same operation of reading and writing at the same time. You can see this operation is ideal for implementing workers based on a pull model, each asking the database for a new job to do and locking it with '$set: {locked: true}'. However, after the number of workers increases a little bit, locks become a problem. Lock duration We cleaned up the working space collection of our MongoDB database by keeping in it only the unfinished jobs, and moving all the rest (completed or failed) to a different collection for archival. As the load increases due to new contracts, we saw the locking time increase as well: the application and the workers were insisting on the same database. The first of the problems was that after reducing the specs of our primary server, we started seeing timeouts of unrelated code even if the CPU and IO usage were low. The locks taken by workers to pick jobs were starting to take seconds or tens of seconds. Moreover, the MongoDB server started filling the logs with: Fri Dec 6 00:01:07 [conn280998] warning: ClientCursor::yield can't unlock b/c of recursive lock... I'm a user, not MongoDB guru but that seems not very good, especially given hundreds of these messages were written every day (although the queues continued to work correctly.) We did not find any explanation for these messages in the documentation, but I suppose they mean some operations are taking so long that they have to yield to make room for others, but in the case of atomic operations they can't to preserve consistency. An easy solution Since MongoDB does not have collection-wide locks yet, we decided to move the job pool and the completed job collections to a different database. In this way, we had a main database with the usual collections and one containing just these two, named with a '_queue' suffix. Note that we're still writing to the same database server: there is still the same number of connections being created by each process. This solution preallocates more space given two databases are involved, but as you know space is cheap nowadays. Both insertion of jobs and worker reads must take place on the same database. Here is where we discovered cohesion pays: if you have this information in a single place it is very easy to change configuration. If you have a singleton database, because "we should only have one database in this application, it will never change" this feature would cost you a lot. Fortunately, in our case it was about 10 lines of code, including the refactoring on the Factory Methods that created MongoDB database objects. Long term This solution is not for the long term, as we know the numbers of machines and their workers pool will increase in the future; a sufficiently high number of workers will saturate the connections available on the MongoDB server and lock the common collection until a pick of a job takes dozens of seconds. The design towards which we are moving includes one "foreman" to each machine, and many workers under his control; only the foreman polls the database and may lock the common collection. Distributing the job pool is not what we want for ease of retrieval of a job in case something goes bad (ever done a query on multiple databases?). Also, we don't want a push solution as it will involve the registration of workers or foremen to a central point of failure that assignes them their jobs. Since most of our servers are shutdown and rebooted according to the user load, we prefer a dynamic solution where a server can start picking jobs whenever it wants and stop without notifying remote machines.

when you build applications based on mule ee (enterprise edition) and you are using maven to build your projects, you will notice you have dependencies to libraries that are not available in the public maven repos. to add these libraries to your local maven repo the mule distribution comes with a script ‘populate_m2_repo’ which is described here how to use it. now that is okay if you are the only developer and you are running your continuous integration on your local machine. in my case we are using artifactory as our company maven repository and also our build server is using it as the maven repo. so what i wanted was not to populate my local repository but the artifactory instance with all mule libraries. to do so i did two things: first make sure that maven is authorised to add libraries to artifactory. you can do this by adding the following to your settings.xml: artifactory admin password second step is to modify the original ‘populate_m2_repo.groovy’ script. replace the following line: mvn(["install:install-file", "-dgroupid=${project.groupid}", "-dartifactid=${project.artifactid}", "-dversion=${version}", "-dpackaging=pom", "-dfile=${localpom.canonicalpath}"]) with mvn(["deploy:deploy-file", "-dgroupid=${project.groupid}", "-dartifactid=${project.artifactid}", "-dversion=${version}", "-dpackaging=pom", "-dfile=${localpom.canonicalpath}", "-drepositoryid=arti", "-durl=http://localhost:8080/artifactory/libs-release-local" ]) and do the same for the line: def args = ["install:install-file", "-dgroupid=${pomprops.groupid}", "-dartifactid=${pomprops.artifactid}", "-dversion=${pomprops.version}", "-dpackaging=jar", "-dfile=${f.canonicalpath}", "-dpomfile=${localpom.canonicalpath}"] by replacing it with: def args = ["deploy:deploy-file", "-dgroupid=${pomprops.groupid}", "-dartifactid=${pomprops.artifactid}", "-dversion=${pomprops.version}", "-dpackaging=jar", "-dfile=${f.canonicalpath}", "-dpomfile=${localpom.canonicalpath}", "-drepositoryid=arti", "-durl=http://localhost:8080/artifactory/libs-release-local" ] now you can run the script with: ./populate_m2_repo bla as you can see it doesn’t really matter what you supply as m2_repo_home here because the libraries are uploaded to artifactory anyway. if you want you can replace the hardcoded url for artifactory in the script with the supplied parameter but in my case this solution was sufficient

eclipse is very workspace centric: it only knows and deals with files in the workspace. so it is easy to compare and merge files present in the workspace: i select both files/folders and compare them with each other: compare with each other but what if the files and folders are not in the workspace? hidden option to compare external files as outlined in this post , it needs a special plugin to search for files outside the eclipse workspace. and doing a file or folder compare outside of the workspace requires a trick as shown in this post . thanks to a tip from john there is another (hidden) way in eclipse to compare external files keyboard shortcut the trick is described in this article and requires a keyboard shortcut assigned. select the menu window > preferences > general > keys and assign a shortcut key for ‘ compare with other resource ‘: compare with other resource key binding (i’m using ctrl+shift+home above). comparing to compare, i have first to select a file, folder or project, then i press my shortcut. then the following dialog shows up (with the selection as default): select resource to compare if the dialog does not show up, then i probably have not selected a file or folder in the eclipse project view. now i can select the external files or folders to compare with: selected external files with this, i can now compare and merge my files and folders with the eclipse compare view: compare view in eclipse i can select two files/folders and then press the shortcut, and it will populate the search dialog values. drag & drop that compare dialog has a nice feature: i can drag&drop files and folders too: c:\programdata\processor expert\cwmcu_pe5_00\examples\frdm-kl25z\frdm-kl25z_rnet\sources summary with ‘ compare with other resource ‘ i have a way to compare files/folders, and i’m not limited to the workspace files. the only disadvantage is that i need to assign a shortcut for it first. beside of that: yet another hidden treasure in eclipse . happy comparing

In a REST API which correctly uses hypermedia, the URLs to contact with HTTP requests by the client are not fixed. They are embedded in responses from previous requests, making their change possible. This style uses standard media types to transmit the links, such as XML in the form of Atom feeds or JSON HAL. Even if we skip the standard media types, we can embed links into HTTP headers of the responses to provide flexibility and discoverability: Location headers in 201 Created responses to POST requests. Location headers in 30X redirects after we notice a POST is not acceptable or If-Match conditions are not satisfied. Link headers to navigate a collection of resources back and forth, and to store bookmarks to the currently most recent page. Today I want to expand this mindset to cover server to server interaction, between two applications that chat through a REST API; actually, through two REST APIs exposed to each other. Context: asynchronous calls Processes are a precious, limited resource; in many stacks such as LAMP a process busy respond to an HTTP request can't do anything else, even if it's blocked waiting for IO from a downstream dependency like a database. Moreover, there is a limit to how many processes can be allocated. For example, the number of Apache processes is limited by configuration; each new process results in a different connection to the database being created or taken by the pool; and it takes a toll in context switches that the processor cores have to perform to manage a number of processes much bigger than them (like a programmer having to work on 3 or 4 user stories at the same time). Still, the number of web processes defined how many concurrent clients you can support: once they are all allocated clients will see their connection hung up while they wait for an accept() system call by Apache. After a timeout, their connection are refused or terminated on the client side. One of the easiest ways to expand this capacity is to make processes terminate as fast as possible (another is front end caching so that they do not even get called). For example, with 64 processes available, taking an average amount of time of 1 second to produce a response, you can deal with exactly 64 concurrent clients before hitting degradation (the 65th client will have to wait until one process frees and this queue time will be added to the demand time taken by the process then to build its response.) If you sacrifice an immediate response, you can make these process terminate as fast as possible by transforming the interface in a asynchronous one. You go from > POST /jobs HTTP/1.1 > param=value < HTTP/1.1 200 OK to > POST /jobs HTTP/1.1 > param=value < HTTP/1.1 202 Accepted The process just puts the job in a queue, possibly performing some low-cost validation immediately. The client application can get a result by being called back on another URL after the job has been processed. Asynchronous calls like this one let you handle larger spikes in load that fill your queue even if it's not capable of guaranteeing that throughput (paying the price of a higher total time for the job to be completed). Going asynchronous is also mandatory when you're calling external systems in the job execution. You don't want to depend on the availability and response times of other systems before returning a response to the user, even a partial one. Calling back The server side takes the initiative after completing a job and transforms itself in a client making an HTTP request to the original client (which acts now like a server, of course). The protocol has to be shared (such as PUT or POST requests with a certain format). However, you don't need this callback URL to be fixed, as it can be passed through the original HTTP request: > POST /jobs HTTP/1.1 > param=value&resource_url=https://www.onebip.com/api/billing/123 < 202 Accepted After completing the job, the callback request will be: PUT /api/billing/123 ...some body... Host: www.onebip.com (HTTP methods are used for explicative purposes here, please do not judge their semantics.) This is similar to what JavaScript and other continuation-passing style languages do. For example, jQuery performing Ajax requests: success = function(response) { ... }; failure = function(response) { ... }; $.ajax(url, ..., success, failure); Erlang processes instead communicate with unidirectional messages handled asynchronously. So to get a response, each message must include the sender in its content (in this case not an URL but a PID): neighbor ! {self(), ...}. This pattern can also be extended to communication between more than two processes, if the main process passes to neighbor someone else's PID instead of its own. Conclusions The dynamicity of not harcoding return routes for messages let us also play with the system for testing purposes. For example, if we are developing a system A that talks to another system B, it's easy to test A from a staging area against a production system B without touching A's production: just configure B urls and pass your own (publicly reachable) URLs. It also becomes easy to support multiple production systems A1, A2, ... An: you basically transform a collaboration between A and B to a B-as-a-service situation where it's easy to drop in new A clients even when there is a return path from B to A.

Applications are at the center of the IT universe. As IT shifts its primary goal from connectivity to experience, it will require tighter collaboration between the various infrastructure elements that support application workloads. There are two philosophical approaches to how this orchestration might take place: through a tightly-integrated system, or through a more loose coupling of heterogeneous components. But how should architects make the choice between these approaches? The principles of architecture tend to be most vehemently argued by the vendors competing to sell the underlying solutions. IT vendors generally (and networking in particular) tend to turn these principle discussions into tit-for-tat FUD wars, arguing in absolution that one approach or another is the right way to go. But the ones who put their careers on the line when they select an architectural approach should understand more fully what drives specific architectural selections. The difference between tightly-integrated systems and more loosely federated components is really performance. Whenever two components come together, that boundary is defined by some interface. If you need to extract performance out of the coupled system, you have to make changes on one or both sides of said interface. As a vendor, if you can twiddle the bits on only one side, you can improve the overall system performance up to but not beyond whatever the other side can do. So when performance is the primary objective, you will tend to see solutions where both sides of that interface are owned (or at least controlled) by the same party. The ability to make changes on both sides of the interface is the only way to maximize performance. When the primary objective is not performance, you will see a generalized interface that sits between a decoupled pairing of solution components. Enter SDN. Or network virtualization. Or NFV. Or DevOps. When we talk about performance as an industry, we usually mean capacity and speed. But performance is more than bandwidth and latency. The whole reason any of the SDN technologies is emerging is to satisfy operational issues. Getting applications provisioned, monitored, troubleshot, billed, upgraded, and so on has taken over the top spot on the pain list for many companies. The question we ought to be asking is what are the operational performance requirements. The answer isn't black or white. What does performance even mean in an operational setting? It seems at least plausible that operational performance translates to things like the rate of change (think provisioning changes per second or call setup and teardown rates, for example) or the rate of polling (queries per second, as with monitoring or billing). For some environments, it might be that the scale of configuration management or data querying is quite high. Any company that is doing fine-grained monitoring or rapid state-based network changes, for example, might have very high operational performance requirements. Meanwhile, most normal networks will likely have a much lower performance bar. For the former, the objective has to be to eke out every bit of operational performance from the system. This will demand a more tightly-integrated solution. Both sides of the resource boundary (network and storage, as an example) might need to be within the same system, and the interface between them should appropriately be very specific to the implementation. For the latter, a more generalized interface between infrastructure elements should be more than sufficient. The primary goal is not to maximize performance but rather enable collaboration between components. In these architectures, the generalized interface is the most important thing as it will optimize choice and flexibility between the individual system elements. Both are absolutely valid use cases; there is no judgment in which is the more noble cause. But architects ought to be clear about what it is they are optimizing for. Selecting a generalized interface merely because it is open could be disastrous if it turns out that the performance requirements exceed what that interface provides. Conversely, selecting a tightly-integrated system might be more costly or limiting than is necessary if the real problem is orchestration rather than performance. So where do architects start? Everything starts with requirements. Is the objective to achieve a specific rate of change? Or is the objective merely to make tasks like provisioning and troubleshooting more coordinated across infrastructure silos? Are you planning to do anything exotic in terms of polling data on the system elements? Or are you expecting data to be accessed at a more casual rate? The real point here is that architects should start to express their orchestration requirements in terms of both capability and performance. We do this instinctively when we think about how we move bits back and forth, or how we access storage, or how we allot cycles on a server. But when it comes to management, because our collective capabilities have been so lacking, we have ignored performance. As SDN and other technologies continue to advance, operational performance will take on a more important role. And without knowing what the requirements are, designers will really be flying blind, making tradeoffs that might not even be necessary. [Today's fun fact: In ancient Rome, it was considered a sign of leadership to be born with a crooked nose. If Mike Tyson were born earlier, we'd call him Emperor.]

A quick overview on how to see your heaps in Eclipse.