once upon a time there was a munich pizza baker who developed a technique to beam pizza out of bright sunshine. he can produce more than a thousand pizzas per second and needs a channel to sell this amount of pizza and decides to build an online shop. mario’s initial idea is to sell pizzas, but now he is thinking about introduction of new product lines like beverages, salads and pasta. before we take a look to the validation of mario´s idea, lets take a short look at the existing online shop. mario’s online shop is based on mongodb , apache wicket and spring . mongodb is a document-oriented nosql-database . mongodb stores records not in tables as a relational database but in bson documents, which is a binary version of json (java script object notation) and very similar to the object structure in mario’s application. the usage of mongodb makes his development easier and deployment faster. the figure shows a json document which is very similar to a java object: a json document property with the according value corresponds to the java object property with the appropriate value. you can add or remove properties in your java object and this will automatically change your database schema. so there is no need to put your java object model into a relational schema via hibernate. mario also decided to build his online shop only with open-source technologies like apache wicket and spring. wicket is a very common lightweight component-based web application framework and it is closely patterned after stateful gui frameworks such as javafx . the spring framework is an open source application framework and inversion of control container for the java platform and does not impose any specific programming model. spring has become popular in the java community as an alternative to, replacement for, or even addition to the enterprise javabean (ejb) model. because of this architecture mario is able to deploy its application in a lightweight application server like tomcat or jetty . this figure shows the system landscape of mario. mario has two major system on the lefthand site there is his online shop and on the righthand site there is ‘pas’ a famous billing system. in the middle is hadoop that connects both systems together. in the business world an application normally does not stand alone. in most cases an application must communicate with others. the lean architecture of marios online shop enables him to connect the billing system ‘pas’ to his online shop. spring for apache hadoop provides this integration between the two systems online shop and ‘pas’. hadoop supports data-intensive distributed applications and implements a computational paradigm named mapreduce, where the computation is divided into many small fragments, each of them may be executed or re-executed on any node in the cluster of commodity hardware. mario uses hadoop as an etl layer that enables him to transfer gigabytes of order information into the billing system. in this case hadoop makes it possible for a financial controller to verify if all orders were billed correctly. in addition to the online shop feature mario has a real-time sales dashboard that enables him to track his sales in real time. the dashboard displays daily and monthly sales statistics for each pizza and contains a map with the geographical overview of customer activity and competitor locations. here is a walkthrough of the shop : now lets talk about mario’s incredible new idea : mario wants to sell even more pizza! and other products as well. mario decides to use lean startup methods in order to test the possible introduction of new product lines and plans an experiment to validate his new idea using a scientific approach and pure facts instead of hunches. mario´s core assumption is that customers wants to buy other products than pizza – drinks, salads and pasta. furthermore he is worried about pricing. mario contacts all customers to complete a survey and provides an incentive for the participation, a free pizza to every customer who responds to the survey. the result of the survey validated mario’s assumption – customers want to buy beverages, salads and pasta. but he also found out that his customers are willing to pay higher prices for high-quality products and that they simply love his easy shopping flow. currently a pizza order can be completed with three clicks only, so there is new riskiest assumption to validate: will a more complex shopping flow affect his sales? the figures shows a validation board. a validation board is a deceptively simple tool for testing out product ideas. furthermore a validation board tracks pivots which follows from customer feedback. mario decides to introduce beverages, salads and pasta product lines and thinks about a possibility, how he can handle the extension of the product line without destroying the easy shopping flow. that’s why mario thinks a recommendation engine is the right way for him. panels for recommendations can be integrated in the online shop without changing the shopping flow. mario hired a statistician to help him implement a recommender system for his online shop for better cross-selling. he also defined new measurement points to validate his new idea . therefore he tracks the conversion rate of orders as well as cross-selling rates and every event in the online shop is already tracked in realtime. so mario can very easily perform further experiments in order to verify more assumptions. follow the blog to see how the story continues or come to mongodb usergroup meetup in munich , february 20, 2013 or mongodb days in berlin , february 26, 2013 to get a live presentation. our talk sheds light on how to build an online recommendation engine based on mongodb and apache mahout. we’ll show which recommenders must be built to reach mario’s goal and how these can be integrated in mario’s shop infrastructure.

ActiveMq is a powerful open source messaging broker, and is very easy and straightforward to use with Spring as the below classes and XML will prove. The example below is the bar minimum needed to get up and running with transactions and message converters. On the sending side, the ActiveMq connection factory needs to be created with the url of the broker. This in turn is used to create the Spring JMS connection factory and as no session cache property is supplied the default cache is one. The template is then used in turn to create the Message Sender class: An example sending class is below. It uses the convertandSend method of the JmsTemplate class. As there is no destination arg, the message will be sent to the default destination which was set up in the XML file: import java.util.Map; import org.springframework.jms.core.JmsTemplate; public class MessageSender { private final JmsTemplate jmsTemplate; public MessageSender(final JmsTemplate jmsTemplate) { this.jmsTemplate = jmsTemplate; } public void send(final Map map) { jmsTemplate.convertAndSend(map); } } On the receiving side, there needs to be a listener container. The simplest example of this is the SimpleMessageListenerContainer. This requires a connection factory, a destination (or destination name) and a message listener. An example of the Spring configuration for the receiving messages is below: The listening/receiving class needs to extend javax.jms.MessageListener and implement the onMessage method: import javax.jms.MapMessage; import javax.jms.Message; import javax.jms.MessageListener; public class MessageReceiver implements MessageListener { public void onMessage(final Message message) { if (message instanceof MapMessage) { final MapMessage mapMessage = (MapMessage) message; // do something } } } To then send a message would be as simple as getting the sending bean from the bean factory as shown in the below code: MessageSender sender = (MessageSender) factory.getBean("messageSender"); Map map = new HashMap(); map.put("Name", "MYNAME"); sender.send(map); Will try to expand and build up JMS and Spring articles with examples of using transactions and other brokers like MQSeries.

usually, one of the first things i see if i launch eclipse is this dialog: select a workspace dialog actually, that ‘workspace’ thing is one of the most important things in eclipse to understand. to mess around it can cause a lot of pain. so i have collected some ‘lessons learned’ around workspaces. the workspace .metadata folder the workspace is where eclipse stores that .metadata folder: workspace metadata folder in this folder, eclipse stores all the workspace settings or preferences i configure e.g. using the menu window > preferences . sometimes that .metadata folder is named ‘framework’ too. e.g. if i’m are asked by eclipse to store some settings in the ‘frame work’ then this means it will be stored in the .metadata. eclipse uses this folder as well to store internal files and data structures. and many plugins store their settings in here to. consider the content of this folder as a ‘black box’: so do not change it, do not touch it unless you *really* know what you are doing! do not copy or move that folder. if you want to copy/share your workspace settings, then do *not* copy the .metadata folder, as typically you cannot use this folder on another machine or for another user. if you want to transfer/copy your settings, then see this post . workspace and eclipse versions as eclipse stores information in the .metadata workspace structure, the data/format might be different from version of eclipse to another (e.g. from one version of codewarrior to another). while using the same workspace with different versions of eclipse might work, it is *not* recommended. the eclipse community tries hard to keep things compatible, but using a different workspace for different eclipse versions is what i recommend. i started to name my workspace(s) like ‘wsp_lecture_10.2′ or ‘wsp_lecture_10.3′ to show that i’m using it for a specific version of codewarrior. workspace and projects this leads to the question: “do i have to duplicate then my projects if using with different versions of eclipse in parallel?” the answer is ‘no’. because the workspace folder does *not* have to have the projects in it (as folders). they can, but it is not needed. for example i have different workspaces (“wsp_10.3″, “wsp_10.2″), but my projects are in the “projects” folder somewhere else on my disk. what i do is to import the projects into each workspace, keep the projects in their original folder location. the menu file > import > general > existing projects into workspace can be used, with ‘copy projects into workspace’ * unchecked *: importing projects into workspace an easy trick is to drag&drop the project folders into eclipse: that’s much faster and simpler in my view than using above dialog. tip: showing the current workspace in the title bar using multiple workspaces can be confusing at some time. see this post how you can show the workspace in the application title: workspace shown in title bar processor expert processor expert has a special setting in the workspace pointing to its ‘data base’. that data base is inside the installation folder, in the mcu\processorexpert folder. if a launch eclipse and use a workspace from a different installation, i get a warning dialog: processor expert workspace warning: current worksapce is configured to use data from another installation of processor expert. that path setting of processor expert (pointing to the installation folder) is in my view the biggest argument to *not* share a workspace between different versions of eclipse. pressing the ‘open preferences’ opens the settings, and with ‘restore defaults’ it will (after a restart of eclipse) use the new installation path: processor expert directory that warning might not come up if using multiple installations of codewarrior. it seems that as long there is a valid path to a processor expert data base, the warning might not show up, and it will use that data base. that can lead to weird behaviour, so better check that the path in the workspace setting is pointing to the right folder. workspace dialog on startup remember that dialog at the beginning of this post? if i want to change if (and what) is shown at eclipse startup, then this is configured with the menu window > preferences > general > startup and shutdown : startup and workspaces in this dialog i can remove items from the list (e.g. if a workspace folder does not exist any more). tabula rasa as eclipse stores a lot of information in the workspace .metadata, that folder can grow to a substantial size (several hundreds of mbytes, depending on usage and configuration). one reason is because eclipse stores local history and undo information into that .metadata folder. just have a look at .metadata\.plugins\org.eclipse.core.resources\.history so from time to time (especially if i think that eclipse is slowing down), i export my workspace settings ( file > export > general > preferences , see copy my workspace settings ) and re-import it again into the new workspace. summary eclipse stores all its workspace settings and files in the .metadata folder. never touch/copy/change/move the .metadata folder. use different workspaces for each eclipse version. store the projects outside of the workspace if you want to share them across different eclipse versions. happy workspacing

the great thing with eclipse is that you can configure a lot. in general, i’m happy with most of the defaults in eclipse and codewarrior. here are my top 10 things i change in eclipse to make it even better: add -showlocation to the eclipse startup command line: show workspace location in the title bar disable the heuristik settings for the indexer : fixing the eclipse index disable build (if required) for the debugger: speeding up the debug launch in codewarrior using spaces and not tabs: spaces vs. tabs in eclipse highlight the selected line : color makes the difference! configuring more hovers : hovering and debugging enabling static software analysis : free static code analysis with eclipse processor expert expert settings: enabling the expert level in processor expert custom dictionary settings: eclipse spell checker show line numbers : eclipse and line numbers i don’t have to waste time to change the settings for each of my workspaces: after i have changed the settings, i can simply export and import them again into another workspace: change my preferences using the menu window > preferences use the menu file > export > general > preferences and save the settings in a file: file export switch to the new workspace use the menu file > import > general > preferences to import the settings from the file: file import happy customizing

one of the nice things of modern ide’s are: they offer many extras for free. many times it is related to programming and coding. but i love as well the ones which makes things easier and better which is not directly related to the executed code. one thing eclipse offers is an on-the-fly spell-checking, similar to microsoft word: spellchecked sources hovering over the text offers me to correct the flagged error: initialization vs. initialisation but wait: is that example not spelled correctly? and indeed, eclipse offers to customize the spell checking. the option page is in the windows > preferences > general > editors > text editors > spelling page: spelling preferences ‘initialization’ vs. ‘initialisation’: that’s an ‘english us’ vs. english uk’ thing, and is easily changed. and i prefer the us english: changing dictionary with this, everything is ok now: not flagged any more after changing the platform dictionary, it usually takes a few minutes until the sources are checked again. but what if eclipse does not know a word? then it offers to add it to a dictionary: adding to the dictionary if i do not have a user dictionary yet, it will prompt a dialog: missing user dictionary if pressing ‘yes’, it will prompt the settings page from above where i can specify my user dictionary file: user defined dictionary the user dictionary is a normal text file with one word on each line. that makes it easy to edit and to have it in a version control system. i have one common dictionary file for all my workspaces. but of course it is possible to have different dictionaries per workspace, as the settings are per workspace too. summary i feel having reasonable spelled comments in the sources is just something an engineer should care about. and the eclipse spelling engine does not have to be as good as the one in ms word (which is pretty good in my view). but for making sources better something like correctly spelled comments is a plus. but only if the code works like a charm :mrgreen: happy spelling

As I've shown it in my previous post JBoss Drools are a very useful rules engine. The only problem is that creating the rules in the Rule language might be pretty complicated for a non-technical person. That's why one can provide an easy way for creating business rules - decision tables created in a spreadsheet! In the following example I will show you a really complicated business rule example converted to a decision table in a spreadsheet. As a backend we will have Drools, Camel and Spring. To begin with let us take a look at our imaginary business problem. Let us assume that we are running a business that focuses on selling products (either Medical or Electronic). We are shipping our products to several countries (PL, USA, GER, SWE, UK, ESP) and depending on the country there are different law regulations concerning the buyer's age. In some countries you can buy products when you are younger than in others. What is more depending on the country from which the buyer and the product comes from and on the quantity of products, the buyer might get a discount. As you can see there is a substantial number of conditions needed to be fullfield in this scenario (imagine the number of ifs needed to program this :P ). Another problem would be the business side (as usual). Anybody who has been working on a project knows how fast the requirements are changing. If one entered all the rules in the code he would have to redeploy the software each time the requirements changed. That's why it is a good practice to divide the business logic from the code itself. Anyway, let's go back to our example. To begin with let us take a look at the spreadsheets (before that it is worth taking a look at the JBoss website with precise description of how the decision table should look like): The point of entry of our program is the first spreadsheet that checks if the given user should be granted with the possibility of buying a product (it will be better if you download the spreadsheets and play with them from Too Much Coding's repository at Bitbucket: user_table.xls and product_table.xls, or Github user_table.xls and product_table.xls): user_table.xls (tables worksheet) Once the user has been approved he might get a discount: product_table.xls (tables worksheet) product_table.xls (lists worksheet) As you can see in the images the business problem is quite complex. Each row represents a rule, and each column represents a condition. Do you remember the rules syntax from my recent post? So you would understand the hidden part of the spreadsheet that is right above the first visible row: The rows from 2 to 6 represent some fixed configuration values such as rule set, imports ( you've already seen that in my recent post) and functions. Next in row number 7 you can find the name of the RuleTable. Then in row number 8 you have in our scenario either a CONDITION or an ACTION - so in other words either the LHS or rhe RHS respectively. Row number 9 is both representation of types presented in the condition and the binding to a variable. In row number 10 we have the exact LHS condition. Row number 11 shows the label of columns. From row number 12 we have the rules one by one. You can find the spreadsheets in the sources. Now let's take a look at the code. Let's start with taking a look at the schemas defining the Product and the User. Person.xsd User.xsd Due to the fact that we are using maven we may use a plugin that will convert the XSD into Java classes. part of the pom.xml org.apache.maven.plugins maven-compiler-plugin 2.5.1 org.codehaus.mojo jaxb2-maven-plugin 1.5 xjc xjc pl.grzejszczak.marcin.drools.decisiontable.model ${project.basedir}/src/main/resources/xsd Thanks to this plugin we have our generated by JAXB classes in the pl.grzejszczak.marcin.decisiontable.model package. Now off to the drools-context.xml file where we've defined all the necessary beans as far as Drools are concerned: As you can see in comparison to the application context from the recent post there are some differences. First instead of passing the DRL file as the resource inside the knowledge base we are providing the Decision table (DTABLE). I've decided to pass in two seperate files but you can provide one file with several worksheets and access those worksheets (through the decisiontable-conf element). Also there is an additional element called node. We have to choose an implementation of the Node interface (Execution, Grid...) for the Camel route to work properly as you will see in a couple of seconds in the Spring application context file. applicationContext.xml As you can see in order to access the Drools Camel Component we have to provide the node through which we will access the proper knowledge session. We have defined two routes - the first one ends at the Drools component that accesses the users knowledge session and the other the products knowledge session. We have a ProductService interface implementation called ProductServiceImpl that given an input User and Product objects pass them through the Camel's Producer Template to two Camel routes each ending at the Drools components. The concept behind this product service is that we are first processing the User if he can even buy the software and then we are checking what kind of a discount he would receive. From the service's point of view in fact we are just sending the object out and waiting for the response. Finally having reveived the response we are passing the User and the Product to the Financial Service implementation that will bill the user for the products that he has bought or reject his offer if needed. ProductServiceImpl.java package pl.grzejszczak.marcin.drools.decisiontable.service; import org.apache.camel.CamelContext; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import pl.grzejszczak.marcin.drools.decisiontable.model.Product; import pl.grzejszczak.marcin.drools.decisiontable.model.User; import static com.google.common.collect.Lists.newArrayList; /** * Created with IntelliJ IDEA. * User: mgrzejszczak * Date: 14.01.13 */ @Component("productServiceImpl") public class ProductServiceImpl implements ProductService { private static final Logger LOGGER = LoggerFactory.getLogger(ProductServiceImpl.class); @Autowired CamelContext camelContext; @Autowired FinancialService financialService; @Override public void runProductLogic(User user, Product product) { LOGGER.debug("Running product logic - first acceptance Route, then discount Route"); camelContext.createProducerTemplate().sendBody("direct:acceptanceRoute", newArrayList(user, product)); camelContext.createProducerTemplate().sendBody("direct:discountRoute", newArrayList(user, product)); financialService.processOrder(user, product); } } Another crucial thing to remember about is that the Camel Drools Component requires the Command object as the input. As you can see, in the body we are sending a list of objects (and these are not Command objects). I did it on purpose since in my opinion it is better not to bind our code to a concrete solution. What if we find out that there is a better solution than Drools? Will we change all the code that we have created or just change the Camel route to point at our new solution? That's why Camel has the TypeConverters. We have our own here as well. First of all let's take a look at the implementation. ProductTypeConverter.java package pl.grzejszczak.marcin.drools.decisiontable.converter; import org.apache.camel.Converter; import org.drools.command.Command; import org.drools.command.CommandFactory; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import pl.grzejszczak.marcin.drools.decisiontable.model.Product; import java.util.List; /** * Created with IntelliJ IDEA. * User: mgrzejszczak * Date: 30.01.13 * Time: 21:42 */ @Converter public class ProductTypeConverter { private static final Logger LOGGER = LoggerFactory.getLogger(ProductTypeConverter.class); @Converter public static Command toCommandFromList(List inputList) { LOGGER.debug("Executing ProductTypeConverter's toCommandFromList method"); return CommandFactory.newInsertElements(inputList); } @Converter public static Command toCommand(Product product) { LOGGER.debug("Executing ProductTypeConverter's toCommand method"); return CommandFactory.newInsert(product); } } There is a good tutorial on TypeConverters on the Camel website - if you needed some more indepth info about it. Anyway, we are annotating our class and the functions used to convert different types into one another. What is important here is that we are showing Camel how to convert a list and a single product to Commands. Due to type erasure this will work regardless of the provided type that is why even though we are giving a list of Product and User, the toCommandFromList function will get executed. In addition to this in order for the type converter to work we have to provide the fully quallified name of our class (FQN) in the /META-INF/services/org/apache/camel/TypeConverter file. TypeConverter pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter In order to properly test our functionality one should write quite a few tests that would verify the rules. A pretty good way would be to have input files stored in the test resources folders that are passed to the rule engine and then the result would be compared against the verified output (unfortunately it is rather impossible to make the business side develop such a reference set of outputs). Anyway let's take a look at the unit test that verifies only a few of the rules and the logs that are produced from running those rules: ProductServiceImplTest.java package pl.grzejszczak.marcin.drools.decisiontable.service.drools; import org.apache.commons.lang.builder.ReflectionToStringBuilder; import org.apache.commons.lang.builder.ToStringStyle; import org.junit.Test; import org.junit.runner.RunWith; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import pl.grzejszczak.marcin.drools.decisiontable.model.*; import pl.grzejszczak.marcin.drools.decisiontable.service.ProductService; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; /** * Created with IntelliJ IDEA. * User: mgrzejszczak * Date: 03.02.13 * Time: 16:06 */ @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration("classpath:applicationContext.xml") public class ProductServiceImplTest { private static final Logger LOGGER = LoggerFactory.getLogger(ProductServiceImplTest.class); @Autowired ProductService objectUnderTest; @Test public void testRunProductLogicUserPlUnderageElectronicCountryPL() throws Exception { int initialPrice = 1000; int userAge = 6; int quantity = 10; User user = createUser("Smith", CountryType.PL, userAge); Product product = createProduct("Electronic", initialPrice, CountryType.PL, ProductType.ELECTRONIC, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); assertTrue(product.getPrice() == initialPrice); assertEquals(DecisionType.REJECTED, user.getDecision()); } @Test public void testRunProductLogicUserPlHighAgeElectronicCountryPLLowQuantity() throws Exception { int initialPrice = 1000; int userAge = 19; int quantity = 1; User user = createUser("Smith", CountryType.PL, userAge); Product product = createProduct("Electronic", initialPrice, CountryType.PL, ProductType.ELECTRONIC, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); assertTrue(product.getPrice() == initialPrice); assertEquals(DecisionType.ACCEPTED, user.getDecision()); } @Test public void testRunProductLogicUserPlHighAgeElectronicCountryPLHighQuantity() throws Exception { int initialPrice = 1000; int userAge = 19; int quantity = 8; User user = createUser("Smith", CountryType.PL, userAge); Product product = createProduct("Electronic", initialPrice, CountryType.PL, ProductType.ELECTRONIC, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); double expectedDiscount = 0.1; assertTrue(product.getPrice() == initialPrice * (1 - expectedDiscount)); assertEquals(DecisionType.ACCEPTED, user.getDecision()); } @Test public void testRunProductLogicUserUsaLowAgeElectronicCountryPLHighQuantity() throws Exception { int initialPrice = 1000; int userAge = 19; int quantity = 8; User user = createUser("Smith", CountryType.USA, userAge); Product product = createProduct("Electronic", initialPrice, CountryType.PL, ProductType.ELECTRONIC, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); assertTrue(product.getPrice() == initialPrice); assertEquals(DecisionType.REJECTED, user.getDecision()); } @Test public void testRunProductLogicUserUsaHighAgeMedicalCountrySWELowQuantity() throws Exception { int initialPrice = 1000; int userAge = 22; int quantity = 4; User user = createUser("Smith", CountryType.USA, userAge); Product product = createProduct("Some name", initialPrice, CountryType.SWE, ProductType.MEDICAL, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); assertTrue(product.getPrice() == initialPrice); assertEquals(DecisionType.ACCEPTED, user.getDecision()); } @Test public void testRunProductLogicUserUsaHighAgeMedicalCountrySWEHighQuantity() throws Exception { int initialPrice = 1000; int userAge = 22; int quantity = 8; User user = createUser("Smith", CountryType.USA, userAge); Product product = createProduct("Some name", initialPrice, CountryType.SWE, ProductType.MEDICAL, quantity); printInputs(user, product); objectUnderTest.runProductLogic(user, product); printInputs(user, product); double expectedDiscount = 0.25; assertTrue(product.getPrice() == initialPrice * (1 - expectedDiscount)); assertEquals(DecisionType.ACCEPTED, user.getDecision()); } private void printInputs(User user, Product product) { LOGGER.debug(ReflectionToStringBuilder.reflectionToString(user, ToStringStyle.MULTI_LINE_STYLE)); LOGGER.debug(ReflectionToStringBuilder.reflectionToString(product, ToStringStyle.MULTI_LINE_STYLE)); } private User createUser(String name, CountryType countryType, int userAge){ User user = new User(); user.setUserName(name); user.setUserCountry(countryType); user.setUserAge(userAge); return user; } private Product createProduct(String name, double price, CountryType countryOfOrigin, ProductType productType, int quantity){ Product product = new Product(); product.setPrice(price); product.setCountryOfOrigin(countryOfOrigin); product.setName(name); product.setType(productType); product.setQuantity(quantity); return product; } } Of course the log.debugs in the tests are totally redundant but I wanted you to quickly see that the rules are operational :) Sorry for the length of the logs but I wrote a few tests to show different combinations of rules (in fact it's better too have too many logs than the other way round :) ) pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@1d48043[ userName=Smith userAge=6 userCountry=PL decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1e8f2a0[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=10 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Sorry, according to your age (< 18) and country (PL) you can't buy this product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:29 Sorry, user has been rejected... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@1d48043[ userName=Smith userAge=6 userCountry=PL decision=REJECTED decisionDescription=Sorry, according to your age (< 18) and country (PL) you can't buy this product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1e8f2a0[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=10 ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@b28f30[ userName=Smith userAge=19 userCountry=PL decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@d6a0e0[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=1 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations, you have successfully bought the product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Sorry, no discount will be granted. pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:25 User has been approved - processing the order... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@b28f30[ userName=Smith userAge=19 userCountry=PL decision=ACCEPTED decisionDescription=Congratulations, you have successfully bought the product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@d6a0e0[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo=Sorry, no discount will be granted. quantity=1 ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@14510ac[ userName=Smith userAge=19 userCountry=PL decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1499616[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=8 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations, you have successfully bought the product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations - you've been granted a 10% discount! pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:25 User has been approved - processing the order... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@14510ac[ userName=Smith userAge=19 userCountry=PL decision=ACCEPTED decisionDescription=Congratulations, you have successfully bought the product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1499616[ name=Electronic type=ELECTRONIC price=900.0 countryOfOrigin=PL additionalInfo=Congratulations - you've been granted a 10% discount! quantity=8 ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@17667bd[ userName=Smith userAge=19 userCountry=USA decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@ad9f5d[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=8 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Sorry, according to your age (< 18) and country (USA) you can't buy this product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:29 Sorry, user has been rejected... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@17667bd[ userName=Smith userAge=19 userCountry=USA decision=REJECTED decisionDescription=Sorry, according to your age (< 18) and country (USA) you can't buy this product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@ad9f5d[ name=Electronic type=ELECTRONIC price=1000.0 countryOfOrigin=PL additionalInfo= quantity=8 ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@9ff588[ userName=Smith userAge=22 userCountry=USA decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1b0d2d0[ name=Some name type=MEDICAL price=1000.0 countryOfOrigin=SWE additionalInfo= quantity=4 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations, you have successfully bought the product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:25 User has been approved - processing the order... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@9ff588[ userName=Smith userAge=22 userCountry=USA decision=ACCEPTED decisionDescription=Congratulations, you have successfully bought the product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@1b0d2d0[ name=Some name type=MEDICAL price=1000.0 countryOfOrigin=SWE additionalInfo= quantity=4 ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@1b27882[ userName=Smith userAge=22 userCountry=USA decision= decisionDescription= ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@5b84b[ name=Some name type=MEDICAL price=1000.0 countryOfOrigin=SWE additionalInfo= quantity=8 ] pl.grzejszczak.marcin.drools.decisiontable.service.ProductServiceImpl:31 Running product logic - first acceptance Route, then discount Route pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations, you have successfully bought the product pl.grzejszczak.marcin.drools.decisiontable.converter.ProductTypeConverter:25 Executing ProductTypeConverter's toCommandFromList method pl.grzejszczak.marcin.drools.decisiontable.service.ProductService:8 Congratulations, you are granted a discount pl.grzejszczak.marcin.drools.decisiontable.service.FinancialServiceImpl:25 User has been approved - processing the order... pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:150 pl.grzejszczak.marcin.drools.decisiontable.model.User@1b27882[ userName=Smith userAge=22 userCountry=USA decision=ACCEPTED decisionDescription=Congratulations, you have successfully bought the product ] pl.grzejszczak.marcin.drools.decisiontable.service.drools.ProductServiceImplTest:151 pl.grzejszczak.marcin.drools.decisiontable.model.Product@5b84b[ name=Some name type=MEDICAL price=750.0 countryOfOrigin=SWE additionalInfo=Congratulations, you are granted a discount quantity=8 ] In this post I've presented how you can push some of your developing work to your BA by giving him a tool which he can be able to work woth - the Decision Tables in a spreadsheet. What is more now you will now how to integrate Drools with Camel. Hopefully you will see how you can simplify (thus minimize the cost of implementing and supporting) the implementation of business rules bearing in mind how prone to changes they are. I hope that this example will even better illustrate how difficult it would be to implement all the business rules in Java than in the previous post about Drools. If you have any experience with Drools in terms of decision tables, integration with Spring and Camel please feel free to leave a comment here or on my blog - let's have a discussion on that :) All the code is available at Too Much Coding repository at Bitbucket and GitHub.

By default, Spring beans are scoped singleton, meaning there’s only one instance for the whole application context. For most applications, this is a sensible default; then sometimes, not so much. This may be the case when using a custom scope, which is the case, on the product I’m currently working on. I’m not at liberty to discuss the details further: suffice to say that it is very painful to configure each and every needed bean with this custom scope. Since being lazy in a smart way is at the core of developer work, I decided to search for a way to ease my burden and found it in the BeanFactoryPostProcessor class. It only has a single method - postProcessBeanFactory(), but it gives access to the bean factory itself (which is at the root of the various application context classes). From this point on, the code is trivial even with no prior experience of the API: public class PrototypeScopedBeanFactoryPostProcessor implements BeanFactoryPostProcessor { @Override public void postProcessBeanFactory(ConfigurableListableBeanFactory factory) throws BeansException { for (String beanName : factory.getBeanDefinitionNames()) { BeanDefinition beanDef = factory.getBeanDefinition(beanName); String explicitScope = beanDef.getScope(); if ("".equals(explicitScope)) { beanDef.setScope("prototype"); } } } } The final touch is to register the post-processor in the context . This is achieved by treating it as a simple anonymous bean: Now, every bean which scope is not explicitly set will be scoped prototype. Sources for this article can be found attached in Eclipse/Maven format.

I am thrilled to announce first version of my Spring Data JDBC repository project. The purpose of this open source library is to provide generic, lightweight and easy to use DAO implementation for relational databases based on JdbcTemplate from Spring framework, compatible with Spring Data umbrella of projects. Design objectives Lightweight, fast and low-overhead. Only a handful of classes, no XML, annotations, reflection This is not full-blown ORM. No relationship handling, lazy loading, dirty checking, caching CRUD implemented in seconds For small applications where JPA is an overkill Use when simplicity is needed or when future migration e.g. to JPA is considered Minimalistic support for database dialect differences (e.g. transparent paging of results) Features Each DAO provides built-in support for: Mapping to/from domain objects through RowMapper abstraction Generated and user-defined primary keys Extracting generated key Compound (multi-column) primary keys Immutable domain objects Paging (requesting subset of results) Sorting over several columns (database agnostic) Optional support for many-to-one relationships Supported databases (continuously tested): MySQL PostgreSQL H2 HSQLDB Derby ...and most likely most of the others Easily extendable to other database dialects via SqlGenerator class. Easy retrieval of records by ID API Compatible with Spring Data PagingAndSortingRepository abstraction, all these methods are implemented for you: public interface PagingAndSortingRepository extends CrudRepository { T save(T entity); Iterable save(Iterable entities); T findOne(ID id); boolean exists(ID id); Iterable findAll(); long count(); void delete(ID id); void delete(T entity); void delete(Iterable entities); void deleteAll(); Iterable findAll(Sort sort); Page findAll(Pageable pageable); } Pageable and Sort parameters are also fully supported, which means you get paging and sorting by arbitrary properties for free. For example say you have userRepository extending PagingAndSortingRepository interface (implemented for you by the library) and you request 5th page of USERS table, 10 per page, after applying some sorting: Page page = userRepository.findAll( new PageRequest( 5, 10, new Sort( new Order(DESC, "reputation"), new Order(ASC, "user_name") ) ) ); Spring Data JDBC repository library will translate this call into (PostgreSQL syntax): SELECT * FROM USERS ORDER BY reputation DESC, user_name ASC LIMIT 50 OFFSET 10 ...or even (Derby syntax): SELECT * FROM ( SELECT ROW_NUMBER() OVER () AS ROW_NUM, t.* FROM ( SELECT * FROM USERS ORDER BY reputation DESC, user_name ASC ) AS t ) AS a WHERE ROW_NUM BETWEEN 51 AND 60 No matter which database you use, you'll get Page object in return (you still have to provide RowMapper yourself to translate from ResultSet to domain object. If you don't know Spring Data project yet, Page is a wonderful abstraction, not only encapsulating List , but also providing metadata such as total number of records, on which page we currently are, etc. Reasons to use You consider migration to JPA or even some NoSQL database in the future. Since your code will rely only on methods defined in PagingAndSortingRepository and CrudRepository from Spring Data Commons umbrella project you are free to switch from JdbcRepository implementation (from this project) to: JpaRepository, MongoRepository, GemfireRepository or GraphRepository. They all implement the same common API. Of course don't expect that switching from JDBC to JPA or MongoDB will be as simple as switching imported JAR dependencies - but at least you minimize the impact by using same DAO API. You need a fast, simple JDBC wrapper library. JPA or even MyBatis is an overkill You want to have full control over generated SQL if needed You want to work with objects, but don't need lazy loading, relationship handling, multi-level caching, dirty checking... You need CRUD and not much more You want to by DRY You are already using Spring or maybe even JdbcTemplate, but still feel like there is too much manual work You have very few database tables Getting started For more examples and working code don't forget to examine project tests. Prerequisites Maven coordinates: com.blogspot.nurkiewicz jdbcrepository 0.1 Unfortunately the project is not yet in maven central repository. For the time being you can install the library in your local repository by cloning it: $ git clone git://github.com/nurkiewicz/spring-data-jdbc-repository.git $ git checkout 0.1 $ mvn javadoc:jar source:jar install In order to start your project must have DataSource bean present and transaction management enabled. Here is a minimal MySQL configuration: @EnableTransactionManagement @Configuration public class MinimalConfig { @Bean public PlatformTransactionManager transactionManager() { return new DataSourceTransactionManager(dataSource()); } @Bean public DataSource dataSource() { MysqlConnectionPoolDataSource ds = new MysqlConnectionPoolDataSource(); ds.setUser("user"); ds.setPassword("secret"); ds.setDatabaseName("db_name"); return ds; } } Entity with auto-generated key Say you have a following database table with auto-generated key (MySQL syntax): CREATE TABLE COMMENTS ( id INT AUTO_INCREMENT, user_name varchar(256), contents varchar(1000), created_time TIMESTAMP NOT NULL, PRIMARY KEY (id) ); First you need to create domain object User mapping to that table (just like in any other ORM): public class Comment implements Persistable { private Integer id; private String userName; private String contents; private Date createdTime; @Override public Integer getId() { return id; } @Override public boolean isNew() { return id == null; } //getters/setters/constructors/... } Apart from standard Java boilerplate you should notice implementing Persistable where Integer is the type of primary key. Persistable is an interface coming from Spring Data project and it's the only requirement we place on your domain object. Finally we are ready to create our CommentRepository DAO: @Repository public class CommentRepository extends JdbcRepository { public CommentRepository() { super(ROW_MAPPER, ROW_UNMAPPER, "COMMENTS"); } public static final RowMapper ROW_MAPPER = //see below private static final RowUnmapper ROW_UNMAPPER = //see below @Override protected Comment postCreate(Comment entity, Number generatedId) { entity.setId(generatedId.intValue()); return entity; } } First of all we use @Repository annotation to mark DAO bean. It enables persistence exception translation. Also such annotated beans are discovered by CLASSPATH scanning. As you can see we extend JdbcRepository which is the central class of this library, providing implementations of all PagingAndSortingRepository methods. Its constructor has three required dependencies: RowMapper , RowUnmapper and table name. You may also provide ID column name, otherwise default "id" is used. If you ever used JdbcTemplate from Spring, you should be familiar with RowMapper interface. We need to somehow extract columns from ResultSet into an object. After all we don't want to work with raw JDBC results. It's quite straightforward: public static final RowMapper ROW_MAPPER = new RowMapper () { @Override public Comment mapRow(ResultSet rs, int rowNum) throws SQLException { return new Comment( rs.getInt("id"), rs.getString("user_name"), rs.getString("contents"), rs.getTimestamp("created_time") ); } }; RowUnmapper comes from this library and it's essentially the opposite of RowMapper : takes an object and turns it into a Map . This map is later used by the library to construct SQL CREATE / UPDATE queries: private static final RowUnmapper ROW_UNMAPPER = new RowUnmapper () { @Override public Map mapColumns(Comment comment) { Map mapping = new LinkedHashMap (); mapping.put("id", comment.getId()); mapping.put("user_name", comment.getUserName()); mapping.put("contents", comment.getContents()); mapping.put("created_time", new java.sql.Timestamp(comment.getCreatedTime().getTime())); return mapping; } }; If you never update your database table (just reading some reference data inserted elsewhere) you may skip RowUnmapper parameter or use MissingRowUnmapper. Last piece of the puzzle is the postCreate() callback method which is called after an object was inserted. You can use it to retrieve generated primary key and update your domain object (or return new one if your domain objects are immutable). If you don't need it, just don't override postCreate() . Check out JdbcRepositoryGeneratedKeyTest for a working code based on this example. By now you might have a feeling that, compared to JPA or Hibernate, there is quite a lot of manual work. However various JPA implementations and other ORM frameworks are notoriously known for introducing significant overhead and manifesting some learning curve. This tiny library intentionally leaves some responsibilities to the user in order to avoid complex mappings, reflection, annotations... all the implicitness that is not always desired. This project is not intending to replace mature and stable ORM frameworks. Instead it tries to fill in a niche between raw JDBC and ORM where simplicity and low overhead are key features. Entity with manually assigned key In this example we'll see how entities with user-defined primary keys are handled. Let's start from database model: CREATE TABLE USERS ( user_name varchar(255), date_of_birth TIMESTAMP NOT NULL, enabled BIT(1) NOT NULL, PRIMARY KEY (user_name) ); ...and User domain model: public class User implements Persistable { private transient boolean persisted; private String userName; private Date dateOfBirth; private boolean enabled; @Override public String getId() { return userName; } @Override public boolean isNew() { return !persisted; } public User withPersisted(boolean persisted) { this.persisted = persisted; return this; } //getters/setters/constructors/... } Notice that special persisted transient flag was added. Contract of CrudRepository.save() from Spring Data project requires that an entity knows whether it was already saved or not ( isNew() ) method - there are no separate create() and update() methods. Implementing isNew() is simple for auto-generated keys (see Comment above) but in this case we need an extra transient field. If you hate this workaround and you only insert data and never update, you'll get away with return true all the time from isNew() . And finally our DAO, UserRepository bean: @Repository public class UserRepository extends JdbcRepository { public UserRepository() { super(ROW_MAPPER, ROW_UNMAPPER, "USERS", "user_name"); } public static final RowMapper ROW_MAPPER = //... public static final RowUnmapper ROW_UNMAPPER = //... @Override protected User postUpdate(User entity) { return entity.withPersisted(true); } @Override protected User postCreate(User entity, Number generatedId) { return entity.withPersisted(true); } } "USERS" and "user_name" parameters designate table name and primary key column name. I'll leave the details of mapper and unmapper (see source code). But please notice postUpdate() and postCreate() methods. They ensure that once object was persisted, persisted flag is set so that subsequent calls to save() will update existing entity rather than trying to reinsert it. Check out JdbcRepositoryManualKeyTest for a working code based on this example. Compound primary key We also support compound primary keys (primary keys consisting of several columns). Take this table as an example: CREATE TABLE BOARDING_PASS ( flight_no VARCHAR(8) NOT NULL, seq_no INT NOT NULL, passenger VARCHAR(1000), seat CHAR(3), PRIMARY KEY (flight_no, seq_no) ); I would like you to notice the type of primary key in Peristable : public class BoardingPass implements Persistable { private transient boolean persisted; private String flightNo; private int seqNo; private String passenger; private String seat; @Override public Object[] getId() { return pk(flightNo, seqNo); } @Override public boolean isNew() { return !persisted; } //getters/setters/constructors/... } Unfortunately we don't support small value classes encapsulating all ID values in one object (like JPA does with @IdClass), so you have to live with Object[] array. Defining DAO class is similar to what we've already seen: public class BoardingPassRepository extends JdbcRepository { public BoardingPassRepository() { this("BOARDING_PASS"); } public BoardingPassRepository(String tableName) { super(MAPPER, UNMAPPER, new TableDescription(tableName, null, "flight_no", "seq_no") ); } public static final RowMapper ROW_MAPPER = //... public static final RowUnmapper UNMAPPER = //... } Two things to notice: we extend JdbcRepository and we provide two ID column names just as expected: "flight_no", "seq_no" . We query such DAO by providing both flight_no and seq_no (necessarily in that order) values wrapped by Object[] : BoardingPass pass = repository.findOne(new Object[] {"FOO-1022", 42}); No doubts, this is cumbersome in practice, so we provide tiny helper method which you can statically import: import static com.blogspot.nurkiewicz.jdbcrepository.JdbcRepository.pk; //... BoardingPass foundFlight = repository.findOne(pk("FOO-1022", 42)); Check out JdbcRepositoryCompoundPkTest for a working code based on this example. Transactions This library is completely orthogonal to transaction management. Every method of each repository requires running transaction and it's up to you to set it up. Typically you would place @Transactional on service layer (calling DAO beans). I don't recommend placing @Transactional over every DAO bean. Caching Spring Data JDBC repository library is not providing any caching abstraction or support. However adding @Cacheable layer on top of your DAOs or services using caching abstraction in Spring is quite straightforward. See also: @Cacheable overhead in Spring. Contributions ..are always welcome. Don't hesitate to submit bug reports and pull requests. Biggest missing feature now is support for MSSQL and Oracle databases. It would be terrific if someone could have a look at it. Testing This library is continuously tested using Travis (). Test suite consists of 265 tests (53 distinct tests each run against 5 different databases: MySQL, PostgreSQL, H2, HSQLDB and Derby. When filling bug reports or submitting new features please try including supporting test cases. Each pull request is automatically tested on a separate branch. Building After forking the official repository building is as simple as running: $ mvn install You'll notice plenty of exceptions during JUnit test execution. This is normal. Some of the tests run against MySQL and PostgreSQL available only on Travis CI server. When these database servers are unavailable, whole test is simply skipped: Results : Tests run: 265, Failures: 0, Errors: 0, Skipped: 106 Exception stack traces come from root AbstractIntegrationTest. Design Library consists of only a handful of classes, highlighted in the diagram below: JdbcRepository is the most important class that implements all PagingAndSortingRepository methods. Each user repository has to extend this class. Also each such repository must at least implement RowMapper and RowUnmapper (only if you want to modify table data). SQL generation is delegated to SqlGenerator. PostgreSqlGenerator. and DerbySqlGenerator are provided for databases that don't work with standard generator. License This project is released under version 2.0 of the Apache License (same as Spring framework).

As NoSQL solutions are getting more and more popular for many kind of problems, more often the modern projects consider to use some (or several) of NoSQLs instead (or side-by-side) of traditional RDBMS. I have already covered my experience with MongoDB in this, this and this posts. In this post I would like to switch gears a bit towards Redis, an advanced key-value store. Aside from very rich key-value semantics, Redis also supports pub-sub messaging and transactions. In this post I am going just to touch the surface and demonstrate how simple it is to integrate Redis into your Spring application. As always, we will start with Maven POM file for our project: 4.0.0 com.example.spring redis 0.0.1-SNAPSHOT jar UTF-8 3.1.0.RELEASE org.springframework.data spring-data-redis 1.0.0.RELEASE cglib cglib-nodep 2.2 log4j log4j 1.2.16 redis.clients jedis 2.0.0 jar org.springframework spring-core ${spring.version} org.springframework spring-context ${spring.version} Spring Data Redis is the another project under Spring Data umbrella which provides seamless injection of Redis into your application. The are several Redis clients for Java and I have chosen the Jedis as it is stable and recommended by Redis team at the moment of writing this post. We will start with simple configuration and introduce the necessary components first. Then as we move forward, the configuration will be extended a bit to demonstrated pub-sub capabilities. Thanks to Java config support, we will create the configuration class and have all our dependencies strongly typed, no XML anymore: package com.example.redis.config; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.data.redis.connection.jedis.JedisConnectionFactory; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.serializer.GenericToStringSerializer; import org.springframework.data.redis.serializer.StringRedisSerializer; @Configuration public class AppConfig { @Bean JedisConnectionFactory jedisConnectionFactory() { return new JedisConnectionFactory(); } @Bean RedisTemplate< String, Object > redisTemplate() { final RedisTemplate< String, Object > template = new RedisTemplate< String, Object >(); template.setConnectionFactory( jedisConnectionFactory() ); template.setKeySerializer( new StringRedisSerializer() ); template.setHashValueSerializer( new GenericToStringSerializer< Object >( Object.class ) ); template.setValueSerializer( new GenericToStringSerializer< Object >( Object.class ) ); return template; } } That's basically everything we need assuming we have single Redis server up and running on localhost with default configuration. Let's consider several common uses cases: setting a key to some value, storing the object and, finally, pub-sub implementation. Storing and retrieving a key/value pair is very simple: @Autowired private RedisTemplate< String, Object > template; public Object getValue( final String key ) { return template.opsForValue().get( key ); } public void setValue( final String key, final String value ) { template.opsForValue().set( key, value ); } Optionally, the key could be set to expire (yet another useful feature of Redis), f.e. let our keys expire in 1 second: public void setValue( final String key, final String value ) { template.opsForValue().set( key, value ); template.expire( key, 1, TimeUnit.SECONDS ); } Arbitrary objects could be saved into Redis as hashes (maps), f.e. let save instance of some class User public class User { private final Long id; private String name; private String email; // Setters and getters are omitted for simplicity } into Redis using key pattern "user:": public void setUser( final User user ) { final String key = String.format( "user:%s", user.getId() ); final Map< String, Object > properties = new HashMap< String, Object >(); properties.put( "id", user.getId() ); properties.put( "name", user.getName() ); properties.put( "email", user.getEmail() ); template.opsForHash().putAll( key, properties); } Respectively, object could easily be inspected and retrieved using the id. public User getUser( final Long id ) { final String key = String.format( "user:%s", id ); final String name = ( String )template.opsForHash().get( key, "name" ); final String email = ( String )template.opsForHash().get( key, "email" ); return new User( id, name, email ); } There are much, much more which could be done using Redis, I highly encourage to take a look on it. It surely is not a silver bullet but could solve many challenging problems very easy. Finally, let me show how to use a pub-sub messaging with Redis. Let's add a bit more configuration here (as part of AppConfig class): @Bean MessageListenerAdapter messageListener() { return new MessageListenerAdapter( new RedisMessageListener() ); } @Bean RedisMessageListenerContainer redisContainer() { final RedisMessageListenerContainer container = new RedisMessageListenerContainer(); container.setConnectionFactory( jedisConnectionFactory() ); container.addMessageListener( messageListener(), new ChannelTopic( "my-queue" ) ); return container; } The style of message listener definition should look very familiar to Spring users: generally, the same approach we follow to define JMS message listeners. The missed piece is our RedisMessageListener class definition: package com.example.redis.impl; import org.springframework.data.redis.connection.Message; import org.springframework.data.redis.connection.MessageListener; public class RedisMessageListener implements MessageListener { @Override public void onMessage(Message message, byte[] paramArrayOfByte) { System.out.println( "Received by RedisMessageListener: " + message.toString() ); } } Now, when we have our message listener, let see how we could push some messages into the queue using Redis. As always, it's pretty simple: @Autowired private RedisTemplate< String, Object > template; public void publish( final String message ) { template.execute( new RedisCallback< Long >() { @SuppressWarnings( "unchecked" ) @Override public Long doInRedis( RedisConnection connection ) throws DataAccessException { return connection.publish( ( ( RedisSerializer< String > )template.getKeySerializer() ).serialize( "queue" ), ( ( RedisSerializer< Object > )template.getValueSerializer() ).serialize( message ) ); } } ); } That's basically it for very quick introduction but definitely enough to fall in love with Redis.

Many enterprise applications require batch processing to process billions of transactions every day. These big transaction sets have to be processed without performance problems. Spring Batch is a lightweight and robust batch framework to process these big data sets. Spring Batch offers ‘TaskletStep Oriented’ and ‘Chunk Oriented’ processing style. In this article, TaskletStep Oriented Processing Model is explained. Let us investigate fundamental Spring Batch components : Job : An entity that encapsulates an entire batch process. Step and Tasklets are defined under a Job Step : A domain object that encapsulates an independent, sequential phase of a batch job. JobInstance : Batch domain object representing a uniquely identifiable job run – it’s identity is given by the pair Job and JobParameters. JobParameters : Value object representing runtime parameters to a batch job. JobExecution : A JobExecution refers to the technical concept of a single attempt to run a Job. An execution may end in failure or success, but the JobInstance corresponding to a given execution will not be considered complete unless the execution completes successfully. JobRepository : An interface which responsible for persistence of batch meta-data entities. In the following sample, an in-memory repository is used via MapJobRepositoryFactoryBean. JobLauncher : An interface exposing run method, which launches and controls the defined jobs. TaskLet : An interface exposing execute method, which will be a called repeatedly until it either returns RepeatStatus.FINISHED or throws an exception to signal a failure. It is used when both readers and writers are not required as the following sample. Let us take a look how to develop Tasklet-Step Oriented Processing Model. Used Technologies : JDK 1.7.0_09 Spring 3.1.3 Spring Batch 2.1.9 Maven 3.0.4 STEP 1 : CREATE MAVEN PROJECT A maven project is created as below. (It can be created by using Maven or IDE Plug-in). STEP 2 : LIBRARIES Firstly, dependencies are added to Maven’ s pom.xml. 3.1.3.RELEASE 2.1.9.RELEASE org.springframework spring-core ${spring.version} org.springframework spring-context ${spring.version} org.springframework.batch spring-batch-core ${spring-batch.version} log4j log4j 1.2.16 maven-compiler-plugin(Maven Plugin) is used to compile the project with JDK 1.7 org.apache.maven.plugins maven-compiler-plugin 3.0 1.7 1.7 The following Maven plugin can be used to create runnable-jar, org.apache.maven.plugins maven-shade-plugin 2.0 package shade 1.7 1.7 com.onlinetechvision.exe.Application META-INF/spring.handlers META-INF/spring.schemas STEP 3 : CREATE SuccessfulStepTasklet TASKLET SuccessfulStepTasklet is created by implementing Tasklet Interface. It illustrates business logic in successful step. package com.onlinetechvision.tasklet; import org.apache.log4j.Logger; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; /** * SuccessfulStepTasklet Class illustrates a successful job * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class SuccessfulStepTasklet implements Tasklet { private static final Logger logger = Logger.getLogger(SuccessfulStepTasklet.class); private String taskResult; /** * Executes SuccessfulStepTasklet * * @param StepContribution stepContribution * @param ChunkContext chunkContext * @return RepeatStatus * @throws Exception * */ @Override public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { logger.debug("Task Result : " + getTaskResult()); return RepeatStatus.FINISHED; } public String getTaskResult() { return taskResult; } public void setTaskResult(String taskResult) { this.taskResult = taskResult; } } STEP 4 : CREATE FailedStepTasklet TASKLET FailedStepTasklet is created by implementing Tasklet Interface. It illustrates business logic in failed step. package com.onlinetechvision.tasklet; import org.apache.log4j.Logger; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; /** * FailedStepTasklet Class illustrates a failed job. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class FailedStepTasklet implements Tasklet { private static final Logger logger = Logger.getLogger(FailedStepTasklet.class); private String taskResult; /** * Executes FailedStepTasklet * * @param StepContribution stepContribution * @param ChunkContext chunkContext * @return RepeatStatus * @throws Exception * */ @Override public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { logger.debug("Task Result : " + getTaskResult()); throw new Exception("Error occurred!"); } public String getTaskResult() { return taskResult; } public void setTaskResult(String taskResult) { this.taskResult = taskResult; } } STEP 5 : CREATE BatchProcessStarter CLASS BatchProcessStarter Class is created to launch the jobs. Also, it logs their execution results. A Completed Job Instance can not be restarted with the same parameter(s) because it already exists in job repository and JobInstanceAlreadyCompleteException is thrown with “A job instance already exists and is complete” description. It can be restarted with different parameter. In the following sample, different currentTime parameter is set in order to restart FirstJob. package com.onlinetechvision.spring.batch; import org.apache.log4j.Logger; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; /** * BatchProcessStarter Class launches the jobs and logs their execution results. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class BatchProcessStarter { private static final Logger logger = Logger.getLogger(BatchProcessStarter.class); private Job firstJob; private Job secondJob; private Job thirdJob; private JobLauncher jobLauncher; private JobRepository jobRepository; /** * Starts the jobs and logs their execution results. * */ public void start() { JobExecution jobExecution = null; JobParametersBuilder builder = new JobParametersBuilder(); try { builder.addLong("currentTime", new Long(System.currentTimeMillis())); getJobLauncher().run(getFirstJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getFirstJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getSecondJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getSecondJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getThirdJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getThirdJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); builder.addLong("currentTime", new Long(System.currentTimeMillis())); getJobLauncher().run(getFirstJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getFirstJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); } catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) { logger.error(e); } } public Job getFirstJob() { return firstJob; } public void setFirstJob(Job firstJob) { this.firstJob = firstJob; } public Job getSecondJob() { return secondJob; } public void setSecondJob(Job secondJob) { this.secondJob = secondJob; } public Job getThirdJob() { return thirdJob; } public void setThirdJob(Job thirdJob) { this.thirdJob = thirdJob; } public JobLauncher getJobLauncher() { return jobLauncher; } public void setJobLauncher(JobLauncher jobLauncher) { this.jobLauncher = jobLauncher; } public JobRepository getJobRepository() { return jobRepository; } public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; } } STEP 6 : CREATE applicationContext.xml Spring Configuration file, applicationContext.xml, is created. It covers Tasklets and BatchProcessStarter definitions. STEP 7 : CREATE jobContext.xml Spring Configuration file, jobContext.xml, is created. Jobs’ flows are the following : FirstJob’ s flow : 1) FirstStep is started. 2) After FirstStep is completed with COMPLETED status, SecondStep is started. 3) After SecondStep is completed with COMPLETED status, ThirdStep is started. 4) After ThirdStep is completed with COMPLETED status, FirstJob execution is completed with COMPLETED status. SecondJob’ s flow : 1) FourthStep is started. 2) After FourthStep is completed with COMPLETED status, FifthStep is started. 3) After FifthStep is completed with COMPLETED status, SecondJob execution is completed with COMPLETED status. ThirdJob’ s flow : 1) SixthStep is started. 2) After SixthStep is completed with COMPLETED status, SeventhStep is started. 3) After SeventhStep is completed with FAILED status, ThirdJob execution is completed FAILED status. FirstJob’ s flow is same with the first execution. STEP 8 : CREATE Application CLASS Application Class is created to run the application. package com.onlinetechvision.exe; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import com.onlinetechvision.spring.batch.BatchProcessStarter; /** * Application Class starts the application. * * @author onlinetechvision.com * @since 27 Nov 2012 * @version 1.0.0 * */ public class Application { /** * Starts the application * * @param String[] args * */ public static void main(String[] args) { ApplicationContext appContext = new ClassPathXmlApplicationContext("jobContext.xml"); BatchProcessStarter batchProcessStarter = (BatchProcessStarter)appContext.getBean("batchProcessStarter"); batchProcessStarter.start(); } } STEP 9 : BUILD PROJECT After OTV_SpringBatch_TaskletStep_Oriented_Processing Project is built, OTV_SpringBatch_TaskletStep-0.0.1-SNAPSHOT.jar will be created. STEP 10 : RUN PROJECT After created OTV_SpringBatch_TaskletStep-0.0.1-SNAPSHOT.jar file is run, the following console output logs will be shown : First Job’ s console output : 25.11.2012 21:29:19 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=firstJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:19 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=0, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:19 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN; exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=firstJob.firstStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.firstStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [firstStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=1 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : First Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=1 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=1, version=3, name=firstStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.firstStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.secondStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [secondStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=2 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Second Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=2 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=2, version=3, name=secondStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.secondStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.thirdStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [thirdStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=3 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Third Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=3, version=3, name=thirdStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.thirdStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.end3 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.end3 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=0, version=1, startTime=Sun Nov 25 21:29:19 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:19 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=firstJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:44) - JobExecution: id=0, version=2, startTime=Sun Nov 25 21:29:19 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{currentTime=1353878959462}], Job=[firstJob]] Second Job’ s console output : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=secondJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=1, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=secondJob.fourthStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.fourthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [fourthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=4 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Fourth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=4, version=3, name=fourthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.fourthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.fifthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [fifthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=5 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Fifth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=5, version=3, name=fifthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.fifthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=secondJob.end5 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=secondJob.end5 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=1, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=secondJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:48) - JobExecution: id=1, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{currentTime=1353878959462}], Job=[secondJob]] Third Job’ s console output : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=thirdJob]] launched with the following parameters: [{currentTime=1353878959462}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=2, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=thirdJob.sixthStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.sixthStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [sixthStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=6 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Sixth Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=6, version=3, name=sixthStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.sixthStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.seventhStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [seventhStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=7 25.11.2012 21:29:20 DEBUG (FailedStepTasklet.java:33) - Task Result : Error occurred! 25.11.2012 21:29:20 DEBUG (TaskletStep.java:456) - Rollback for Exception: java.lang.Exception: Error occurred! 25.11.2012 21:29:20 DEBUG (TransactionTemplate.java:152) - Initiating transaction rollback on application exception ... 25.11.2012 21:29:20 DEBUG (AbstractPlatformTransactionManager.java:821) - Initiating transaction rollback 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:54) - Rolling back resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@40874c04] 25.11.2012 21:29:20 DEBUG (RepeatTemplate.java:291) - Handling exception: java.lang.Exception, caused by: java.lang.Exception: Error occurred! 25.11.2012 21:29:20 DEBUG (RepeatTemplate.java:251) - Handling fatal exception explicitly (rethrowing first of 1): java.lang.Exception: Error occurred! 25.11.2012 21:29:20 ERROR (AbstractStep.java:222) - Encountered an error executing the step ... 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:34) - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@66a7d863] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=7, version=2, name=seventhStep, status=FAILED, exitStatus=FAILED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=0, rollbackCount=1 25.11.2012 21:29:20 DEBUG (ResourcelessTransactionManager.java:34) - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@156f803c] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.seventhStep with status=FAILED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=thirdJob.fail8 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=thirdJob.fail8 with status=FAILED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=2, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=FAILED, exitStatus=exitCode=FAILED;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=thirdJob]] completed with the following parameters: [{currentTime=1353878959462}] and the following status: [FAILED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:52) - JobExecution: id=2, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=FAILED, exitStatus=exitCode=FAILED; exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{currentTime=1353878959462}], Job=[thirdJob]] First Job’ s console output after restarting : 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=firstJob]] launched with the following parameters: [{currentTime=1353878960660}] 25.11.2012 21:29:20 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=3, version=0, startTime=null, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:135) - Resuming state=firstJob.firstStep with status=UNKNOWN 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.firstStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [firstStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=8 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : First Task is executed... 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=8 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=8, version=3, name=firstStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.firstStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.secondStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [secondStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=9 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Second Task is executed... 25.11.2012 21:29:20 DEBUG (TaskletStep.java:417) - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=9 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=9, version=3, name=secondStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.secondStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.thirdStep 25.11.2012 21:29:20 INFO (SimpleStepHandler.java:133) - Executing step: [thirdStep] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:180) - Executing: id=10 25.11.2012 21:29:20 DEBUG (SuccessfulStepTasklet.java:33) - Task Result : Third Task is executed... 25.11.2012 21:29:20 DEBUG (TaskletStep.java:417) - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING] 25.11.2012 21:29:20 DEBUG (AbstractStep.java:209) - Step execution success: id=10 25.11.2012 21:29:20 DEBUG (AbstractStep.java:273) - Step execution complete: StepExecution: id=10, version=3, name=thirdStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.thirdStep with status=COMPLETED 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:143) - Handling state=firstJob.end3 25.11.2012 21:29:20 DEBUG (SimpleFlow.java:156) - Completed state=firstJob.end3 with status=COMPLETED 25.11.2012 21:29:20 DEBUG (AbstractJob.java:294) - Job execution complete: JobExecution: id=3, version=1, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=null, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] 25.11.2012 21:29:20 INFO (SimpleJobLauncher.java:121) - Job: [FlowJob: [name=firstJob]] completed with the following parameters: [{currentTime=1353878960660}] and the following status: [COMPLETED] 25.11.2012 21:29:20 DEBUG (BatchProcessStarter.java:57) - JobExecution: id=3, version=2, startTime=Sun Nov 25 21:29:20 GMT 2012, endTime=Sun Nov 25 21:29:20 GMT 2012, lastUpdated=Sun Nov 25 21:29:20 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=3, version=0, JobParameters=[{currentTime=1353878960660}], Job=[firstJob]] STEP 11 : DOWNLOAD https://github.com/erenavsarogullari/OTV_SpringBatch_TaskletStep REFERENCES : Spring Batch – Reference Documentation Spring Batch – API Documentation

Spring 3.1 introduced great caching abstraction layer. Finally we can abandon all home-grown aspects, decorators and code polluting our business logic related to caching. Since then we can simply annotate heavyweight methods and let Spring and AOP machinery do the work: @Cacheable("books") public Book findBook(ISBN isbn) {...} "books" is a cache name, isbn parameter becomes cache key and returned Book object will be placed under that key. The meaning of cache name is dependant on the underlying cache manager (EhCache, concurrent map, etc.) - Spring makes it easy to plug different caching providers. But this post won't be about caching feature in Spring... Some time ago my teammate was optimizing quite low-level code and discovered an opportunity for caching. He quickly applied @Cacheable just to discover that the code performed worse then it used to. He got rid of the annotation and implemented caching himself manually, using good old java.util.ConcurrentHashMap. The performance was much better. He blamed @Cacheable and Spring AOP overhead and complexity. I couldn't believe that a caching layer can perform so poorly until I had to debug Spring caching aspects few times myself (some nasty bug in my code, you know, cache invalidation is one of the two hardest things in CS). Well, the caching abstraction code is much more complex than one would expect (after all it's just get and put!), but it doesn't necessarily mean it must be that slow? In science we don't believe and trust, we measure and benchmark. So I wrote a benchmark to precisely measure the overhead of @Cacheable layer. Caching abstraction layer in Spring is implemented on top of Spring AOP, which can further be implemented on top of Java proxies, CGLIB generated subclasses or AspectJ instrumentation. Thus I'll test the following configurations: no caching at all - to measure how fast the code is with no intermediate layer manual cache handling using ConcurrentHashMap in business code @Cacheable with CGLIB implementing AOP @Cacheable with java.lang.reflect.Proxy implementing AOP @Cacheable with AspectJ compile time weaving (as similar benchmark shows, CTW is slightly faster than LTW) Home-grown AspectJ caching aspect - something between manual caching in business code and Spring abstraction Let me reiterate: we are not measuring the performance gain of caching and we are not comparing various cache providers. That's why our test method is as fast as it can be and I will be using simplest ConcurrentMapCacheManager from Spring. So here is a method in question: public interface Calculator { int identity(int x); } public class PlainCalculator implements Calculator { @Cacheable("identity") @Override public int identity(int x) { return x; } } I know, I know there is no point in caching such a method. But I want to measure the overhead of caching layer (during cache hit to be specific). Each caching configuration will have its own ApplicationContext as you can't mix different proxying modes in one context: public abstract class BaseConfig { @Bean public Calculator calculator() { return new PlainCalculator(); } } @Configuration class NoCachingConfig extends BaseConfig {} @Configuration class ManualCachingConfig extends BaseConfig { @Bean @Override public Calculator calculator() { return new CachingCalculatorDecorator(super.calculator()); } } @Configuration abstract class CacheManagerConfig extends BaseConfig { @Bean public CacheManager cacheManager() { return new ConcurrentMapCacheManager(); } } @Configuration @EnableCaching(proxyTargetClass = true) class CacheableCglibConfig extends CacheManagerConfig {} @Configuration @EnableCaching(proxyTargetClass = false) class CacheableJdkProxyConfig extends CacheManagerConfig {} @Configuration @EnableCaching(mode = AdviceMode.ASPECTJ) class CacheableAspectJWeaving extends CacheManagerConfig { @Bean @Override public Calculator calculator() { return new SpringInstrumentedCalculator(); } } @Configuration @EnableCaching(mode = AdviceMode.ASPECTJ) class AspectJCustomAspect extends CacheManagerConfig { @Bean @Override public Calculator calculator() { return new ManuallyInstrumentedCalculator(); } } Each @Configuration class represents one application context. CachingCalculatorDecorator is a decorator around real calculator that does the caching (welcome to the 1990s): public class CachingCalculatorDecorator implements Calculator { private final Map cache = new java.util.concurrent.ConcurrentHashMap(); private final Calculator target; public CachingCalculatorDecorator(Calculator target) { this.target = target; } @Override public int identity(int x) { final Integer existing = cache.get(x); if (existing != null) { return existing; } final int newValue = target.identity(x); cache.put(x, newValue); return newValue; } } SpringInstrumentedCalculator and ManuallyInstrumentedCalculator are exactly the same as PlainCalculator but they are instrumented by AspectJ compile-time weaver with Spring and custom aspect accordingly. My custom caching aspect looks like this: public aspect ManualCachingAspect { private final Map cache = new ConcurrentHashMap(); pointcut cacheMethodExecution(int x): execution(int com.blogspot.nurkiewicz.cacheable.calculator.ManuallyInstrumentedCalculator.identity(int)) && args(x); Object around(int x): cacheMethodExecution(x) { final Integer existing = cache.get(x); if (existing != null) { return existing; } final Object newValue = proceed(x); cache.put(x, (Integer)newValue); return newValue; } } After all this preparation we can finally write the benchmark itself. At the beginning I start all the application contexts and fetch Calculator instances. Each instance is different. For example noCaching is a PlainCalculator instance with no wrappers, cacheableCglib is a CGLIB generated subclass while aspectJCustom is an instance of ManuallyInstrumentedCalculator with my custom aspect woven. private final Calculator noCaching = fromSpringContext(NoCachingConfig.class); private final Calculator manualCaching = fromSpringContext(ManualCachingConfig.class); private final Calculator cacheableCglib = fromSpringContext(CacheableCglibConfig.class); private final Calculator cacheableJdkProxy = fromSpringContext(CacheableJdkProxyConfig.class); private final Calculator cacheableAspectJ = fromSpringContext(CacheableAspectJWeaving.class); private final Calculator aspectJCustom = fromSpringContext(AspectJCustomAspect.class); private static Calculator fromSpringContext(Class config) { return new AnnotationConfigApplicationContext(config).getBean(Calculator.class); } I'm going to exercise each Calculator instance with the following test. The additional accumulator is necessary, otherwise JVM might optimize away the whole loop (!): private int benchmarkWith(Calculator calculator, int reps) { int accum = 0; for (int i = 0; i < reps; ++i) { accum += calculator.identity(i % 16); } return accum; } Here is the full caliper test without parts already discussed: public class CacheableBenchmark extends SimpleBenchmark { //... public int timeNoCaching(int reps) { return benchmarkWith(noCaching, reps); } public int timeManualCaching(int reps) { return benchmarkWith(manualCaching, reps); } public int timeCacheableWithCglib(int reps) { return benchmarkWith(cacheableCglib, reps); } public int timeCacheableWithJdkProxy(int reps) { return benchmarkWith(cacheableJdkProxy, reps); } public int timeCacheableWithAspectJWeaving(int reps) { return benchmarkWith(cacheableAspectJ, reps); } public int timeAspectJCustom(int reps) { return benchmarkWith(aspectJCustom, reps); } } I hope you are still following our experiment. We are now going to execute Calculate.identity() millions of times and see which caching configuration performs best. Since we only call identity() with 16 different arguments, we hardly ever touch the method itself as we always get cache hit. Curious to see the results? benchmark ns linear runtime NoCaching 1.77 = ManualCaching 23.84 = CacheableWithCglib 1576.42 ============================== CacheableWithJdkProxy 1551.03 ============================= CacheableWithAspectJWeaving 1514.83 ============================ AspectJCustom 22.98 = Interpretation Let's go step by step. First of all calling a method in Java is pretty darn fast! 1.77 nanoseconds, we are talking here about 3 CPU cycles on my Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz! If this doesn't convince you that Java is fast, I don't know what will. But back to our test. Hand-made caching decorator is also pretty fast. Of course it's slower by an order of magnitude compared to pure function call, but still blazingly fast compared to all @Scheduled benchmarks. We see a drop by 3 orders of magnitude, from 1.8 ns to 1.5 μs. I'm especially disappointed by the @Cacheable backed by AspectJ. After all caching aspect is precompiled directly into my Java .class file, I would expect it to be much faster compared to dynamic proxies and CGLIB. But that doesn't seem to be the case. All three Spring AOP techniques are similar. The greatest surprise is my custom AspectJ aspect. It's even faster than CachingCalculatorDecorator! maybe it's due to polymorphic call in the decorator? I strongly encourage you to clone this benchmark on GitHub and run it (mvn clean test, takes around 2 minutes) to compare your results. Conclusions You might be wondering why Spring abstraction layer is so slow? Well, first of all, check out the core implementation in CacheAspectSupport - it's actually quite complex. Secondly, is it really that slow? Do the math - you typically use Spring in business applications where database, network and external APIs are the bottleneck. What latencies do you typically see? Milliseconds? Tens or hundreds of milliseconds? Now add an overhead of 2 μs (worst case scenario). For caching database queries or REST calls this is completely negligible. It doesn't matter which technique you choose. But if you are caching very low-level methods close to the metal, like CPU-intensive, in-memory computations, Spring abstraction layer might be an overkill. The bottom line: measure! PS: both benchmark and contents of this article in Markdown format are freely available.

Apache Camel provides an event notifier support class which allows you to keep information about what happened on Exchange, Route and Endpoint. One of the benefits of this class is that you can easily audit messages created in Camel Routes, collect information and report that in log by example. When developing an application, it is very important to calculate/measure elapsed time on the platform to find which part of your code, processor or system integrated which is the bad duck and must be improved. In three steps, I would show you How to enable this mechanism to report : - Time elapsed to call an endpoint (could be another camel route, web service, ...) - Time elapsed on the route exchange STEP 1 - Create a Class implementing the EventNotifierSupport public class AuditEventNotifier extends EventNotifierSupport { public void notify(EventObject event) throws Exception { if (event instanceof ExchangeSentEvent) { ExchangeSentEvent sent = (ExchangeSentEvent) event; log.info(">>> Took " + sent.getTimeTaken() + " millis to send to external system : " + sent.getEndpoint()); } if (event instanceof ExchangeCompletedEvent) {; ExchangeCompletedEvent exchangeCompletedEvent = (ExchangeCompletedEvent) event; Exchange exchange = exchangeCompletedEvent.getExchange(); String routeId = exchange.getFromRouteId(); Date created = ((ExchangeCompletedEvent) event).getExchange().getProperty(Exchange.CREATED_TIMESTAMP, Date.class); // calculate elapsed time Date now = new Date(); long elapsed = now.getTime() - created.getTime(); log.info(">>> Took " + elapsed + " millis for the exchange on the route : " + routeId); } } public boolean isEnabled(EventObject event) { return true; } protected void doStart() throws Exception { // filter out unwanted events setIgnoreCamelContextEvents(true); setIgnoreServiceEvents(true); setIgnoreRouteEvents(true); setIgnoreExchangeCreatedEvent(true); setIgnoreExchangeCompletedEvent(false); setIgnoreExchangeFailedEvents(true); setIgnoreExchangeRedeliveryEvents(true); setIgnoreExchangeSentEvents(false); } protected void doStop() throws Exception { // noop } } Not really complicated and the code is explicit. Check the doStart() method to enable/disable the events for which you would like to gather information. This example uses only Exchange.CREATED_TIMESTAMP property but the next version of Camel 2.7.0 will provide you the property exchange.RECEIVED_TIMESTAMP and so you will be able to calculate more easily the time spend by the exchange to call the different processors till it arrives at the end of the route. This example collects Date information but you can imagine to use this mechanism to check if your route processes the message according to SLA, .... STEP 2 - Instantiate the bean in Camel Spring XML By adding this bean definition, Camel will automatically register it to the CamelContext created. STEP 3 - Collect info into the log 18:10:46,060 | INFO | tp1238469515-285 | AuditEventNotifier | ? ? | 68 - org.apache.camel.camel-core - 2.6.0.fuse-00-00 | >>> Took 3 millis for the exchange on the route : mock-HTTP-Server 18:10:46,062 | INFO | tp2056154542-293 | AuditEventNotifier | ? ? | 68 - org.apache.camel.camel-core - 2.6.0.fuse-00-00 | >>> Took 25 millis to send to external system : Endpoint[http://localhost:9191/sis] 18:10:46,077 | INFO | tp2056154542-293 | AuditEventNotifier | ? ? | 68 - org.apache.camel.camel-core - 2.6.0.fuse-00-00 | >>> Took 103 millis for the exchange on the route : ws-to-sis

developing with maven and pure java libraries all the time, i never thought it could be a problem to issue a few log statements when developing an eclipse plugin. but it looks like in the imaginary of an eclipse developer everything is always inside the eclipse environment and nothing is outside the eclipse universe. if you search for the above headline using google, one of the first articles you’ll find is one about the “platform logging facility”. but what about 3rd libraries? they cannot use an eclipse-based logging framework. in my libraries i use the slf4j api and leave it up to the user to decide what logging implementation (log4j, logback, jdk) he or she wants to use. and that’s exactly what i want to do in eclipse. it was hard to figure out exactly how to do it, but here are the pieces of that puzzle. phase 1: development this describes the steps during the development phase of your custom plugin. step 1: get your libaries into a p2 repository everything you want to use in eclipse has to be installed from a p2 repository. but most of the libaries i use are in a maven repository. as far as i know there is no such thing as a main p2 repository similar to the “maven central,” and all libraries i found in p2 repositories were pretty old. so you have to create one by yourself. luckily there is a maven plugin called p2-maven-plugin that converts all your maven jars into a single p2 repository. you can upload the plugin to a folder of your website or simply install it from your local hard drive. for this example you’ll need the following libraries: org.slf4j:slf4j-api:1.6.6 org.slf4j:slf4j-log4j12:1.6.6 log4j:log4j:1.2.17 org.ops4j.pax.logging:pax-logging-api:1.7.0 org.ops4j.pax.logging:pax-logging-service:1.7.0 org.ops4j.pax.confman:pax-confman-propsloader:0.2.2 format “groupid:artifactid:version” is as used for the “p2-maven-plugin.” to skip this step you could also use http://www.fuin.org/p2-repository/ . step 2: install the slf4j api in the eclipse ide select “help / install new software…”. add the p2 repository url and install the “slf4j-api”—you could directly use the folder from step 1 with a file url like this: “file:/pathtoyour/p2-repository/”. add the freshly installed “slf4j.api” to your manifest.mf. start using slf4j logs in your code as usual. phase 2: production this describes the tasks a user of your custom plugin has to complete to start logging with log4j. the following assumes that your custom plugin is already installed. step 1: install the log libraries in the eclipse ide select “help / install new software…”. install the “equinox target components” from the eclipse update site. add the p2 repository url and install the following plugins: apache log4j ops4j pax confman–properties loader ops4j pax logging–api ops4j pax logging–service step 2: configure pax logging set the location for your log configuration in the “eclipse.ini” as “vmarg" … -vmargs -xms40m -xmx512m -dbundles.configuration.location= … create a folder named “services” in the above “config-dir.” create log4j properties named “org.ops4j.pax.logging.properties” in “services.” log4j.rootlogger=info, file log4j.appender.file=org.apache.log4j.fileappender log4j.appender.file.file=/example.log log4j.appender.file.layout=org.apache.log4j.patternlayout log4j.appender.file.layout.conversionpattern=%d{yyyy/mm/dd hh:mm:ss,sss} [%t] %-5p %c %x - %m%n log4j.logger.your.package=debug step 3: activate pax logging open the “console” view. select the “host osgi console.” start the following bundles: start org.eclipse.equinox.cm start org.ops4j.pax.logging.pax-logging-api start org.ops4j.pax.logging.pax-logging-service start org.ops4j.pax.configmanager now you should be able to see your log statements in the configured “example.log” file. step 4: changing the configuration if you want to change the configuration in the “org.ops4j.pax.logging.properties”, simply restart the pax configmanager in the osgi console stop org.ops4j.pax.configmanager start org.ops4j.pax.configmanager happy logging!

Big Data Sets’ Processing is one of the most important problem in the software world. Spring Batch is a lightweight and robust batch framework to process the data sets. Spring Batch Framework offers ‘TaskletStep Oriented’ and ‘Chunk Oriented’ processing style. In this article, Chunk Oriented Processing Model is explained. Also, TaskletStep Oriented Processing in Spring Batch Article is definitely suggested to investigate how to develop TaskletStep Oriented Processing in Spring Batch. Chunk Oriented Processing Feature has come with Spring Batch v2.0. It refers to reading the data one at a time, and creating ‘chunks’ that will be written out, within a transaction boundary. One item is read from an ItemReader, handed to an ItemProcessor, and written. Once the number of items read equals the commit interval, the entire chunk is written out via the ItemWriter, and then the transaction is committed. Basically, this feature should be used if at least one data item’ s reading and writing is required. Otherwise, TaskletStep Oriented processing can be used if the data item’ s only reading or writing is required. Chunk Oriented Processing model exposes three important interface as ItemReader, ItemProcessor and ItemWriter via org.springframework.batch.item package. ItemReader : This interface is used for providing the data. It reads the data which will be processed. ItemProcessor : This interface is used for item transformation. It processes input object and transforms to output object. ItemWriter : This interface is used for generic output operations. It writes the datas which are transformed by ItemProcessor. For example, the datas can be written to database, memory or outputstream (etc). In this sample application, we will write to database. Let us take a look how to develop Chunk Oriented Processing Model. Used Technologies : JDK 1.7.0_09 Spring 3.1.3 Spring Batch 2.1.9 Hibernate 4.1.8 Tomcat JDBC 7.0.27 MySQL 5.5.8 MySQL Connector 5.1.17 Maven 3.0.4 Step 1 : Create Maven Project A maven project is created as below. (It can be created by using Maven or IDE Plug-in). Step 2: Libraries A new USER Table is created by executing below script: CREATE TABLE ONLINETECHVISION.USER ( id int(11) NOT NULL AUTO_INCREMENT, name varchar(45) NOT NULL, surname varchar(45) NOT NULL, PRIMARY KEY (`id`) ); Step 3: Libraries Firstly, dependencies are added to Maven’ s pom.xml. 3.1.3.RELEASE 2.1.9.RELEASE org.springframework spring-core ${spring.version} org.springframework spring-context ${spring.version} org.springframework spring-tx ${spring.version} org.springframework spring-orm ${spring.version} org.springframework.batch spring-batch-core ${spring-batch.version} org.hibernate hibernate-core 4.1.8.Final org.apache.tomcat tomcat-jdbc 7.0.27 mysql mysql-connector-java 5.1.17 log4j log4j 1.2.16 maven-compiler-plugin(Maven Plugin) is used to compile the project with JDK 1.7 org.apache.maven.plugins maven-compiler-plugin 3.0 1.7 1.7 The following Maven plugin can be used to create runnable-jar, org.apache.maven.plugins maven-shade-plugin 2.0 package shade 1.7 1.7 com.onlinetechvision.exe.Application META-INF/spring.handlers META-INF/spring.schemas Step 4 : Create User Entity User Entity is created. This entity will be stored after processing. package com.onlinetechvision.user; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.Table; /** * User Entity * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ @Entity @Table(name="USER") public class User { private int id; private String name; private String surname; @Id @GeneratedValue(strategy=GenerationType.AUTO) @Column(name="ID", unique = true, nullable = false) public int getId() { return id; } public void setId(int id) { this.id = id; } @Column(name="NAME", unique = true, nullable = false) public String getName() { return name; } public void setName(String name) { this.name = name; } @Column(name="SURNAME", unique = true, nullable = false) public String getSurname() { return surname; } public void setSurname(String surname) { this.surname = surname; } @Override public String toString() { StringBuffer strBuff = new StringBuffer(); strBuff.append("id : ").append(getId()); strBuff.append(", name : ").append(getName()); strBuff.append(", surname : ").append(getSurname()); return strBuff.toString(); } } Step 5 : Create IUserDAO Interface IUserDAO Interface is created to expose data access functionality. package com.onlinetechvision.user.dao; import java.util.List; import com.onlinetechvision.user.User; /** * User DAO Interface * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public interface IUserDAO { /** * Adds User * * @param User user */ void addUser(User user); /** * Gets User List * */ List getUsers(); } Step 6 : Create UserDAO IMPL UserDAO Class is created by implementing IUserDAO Interface. package com.onlinetechvision.user.dao; import java.util.List; import org.hibernate.SessionFactory; import com.onlinetechvision.user.User; /** * User DAO * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class UserDAO implements IUserDAO { private SessionFactory sessionFactory; /** * Gets Hibernate Session Factory * * @return SessionFactory - Hibernate Session Factory */ public SessionFactory getSessionFactory() { return sessionFactory; } /** * Sets Hibernate Session Factory * * @param SessionFactory - Hibernate Session Factory */ public void setSessionFactory(SessionFactory sessionFactory) { this.sessionFactory = sessionFactory; } /** * Adds User * * @param User user */ @Override public void addUser(User user) { getSessionFactory().getCurrentSession().save(user); } /** * Gets User List * * @return List - User list */ @SuppressWarnings({ "unchecked" }) @Override public List getUsers() { List list = getSessionFactory().getCurrentSession().createQuery("from User").list(); return list; } } Step 7 : Create IUserService Interface IUserService Interface is created for service layer. package com.onlinetechvision.user.service; import java.util.List; import com.onlinetechvision.user.User; /** * * User Service Interface * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public interface IUserService { /** * Adds User * * @param User user */ void addUser(User user); /** * Gets User List * * @return List - User list */ List getUsers(); } Step 8 : Create UserService IMPL UserService Class is created by implementing IUserService Interface. package com.onlinetechvision.user.service; import java.util.List; import org.springframework.transaction.annotation.Transactional; import com.onlinetechvision.user.User; import com.onlinetechvision.user.dao.IUserDAO; /** * * User Service * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ @Transactional(readOnly = true) public class UserService implements IUserService { IUserDAO userDAO; /** * Adds User * * @param User user */ @Transactional(readOnly = false) @Override public void addUser(User user) { getUserDAO().addUser(user); } /** * Gets User List * */ @Override public List getUsers() { return getUserDAO().getUsers(); } public IUserDAO getUserDAO() { return userDAO; } public void setUserDAO(IUserDAO userDAO) { this.userDAO = userDAO; } } Step 9 : Create TestReader IMPL TestReader Class is created by implementing ItemReader Interface. This class is called in order to read items. When read method returns null, reading operation is completed. The following steps explains with details how to be executed firstJob. The commit-interval value of firstjob is 2 and the following steps are executed : 1) firstTestReader is called to read first item(firstname_0, firstsurname_0) 2) firstTestReader is called again to read second item(firstname_1, firstsurname_1) 3) testProcessor is called to process first item(FIRSTNAME_0, FIRSTSURNAME_0) 4) testProcessor is called to process second item(FIRSTNAME_1, FIRSTSURNAME_1) 5) testWriter is called to write first item(FIRSTNAME_0, FIRSTSURNAME_0) to database 6) testWriter is called to write second item(FIRSTNAME_1, FIRSTSURNAME_1) to database 7) first and second items are committed and the transaction is closed. 8) firstTestReader is called to read third item(firstname_2, firstsurname_2) 9) maxIndex value of firstTestReader is 3. read method returns null and item reading operation is completed. 10) testProcessor is called to process third item(FIRSTNAME_2, FIRSTSURNAME_2) 11) testWriter is called to write first item(FIRSTNAME_2, FIRSTSURNAME_2) to database 12) third item is committed and the transaction is closed. firstStep is completed with COMPLETED status and secondStep is started. secondJob and thirdJob are executed in the same way. package com.onlinetechvision.item; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.NonTransientResourceException; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; import com.onlinetechvision.user.User; /** * TestReader Class is created to read items which will be processed * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class TestReader implements ItemReader { private int index; private int maxIndex; private String namePrefix; private String surnamePrefix; /** * Reads items one by one * * @return User * * @throws Exception * @throws UnexpectedInputException * @throws ParseException * @throws NonTransientResourceException * */ @Override public User read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { User user = new User(); user.setName(getNamePrefix() + "_" + index); user.setSurname(getSurnamePrefix() + "_" + index); if(index > getMaxIndex()) { return null; } incrementIndex(); return user; } /** * Increments index which defines read-count * * @return int * */ private int incrementIndex() { return index++; } public int getMaxIndex() { return maxIndex; } public void setMaxIndex(int maxIndex) { this.maxIndex = maxIndex; } public String getNamePrefix() { return namePrefix; } public void setNamePrefix(String namePrefix) { this.namePrefix = namePrefix; } public String getSurnamePrefix() { return surnamePrefix; } public void setSurnamePrefix(String surnamePrefix) { this.surnamePrefix = surnamePrefix; } } Step 10 : Create FailedCaseTestReader IMPL FailedCaseTestReader Class is created in order to simulate the failed job status. In this sample application, when thirdJob is processed at fifthStep, failedCaseTestReader is called and exception is thrown so its status will be FAILED. package com.onlinetechvision.item; import org.springframework.batch.item.ItemReader; import org.springframework.batch.item.NonTransientResourceException; import org.springframework.batch.item.ParseException; import org.springframework.batch.item.UnexpectedInputException; import com.onlinetechvision.user.User; /** * FailedCaseTestReader Class is created in order to simulate the failed job status. * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class FailedCaseTestReader implements ItemReader { private int index; private int maxIndex; private String namePrefix; private String surnamePrefix; /** * Reads items one by one * * @return User * * @throws Exception * @throws UnexpectedInputException * @throws ParseException * @throws NonTransientResourceException * */ @Override public User read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException { User user = new User(); user.setName(getNamePrefix() + "_" + index); user.setSurname(getSurnamePrefix() + "_" + index); if(index >= getMaxIndex()) { throw new Exception("Unexpected Error!"); } incrementIndex(); return user; } /** * Increments index which defines read-count * * @return int * */ private int incrementIndex() { return index++; } public int getMaxIndex() { return maxIndex; } public void setMaxIndex(int maxIndex) { this.maxIndex = maxIndex; } public String getNamePrefix() { return namePrefix; } public void setNamePrefix(String namePrefix) { this.namePrefix = namePrefix; } public String getSurnamePrefix() { return surnamePrefix; } public void setSurnamePrefix(String surnamePrefix) { this.surnamePrefix = surnamePrefix; } } Step 11 : Create TestProcessor IMPL TestProcessor Class is created by implementing ItemProcessor Interface. This class is called to process items. User item is received from TestReader, processed and returned to TestWriter. package com.onlinetechvision.item; import java.util.Locale; import org.springframework.batch.item.ItemProcessor; import com.onlinetechvision.user.User; /** * TestProcessor Class is created to process items. * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class TestProcessor implements ItemProcessor { /** * Processes items one by one * * @param User user * @return User * @throws Exception * */ @Override public User process(User user) throws Exception { user.setName(user.getName().toUpperCase(Locale.ENGLISH)); user.setSurname(user.getSurname().toUpperCase(Locale.ENGLISH)); return user; } } Step 12 : Create TestWriter IMPL TestWriter Class is created by implementing ItemWriter Interface. This class is called to write items to DB, memory etc… package com.onlinetechvision.item; import java.util.List; import org.springframework.batch.item.ItemWriter; import com.onlinetechvision.user.User; import com.onlinetechvision.user.service.IUserService; /** * TestWriter Class is created to write items to DB, memory etc... * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class TestWriter implements ItemWriter { private IUserService userService; /** * Writes items via list * * @throws Exception * */ @Override public void write(List userList) throws Exception { for(User user : userList) { getUserService().addUser(user); } System.out.println("User List : " + getUserService().getUsers()); } public IUserService getUserService() { return userService; } public void setUserService(IUserService userService) { this.userService = userService; } } Step 13 : Create FailedStepTasklet Class FailedStepTasklet is created by implementing Tasklet Interface. It illustrates business logic in failed step. package com.onlinetechvision.tasklet; import org.apache.log4j.Logger; import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; /** * FailedStepTasklet Class illustrates a failed job. * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class FailedStepTasklet implements Tasklet { private static final Logger logger = Logger.getLogger(FailedStepTasklet.class); private String taskResult; /** * Executes FailedStepTasklet * * @param StepContribution stepContribution * @param ChunkContext chunkContext * @return RepeatStatus * @throws Exception * */ public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception { logger.debug("Task Result : " + getTaskResult()); throw new Exception("Error occurred!"); } public String getTaskResult() { return taskResult; } public void setTaskResult(String taskResult) { this.taskResult = taskResult; } } Step 14 : Create BatchProcessStarter Class BatchProcessStarter Class is created to launch the jobs. Also, it logs their execution results. package com.onlinetechvision.spring.batch; import org.apache.log4j.Logger; import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParametersBuilder; import org.springframework.batch.core.JobParametersInvalidException; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException; import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; import org.springframework.batch.core.repository.JobRepository; import org.springframework.batch.core.repository.JobRestartException; /** * BatchProcessStarter Class launches the jobs and logs their execution results. * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class BatchProcessStarter { private static final Logger logger = Logger.getLogger(BatchProcessStarter.class); private Job firstJob; private Job secondJob; private Job thirdJob; private JobLauncher jobLauncher; private JobRepository jobRepository; /** * Starts the jobs and logs their execution results. * */ public void start() { JobExecution jobExecution = null; JobParametersBuilder builder = new JobParametersBuilder(); try { getJobLauncher().run(getFirstJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getFirstJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getSecondJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getSecondJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); getJobLauncher().run(getThirdJob(), builder.toJobParameters()); jobExecution = getJobRepository().getLastJobExecution(getThirdJob().getName(), builder.toJobParameters()); logger.debug(jobExecution.toString()); } catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) { logger.error(e); } } public Job getFirstJob() { return firstJob; } public void setFirstJob(Job firstJob) { this.firstJob = firstJob; } public Job getSecondJob() { return secondJob; } public void setSecondJob(Job secondJob) { this.secondJob = secondJob; } public Job getThirdJob() { return thirdJob; } public void setThirdJob(Job thirdJob) { this.thirdJob = thirdJob; } public JobLauncher getJobLauncher() { return jobLauncher; } public void setJobLauncher(JobLauncher jobLauncher) { this.jobLauncher = jobLauncher; } public JobRepository getJobRepository() { return jobRepository; } public void setJobRepository(JobRepository jobRepository) { this.jobRepository = jobRepository; } } Step 15 : Create dataContext.xml jdbc.properties, is created. It defines data-source informations and is read via dataContext.xml jdbc.db.driverClassName=com.mysql.jdbc.Driver jdbc.db.url=jdbc:mysql://localhost:3306/onlinetechvision jdbc.db.username=root jdbc.db.password=root jdbc.db.initialSize=10 jdbc.db.minIdle=3 jdbc.db.maxIdle=10 jdbc.db.maxActive=10 jdbc.db.testWhileIdle=true jdbc.db.testOnBorrow=true jdbc.db.testOnReturn=true jdbc.db.initSQL=SELECT 1 FROM DUAL jdbc.db.validationQuery=SELECT 1 FROM DUAL jdbc.db.timeBetweenEvictionRunsMillis=30000 Step 16 : Create dataContext.xml Spring Configuration file, dataContext.xml, is created. It covers dataSource, sessionFactory and transactionManager definitions. com.onlinetechvision.user.User org.hibernate.dialect.MySQLDialect true Step 17 : Create jobContext.xml Spring Configuration file, jobContext.xml, is created. It covers jobRepository, jobLauncher, item reader, item processor, item writer, tasklet and job definitions. Step 18 : Create applicationContext.xml Spring Configuration file, applicationContext.xml, is created. It covers bean definitions. Step 19 : Create Application Class Application Class is created to run the application. package com.onlinetechvision.exe; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import com.onlinetechvision.spring.batch.BatchProcessStarter; /** * Application Class starts the application. * * @author onlinetechvision.com * @since 10 Dec 2012 * @version 1.0.0 * */ public class Application { /** * Starts the application * * @param String[] args * */ public static void main(String[] args) { ApplicationContext appContext = new ClassPathXmlApplicationContext("applicationContext.xml"); BatchProcessStarter batchProcessStarter = (BatchProcessStarter)appContext.getBean("batchProcessStarter"); batchProcessStarter.start(); } } Step 20 : Build Project After OTV_SpringBatch_Chunk_Oriented_Processing Project is built, OTV_SpringBatch_Chunk_Oriented_Processing-0.0.1-SNAPSHOT.jar will be created. STEP 21 : RUN PROJECT After created OTV_SpringBatch_Chunk_Oriented_Processing-0.0.1-SNAPSHOT.jar file is run, the following database and console output logs will be shown : Database screenshot : First Job’ s console output : 16.12.2012 19:30:41 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=firstJob]] launched with the following parameters: [{}] 16.12.2012 19:30:41 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=0, version=0, startTime=null, endTime=null, lastUpdated=Sun Dec 16 19:30:41 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{}], Job=[firstJob]] User List : [id : 181, name : FIRSTNAME_0, surname : FIRSTSURNAME_0, id : 182, name : FIRSTNAME_1, surname : FIRSTSURNAME_1, id : 183, name : FIRSTNAME_2, surname : FIRSTSURNAME_2, id : 184, name : SECONDNAME_0, surname : SECONDSURNAME_0, id : 185, name : SECONDNAME_1, surname : SECONDSURNAME_1, id : 186, name : SECONDNAME_2, surname : SECONDSURNAME_2] 16.12.2012 19:30:42 DEBUG (BatchProcessStarter.java:43) - JobExecution: id=0, version=2, startTime=Sun Dec 16 19:30:41 GMT 2012, endTime=Sun Dec 16 19:30:42 GMT 2012, lastUpdated=Sun Dec 16 19:30:42 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=0, version=0, JobParameters=[{}], Job=[firstJob]] Second Job’ s console output : 16.12.2012 19:30:42 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=secondJob]] launched with the following parameters: [{}] 16.12.2012 19:30:42 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=1, version=0, startTime=null, endTime=null, lastUpdated=Sun Dec 16 19:30:42 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{}], Job=[secondJob]] User List : [id : 181, name : FIRSTNAME_0, surname : FIRSTSURNAME_0, id : 182, name : FIRSTNAME_1, surname : FIRSTSURNAME_1, id : 183, name : FIRSTNAME_2, surname : FIRSTSURNAME_2, id : 184, name : SECONDNAME_0, surname : SECONDSURNAME_0, id : 185, name : SECONDNAME_1, surname : SECONDSURNAME_1, id : 186, name : SECONDNAME_2, surname : SECONDSURNAME_2, id : 187, name : THIRDNAME_0, surname : THIRDSURNAME_0, id : 188, name : THIRDNAME_1, surname : THIRDSURNAME_1, id : 189, name : THIRDNAME_2, surname : THIRDSURNAME_2, id : 190, name : THIRDNAME_3, surname : THIRDSURNAME_3, id : 191, name : FOURTHNAME_0, surname : FOURTHSURNAME_0, id : 192, name : FOURTHNAME_1, surname : FOURTHSURNAME_1, id : 193, name : FOURTHNAME_2, surname : FOURTHSURNAME_2, id : 194, name : FOURTHNAME_3, surname : FOURTHSURNAME_3] 16.12.2012 19:30:42 DEBUG (BatchProcessStarter.java:47) - JobExecution: id=1, version=2, startTime=Sun Dec 16 19:30:42 GMT 2012, endTime=Sun Dec 16 19:30:42 GMT 2012, lastUpdated=Sun Dec 16 19:30:42 GMT 2012, status=COMPLETED, exitStatus=exitCode=COMPLETED;exitDescription=, job=[JobInstance: id=1, version=0, JobParameters=[{}], Job=[secondJob]] Third Job’ s console output : 16.12.2012 19:30:42 INFO (SimpleJobLauncher.java:118) - Job: [FlowJob: [name=thirdJob]] launched with the following parameters: [{}] 16.12.2012 19:30:42 DEBUG (AbstractJob.java:278) - Job execution starting: JobExecution: id=2, version=0, startTime=null, endTime=null, lastUpdated=Sun Dec 16 19:30:42 GMT 2012, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{}], Job=[thirdJob]] 16.12.2012 19:30:42 DEBUG (TransactionTemplate.java:159) - Initiating transaction rollback on application exception org.springframework.batch.repeat.RepeatException: Exception in batch process; nested exception is java.lang.Exception: Unexpected Error! ... 16.12.2012 19:30:43 DEBUG (BatchProcessStarter.java:51) - JobExecution: id=2, version=2, startTime=Sun Dec 16 19:30:42 GMT 2012, endTime=Sun Dec 16 19:30:43 GMT 2012, lastUpdated=Sun Dec 16 19:30:43 GMT 2012, status=FAILED, exitStatus=exitCode=FAILED;exitDescription=, job=[JobInstance: id=2, version=0, JobParameters=[{}], Job=[thirdJob]] Step 22 : Download https://github.com/erenavsarogullari/OTV_SpringBatch_Chunk_Oriented_Processing REFERENCES : Chunk Oriented Processing in Spring Batch

It is easy to do - a few button clicks (generally) - but the button location is damn unintuitive. So, this is what you have got to do Go to "Help" menu item. Click on "About ..." button (why on earth should I click that when I am trying to un-install a plugin. By the way, the menu item just above "About ..." is "Install New Software ...". Would it have been too much pain to have a "Manage plugins" and / or "Un-install plugins" right underneath it?) A form opens up. At the bottom of it there is button "Installation details". Click that. (Again, why on earth would anyone think "Installation details" would have anything to do with un-installing stuff. I would have expected only a static display of stuff that are already installed.) Another multi tabbed form opens up (Anyone keeping count of the number of windows opened already. This is the 3rd window by now, including the parent editor window) which shows all the installed plugins. If you select any of the installed plugins, a button to "uninstall" becomes available. Click that and you should be able to un-install and after a restart everything should be fine. My interest in software and IT has always been much more than a 9 to 5 job (and I am sure there is a huge population that it holds equally true for). I have always wanted software to be efficient and beautiful apart from doing it's job. However, it took an excellent session on usability (which I joined only with casual curiosity but left with renewed interest in the subject and admiration for David Travis who delivered the course) to get me to start looking at all software with an "user's" perspective. And I was surprised with what I found and how it changed my coding. I have been using Eclipse and STS for years now (nearing a decade now) and I absolutely love these software. However when you start looking at them as a "user" and not only as a developer, there are quite a few usability opportunities of improvement that meets the eye. This article - apart from helping folks looking to un-install plugins in Eclipse - is also intended at folks who design Eclipse - just a humble request to consider this also as a usability improvement.

When you deploy your Java web application to the Apache Tomcat server, via Eclipse, by default the web app will be deployed under {YOUR_ECLIPSE_WORKSPACE}\.metadata\.plugins\org.eclipse.wst.server.core\tmp{a-number}\wtpwebapps. Suppose if you want to deploy your web app to a location that is easily navigable, follow these steps. First make sure you have removed all the web apps that are currently added to your server instance (In servers view, right click on the server name and then Add and Remove). And then double click on the server instance in servers view which will open up that server’s configuration page. On that page, see under Server Locations and select either the option Use Tomcat Installation to deploy the web app under the directory where the Tomcat server is installed or Use custom location to manually specify. Save, Re-add the web application and then Publish. Now the deployed web app will be under the directory of your choice.

In this example I will show how to test Spring Integration flow using Mock SftpServer.

for people in hurry, get the latest code and the steps in github . to run the junit test, run “mvn test” and understand the test flow. introduction: fakeftpserver in this spring integration fakeftpserver example, i will demonstrate using spring fakeftpserver to junit test a spring integration flow. this is an interesting topic, and there are few articles on unit testing file transfers , which gives some insight on this topic. in this blog, we will test a spring integration flow which checks for a list of files, apply a splitter to separate each file and start downloading them into a local location. once the download is complete, it will delete the files on the ftp server. in my next blog, i will show how to do junit testing of spring integration flow with sftp server. spring integration flow spring integration fakeftpserver example in order to use fakeftpserver we need to have maven dependency as below, org.mockftpserver mockftpserver 2.3 test the first step to this is to create a fakeftpserver before every test runs as below, @before public void setup() throws exception { fakeftpserver = new fakeftpserver(); fakeftpserver.setservercontrolport(9999); // use any free port filesystem filesystem = new unixfakefilesystem(); filesystem.add(new fileentry(file, contents)); fakeftpserver.setfilesystem(filesystem); useraccount useraccount = new useraccount("user", "password", home_dir); fakeftpserver.adduseraccount(useraccount); fakeftpserver.start(); } @after public void teardown() throws exception { fakeftpserver.stop(); } finally run the junit test case as seen below, @autowired private filedownloadutil downloadutil; @test public void testftpdownload() throws exception { file file = new file("src/test/resources/output"); delete(file); ftpclient client = new ftpclient(); client.connect("localhost", 9999); client.login("user", "password"); string files[] = client.listnames("/dir"); client.help(); logger.debug("before delete" + files[0]); assertequals(1, files.length); downloadutil.downloadfilesfromremotedirectory(); logger.debug("after delete"); files = client.listnames("/dir"); client.help(); assertequals(0, files.length); assertequals(1, file.list().length); } i hope this blog helped.

Use jQuery's UI To drag and drop within a single or multiple lists in AngularJS.

What is a Theory? Functionally, a theory is an alternative to JUnit's parameterized tests. Semantically, a theory encapsulates the tester's understanding of an object's universal behavior. That is, whatever it is that a theory asserts, it is expected to be true for all data. Theories should be especially useful for finding bugs in edge cases. Contrast this with a typical unit test, which asserts that a specific data point will have a specific outcome, and only asserts that. (For this reason, typical unit tests are sometimes called example-based tests to contrast them with theories.) This is very nice in theory, but... A @Theory needs a special JUnit runner (Theories.class). So if you want to use Spring and/or Mockito together with theories, you have a problem. All of these features need a different runner and you can only use one on each test class. The solution For Mockito is easy. Instead of using the @Mock annotiation, you can use the static createMock method. One problem solved. For Spring is a little bit trickier. First of all, you have to use @ContextConfiguration to declare the XML with the bean definitions that you need. But the trickiest part is that you have to tell Spring how to do the autowiring without using its own runner. This can be accomplish adding this line to the @Before method: new TestContextManager(getClass()).prepareTestInstance(this); Basic Usage Example package org.mackenzine.theories; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; import static org.mockito.Mockito.when; import java.util.Date; import org.junit.Before; import org.junit.experimental.theories.DataPoints; import org.junit.experimental.theories.Theories; import org.junit.experimental.theories.Theory; import org.junit.runner.RunWith; import org.mockito.Mockito; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.TestContextManager; @RunWith(Theories.class) @ContextConfiguration("classpath:parser.xml") public class QuoteTheoriesTest { private static String deleteMessage = "deleteMessage"; private static String updateMessage = "updateMessage"; private QuoteFactory factory; private final Event event = Mockito.mock(Event.class); private final Contract contract = Mockito.mock(Contract.class); private final Commodity commodity = Mockito.mock(Commodity.class); @Autowired private Parser parser; @Before public void setUp() throws Exception { factory = new QuoteFactory(); new TestContextManager(getClass()).prepareTestInstance(this); } @DataPoints public static String[] getEventTypes() { return new String[] { updateMessage, deleteMessage }; } @Theory public void shouldCreateQuote(final String message) throws Exception { Date now = new Date(); when(event.getParsedMessage()).thenReturn(parser.parse(message)); when(event.getContract()).thenReturn(contract); when(event.getTradeDate()).thenReturn(now); when(contract.getExternalCode()).thenReturn("externalCode"); when(contract.getCommodity()).thenReturn(commodity); when(commodity.getCommodityCode()).thenReturn("code"); Quote quote = factory.createQuote(event); assertNotNull(quote); assertEquals("code", quote.getCommodityCode()); assertEquals(now, quote.getTradeDate()); } } Sources Definition of Theories: https://blogs.oracle.com/jacobc/entry/junit_theories Original Idea for Parameterized Tests: http://stackoverflow.com/questions/8974977/spring-parameterized-theories-junit-tests Thread on SpringSource: http://forum.springsource.org/showthread.php?78929-Is-Theory-supported Open Issue in SpringSource for Parameterized Tests (not for Theories): https://jira.springsource.org/browse/SPR-5292