When I was learning programming, back in the days of Turbo Pascal, I managed to list files in directory using FindFirst,FindNext and FindClose functions. First I came up with a procedure printing contents of a given directory. You can imagine how proud I was to discover I can actually call that procedure from itself to traverse file system recursively. Well, I didn't know the term recursion back then, but it worked. Similar code in Java would look something like this: public void printFilesRecursively(final File folder) { for (final File entry : listFilesIn(folder)) { if (entry.isDirectory()) { printFilesRecursively(entry); } else { System.out.println(entry.getAbsolutePath()); } } } private File[] listFilesIn(File folder) { final File[] files = folder.listFiles(); return files != null ? files : new File[]{}; } Didn't know File.listFiles() can return null, did ya? That's how it signals I/O errors, like if IOException never existed. But that's not the point. System.out.println() is rarely what we need, thus this method is neither reusable nor composable. It is probably the best counterexample of Open/Closed principle. I can imagine several use cases for recursive traversal of file system: Getting a complete list of all files for display purposes Looking for all files matching given pattern/property (also check out File.list(FilenameFilter)) Searching for one particular file Processing every single file, e.g. sending it over network Every use case above has a unique set of challenges. For example we don't want to build a list of all files because it will take a significant amount of time and memory before we can start processing it. We would like to process files as they are discovered and lazily - by pipe-lining computation (but without clumsy visitor pattern). Also we want to short-circuit searching to avoid unnecessary I/O. Luckily in Java 8 some of these issues can be addressed with streams: final File home = new File(FileUtils.getUserDirectoryPath()); final Stream files = Files.list(home.toPath()); files.forEach(System.out::println); Remember that Files.list(Path) (new in Java 8) does not look into subdirectories - we'll fix that later. The most important lesson here is: Files.list() returns a Stream - a value that we can pass around, compose, map, filter, etc. It's extremely flexible, e.g. it's fairly simple to count how many files I have in a directory per extension: import org.apache.commons.io.FilenameUtils; //... final File home = new File(FileUtils.getUserDirectoryPath()); final Stream files = Files.list(home.toPath()); final Map> byExtension = files .filter(path -> !path.toFile().isDirectory()) .collect(groupingBy(path -> getExt(path))); byExtension. forEach((extension, matchingFiles) -> System.out.println( extension + "\t" + matchingFiles.size())); //... private String getExt(Path path) { return FilenameUtils.getExtension(path.toString()).toLowerCase(); } OK, just another API, you might say. But it becomes really interesting once we need to go deeper, recursively traversing subdirectories. One amazing feature of streams is that you can combine them with each other in various ways. Old Scala saying "flatMap that shit" is applicable here as well, check out this recursive Java 8 code: //WARNING: doesn't compile, yet: private static Stream filesInDir(Path dir) { return Files.list(dir) .flatMap(path -> path.toFile().isDirectory() ? filesInDir(path) : singletonList(path).stream()); } Stream lazily produced by filesInDir() contains all files within directory including subdirectories. You can use it as any other stream by calling map(), filter(), anyMatch(), findFirst(), etc. But how does it really work?flatMap() is similar to map() but while map() is a straightforward 1:1 transformation, flatMap() allows replacing single entry in input Stream with multiple entries. If we had used map(), we would have end up with Stream>(or maybe Stream>). But flatMap() flattens this structure, in a way exploding inner entries. Let's see a simple example. Imagine Files.list() returned two files and one directory. For files flatMap() receives a one-element stream with that file. We can't simply return that file, we have to wrap it, but essentially this is no-operation. It gets way more interesting for a directory. In that case we call filesInDir() recursively. As a result we get a stream of contents of that directory, which we inject into our outer stream. Code above is short, sweet and... doesn't compile. These pesky checked exceptions again. Here is a fixed code, wrapping checked exceptions for sanity: public static Stream filesInDir(Path dir) { return listFiles(dir) .flatMap(path -> path.toFile().isDirectory() ? filesInDir(path) : singletonList(path).stream()); } private static Stream listFiles(Path dir) { try { return Files.list(dir); } catch (IOException e) { throw Throwables.propagate(e); } } Unfortunately this quite elegant code is not lazy enough. flatMap() evaluates eagerly, thus it always traverses all subdirectories, even if we barely ask for first file. You can try with my tiny LazySeq library that tries to provide even lazier abstraction, similar to streams in Scala or lazy-seq in Clojure. But even standard JDK 8 solution might be really helpful and simplify your code significantly.

We had a lengthy discussion recently during code review whether scala.Option.fold() is idiomatic and clever or maybe unreadable and tricky? Let's first describe what the problem is. Option.fold does two things: maps a function f overOption's value (if any) or returns an alternative alt if it's absent. Using simple pattern matching we can implement it as follows: val option: Option[T] = //... def alt: R = //... def f(in: T): R = //... val x: R = option match { case Some(v) => f(v) case None => alt } If you prefer one-liner, fold is actually a combination of map and getOrElse val x: R = option map f getOrElse alt Or, if you are a C programmer that still wants to write in C, but using Scala compiler: val x: R = if (option.isDefined) f(option.get) else alt Interestingly this is similar to how fold() is actually implemented, but that's an implementation detail. OK, all of the above can be replaced with single Option.fold(): val x: R = option.fold(alt)(f) Technically you can even use /: and \: operators (alt /: option) - but that would be simply masochistic. I have three problems with option.fold() idiom. First of all - it's anything but readable. We are folding (reducing) over Option - which doesn't really make much sense. Secondly it reverses the ordinary positive-then-negative-case flow by starting with failure (absence, alt) condition followed by presence block (f function; see also: Refactoring map-getOrElse to fold). Interestingly this method would work great for me if it was named mapOrElse: ** * Hypothetical in Option */ def mapOrElse[B](f: A => B, alt: => B): B = this map f getOrElse alt Actually there is already such method in Scalaz, called OptionW.cata. cata. Here is whatMartin Odersky has to say about it: "I personally find methods like cata that take two closures as arguments are often overdoing it. Do you really gain in readability over map + getOrElse? Think of a newcomer to your code[...]"While cata has some theoretical background, Option.fold just sounds like a random name collision that doesn't bring anything to the table, apart from confusion. I know what you'll say, that TraversableOnce has fold and we are sort-of doing the same thing. Why it's a random collision rather than extending the contract described inTraversableOnce? fold() method in Scala collections typically just delegates to one offoldLeft()/foldRight() (the one that works better for given data structure), thus it doesn't guarantee order and folding function has to be associative. But inOption.fold() the contract is different: folding function takes just one parameter rather than two. If you read my previous article about folds you know that reducing function always takes two parameters: current element and accumulated value (initial value during first iteration). But Option.fold() takes just one parameter: current Option value! This breaks the consistency, especially when realizing Option.foldLeft() andOption.foldRight() have correct contract (but it doesn't mean they are more readable). The only way to understand folding over option is to imagine Option as a sequence with0 or 1 elements. Then it sort of makes sense, right? No. def double(x: Int) = x * 2 Some(21).fold(-1)(double) //OK: 42 None.fold(-1)(double) //OK: -1 but: Some(21).toList.fold(-1)(double) : error: type mismatch; found : Int => Int required: (Int, Int) => Int Some(21).toList.fold(-1)(double) ^ If we treat Option[T] as a List[T], awkward Option.fold() breaks because it has different type than TraversableOnce.fold(). This is my biggest concern. I can't understand why folding wasn't defined in terms of the type system (trait?) and implemented strictly. As an example take a look at: Data.Foldable in Haskell (advanced) Data.Foldable typeclass describes various flavours of folding in Haskell. There are familiar foldl/foldr/foldl1/foldr1, in Scala namedfoldLeft/foldRight/reduceLeft/reduceRight accordingly. They have the same type as Scala and behave unsurprisingly with all types that you can fold over, including Maybe, lists, arrays, etc. There is also a function named fold, but it has a completely different meaning: class Foldable t where fold :: Monoid m => t m -> m While other folds are quite complex, this one barely takes a foldable container of ms (which have to be Monoids) and returns the same Monoid type. A quick recap: a type can be aMonoid if there exists a neutral value of that type and an operation that takes two values and produces just one. Applying that function with one of the arguments being neutral value yields the other argument. String ([Char]) is a good example with empty string being neutral value (mempty) and string concatenation being such operation (mappend). Notice that there are two different ways you can construct monoids for numbers: under addition with neutral value being 0 (x + 0 == 0 + x == x for any x) and under multiplication with neutral 1 (x * 1 == 1 * x == x for any x). Let's stick to strings. If I fold empty list of strings, I'll get an empty string. But when a list contains many elements, they are being concatenated: > fold ([] :: [String]) "" > fold [] :: String "" > fold ["foo", "bar"] "foobar" In the first example we have to explicitly say what is the type of empty list []. Otherwise Haskell compiler can't figure out what is the type of elements in a list, thus which monoid instance to choose. In second example we declare that whatever is returned from fold [], it should be a String. From that the compiler infers that [] actually must have a type of [String]. Last fold is the simplest: the program folds over elements in list and concatenates them because concatenation is the operation defined in Monoid Stringtypeclass instance. Back to options (or more precisely Maybe). Folding over Maybe monad having type parameter being Monoid (I can't believe I just said it) has an interesting interpretation: it either returns value inside Maybe or a default Monoid value: > fold (Just "abc") "abc" > fold Nothing :: String "" Just "abc" is same as Some("abc") in Scala. You can see here that if Maybe Stringis Nothing, neutral String monoid value is returned, that is an empty string. Summary Haskell shows that folding (also over Maybe) can be at least consistent. In ScalaOption.fold is unrelated to List.fold, confusing and unreadable. I advise avoiding it and staying with slightly more verbose map/getOrElse transformations or pattern matching. PS: Did I mention there is also Either.fold() (with even different contract) but noTry.fold()?

This article was originally written by Edwin Dalorzo at the Informatech CR Blog. One of those aspects of Node.js that took me a while to fully understand initially is how to properly declare modules. At first it looks kind of obvious and intuitively simple, but later, you realize you can expose different types of objects like functions, constructors, properties or entire object instances. So I have decided to write this article with different examples and techniques I learned while writing different type of modules in Node.js. On Modules, Import and Export Let’s start by the most obvious and simple thing. Something probably everyone learns since the first day of work with Node: every code file is considered a module. The variables, properties, functions, constructors that we declared in it are private to the module and other modules cannot gain access to them or use them unless the programmer of the module explicitly expose them to the public; namely everything we declare inside a module is encapsulated and hidden from the outside world by default unless explicitly stated otherwise. To expose something the programmer has access to a special object called module, which has a special property called exports. Everything that you publish in the module.exports object is made publicly available to other modules. For instance, in the code below, the variable pi is inaccessible to any other modules but foo.js, whereas the property named bar is made publicly available to any other modules importing the module foo.js. Note that this is a fundamental difference from JavaScript in Node.js when compared with JavaScript as executed in a browser where objects are publicly exposed in a global object (i.e. window). //module foo.js var pi = 3.14; module.exports.bar = 'Hello World'; Now a second module baz.js can “import” the module foo.js and gain access to the property bar. In Node, we achieve this effect by means of using a global function named require. Somewhat as follows: //module baz.js var foo = require('./foo'); console.log(foo.bar); //yields Hello World Technique 1 – Extending exports Object with Additional Functionality So, one technique to expose the functionality in a module consists in adding functions and properties to the module.exports object. When this is the case, Node provides a direct access to the exports object to make things simpler for us. For instance: //module foo.js exports.serviceOne = function(){ }; exports.serviceTwo = function(){ }; exports.serviceThree = function(){ }; And as you might expect, the users of this module, at importing it, would obtain a reference to the exports object and by this they would gain access to all the functionality exposed in it. //module bar.js var foo = require('./foo'); foo.serviceOne(); foo.serviceTwo(); foo.serviceThree(); Technique 2 – Substitute Default exports Object with Another Object By this point you probably suspect that given the fact that module.exports is just an object that exposes the public part of a module then we could probably define our own object and then replace the default module.exports object with our own. For instance: //module foo.js var service = { serviceOne: function(){ }, serviceTwo: function(){ }, serviceThree = function(){ } }; module.exports = service; The code in this last example would behave exactly as the code in the previous example, it’s just that this time we have explicitly created our exported object instead of using the one provided by default by Node. Technique 3 – Substitute Default exports Object with a Constructor Function In the examples so far we have always used an instance of an object as our exposed target. However there are occasions in which it may seem more convenient to allow the user to create as many instances of a given type as she wants. Nothing prevents us from replacing the module.exports object with other types of objects like a constructor function. In the example below we expose a constructor which the user can use to create many instances of the Foo type. //module Foo.js function Foo(name){ this.name = name; } Foo.prototype.serviceOne = function(){ }; Foo.prototype.serviceTwo = function(){ }; Foo.prototype.serviceThree = function(){ }; module.exports = Foo; And the user of this module can simply do something like this: //module bar.js var Foo = require('./Foo'); var foo = new Foo('Obi-wan'); foo.serviceOne(); foo.serviceTwo(); foo.serviceThree(); Technique 4 – Substitute Default exports Object with Plain Old Function It is easy to imagine now that if we can use a constructor function then we might just as well be able to use any other plain old JavaScript function as the target exposed in module.exports. As in the following example in which our exported function allows the user of this module to gain access to one of several other encapsulated service objects. //foo.js var serviceA = {}; serviceA.serviceOne = function(){ }; serviceA.serviceTwo = function(){ }; serviceA.serviceThree = function(){ }; var serviceB = {}; serviceB.serviceOne = function(){ }; serviceB.serviceTwo = function(){ }; serviceB.serviceThree = function(){ }; module.exports = function(name){ switch(name){ case 'A': return serviceA; case 'B': return serviceB; default: throw new Error('Unknown service name: ' + name); } }; Now the user that imports this module receives a reference to our anonymous function declared above and then she can simply invoke the function to gain access to one of our encapsulated objects. For instance: //module bar.js var foo = require('./foo'); var obj = foo('A'); obj.serviceOne(); obj.serviceTwo(); obj.serviceThree(); Many programmers ordinarily invoke the function immediately returned by require instead of assigning it to a reference first. For instance: //module bar.js var foo = require('./foo')('A'); foo.serviceOne(); foo.serviceTwo(); foo.serviceThree(); So, in summary, it is as simple as follows: everything that we expose in module.exports is what we get when we invoke require. And using different techniques we could expose objects, constructors functions, properties, etc. About Modules and the use Global State An interesting aspect of modules is the way they are evaluated. The module is evaluated the first time it is required and then it is cached. This means that after it has been evaluated no matter how many times we require it, we will always get the same exported object back. This means that, although Node provides a global object, it is probably better to use modules to store shared stated instead of putting it directly into the global object. For instance, the following module exposes the configuration of a Mongo database. //module config.js dbConfig = { url:'mongodb://foo', user: 'anakin', password: '*******' } We can easily share this module with as many other modules as we want, and everyone of them will get the exact same instance of the configuration object since the module is evaluated only once and the exported object is cached from there on. //foo.js var dbConfig1 = require('./config'); var dbConfig2 = require('./config'); var assert = require('assert'); assert(dbConfig1==dbConfi2);

One more of the many cool features of thymeleaf is the ability to render fragments of templates - I have found this to be an especially useful feature to use with AngularJs. AngularJS $routeProvider or AngularUI router can be configured to return partial views for different "paths", using thymeleaf to return these partial views works really well. Consider a simple CRUD flow, with the AngularUI router views defined this way: app.config(function ($stateProvider, $urlRouterProvider) { $urlRouterProvider.otherwise("list"); $stateProvider .state('list', { url:'/list', templateUrl: URLS.partialsList, controller: 'HotelCtrl' }) .state('edit', { url:'/edit/:hotelId', templateUrl: URLS.partialsEdit, controller: 'HotelEditCtrl' }) .state('create', { url:'/create', templateUrl: URLS.partialsCreate, controller: 'HotelCtrl' }); }); The templateUrl above is the partial view rendered when the appropriate state is activated, here these are defined using javascript variables and set using thymeleaf templates this way(to cleanly resolve the context path of the deployed application as the root path): Now, consider one of the fragment definitions, say the one handling the list: file: templates/hotels/partialList.html Hotels IDNameAddressZipAction{{hotel.id}{{hotel.name}{{hotel.address}{{hotel.zip}Edit | Delete New Hotel The great thing about thymeleaf here is that this view can be opened up in a browser and previewed. To return the part of the view, which in this case is the section which starts with "th:fragment="content"", all I have to do is to return the name of the view as "hotels/partialList::content"! The same approach can be followed for the update and the create views. One part which I have left open is about how the uri in the UI which is "/hotels/partialsList" maps to "hotels/partialList::content", with Spring MVC this can be easily done through a View Controller, which is essentially a way to return a view name without needing to go through a Controller and can be configured this way: @Configuration public class WebConfig extends WebMvcConfigurerAdapter { @Override public void addViewControllers(ViewControllerRegistry registry) { registry.addViewController("/hotels/partialsList").setViewName("hotels/partialsList::content"); registry.addViewController("/hotels/partialsCreate").setViewName("hotels/partialsCreate::content"); registry.addViewController("/hotels/partialsEdit").setViewName("hotels/partialsEdit::content"); } } So to summarize, you create a full html view using thymeleaf templates which can be previewed and any rendering issues fixed by opening the view in a browser during development time and then return the fragment of the view at runtime purely by referring to the relevant section of the html page. A sample which follows this pattern is available at this github location: https://github.com/bijukunjummen/spring-boot-mvc-test

at data geekery , we love java. and as we’re really into jooq’s fluent api and query dsl , we’re absolutely thrilled about what java 8 will bring to our ecosystem. java 8 friday every friday, we’re showing you a couple of nice new tutorial-style java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. you’ll find the source code on github . 10 subtle mistakes when using the streams api we’ve done all the sql mistakes lists: 10 common mistakes java developers make when writing sql 10 more common mistakes java developers make when writing sql yet another 10 common mistakes java developers make when writing sql (you won’t believe the last one) but we haven’t done a top 10 mistakes list with java 8 yet! for today’s occasion ( it’s friday the 13th ), we’ll catch up with what will go wrong in your application when you’re working with java 8. (it won’t happen to us, as we’re stuck with java 6 for another while) 1. accidentally reusing streams wanna bet, this will happen to everyone at least once. like the existing “streams” (e.g. inputstream ), you can consume streams only once. the following code won’t work: intstream stream = intstream.of(1, 2); stream.foreach(system.out::println); // that was fun! let's do it again! stream.foreach(system.out::println); you’ll get a java.lang.illegalstateexception: stream has already been operated upon or closed so be careful when consuming your stream. it can be done only once 2. accidentally creating “infinite” streams you can create infinite streams quite easily without noticing. take the following example: // will run indefinitely intstream.iterate(0, i -> i + 1) .foreach(system.out::println); the whole point of streams is the fact that they can be infinite, if you design them to be. the only problem is, that you might not have wanted that. so, be sure to always put proper limits: // that's better intstream.iterate(0, i -> i + 1) .limit(10) .foreach(system.out::println); 3. accidentally creating “subtle” infinite streams we can’t say this enough. you will eventually create an infinite stream, accidentally. take the following stream, for instance: intstream.iterate(0, i -> ( i + 1) % 2) .distinct() .limit(10) .foreach(system.out::println); so… we generate alternating 0′s and 1′s then we keep only distinct values, i.e. a single 0 and a single 1 then we limit the stream to a size of 10 then we consume it well… the distinct() operation doesn’t know that the function supplied to the iterate() method will produce only two distinct values. it might expect more than that. so it’ll forever consume new values from the stream, and the limit(10) will never be reached. tough luck, your application stalls. 4. accidentally creating “subtle” parallel infinite streams we really need to insist that you might accidentally try to consume an infinite stream. let’s assume you believe that the distinct() operation should be performed in parallel. you might be writing this: intstream.iterate(0, i -> ( i + 1) % 2) .parallel() .distinct() .limit(10) .foreach(system.out::println); now, we’ve already seen that this will turn forever. but previously, at least, you only consumed one cpu on your machine. now, you’ll probably consume four of them, potentially occupying pretty much all of your system with an accidental infinite stream consumption. that’s pretty bad. you can probably hard-reboot your server / development machine after that. have a last look at what my laptop looked like prior to exploding: if i were a laptop, this is how i’d like to go. 5. mixing up the order of operations so, why did we insist on your definitely accidentally creating infinite streams? it’s simple. because you may just accidentally do it. the above stream can be perfectly consumed if you switch the order of limit() and distinct() : intstream.iterate(0, i -> ( i + 1) % 2) .limit(10) .distinct() .foreach(system.out::println); this now yields: 0 1 why? because we first limit the infinite stream to 10 values (0 1 0 1 0 1 0 1 0 1), before we reduce the limited stream to the distinct values contained in it (0 1). of course, this may no longer be semantically correct, because you really wanted the first 10 distinct values from a set of data (you just happened to have “forgotten” that the data is infinite). no one really wants 10 random values, and only then reduce them to be distinct. if you’re coming from a sql background, you might not expect such differences. take sql server 2012, for instance. the following two sql statements are the same: -- using top selectdistincttop10 * fromi orderby.. -- using fetch select* fromi orderby.. offset 0 rows fetchnext10 rowsonly so, as a sql person, you might not be as aware of the importance of the order of streams operations. 6. mixing up the order of operations (again) speaking of sql, if you’re a mysql or postgresql person, you might be used to the limit .. offset clause. sql is full of subtle quirks, and this is one of them. the offset clause is applied first , as suggested in sql server 2012′s (i.e. the sql:2008 standard’s) syntax. if you translate mysql / postgresql’s dialect directly to streams, you’ll probably get it wrong: intstream.iterate(0, i -> i + 1) .limit(10) // limit .skip(5) // offset .foreach(system.out::println); the above yields 5 6 7 8 9 yes. it doesn’t continue after 9 , because the limit() is now applied first , producing (0 1 2 3 4 5 6 7 8 9). skip() is applied after, reducing the stream to (5 6 7 8 9). not what you may have intended. beware of the limit .. offset vs. "offset .. limit" trap! 7. walking the file system with filters we’ve blogged about this before . what appears to be a good idea is to walk the file system using filters: files.walk(paths.get(".")) .filter(p -> !p.tofile().getname().startswith(".")) .foreach(system.out::println); the above stream appears to be walking only through non-hidden directories, i.e. directories that do not start with a dot. unfortunately, you’ve again made mistake #5 and #6. walk() has already produced the whole stream of subdirectories of the current directory. lazily, though, but logically containing all sub-paths. now, the filter will correctly filter out paths whose names start with a dot “.”. e.g. .git or .idea will not be part of the resulting stream. but these paths will be: .\.git\refs , or .\.idea\libraries . not what you intended. now, don’t fix this by writing the following: files.walk(paths.get(".")) .filter(p -> !p.tostring().contains(file.separator + ".")) .foreach(system.out::println); while that will produce the correct output, it will still do so by traversing the complete directory subtree, recursing into all subdirectories of “hidden” directories. i guess you’ll have to resort to good old jdk 1.0 file.list() again. the good news is, filenamefilter and filefilter are both functional interfaces. 8. modifying the backing collection of a stream while you’re iterating a list , you must not modify that same list in the iteration body. that was true before java 8, but it might become more tricky with java 8 streams. consider the following list from 0..9: // of course, we create this list using streams: list list = intstream.range(0, 10) .boxed() .collect(tocollection(arraylist::new)); now, let’s assume that we want to remove each element while consuming it: list.stream() // remove(object), not remove(int)! .peek(list::remove) .foreach(system.out::println); interestingly enough, this will work for some of the elements! the output you might get is this one: 0 2 4 6 8 null null null null null java.util.concurrentmodificationexception if we introspect the list after catching that exception, there’s a funny finding. we’ll get: [1, 3, 5, 7, 9] heh, it “worked” for all the odd numbers. is this a bug? no, it looks like a feature. if you’re delving into the jdk code, you’ll find this comment in arraylist.arralistspliterator : /* * if arraylists were immutable, or structurally immutable (no * adds, removes, etc), we could implement their spliterators * with arrays.spliterator. instead we detect as much * interference during traversal as practical without * sacrificing much performance. we rely primarily on * modcounts. these are not guaranteed to detect concurrency * violations, and are sometimes overly conservative about * within-thread interference, but detect enough problems to * be worthwhile in practice. to carry this out, we (1) lazily * initialize fence and expectedmodcount until the latest * point that we need to commit to the state we are checking * against; thus improving precision. (this doesn't apply to * sublists, that create spliterators with current non-lazy * values). (2) we perform only a single * concurrentmodificationexception check at the end of foreach * (the most performance-sensitive method). when using foreach * (as opposed to iterators), we can normally only detect * interference after actions, not before. further * cme-triggering checks apply to all other possible * violations of assumptions for example null or too-small * elementdata array given its size(), that could only have * occurred due to interference. this allows the inner loop * of foreach to run without any further checks, and * simplifies lambda-resolution. while this does entail a * number of checks, note that in the common case of * list.stream().foreach(a), no checks or other computation * occur anywhere other than inside foreach itself. the other * less-often-used methods cannot take advantage of most of * these streamlinings. */ now, check out what happens when we tell the stream to produce sorted() results: list.stream() .sorted() .peek(list::remove) .foreach(system.out::println); this will now produce the following, “expected” output 0 1 2 3 4 5 6 7 8 9 and the list after stream consumption? it is empty: [] so, all elements are consumed, and removed correctly. the sorted() operation is a “stateful intermediate operation” , which means that subsequent operations no longer operate on the backing collection, but on an internal state. it is now “safe” to remove elements from the list! well… can we really? let’s proceed with parallel() , sorted() removal: list.stream() .sorted() .parallel() .peek(list::remove) .foreach(system.out::println); this now yields: 7 6 2 5 8 4 1 0 9 3 and the list contains [8] eek. we didn’t remove all elements!? free beers ( and jooq stickers ) go to anyone who solves this streams puzzler! this all appears quite random and subtle, we can only suggest that you never actually do modify a backing collection while consuming a stream. it just doesn’t work. 9. forgetting to actually consume the stream what do you think the following stream does? intstream.range(1, 5) .peek(system.out::println) .peek(i -> { if(i == 5) thrownewruntimeexception("bang"); }); when you read this, you might think that it will print (1 2 3 4 5) and then throw an exception. but that’s not correct. it won’t do anything. the stream just sits there, never having been consumed. as with any fluent api or dsl, you might actually forget to call the “terminal” operation. this might be particularly true when you use peek() , as peek() is an aweful lot similar to foreach() . this can happen with jooq just the same, when you forget to call execute() or fetch() : dsl.using(configuration) .update(table) .set(table.col1, 1) .set(table.col2, "abc") .where(table.id.eq(3)); oops. no execute() yes, the “best” way – with 1-2 caveats ;-) 10. parallel stream deadlock this is now a real goodie for the end! all concurrent systems can run into deadlocks, if you don’t properly synchronise things. while finding a real-world example isn’t obvious, finding a forced example is. the following parallel() stream is guaranteed to run into a deadlock: object[] locks = { newobject(), newobject() }; intstream .range(1, 5) .parallel() .peek(unchecked.intconsumer(i -> { synchronized(locks[i % locks.length]) { thread.sleep(100); synchronized(locks[(i + 1) % locks.length]) { thread.sleep(50); } } })) .foreach(system.out::println); note the use of unchecked.intconsumer() , which transforms the functional intconsumer interface into a org.jooq.lambda.fi.util.function.checkedintconsumer , which is allowed to throw checked exceptions. well. tough luck for your machine. those threads will be blocked forever :-) the good news is, it has never been easier to produce a schoolbook example of a deadlock in java! for more details, see also brian goetz’s answer to this question on stack overflow . conclusion with streams and functional thinking, we’ll run into a massive amount of new, subtle bugs. few of these bugs can be prevented, except through practice and staying focused. you have to think about how to order your operations. you have to think about whether your streams may be infinite. streams (and lambdas) are a very powerful tool. but a tool which we need to get a hang of, first.

With the web being used on so many different devices now it's very important that you can change your design to fit on different screen sizes. The best way of changing your display depending on screen size is to use media queries to find out the size viewport of the screen and allowing you to change the design depending on what screen size is on. You will mainly make these changes in the CSS file as you can define the media query break points to change the design on different devices like this. /* Smartphones (portrait) ----------- */ @media only screen and (max-width : 320px) { } /* Desktops and laptops ----------- */ @media only screen and (min-width : 1224px) { } /* Large screens ----------- */ @media only screen and (min-width : 1824px) { } The above code will allow you to make styling completely different on different devices, but what if you wanted to change the functionality of the site depending on the screen size? What if you needed to use some Javascript code on different screen sizes, for example to create a slide down navigation button. How Do You Use Media Queries With jQuery Media queries will be checking the width of the window to see what the size of the device is so you would think that you can use a method like .width() on the window object like this. if($(window).width() < 767) { // change functionality for smaller screens } else { // change functionality for larger screens } But this will not return the true value of the window as it takes into effect things like body padding and scroll bars on the window. The other option you have when checking the media size is to use a Javascript method of .matchMedia() on the window object. var window_size = window.matchMedia('(max-width: 768px)')); This works the same way as media queries and is supported on many browsers apart from IE9 and lower. Can I Use window.matchMedia To use matchMedia you need to pass in the min or max values you want to check (like media queries) and see if the viewport matches this. if (window.matchMedia('(max-width: 768px)').matches) { // do functionality on screens smaller than 768px } Now you can use this to add a click event on to a sub-menu for screens smaller than 768px. The below code is an example of how you can add some Javascript code which will only be run on screens smaller than 768px. if (window.matchMedia('(max-width: 768px)').matches) { $('.sub-menu-button').on('click', function(e) { var subMenu = $(this).next('.sub-navigation'); if(subMenu.is(':visible')) { subMenu.slideUp(); } else { subMenu.slideDown(); } return false; }); }

as software developers we spend a large amount of time learning. there is always a new framework or technology to learn. it can seem impossible to keep up with everything when there is something new every day. so, it should be no surprise to you that learning quickly and gaining a deeper understanding of what you learn is very important. and that is exactly why–if you are not doing so already– you need to incorporate teaching into your learning. why teaching is such an effective learning tool when we learn something, most of us learn it in bits and pieces. typically, if you read a book, you’ll find the material in that book organized in a sensible way. the same goes for others mediums like video or online courses. but, unfortunately, the material doesn’t go into your head in the same way. what happens instead is that you absorb information in jumbled bits and pieces. you learn something, but don’t completely “get it” until you learn something else later on. the earlier topic becomes more clear, but the way that data is structured in your mind is not very well organized–regardless of how organized the source of that information was. even now, as i write this blog post, i am struggling with taking the jumbled mess of information i have in my head about how teaching helps you learn and figuring out how to present it in an organized way. i know what i want to say, but i don’t yet know how to say it. only the process of putting my thoughts on paper will force me to reorganize them; to sort them out and make sense of them. when you try to teach something to someone else, you have to go through this process in your own mind. you have to take that mess of data, sort it out, repackage it and organize it in a way that someone else can understand. this process forces you to reorganize the way that data is stored in your own head. also, as part of this process, you’ll inevitably find gaps in your own understanding of whatever subject you are trying to teach. when we learn something we have a tendency to gloss over many things we think we understand. you might be able to solve a math problem in a mechanical way, and the steps you use to solve the math problem might be sufficient for what you are trying to do, but just knowing how to solve a problem doesn’t mean you understand how to solve a problem. knowledge is temporary. it is easily lost. understanding is much more permanent. it is rare that we forget something we understand thoroughly. when we are trying to explain something to someone else, we are forced to ask ourselves the most important question in leaning… in gaining true understanding… “why.” when we have to answer the question “why,” superficial understanding won’t do. we have to know something deeply in order to not just say how, but why. that means we have to explore a subject deeply ourselves. sometimes this involves just sitting and thinking about it clearly before you try to explain it to someone else. sometimes just the act of writing, speaking or drawing something causes you to make connections you didn’t make before, instantly deepening your knowledge. (ever had one of those moments when you explained something to someone else and you suddenly realized that before you tried to explain it you didn’t really understand it yourself, but now you magically do?) and, sometimes, you have to go back to the drawing board and do more research to fill in those gaps in your own knowledge you just uncovered when you tried to explain it to someone else. becoming a teacher so, you perhaps you agree with me so far, but you’ve got one problem–you’re not a teacher. well, i have good news for you. we are all teachers. teaching doesn’t have to be some formal thing where you have books and a classroom. teaching is simply repackaging information in a way that someone else can understand. the most effective teaching takes place when you can explain something to someone in terms of something else they already understand. (want a great book on the subject that might make your brain hurt? surfaces and essences: analogy as the fuel and fire of thinking. an excellent book by douglas hofstadter, author of godel, escher, bach: an eternal golden braid. both books are extremely difficult reads, but very rewarding.) as human beings, we do this all the time. whenever we communicate with someone else and tell them about something we learned or explain how to do something, we are teaching. of course, the more you do it, the better you get at it, and adding a little more formalization to your practice doesn’t hurt, but at heart, you–yes, you–are a teacher. one of the best ways to start teaching–that may even benefit your career–is to start a blog. many developers i talk to assume that they have to already be experts at something in order to blog about it. the truth is, you only have to be one step ahead of someone for them to learn from you. so, don’t be afraid to blog about what you are learning as you are learning it. there will always be someone out there who could benefit from your knowledge–even if you consider yourself a beginner. and don’t worry about blogging for someone else–at least not at first. just blog for yourself. the act of taking your thoughts and putting them into words will gain you the benefits of increasing your own understanding and reorganizing thoughts in your mind. i won’t pretend the process isn’t painful. when you first start writing, it doesn’t usually come easily. but, don’t worry too much about quality. worry about communicating your ideas. with time, you’ll eventually get better and you’ll find the process of converting the ideas in your head to words becomes easier and easier. of course, creating a blog isn’t the only way to start teaching. you can simply have a conversation with a friend, a coworker, or even your spouse about what you are learning. just make sure you express what you are learning externally in one form or another if you really want to gain a deep understanding of the subject. you can also record videos or screencasts, speak on a subject or even give a presentation at work. whatever you do, make sure that teaching is part of your learning process. for more posts like this and some content i only deliver via email, sign up here.

A common tactic to help speed up your website is to use a technique called lazy loading which means that instead of loading everything your page needs at the start it will only load resources as and when it needs them. For example you can lazy load images so you can start the page off only with the images you need to view the page correctly, then other images that are out of view you won't need to load straight away. As the user scrolls down the page you can then search to see if the images are about to come into view and lazy load the images when they are needed. You can do the same with other resources such as JavaScript or CSS files, you can make sure you only load in the script as and when they need them to be used. An example of this I have used in the past is loading Disqus comments on the click event of a button, this jQuery code will then load in the Disqus Javascript file and initialise the Disqus code on the selected div. $j=jQuery.noConflict(); $j(document).ready(function() { $j('.showDisqus').on('click', function(){ // click event of the show comments button var disqus_shortname = 'enter_your_disqus_user_name'; // Enter your disqus user name // ajax request to load the disqus javascript $j.ajax({ type: "GET", url: "http://" + disqus_shortname + ".disqus.com/embed.js", dataType: "script", cache: true }); $j(this).fadeOut(); // remove the show comments button }); }); Ajax Method As you can see from the code above we have a click event of the .showDisqus button, inside this using the jQuery method .ajax() which is making a GET request for the script of embedding Disqus into your application. The ajax method is normally used to make basic http requests to a server side script and return the output of the script. On this occasion we are making a GET request and setting the dataType to be a script. This tells jQuery to treat the return as if we are including a new JavaScript file, this will disable browser caching on the script by adding a timestamp parameter to the end of the script. If you would like to enable caching of the script then you need to make sure you include a cache: true parameter. $.ajax({ type: "GET", url: "http://test_script.js", dataType: "script", cache: true }); Get Script Method Another option to get the script via GET ajax is to use the method getScript() this is simply a wrapper for the above ajax method. $.ajax({ url: url, dataType: "script", success: success }); This allows you to reduce the amount of code you are writing by simply using this method. $.getScript( "http://test_script.js" ) .done(function( script, textStatus ) { alert('Successfully loaded script'); }) .fail(function( jqxhr, settings, exception ) { alert('Failed to load script'); }); The problem with using getScript() is that it will never cache the script as it will always add the timestamp querystring to the end of the JavaScript file. As the ajax() method allows you to choose if you cache the script or not it is better to use this method when loading in a script that will not change.

ken fogel is the program coordinator and chairperson of the computer science technology program at dawson college in montreal, canada. he is also a program consultant to and part-time instructor in the computer institute of concordia university 's school of extended learning. he blogs at omniprogrammer.com and tweets @omniprof . his regular columns about netbeans in education are listed here . i am old enough to remember commercial applications that had a console interface. my students have no experience with the console. when java was introduced the era of such applications was over. instead you used awt, then swing and now javafx. for the student starting out in programming these gui frameworks add complexity they may not be ready for. therefore i start my classes using console input and output. input can be divided into two categories. the first is values. the primitive types such as int and double are value types. as i explain it, a value is a something you can perform both mathematical and logical operations on. the second is strings. a string does not have a value in the mathematical sense. instead it has a state. therefore it cannot be used in a mathematical expression. state can be compared and so strings can be used is logical expressions such as determining if two strings are the same. if you are ordering strings then you can also determine if one string is greater or less than the other. java does not have any console input routines for values. to java everything you type is a string. this has meant that for many years every java textbook author provided a console input routine that could convert strings to values. java 1.5 made all these routines redundant with the introduction of the scanner class. i could now teach input without needing to go into detail about wrapper classes or the numberformatexception in a standard approach. /** * ask the user for their weight */ public void inputweight() { scanner sc = new scanner(system.in); system.out.print("please enter your weight: "); int weight = sc.nextint(); system.out.println("weight = " + weight); } there is a problem with this code. if the user enters a string that cannot be converted we get the numberformatexception we want to avoid. scanner provides a solution with its ‘hasnext. . .’ methods. these methods determine if the user input can be converted to the target value. you can reject input and avoid the exception. /** * ask the user for their weight */ public void inputweight() { scanner sc = new scanner(system.in); system.out.print("please enter your weight: "); int weight; // accept input and test to see if it’s an integer if (sc.hasnextint()) { // success, it was an integer so get it weight = sc.nextint(); system.out.println("weight = " + weight); } else { // failure so retrieve the input as a string string bad = sc.next(); // inform the user what went wrong system.out.println(bad + " cannot be converted to an integer"); } } now when you run the program you get the following for a good input. this is what you get for a bad input. i will return to numeric input in another article as you do not want a program to end when the user makes one mistake. character input to make my assignments more interesting i have the students create a menu from which they must choose an action. i like to use letters as the menu choice. the problem is that there is no nextchar() method in scanner. instead you need to use next() which will accept anything that you enter. public void displaymenu() { scanner sc = new scanner(system.in); system.out.println("select an account"); system.out.println("a: savings"); system.out.println("b: checking"); system.out.println("c: exit"); system.out.print("please choose a, b or c: "); string choice = sc.next(); system.out.println("you chose: " + choice); } which could result in the following: the first improvement is to convert the string into a character. to do this the string must only be one character long. rather than checking the length you can just extract the first character in the string using the string class charat method. string choice = sc.next(); char letter = choice.charat(0); system.out.println("you chose: " + letter); this will work and if you enter “a” you will get ‘a’. the problem now is that the case of the character you entered should not matter. this is easy to solve by adding one more method from string into the mix. char letter = choice.touppercase().charat(0); almost perfect but since we are using string input the user can enter any string they want. so “bacon” will result in ‘b’. we also need some way to control the allowable letters. in this menu the choices are a, b and c. the solution is to use the ‘hasnext’ method for ‘next’. not the normal, no parameter version, but the version that takes a pattern. pattern is a polite way of saying regular expression. from https://xkcd.com/208/ just as we did with integer input we first test the string that the user has entered. the regular expression will be the pattern that will be used to determine if the string is correct. this regular expression is probably the simplest one you can write. we want a single character, either upper or lower case and is one of the three allowable characters. here it is: [abcabc] the square brackets indicate that this is a character class expression that can match only a single character. what goes inside the brackets are the allowable characters, in this case a, b, c, a, b or c. since these characters are adjacent in the ascii/unicode table they can also be written as a range. [a-ca-c] putting this all together we get a routine that will only accept a single letter. public void displaymenu() { scanner sc = new scanner(system.in); system.out.println("select an account"); system.out.println("a: savings"); system.out.println("b: checking"); system.out.println("c: exit"); system.out.print("please choose a, b or c: "); char letter; if (sc.hasnext("[a-ca-c]")) { string choice = sc.next(); letter = choice.touppercase().charat(0); system.out.println("you chose: " + letter); } else { string choice = sc.next(); system.out.println(choice + " is not valid input"); } } now if you enter ‘a’: but if you enter ‘aardvark’: in my next article i will look at console input a bit more and i will introduce you to the beethoven test.

Learn about string interning, a method of storing only one copy of each distinct string value, which must be immutable.

When the first early access versions of Java 8 were made available, what seemed the most important (r)evolution were lambdas. This is now changing and many developers seem to think now that streams are the most valuable Java 8 feature. And this is because they believe that by changing a single word in their programs (replacing stream with parallelStream) they will make these programs work in parallel. Many Java 8 evangelists have demonstrated amazing examples of this. Is there something wrong with this? No. Not something. Many things: Running in parallel may or may not be a benefit. It depends what you are using this feature for. Java 8 parallel streams may make your programs run faster. Or not. Or even slower. Thinking about streams as a way to achieve parallel processing at low cost will prevent developers to understand what is really happening. Streams are not directly linked to parallel processing. Most of the above problems are based upon a misunderstanding: parallel processing is not the same thing as concurrent processing. And most examples shown about “automatic parallelization” with Java 8 are in fact examples of concurrent processing. Thinking about map, filter and other operations as “internal iteration” is a complete nonsense (although this is not a problem with Java 8, but with the way we use it). So, what are streams According to Wikipedia: “a stream is a potentially infinite analog of a list, given by the inductive definition: data Stream a = Cons a (Stream a) Generating and computing with streams requires lazy evaluation, either implicitly in a lazily evaluated language or by creating and forcing thunks in an eager language.” One most important think to notice is that Java is what Wikipedia calls an “eager” language, which means Java is mostly strict (as opposed to lazy) in evaluating things. For example, if you create a List in Java, all elements are evaluated when the list is created. This may surprise you, since you may create an empty list and add elements after. This is only because either the list is mutable (and you are replacing a null reference with a reference to something) or you are creating a new list from the old one appended with the new element. Lists are created from something producing its elements. For example: List list = Arrays.asList(1, 2, 3, 4, 5); Here the producer is an array, and all elements of the array are strictly evaluated. It is also possible to create a list in a recursive way, for example the list starting with 1 and where all elements are equals to 1 plus the previous element and smaller than 6. In Java < 8, this translates into: List list = new ArrayList(); for(int i = 0; i < 6; i++) { list.add(i); } One may argue that the for loop is one of the rare example of lazy evaluation in Java, but the result is a list in which all elements are evaluated. What happens if we want to apply a function to all elements of this list? We may do this in a loop. For example, if with want to increase all elements by 2, we may do this: for(int i = 0; i < list.size(); i++) { list.set(i, list.get(i) * 2); } However, this does not allow using an operation that changes the type of the elements, for example increasing all elements by 10%. The following solution solves this problem: List list2 = new ArrayList(); for(int i = 0; i < list.size(); i++) { list2.add(list.get(i) * 1.2); } This form allows the use of a the Java 5 for each syntax: List list2 = new ArrayList<>(); for(Integer i : list) { list2.add(i * 1.2); } or the Java 8 syntax: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2)); So far, so good. But what if we want to increase the value by 10% and then divide it by 3? The trivial answer would be to do: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2)); List list3 = new ArrayList<>(); list2.forEach(x -> list3.add(x / 3)); This is far from optimal because we are iterating twice on the list. A much better solution is: List list2 = new ArrayList<>(); for(Integer i : list) { list2.add(i * 1.2 / 3); } Let aside the auto boxing/unboxing problem for now. In Java 8, this can be written as: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2 / 3)); But wait... This is only possible because we see the internals of the Consumer bound to the list, so we are able to manually compose the operations. If we had: List list2 = new ArrayList<>(); list.forEach(consumer1); List list3 = new ArrayList<>(); list2.forEach(consumer2); How could we know how to compose them? No way. In Java 8, the Consumer interface has a default method andThen. We could be tempted to compose the consumers this way: list.forEach(consumer1.andThen(consumer2)); but this will result in an error, because andThen is defined as: default Consumer andThen(Consumer after) { Objects.requireNonNull(after); return (T t) -> { accept(t); after.accept(t); }; } This means that we can't use andThen to compose consumers of different types. In fact, we have it all wrong since the beginning. What we need is to bind the list to a function in order to get a new list, such as: Function function1 = x -> x * 1.2; Function function2 = x -> x / 3; list.bind(function1).bind(function2); where the bind method would be defined in a special FList class like: public class FList { final List list; public FList(List list) { this.list = list; } public FList bind(Function f) { List newList = new ArrayList(); for (T t : list) { newList.add(f.apply(t)); } return new FList(newList); } } and we would use it as in the following example: new Flist<>(list).bind(function1).bind(function2); The only trouble we have then is that binding twice would require iterating twice on the list. This is because bind is evaluated strictly. What we would need is a lazy evaluation, so that we could iterate only once. The problem here is that the bind method is not a real binding. It is in reality a composition of a real binding and a reduce. "Reducing" is applying an operation to each element of the list, resulting in the combination of this element and the result of the same operation applied to the previous element. As there is no previous element when we start from the first element, we start with an initial value. For example, applying (x) -> r + x, where r is the result of the operation on the previous element, or 0 for the first element, gives the sum of all elements of the list. Applying () -> r + 1 to each element, starting with r = 0 gives the length of the list. (This may not be the more efficient way to get the length of the list, but it is totally functional!) Here, the operation is add(element) and the initial value is an empty list. And this occurs only because the function application is strictly evaluated. What Java 8 streams give us is the same, but lazily evaluated, which means that when binding a function to a stream, no iteration is involved! Binding a Function to a Stream gives us a Stream with no iteration occurring. The resulting Stream is not evaluated, and this does not depend upon the fact that the initial stream was built with evaluated or non evaluated data. In functional languages, binding a Function to a Stream is itself a function. In Java 8, it is a method, which means it's arguments are strictly evaluated, but this has nothing to do with the evaluation of the resulting stream. To understand what is happening, we can imagine that the functions to bind are stored somewhere and they become part of the data producer for the new (non evaluated) resulting stream. In Java 8, the method binding a function T -> U to a Stream, resulting in a Stream is called map. The function binding a function T -> Stream to a Stream, resulting in a Stream is called flatMap. Where is flatten? Most functional languages also offer a flatten function converting a Stream> into a Stream, but this is missing in Java 8 streams. It may not look like a big trouble since it is so easy to define a method for doing this. For example, given the following function: Function> f = x -> Stream.iterate(1, y -> y + 1).limit(x); Stream stream = Stream.iterate(1, x -> x + 1); Stream stream2 = stream.limit(5).flatMap(f); System.out.println(stream2.collect(toList())) to produce: [1, 1, 2, 1, 2, 3, 1, 2, 3, 4, 1, 2, 3, 4, 5] Using map instead of flatMap: Stream stream = Stream.iterate(1, x -> x + 1); Stream stream2 = stream.limit(5).map(f); System.out.println(stream2.collect(toList())) will produce a stream of streams: [java.util.stream.SliceOps$1@12133b1, java.util.stream.SliceOps$1@ea2f77, java.util.stream.SliceOps$1@1c7353a, java.util.stream.SliceOps$1@1a9515, java.util.stream.SliceOps$1@f49f1c] Converting this stream of streams of integers to a stream of integers is very straightforward using the functional paradigm: one just need to flatMap the identity function to it: System.out.println(stream2.flatMap(x -> x).collect(toList())); It is however strange that a flatten method has not been added to the stream, knowing the strong relation that ties map, flatMap, unit and flatten, where unit is the function from T to Stream, represented by the method: Stream Stream.of(T... t) When are stream evaluated? Streams are evaluated when we apply to them some specific operations called terminal operation. This may be done only once. Once a terminal operation is applied to a stream, is is no longer usable. Terminal operations are: forEach forEachOrdered toArray reduce collect min max count anyMatch allMatch noneMatch findFirst findAny iterator spliterator Some of these methods are short circuiting. For example, findFirst will return as soon as the first element will be found. Non terminal operations are called intermediate and can be stateful (if evaluation of an element depends upon the evaluation of the previous) or stateless. Intermediate operations are: filter map mapTo... (Int, Long or Double) flatMap flatMapTo... (Int, Long or Double) distinct sorted peek limit skip sequential parallel unordered onClose Several intermediate operations may be applied to a stream, but only one terminal operation may be use. So what about parallel processing? One most advertised functionality of streams is that they allow automatic parallelization of processing. And one can find the amazing demonstrations on the web, mainly based of the same example of a program contacting a server to get the values corresponding to a list of stocks and finding the highest one not exceeding a given limit value. Such an example may show an increase of speed of 400 % and more. But this example as little to do with parallel processing. It is an example of concurrent processing, which means that the increase of speed will be observed also on a single processor computer. This is because the main part of each “parallel” task is waiting. Parallel processing is about running at the same time tasks that do no wait, such as intensive calculations. Automatic parallelization will generally not give the expected result for at least two reasons: The increase of speed is highly dependent upon the kind of task and the parallelization strategy. And over all things, the best strategy is dependent upon the type of task. The increase of speed in highly dependent upon the environment. In some environments, it is easy to obtain a decrease of speed by parallelizing. Whatever the kind of tasks to parallelize, the strategy applied by parallel streams will be the same, unless you devise this strategy yourself, which will remove much of the interest of parallel streams. Parallelization requires: A pool of threads to execute the subtasks, Dividing the initial task into subtasks, Distributing subtasks to threads, Collating the results. Without entering the details, all this implies some overhead. It will show amazing results when: Some tasks imply blocking for a long time, such as accessing a remote service, or There are not many threads running at the same time, and in particular no other parallel stream. If all subtasks imply intense calculation, the potential gain is limited by the number of available processors. Java 8 will by default use as many threads as they are processors on the computer, so, for intensive tasks, the result is highly dependent upon what other threads may be doing at the same time. Of course, if each subtask is essentially waiting, the gain may appear to be huge. The worst case is if the application runs in a server or a container alongside other applications, and subtasks do not imply waiting. In such a case, (for example running in a J2EE server), parallel streams will often be slower that serial ones. Imagine a server serving hundreds of requests each second. There are great chances that several streams might be evaluated at the same time, so the work is already parallelized. A new layer of parallelization at the business level will most probably make things slower. Worst: there are great chances that the business applications will see a speed increase in the development environment and a decrease in production. And that is the worst possible situation. Edit: for a better understanding of why parallel streams in Java 8 (and the Fork/Join pool in Java 7) are broken, refer to these excellent articles by Edward Harned: A Java Fork-Join Calamity A Java Parallel Calamity What streams are good for Stream are a useful tool because they allow lazy evaluation. This is very important in several aspect: They allow functional programming style using bindings. They allow for better performance by removing iteration. Iteration occurs with evaluation. With streams, we can bind dozens of functions without iterating. They allow easy parallelization for task including long waits. Streams may be infinite (since they are lazy). Functions may be bound to infinite streams without problem. Upon evaluation, there must be some way to make them finite. This is often done through a short circuiting operation. What streams are not good for Streams should be used with high caution when processing intensive computation tasks. In particular, by default, all streams will use the same ForkJoinPool, configured to use as many threads as there are cores in the computer on which the program is running. If evaluation of one parallel stream results in a very long running task, this may be split into as many long running sub-tasks that will be distributed to each thread in the pool. From there, no other parallel stream can be processed because all threads will be occupied. So, for computation intensive stream evaluation, one should always use a specific ForkJoinPool in order not to block other streams. To do this, one may create a Callable from the stream and submit it to the pool: List list = // A list of objects Stream stream = list.parallelStream().map(this::veryLongProcessing); Callable> task = () -> stream.collect(toList()); ForkJoinPool forkJoinPool = new ForkJoinPool(4); List newList = forkJoinPool.submit(task).get() This way, other parallel streams (using their own ForkJoinPool) will not be blocked by this one. In other words, we would need a pool of ForkJoinPool in order to avoid this problem. If a program is to be run inside a container, one must be very careful when using parallel streams. Never use the default pool in such a situation unless you know for sure that the container can handle it. In a Java EE container, do not use parallel streams. Previous articles What's Wrong with Java 8, Part I: Currying vs Closures What's Wrong in Java 8, Part II: Functions & Primitives

In a previous post, we had shared a small function that generated random string in Java. It turns out that similar functionality is available from a class in the extremely useful apache commons lang library. If you are using maven, download the jar using the following dependency: commons-lang commons-lang 20030203.000129 The class we are interested in is RandomStringUtils. Listed below are some functions you may find useful. Generate and print a random string of length 5 from all characters available System.out.println(RandomStringUtils.random(5)); Generate and print random string of length 10 from upper and lower case alphabets System.out.println(RandomStringUtils.randomAlphabetic(10)); Generate and print a random number of length 12 System.out.println(RandomStringUtils.randomNumeric(12)); Generate and print a random string of length 5 using only a, b, c and d characters System.out.println(RandomStringUtils.random(10,new char[]{'a','b','c','d'}));

Facts and Terminology As you probably know, Java uses UTF-16 to represent Strings. In order to understand the confusion about String.length(), you need to be familiar with some Encoding/Unicode terms. Code Point: A unique integer value which represents a character in the code space. Code Unit: A bit sequence used to encode characters (Code Points). One or more Code Units may be required to represent a Code Point. UTF-16 Unicode Code Points are logically divided into 17 planes. The first plane, the Basic Multilingual Plane (BMP) contains the “classic” characters (from U+0000 to U+FFFF). The other planes contain the supplementary characters (from U+10000 to U+10FFFF). Characters (Code Points) from the first plane are encoded in one 16-bit Code Unit with the same value. Supplementary characters (Code Points) are encoded in two Code Units (encoding-specific, see Wiki for the explanation). Example Character: A Unicode Code Point: U+0041 UTF-16 Code Unit(s): 0041 Character: Mathematical double-struck capital A Unicode Code Point: U+1D538 UTF-16 Code Unit(s): D835 DD38 As you can see here, there are characters which are encoded in two Code Units. String.length() Let’s take a look at the Javadoc of the length() method: public int length() Returns the length of this string. The length is equal to the number of Unicode code units in the string. So if you have one supplementary character which consists of two code units, the length of that single character is two. // Mathematical double-struck capital A String str = "\uD835\uDD38"; System.out.println(str); System.out.println(str.length()); //prints 2 Which is correct according to the documentation, but maybe it’s not expected. ~Solution You need to count the code points not the code units: String str = "\uD835\uDD38"; System.out.println(str); System.out.println(str.codePointCount(0, str.length())); See: codePointCount(int beginIndex, int endIndex) References/Sources The Java Language Specification Unicode Glossary: Code Point Wiki: Code Point Unicode Glossary: Code Unit Wiki: Code Unit Wiki: Unicode Wiki: UTF-16 Supplementary Characters in the Java Platform Wiki: Unicode Planes

If you need to compare the java.io.file.Path object, be aware that Path.endsWith(String) will ONLY match another sub-element of Path object in your original path, not the path name string portion! If you want to match the string name portion, you would need to call the Path.toString() first. For example // Match all jar files. Files.walk(dir).forEach(path -> { if (path.toString().endsWith(".jar")) System.out.println(path); }); With out the "toString()" you will spend many fruitless hours wonder why your program didn't work.

Before some time I have written a blog post – Converting a C# object into JSON string in that post one of reader, Thomas Levesque commented that mostly people are using JSON.NET a popular high performance JSON for creating for .NET Created by James Newton- King. I agree with him if we are using .NET Framework 4.0 or higher version for earlier version still JavaScriptSerializer is good. So in this post we are going to learn How we can convert C# object into JSON string with JSON.NET framework. What is JSON.NET: JSON.NET is a very high performance framework compared to other serializer for converting C# object into JSON string. It is created by James Newton-Kind. You can find more information about this framework from following link. http://james.newtonking.com/json How to convert C# object into JSON string with JSON.NET framework: For this I am going to use old application that I have used in previous post. Following is a employee class with two properties first name and last name. public class Employee { public string FirstName { get; set; } public string LastName { get; set; } } I have created same object of “Employee” class as I have created in previous post like below. Employee employee=new Employee {FirstName = "Jalpesh", LastName = "Vadgama"}; Now it’s time to add JSON.NET Nuget package. You install Nuget package via following command. I have installed like below. Now we are done with adding NuGet package. Following is code I have written to convert C# object into JSON string. string jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(employee); Console.WriteLine(jsonString); Let's run application and following is a output as expected. That’s it. It’s very easy. Hope you like it. Stay tuned for more.

Recently I had to integrate a Node.js based server application with a C# DLL. Our software (a web-app) offers the possibility to execute payments over a POS terminal. This latter one is controllable through a dedicated DLL which exposes interfaces like ExecutePayment(operation, amount) and so on. As I mentioned, there is the Node.js server that somehow exposes the functionality of the POS (and some more) as a REST api. (The choice for using Node.js had specific reasons which I wouldn't want to outline right now). When you start with such an undertaking, then there are different possibilities. One is to use Edge.js which allows you to embed, reference and invoke .Net CLR objects from within your Node.js based applications. Something like this: var hello = require('edge').func({ assemblyFile: 'My.Edge.Samples.dll', typeName: 'Samples.FooBar.MyType', methodName: 'MyMethod' // Func> }); hello('Node.js', function (error, result) { ... }); Edge is a very interesting project and has a lot of potential. In fact, I just tried it quickly with a simple DLL and it worked right away. However, when using it from my Node app within node-webkit it didn't work. I'm not yet sure whether it was related to node-webkit or the POS DLL itself (because it might be COM exposed etc..). However, if you need simple integrations this might work well for you. Process invocation A second option that came to my mind is to design the DLL as a self-contained process and to invoke it using Node.js's process api. Turns out this is quite simple. Just prepare your C# application to read it's invocation arguments s.t. you can do something like.. IntegrationConsole.exe ExecutePayment 1 100 ..to "ExecutePayment" with operation number 1 and an amount of 1€. The C# console application needs to communicate it's return values to the STDOUT (you may use JSON for creating a more structured information exchange protocol format). Once you have this, you can simply execute the process from Node.js and read the according STDOUT: var process = require('child_process'); ... process.exec(execCmd, function (error, stdout, stderr) { var result = stdout; ... writeToResponse(stdout); }); execCmd is holds the instructions required to launch the EXE with the required invocation arguments. In this approach you execute the process, it does its job, returns the response and terminates. If for some reason however, you need to keep the process running for having a longer, kind of more interactive communication between the two components, you can communicate through the STDIN/STDOUT of the process. Your C# console application starts and listens on the STDIN.. static void Main(string[] args) { ... string line; do { line = Console.ReadLine(); try { // do something meaningful with the input // write to STDOUT to respond to the caller } catch (Exception e) { Console.WriteLine(e.Message); } } while (line != null); } On the Node.js side you do not exec your process, but instead you spawn a child process. var spawn = require('child_process').spawn; ... var posProc = spawn('IntegrationConsole.exe', ['ExecutePayment', 1, 100]); For getting the responses, you simply register on the STDOUT of the process... posProc.stdout.once('data', function (data) { // write it back on the response object writeToResponse(data); }); ..and you may also want to listen for when the process dies to eventually perform some cleanup. posProc.on('exit', function (code) { ... }); Writing to the STDIN of the process is simple as well: posProc.stdin.setEncoding ='utf-8'; posProc.stdin.write('...'); In this way you have a more interactive, "stateful communication", where you send a command to the EXE which responds (STDOUT) and based on the response you again react and send some other command (STDIN). Embedding this in the Request/Response pattern To expose everything as a REST api (on Node), you need to pay some attention on the registration of the event handlers on STDOUT. Suppose you do something like app.post('/someEndpoint',function(req, res){ posProc = spawn('IntegrationConsole.exe',['ExecutePayment',1,100]);... posProc.stdout.on('data',function(data){// return the result of this execution// on the response});}), app.post('/someOtherEndpoint',function(req, res){... posProc.stdout.on('data',function(data){// return the result of this execution// on the response});// write to the stdin of the before created child process posProc.stdin.setEncoding ='utf-8'; posProc.stdin.write('...');}); I excluded proper edge case handling like what happens if your process died before etc.. but the key point here is that you cannot register your events by using on(..), as otherwise you'll end up having multiple data event handlers on the stdout. So you can either register and de-register the event by using the removeListener('event name', callback) syntax or use the more handy once registration mechanism (as I did already in my samples at the beginning of the article): posProc.stdout.once('data',function(data){// write it back on the response object writeToResponse(data);});

To convert a byte[] array to a String we can simply use the new String(byte[]) constructor. But if the array contains non-printable bytes we don't get a good representation. In Groovy we can use the method encodeHex() to transform a byte[] array to a hex String value. The byteelements are converted to their hexadecimal equivalents. final byte[] printable = [109, 114, 104, 97, 107, 105] // array with non-printable bytes 6, 27 (ACK, ESC) final byte[] nonprintable = [109, 114, 6, 27, 104, 97, 107, 105] assert new String(printable) == 'mrhaki' assert new String(nonprintable) != 'mr haki' // encodeHex() returns a Writable final Writable printableHex = printable.encodeHex() assert printableHex.toString() == '6d7268616b69' final nonprintableHex = nonprintable.encodeHex().toString() assert nonprintableHex == '6d72061b68616b69' // Convert back assert nonprintableHex.decodeHex() == nonprintable Code written with Groovy 2.2.1

Gradle is a build tool on the JVM platform that’s been gaining prominence over the last few years. Gradle’s reach doesn’t stop with JVM languages though. The most recent releases, 1.10 and 1.11, have improved support for compiling native code (C / C++ / Objective C) and introduced support for the .NET platform. For Android projects, Google have made Gradle the de facto build tool. It’s getting a name for its flexibility, so when the opportunity came to build a single page Javascript webapp we decided to put the latest addition to the polyglot build tool arsenal through its paces. In this post I’m going to look at why we chose Gradle over a native JS build tool, provide a quick tour of the JS/CSS plugins, look at the embedded Jetty webserver plugin, discuss testing your code in CI, and a variety of other tips and gotchyas found in our experience during the project. Why not {{JavaScript_build_tool_flavour_of_the_month}? The momentum around JavaScript development is growing rapidly. So rapidly in fact that it feels like the book 1984 where one day your nation is fighting a war with one country, then all of a sudden wakes up, and it’s announced it has pledged an allegiance to a former enemy; your former ally becoming the new enemy. Twitter and the blogosphere recently became alive with talk about Gulp, a new JS build tool. This blog post ‘And just like that, Grunt and RequireJS are out – its all about Gulp and Browsify now’ and this Twitter post felt like the shifting of alliances in 1984. This is something you’ll have to factor into your JS build tool decision making. It also means however there is still room for contenders in this space. Having the same build tool for both client side and server side builds helps developers understand the project better. Our team had Groovy and Gradle exposure already and whilst we were happy to try a JS build tool, it would be nice to consider the same tools for both aspects of the project. A tool that works with your existing infrastructure is also important. The straw that broke the camel’s back was our client’s firewall. Firewalls are part and parcel of something we have to work around as consultants. Sometimes you win, sometimes you lose. Simply put, our client’s firewall made npm cry as the https proxy had some confusing message about SSL certificate algorithms. After trying to work around this for a while, I noticed we had a perfectly fine build tool that already worked fine with Java and our proxy. The question was asked: Are there any JavaScript plugins for Gradle? Reinventing the wheel I heard at a JavaScript BOF at JavaOne someone remark that ‘JavaScript had been rapidly climbing the tree of software development, reinventing tools that had already existed on established platforms, then falling off the tree taking each branch along with it on the way down.’ Although tongue-in-cheek, it did capture the frustrations of tools already built that were largely ignored, or unknown by the JavaScript community. From what I’ve seen so far, the other popular JVM build tool, Maven, has a large number of plugins to aid in the preparation of JavaScript files. Most of these use the Rhino JavaScript Engine that comes with Java in order to execute these utilities on the JVM platform where the build and testing is performed. The eco-system of running JavaScript utilities out of Rhino with ScriptEngineManager is well established. Additionally there are a lot of other tools written in Java to validate and compress JavaScript files – the Google Closure Compiler is one that is used by the Gradle JS and CSS plugin. It appears to have been relatively straight forward for plugin devs to repackage all these tools as Gradle plugins that kick off the same scripts from within. Where there isn’t a Java port or Jar, Gradle plugins or tasks can always launch JS tools from the command line. Kicking the tires So just what can Gradle with the available JS plugins do? Here’s a whirlwind tour of the available features. The Gradle JS and CSS plugins The Gradle JS plugin is a third party Gradle plugin written by Eric Wendelin. It can do file combination, minification, Gzip, JSHint and more. And although we didn’t need it for this project, you’ll be pleased to know it has require.js support as well. The Gradle CSS Plugin also written by Eric Wendelin, again supports combining and minification plus supports less compilation and csslint. To add these plugins to your build and associated tasks, all you have to do is add this to your build.gradle apply plugin: 'js' apply plugin: 'css' // define the dependencies gradle buildscript will use to build // the app (not the app itself) buildscript { repositories { mavenLocal() mavenCentral() } dependencies { classpath 'com.eriwen:gradle-css-plugin:1.8.0' classpath 'com.eriwen:gradle-js-plugin:1.9.0' } } You can then configure the tasks the plugin provides, combineJs, minifyJs, gzipJs as follows: combineJs { // pull together the source from string & file lists // eg. def core = ["$webAppDirName/init.js",...] source = core + application + devmode + bigquery + javascript.source.externalLibs.js.files + dfpFilters + chartdef + uxFilters // show the resolved files when gradle is run with -d source.each{ logger.debug ("$it") } dest = file("${buildDir}/all.js") } minifyJs { source = combineJs dest = file("${buildDir}/all.min.js") sourceMap = file("${buildDir}/all.sourcemap.json") closure { warningLevel = 'QUIET' compilerOptions.defineReplacements = ['MY_DBUG_FLAG':false] } } gzipJs { source = minifyJs.dest dest = file("${buildDir}/all.min.js.gz") } combineJs has a source property which takes a collection. We split our app into groups of JavaScript files and combine them with a list of files from a Gradle javascript sourceset called externLibs that references bootstrap and jquery in a separate dir in our project. The combine is done in the order you specify when specified in an ArrayList. For Gradle aficionados, this is why I’m not using sourcesets for our files – see here. The other reason is that all our client JS was in the same folder so lists were the most concise way to declare these file groups. You can see we can also debug the list to make sure we’ve pulled in the files we think we’ve asked for by using standard groovy code to iterate through the source collection, making use of the logger that is injected into the build file by Gradle. One of the neatest things about Gradle is that each task has a set of Input files or dirs, plus output files. Gradle can do some very intelligent introspection into whether or not a file has been modified and what dependent tasks need to be rerun. Spending a little effort to specify these properly means that your build won’t have to do unnecessary stuff repeatedly. The minifyJs task takes advantage of this. We can then declare the input for its source property as the dest outputfrom the previous task. Here we set source to combineJs and gradle figures out we mean the output of that task (we could have also said combineJs.dest). The cool thing about this is that Gradle will be lazy. If it notices that the output on the combineJs hasn’t changed, thus the input on minifyJs hasn’t changed, it will skip running that task. BTW, you can always force Gradle to run each task with the --rerun-tasks command line parameter. To help things along we can define the dependency chain on these tasks tasks.minifyJs.dependsOn tasks.combineJs tasks.gzipJs.dependsOn tasks.minifyJs and run gzipJs each time being confident it will only run the earlier tasks if their task inputs have changed. I’m not sure why the dependency chain isn’t configured by default in the JS plugin – perhaps it could be a reasonable enhancement for a future release? I mentioned that minifyJs uses the Google Closure Compiler under the hood and so I wanted to show our use of configuring it with closure config block, in particular the compilerOptions.defineReplacements option which wasn’t documented in the JS plugins docs. In our JS code we define a constant to indicate some developer mode functionality like so: // config.js /** * Flag to indicate console and extra logging throughout the app * @define {boolean} allow the value of this bool to be * overwritten at closure compiler / minification time * @const * @type {boolean} */ var MY_DBUG_FLAG = true; Then elsewhere throughout our code base, we have code like this which modifies the DOM and adds some helpful info about the applications state to the UI. if (MY_DBUG_FLAG) { $('.dashboard').append( 'Here's some dev info you wouldn't normally see' ); } defineReplacements allows the Google Closure compiler to redefine the MY_DBUG_FLAG constant at build time to false, and then through its optimisation, remove the if (MY_DBUG_FLAG) blocks from the resultant all.min.js entirely (since we’ve forced MY_DBUG_FLAG to false). There are plenty of other configuration options for the compiler but it’s not immediately apparent where these are set. Take a look at CompilerOptions.java in the Google Closure compiler source to see the properties it takes. CSS The CSS plugin adds combineCss, minifyCss, and gzipCss tasks. Here’s the config for the plugin & how we configured it. String[] cssDirs = ["src/main/webapp/$bootstrapPath/css", "src/main/webapp/libs/bootstrap-typeahead/0.9.3/css", "src/main/webapp/libs/bootstrap-datepicker/1.2.0/css", "src/main/webapp/$chosenPath", "src/main/webapp/$ajaxLoaderPath", "src/main/webapp/styles"] css.source { dev { css { srcDirs cssDirs include "*.css" include "*.min.css" exclude ".#*.css" // VCS creates these annoying files } } } Here’s another example of using the Gradle sourcesets, we have a list of our dependent CSS libs + our own CSS, using srcDirs to pull together these directories and let the source set definition define what files to include and exclude. Then pass it off to a combineCss source property. combineCss { source = css.source.dev.css.files dest = "${buildDir}/all.css" } combineCss << { source.each { logger.info "Combining $it"} } A small change made in this example is to add a doLast (aliased <<) closure on the combineCss task to do the non-essential logging of the files found after the task has been configured and actioned. I think it helps keep the build file a bit cleaner. The other thing you probably have noticed is the interleaving of properties such as ${buildDir}, $bootstrapPath, $webAppDirName. This is just Groovy syntax to refer to variables. Gradle does some binding magic to bind those variables to properties, either defined in the plugins, or within whatever you put in your gradle.properties file that lives in the same directory as build.gradle. # Path to Javascript files jsSrcDir=src/main/webapp/js # Path to json files used to describe UI controls jsonPath=src/main/webapp/json # Local Deployment wwwPath=/var/www openBrowserPostLocalDeploy=true # Library Paths jqueryPath=libs/jquery/2.0.3 ajaxLoaderPath=libs/ajaxloader/1.0 chosenPath=libs/chosen/1.0.0 bootstrapPath=libs/bootstrap/3.0.2 Just for completeness here are the minify and gzipCss tasks but now you are down with Gradle and Js, the below should be self-explanatory. minifyCss { source = combineCss dest = "${buildDir}/all.min.css" yuicompressor { // Optional lineBreakPos = -1 } } gzipCss { source = minifyCss dest = "${buildDir}/all.min.css.gz" } Deploy against the machine So we got to generating some minified, gzipped files. Great. Now we need to see our changes with an apache or nginx install. It’s a relatively short task to install through a couple of sudo apt-get commands on Ubuntu, or re-enabling web-sharing on OS X (google it) but admittedly a few hoops need to be jumped through. You also need to set some file permissions so you can let your build tool write to Apache’s /var/www dir. Once the formalities are out of the way though we can automate this deploy. Like any decent build tool, Gradle itself comes with tasks to copy, archive and manipulate files. Gradle copy tasks take a copy configuration closure which let you define properties on which file patterns you wish to include, exclude and which dirs (much like the source sets shown above). The copy tasks are also smart enough to know what’s already been copied and only copied changed files which can save a bit of time when it comes to deploying. Here are tasks for copying the all.min.js and css (plus gzipped versions and sourcemaps) task deployCombinedJsLocally(dependsOn: 'gzipJs', type:Copy) { from files(minifyJs.dest, minifyJs.sourceMap, gzipJs.dest) into wwwPath } task deployCombinedCssLocally(dependsOn: 'gzipCss', type: Copy) { doFirst { logger.quiet("Copying ${files(minifyCss.dest, gzipCss.dest)} ... to ${wwwPath}/css") } from files(minifyCss.dest, gzipCss.dest) into file("$wwwPath/css") } For the css task, I wanted to show an example of logging the files that would be copied but this time using Gradle’s default quiet log level so it would be printed out without needing to specify this flag. Also this logging happens within a doFirst closure block which will happen during the evaluation stage of the gradle build before the task executes. It would also be nice to also have a task to copy my un-minified files to the server: task deployJsLocally(type: Copy) { from files(javascript.source.pcr.js.files) into "$wwwPath/js" } task deployCssLocally(type: Copy) { from css.source.dev.css.files into "$wwwPath/css" } Finally we’d have a deploy task that referenced these and some copy tasks to pull in images, html and other non-css / js artefacts. task deployLocalMinified( description: 'Copy webapp artifacts to local apache', dependsOn: ['copyBootstrapFonts', 'copyJson', 'copyImages', 'addDevBootLibs', 'copyAllHtml', 'deployCombinedJsLocally', 'deployCombinedCssLocally'] ) {} An embedded web server Using a local Apache instance was all good and well, but as I mentioned it did require a developer to sort out their own installation. Gradle comes with a Jetty plugin out of the box and so we utilised this to provide a webserver which served our single page webapp. There was no need to provide a WEB-INF/web.xml, the default Jetty configuration is already configured to look in your webAppDir(src/main/webapp) and serve resources from that. Because it’s running in Jetty and out of the source folder as well, we could change the code out of our IDEs and have it update on the next reload – much better than having a grunt task to keep your source folders and apache in sync. apply plugin: 'jetty' // PROPERTIES (Common to jettyRun/RunWar/Stop tasks) // stop a running jetty stopPort = jettyStopPort as Integer stopKey = jettyStopKey jettyRun.doFirst { logger.quiet('Starting Jetty...') } jettyRun { scanIntervalSeconds = 5 contextPath = '/my-web-app' } IntelliJ has a neat feature that comes from its Webstorm IDE. If you install the LiveEdit plugin and the corresponding Chrome or Firefox extension, your code will update in the IDE whilst you are editing it. This saves a lot of time. You also get access to the debugging tools in IntelliJ as opposed to Chrome Dev Tools. The only tricky thing about setting this up was telling the Debug task in IntelliJ that http://localhost:8080/my-web-app/ was equal to /src/main/webapp so that breakpoints would work. Now any developer can simply check out our project from source control and start running the app by executing gradle jettyRun.It’s an important win – hence I’ve highlighted it. Multiple build files At this stage our build only looks after 3 concerns (CSS build, JS build and deployment) but our build.gradle file is starting to become a bit long. Gradle supports multiple build files so we can organise each concern into a separate build.gradle file to avoid clutter. Each buildscript has its own scope, applying its own plugins. That means that variables, methods and tasks that you or your plugins add are only in the scope of each child file. There isn’t a chance of other plugins polluting the namespace and causing issues you have to spend a lot of time to troubleshoot. They still have access to the project variable added by gradle to get access to configurations in other files if need be. We can define dependencies between our JS and CSS tasks. Our main build.gradle is now very short, with some apply commands like so: apply from: 'gradle/build-css.gradle' apply from: 'gradle/build-js.gradle' apply from: 'gradle/build-webserver.gradle' The buildscript‘s repository and dependency sections that define where to get the 3rd party CSS and JS plugins from can be moved into the relevant build-css and build-js.gradle files. However it’s not quite perfect. I did encounter a classpath issue with one of the JS plugins’ task and simply redefined the dependency in both the main projects build.gradle and the JS file. Overall though, keeping the concerns separate is much nicer for maintainability. Tests We used Jasmine with the JSTestDriver Jasmine plugin for this project. There was a Gradle Jasmine plugin but it’s no longer maintained and doesn’t work with the later versions of Gradle. That said, IntelliJ came to the rescue here and has a built-in Jasmine runner that automatically listens for source and test changes and reruns tests. For both our CI server and for kicking off tests from the command line, the gradle-js-plugin author wrote a task to kick off JSTD with Firefox in a headless browser. We adapted this for our build as follows: task jstd(type: Exec, description: 'runs JS tests through JsTestDriver') { commandLine = ['java', "-jar", "${configurations.testRuntime.files.find()}", '--config', "${projectDir}/jsTestDriverWithJasmine.jstd", '--port','9876', '--tests', 'all', '--testOutput', buildDir, '--browser', 'firefox'] // Then add --verbose flag to commandLine based on // the gradle log level. See later. } There are other JS testing frameworks supported. If you use buster, the Gradle plugin for buster.js keeps a warm VM running in the background checking for changes and re-running your tests. There is a grunt plugin, so you could just run whatever JavaScript tools you want. But as you learnt above, Grunt isn’t the new hotness anymore, so it may have limited appeal. ;-) For us, Grunt still relied on NPM so we left it alone. Quick start and CI friendly The other neat thing that Gradle has is a wrapper script that pulls down the version of Gradle you used to write your Gradle build script with. This is not only good for developers but for any build agents in your CI environment. You don’t need to have Gradle installed, you just clone the project and do gradlew jettyRun and it will go find Gradle, download and install it if it’s not there and then run gradle jettyRun as if you had it right from the beginning. It’s Groovy and it’s groovy If you are already a Javascript dev then much of the Groovy GDK will be easy to relate to. Groovy is dynamically typed as well, but the optional typing will be a welcome facility if you want to take it up. The tooling for Gradle and Groovy out of IntelliJ Community Edition (free) is quite neat. Netbeans (free) has recently made big strides in HTML5 and Javascript support and has strong Groovy support too. This may be a good opportunity to pick up another tool as well that will make your Javascript development easier too. Write your own tasks to do things in the build It’s easy to write your own Gradle tasks. We wrote our own task to modify the base HTML page of our single page webapp to load in different entry point scripts for development and production. We could then wire this task into the build, declare it as a dependency for producing the HTML for our war file or production like jetty deployment that referenced our combined minified gzipped CSS and JS files, or as a dependency to add in the head.js script that called all the individual dev scripts and css. enum DevMode { DEV, PROD, ORIGINAL }; task addProdBootLibs(type: WebAppBootLibTask, description: 'Modify the html of a single page app to include the dev or prod scripts and css') { html = file(webAppDirName + '/index.html') devMode = DevMode.PROD } task addDevBootLibs(type: WebAppBootLibTask, description: 'Modify the html of a single page app to include the dev or prod scripts and css') { html = file(webAppDirName + '/index.html') devMode = DevMode.DEV } task revertBootLibs(type: WebAppBootLibTask, description: 'Revert html to use a placeholder message to remind devs to run the correct script') { html = file(webAppDirName + '/index.html') devMode = DevMode.ORIGINAL } class WebAppBootLibTask extends DefaultTask { final def tag = "" final def endTag = "" @InputFile File html @Input DevMode devMode @OutputFile @Optional File outputHtml def devModeToInclude = [(DevMode.ORIGINAL): ORIGINAL_INCLUDE, (DevMode.DEV): DEV_INCLUDE, (DevMode.PROD): PROD_INCLUDE] final static def ORIGINAL_INCLUDE = """ Libraries Required Run the addDev/ProdBootLibs gradle task to add the correct libraries for the target environment to the project. """ final static def DEV_INCLUDE = """ """ final static def PROD_INCLUDE = """ """ @TaskAction def void applyScriptAndStylesheetTags() { if (html.exists()) { def htmlReplacement = html.getText().replaceAll( "$tag\\p{all}*$endTag", "$tag${devModeToInclude[devMode]}$endTag") logger.info(htmlReplacement) if (outputHtml == null) outputHtml = html outputHtml.setText(htmlReplacement, 'UTF-8') } } } Open your webapp after deploy Because it was Java, we could use the JDK’s java.awt.Desktop to open links to the webapp after a deploy was kicked off locally. import static java.awt.Desktop.* deployLocalMinified << { // open the webapp in your browser desktop.browse "http://localhost/report.html".toURI() } SCP, SSH Because Gradle comes installed with its own ant, it can run any Ant task. We used the ant SSH plugin to SCP our packed distributions to our build box and another task to execute an SSH script remotely to deploy. The deep dark Gradle cookie jar It can get easy to spoil yourself with all of Gradle’s features. The simple stuff is simple but it did take me a few reads of the manual to get my head around it. You can pick out the first 10 chapters and get the basics which should be enough to keep you out of trouble for a while. However venture a little bit out of the norm and then you have to start really thinking about reading a few more chapters. This isn’t a bad thing. The tool in itself is quite broad and the Gradleware team have done a good job of breaking the chapters up to focus on just functionality that you need. But you do get to a point where you start realising that you need to understand more of the Gradle build lifecycle, how external properties work, the different lifecycles of tasks. And I won’t sugarcoat it, since JavaScript is not an official plugin from Gradleware (yet) and not maintained full-time it still has a few quirks both in documentation and the occasional bug. The community around it is good, but you just have to be prepared to roll your sleeves up. The Open Source culture is alive and well both in the Gradle and Groovy communities. When getting stuck it’s pretty easy to clone a plugins git repo to understand how a plugin works. The issues page for the CSS/JS plugins do list a few workarounds too so it’s worth looking there. However, I appreciate that this is not for everyone, especially if you are a Javascript dev with no Java/Groovy experience. Learning this used up more time than I initially planned for Gradle but I do hope JavaScript devs can benefit from this blog to take it up. Also, the experience has been great to learn more about Gradle anyway and thus helped make me aware of things we can apply in our Java builds too. Other lessons learnt So I listed a whole bunch of positives earlier, here were some of the things that irked myself and my colleagues as we picked up the tool. Tasks. When to use << and the task keyword prefix. Some plugins add default tasks. In order to configure them you just go lesscss { source = css.dev.css.files // css sourceset dest = ”$buildDir/out.css” } But the properties are executed every time the script is run for any task. So say I wanted to log what my sourceset had evaluated to, my instinct would be to stick a logging message lesscss { source = css.dev.css.files// css sourceset dest=”$buildDir/out.css” logger.quiet(“Compiling these files $source into $dest”) } If you go run another task, you’ll see that this closure gets evaluated regardless of whether you asked it to or not. gradle –q combineCss Compiling these files [‘src/main/javascript/styles/foo.css’,’src/main/javascript/styles/bar.css’] into /home/username/projects/gradle-js-blog/build … :combineCss task executed successfully. ‘Compiling these files…’? What, thats from lesscss???? Why am I seeing another build task output? What you need is to define the task with doFirst or doLast (shorthand <<) in order to tell Gradle that block isn’t to run during build evaluation. It turns out to be is that lesscss isn’t a task at all. This is called a convention mapping and we are configuring properties in the build evaluation step of Gradle rather than steps to run as part of running the task. It’s a bit confusing since this is also a Groovy bit of code. You can do manipulation on those properties at build evaluation time which is neat but the difference isn’t immediately apparent. Getting the log level flag passed into Gradle Gradle comes with three logging flags. --quiet, --info and --debug. By default Gradle will only show if a task passed, failed or was skipped. Much unlike Maven. Thus, you’ll only use info or debug if you want to see the output of the steps. It would be nice for JS tools to mimic the log level that was passed into the Gradle build. The project object that’s available in every buildscript exposes its logger however calling logger.level returned null. A workaround described on the Gradle forums indicated that using the logger.isXXXXXXEnabled methods worked, so we wrote a task to query this and apply the correct loglevel to the task. task someTaskIdLikeToReuseTheLogLevelPassedToGradle(type: Exec) { // setup commandLine array with path to tool and other flags String runnerMode = getRunnerMode() if (runnerMode) commandLine += ['--verbose', '--runnerMode', runnerMode ] } private String getRunnerMode() { def runnerMode = null if (logger.isDebugEnabled()) { runnerMode = 'DEBUG' } else if (logger.isInfoEnabled()) { runnerMode = 'INFO' } else if (logger.isQuietEnabled()) { runnerMode = 'QUIET' } runnerMode } More to come from the official gradle guys There is more JS support coming. There is a large thread on the public gradle mailing list discussing putting more JS in. Already there are undocumented plugins for Rhino and Coffeescript (you can find these in the Javadoc for Gradle but missing from the users guide and DSL which indicate that they are in progress). File watching is also on the road map so hopefully more plugins can have the watch/build/reload cycle that the buster.js and grunt plugins afford. Haven’t yet covered Artefact resolution from a public repository like Github or Maven Central for JS libs is still a popular topic on the forums. The gradle-js-plugin covers the former with experimental support, and the Gradleware team are working on the later. Because of the problems I had getting npm to work over our proxy I stayed away from this anyway. However require.js support in the gradle-js-plugin out of the box which will get you some of the way there. Build Successful Projects with JavaScript only web front ends are ubiquitous nowadays. Being able to use the same tools to build your front and back end infrastructure allows your team to have a better sense of the project as a whole. There are a number of things that make Gradle attractive – an out of the box web platform to develop with, smart tasks that are more aware of their dependencies, the ability to integrate with the JDK and existing Java tools and the ability to fall back to calling JS command line tools – all provide a huge amount of flexibility. As a Java and Groovy dev, using a familiar tool in Gradle to build for a completely different platform was both a learning and rewarding experience. It did come with its frustrations particularly as you step from pre-configured conventional Gradle build tasks to more specific ones but I think we became better build masters for it. Although I haven’t touched on it in this blog, being able to integrate with a Java web container also availed us to some quick wins when it came to implementing security for our single page webapp. We could rely on the Java Servlet 3 spec to make light work of the authentication and authorisation aspects of our app. In addition, we can package our JavaScript app up as a war and give it to our clients deployment team who just think they are deploying a Java app. Since that’s the infrastructure they maintain and are used to then that works out well for everyone. There is no stopping the JavaScript build tool train, at least for a while. For JavaScript devs looking at their next build tool replacement, Gradle is a worthy choice for any platform. Sure you have to learn Groovy, but it’s not a bad language and JS devs shouldn’t have much trouble adjusting. You may find that if you try Gradle, Groovy and a Java based IDEs and then decide it’s not for you, at least you’ve got some exposure to new tools you didn’t expect to have in your tool box as well as an awareness of how other JS build tools could behave. And that, in my mind at least, is a nice prize for any software practitioner to come away with. This blog was written with Gradle 1.10

over the past few months i've had a series of articles ( part 1 , part 2 , part 3 ) discussing indexeddb. in the last article i built a full, if rather simple, application that let you write notes. (i'm a sucker for note taking applications.) when i built the application, i intentionally did not use a framework. i tried to write nice, clear code of course, but i wanted to avoid anything that wasn't 100% necessary to demonstrate the application and indexeddb. in the perspective of an article, i think this was the right decision to make. i wanted my readers to focus on the feature and not anything else. but i thought this would be an excellent opportunity to try angularjs again. for the most part, this conversion worked perfectly. this may sound lame, but i found myself grinning as i built this application. i'm a firm believer that if something makes you happy then it is probably good for you. ;) i still find myself a bit... not confused... but slowed down by the module system and dependency injection. these are both things i grasp in general, but in angularjs they feel a bit awkward to me. it feels like something i'll never be able to code from memory, but will need to reference older applications to remind me. i'm not saying they are wrong of course, they just don't feel natural to me yet. on the flip side, the binding support is incredible. i love working with html templates and $scope. it feels incredibly powerful. heck, being able to add an input field and use it as a filter in approximately 30 seconds was mind blowing. one issue i ran into and i'm not convinced i created the best solution for was the async nature of indexeddb's database open logic. angularjs has a promises library built in and it works incredibly well for my application in general. but i needed the entire application to be bootstrapped to an async call for database startup. i got around that with two things that felt a bit like a hack. first, my home view (get all notes) ran a call to an init function to ensure the db was already open. so consider this init(): function init() { var deferred = $q.defer(); if(setup) { deferred.resolve(true); return deferred.promise; } var openrequest = window.indexeddb.open("indexeddb_angular",1); openrequest.onerror = function(e) { console.log("error opening db"); console.dir(e); deferred.reject(e.tostring()); }; openrequest.onupgradeneeded = function(e) { var thisdb = e.target.result; var objectstore; //create note os if(!thisdb.objectstorenames.contains("note")) { objectstore = thisdb.createobjectstore("note", { keypath: "id", autoincrement:true }); objectstore.createindex("titlelc", "titlelc", { unique: false }); objectstore.createindex("tags","tags", {unique:false,multientry:true}); } }; openrequest.onsuccess = function(e) { db = e.target.result; db.onerror = function(event) { // generic error handler for all errors targeted at this database's // requests! deferred.reject("database error: " + event.target.errorcode); }; setup=true; deferred.resolve(true); }; return deferred.promise; } this logic is similar to what i had in the non-framework app but i've made use of promises and a flag to remember when i've already opened the database. this lets me then tie to init() in my getnotes logic. function getnotes() { var deferred = $q.defer(); init().then(function() { var result = []; var handleresult = function(event) { var cursor = event.target.result; if (cursor) { result.push({key:cursor.key, title:cursor.value.title, updated:cursor.value.updated}); cursor.continue(); } }; var transaction = db.transaction(["note"], "readonly"); var objectstore = transaction.objectstore("note"); objectstore.opencursor().onsuccess = handleresult; transaction.oncomplete = function(event) { deferred.resolve(result); }; }); return deferred.promise; } all of this worked ok - but i ran into an issue on the other pages of my application. if for example you bookmarked the edit link for a note, you would run into an error. i could have applied the same fix in my service layer (run init first), but it just felt wrong. so instead i did this in my app.js: $rootscope.$on("$routechangestart", function(event,currentroute, previousroute){ if(!persistanceservice.ready() && $location.path() != '/home') { $location.path('/home'); }; }); the ready call was simply a wrapper to the flag variable. so yeah, this worked for me, but i still think there is (probably) a nicer solution. anyway, if you want to check it out, just hit the demo link below. i want to give a shoutout to sharon diorio for giving me a lot of help/tips/support while i built this app. p.s. i assume this is obvious, but i'm not really offering this up as a "best practices" angularjs application. i assume i could have done about every part better. ;)

Note: the following article is purely theoretical. I don’t know if it fits a real-life use-case, but the point is just too good to miss Java’s List sorting has two flavors: one follows the natural ordering of collection objects, the other requires an external comparator. In the first case, Java assumes objects are naturally ordered. From a code point of view, this means types of objects in the list must implement the Comparable interface. For example, such is the case for String and Date objects. If this is not the case, or if objects cannot be compared to one another (because perhapsthey belong to incompatible type as both String and Date). The second case happens when the natural order is not relevant and a comparator has to be implemented. For example, strings are sorted according to the character value, meaning case is relevant. When the use-case requires a case-insensitive sort, the following code will do (using Java 8 enhanced syntax): Collections.sort(strings, (s1, s2) -> s1.compareToIgnoreCase(s2)); The Comparable approach is intrinsic, the Comparator extrinsic; the former case rigid, the latter adaptable to the required context. What applies to lists, however, cannot be applied to Java sets. Objects added to sets have to define equals() and hashCode() and both properties (one could say that it’s only one since they are so coupled together) are intrinsic. There is no way to define an equality that can change depending on the context in the JDK. Enters Trove: The Trove library provide primitive collections with similar APIs to the above. This gap in the JDK is often addressed by using the “wrapper” classes (java.lang.Integer, java.lang.Float, etc.) with Object-based collections. For most applications, however, collections which store primitives directly will require less space and yield significant performance gains. Let’s be frank, Trove is under-documented. However, it offers what is missing regarding extrinsic equality: it provides a dedicated set implementation, that accepts its own extrinsic equality abstraction. A sample code would look like that: HashingStrategy strategy = new MyCustomStrategy(); Set dates = new TCustomHashSet(strategy); A big bonus for using Trove is performance, though: It probably is the first argument to use Trove I never tested that in any context To go further, just have a look at Trove for yourself.