Continuing my Spring-Cloud learning journey, earlier I had covered how to write the infrastructure components of a typical Spring-Cloud and Netflix OSS based micro-services environment - in this specific instance two critical components, Eureka to register and discover services and Spring Cloud Configuration to maintain a centralized repository of configuration for a service. Here I will be showing how I developed two dummy micro-services, one a simple "pong" service and a "ping" service which uses the "pong" service. Sample-Pong microservice The endpoint handling the "ping" requests is a typical Spring MVC based endpoint: @RestController public class PongController { @Value("${reply.message}") private String message; @RequestMapping(value = "/message", method = RequestMethod.POST) public Resource pongMessage(@RequestBody Message input) { return new Resource<>( new MessageAcknowledgement(input.getId(), input.getPayload(), message)); } } It gets a message and responds with an acknowledgement. Here the service utilizes the Configuration server in sourcing the "reply.message" property. So how does the "pong" service find the configuration server, there are potentially two ways - directly by specifying the location of the configuration server, or by finding the Configuration server via Eureka. I am used to an approach where Eureka is considered a source of truth, so in this spirit I am using Eureka to find the Configuration server. Spring Cloud makes this entire flow very simple, all it requires is a "bootstrap.yml" property file with entries along these lines: --- spring: application: name: sample-pong cloud: config: discovery: enabled: true serviceId: SAMPLE-CONFIG eureka: instance: nonSecurePort: ${server.port:8082} client: serviceUrl: defaultZone: http://${eureka.host:localhost}:${eureka.port:8761}/eureka/ The location of Eureka is specified through the "eureka.client.serviceUrl" property and the "spring.cloud.config.discovery.enabled" is set to "true" to specify that the configuration server is discovered via the specified Eureka server. Just a note, this means that the Eureka and the Configuration server have to be completely up before trying to bring up the actual services, they are the pre-requisites and the underlying assumption is that the Infrastructure components are available at the application boot time. The Configuration server has the properties for the "sample-pong" service, this can be validated by using the Config-servers endpoint - http://localhost:8888/sample-pong/default, 8888 is the port where I had specified for the server endpoint, and should respond with a content along these lines: "name": "sample-pong", "profiles": [ "default" ], "label": "master", "propertySources": [ { "name": "classpath:/config/sample-pong.yml", "source": { "reply.message": "Pong" } } ] } As can be seen the "reply.message" property from this central configuration server will be used by the pong service as the acknowledgement message Now to set up this endpoint as a service, all that is required is a Spring-boot based entry point along these lines: @SpringBootApplication @EnableDiscoveryClient public class PongApplication { public static void main(String[] args) { SpringApplication.run(PongApplication.class, args); } } and that completes the code for the "pong" service. Sample-ping micro-service So now onto a consumer of the "pong" micro-service, very imaginatively named the "ping" micro-service. Spring-Cloud and Netflix OSS offer a lot of options to invoke endpoints on Eureka registered services, to summarize the options that I had: 1. Use raw Eureka DiscoveryClient to find the instances hosting a service and make calls using Spring's RestTemplate. 2. Use Ribbon, a client side load balancing solution which can use Eureka to find service instances 3. Use Feign, which provides a declarative way to invoke a service call. It internally uses Ribbon. I went with Feign. All that is required is an interface which shows the contract to invoke the service: package org.bk.consumer.feign; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.springframework.cloud.netflix.feign.FeignClient; import org.springframework.http.MediaType; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.ResponseBody; @FeignClient("samplepong") public interface PongClient { @RequestMapping(method = RequestMethod.POST, value = "/message", produces = MediaType.APPLICATION_JSON_VALUE, consumes = MediaType.APPLICATION_JSON_VALUE) @ResponseBody MessageAcknowledgement sendMessage(@RequestBody Message message); } The annotation @FeignClient("samplepong") internally points to a Ribbon "named" client called "samplepong". This means that there has to be an entry in the property files for this named client, in my case I have these entries in my application.yml file: samplepong: ribbon: DeploymentContextBasedVipAddresses: sample-pong NIWSServerListClassName: com.netflix.niws.loadbalancer.DiscoveryEnabledNIWSServerList ReadTimeout: 5000 MaxAutoRetries: 2 The most important entry here is the "samplepong.ribbon.DeploymentContextBasedVipAddresses" which points to the "pong" services Eureka registration address using which the service instance will be discovered by Ribbon. The rest of the application is a routine Spring Boot application. I have exposed this service call behind Hystrix which guards against service call failures and essentially wraps around this FeignClient: package org.bk.consumer.service; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import org.bk.consumer.domain.Message; import org.bk.consumer.domain.MessageAcknowledgement; import org.bk.consumer.feign.PongClient; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Service; @Service("hystrixPongClient") public class HystrixWrappedPongClient implements PongClient { @Autowired @Qualifier("pongClient") private PongClient feignPongClient; @Override @HystrixCommand(fallbackMethod = "fallBackCall") public MessageAcknowledgement sendMessage(Message message) { return this.feignPongClient.sendMessage(message); } public MessageAcknowledgement fallBackCall(Message message) { MessageAcknowledgement fallback = new MessageAcknowledgement(message.getId(), message.getPayload(), "FAILED SERVICE CALL! - FALLING BACK"); return fallback; } } Boot"ing up I have dockerized my entire set-up, so the simplest way to start up the set of applications is to first build the docker images for all of the artifacts this way: mvn clean package docker:build -DskipTests and bring all of them up using the following command, the assumption being that both docker and docker-compose are available locally: docker-compose up Assuming everything comes up cleanly, Eureka should show all the registered services, at http://dockerhost:8761 url - The UI of the ping application should be available at http://dockerhost:8080 url - Additionally a Hystrix dashboard should be available to monitor the requests to the "pong" app at this url http://dockerhost:8989/hystrix/monitor?stream=http%3A%2F%2Fsampleping%3A8080%2Fhystrix.stream: References 1. The code is available at my github location - https://github.com/bijukunjummen/spring-cloud-ping-pong-sample 2. Most of the code is heavily borrowed from the spring-cloud-samples repository - https://github.com/spring-cloud-samples

When creating a camel route using http, the destination might require a ssl connection with a self signed certificate. Therefore on our http client we should register a TrustManager that suports the certificate. In our case we will use the https4 component of Apache Camel Therefore we should configure the routes and add them to the camel context RouteBuilder routeBuilder = new RouteBuilder() { @Override public void configure() throws Exception { from("http://localhost") .to("https4://securepage"); } }; routeBuilder.addRoutesToCamelContext(camelContext); But before we proceed on starting the camel context we should register the trust store on the component we are going to use. Therefore we should implement a function for creating an ssl context with the trustore. Supposed the jks file that has the certificate imported is located on the root of our classpath. private void registerTrustStore(CamelContext camelContext) { try { KeyStore truststore = KeyStore.getInstance("JKS"); truststore.load(getClass().getClassLoader().getResourceAsStream("example.jks"), "changeit".toCharArray()); TrustManagerFactory trustFactory = TrustManagerFactory.getInstance("SunX509"); trustFactory.init(truststore); SSLContext sslcontext = SSLContext.getInstance("TLS"); sslcontext.init(null, trustFactory.getTrustManagers(), null); SSLSocketFactory factory = new SSLSocketFactory(sslcontext, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER); SchemeRegistry registry = new SchemeRegistry(); final Scheme scheme = new Scheme("https4", 443, factory); registry.register(scheme); HttpComponent http4 = camelContext.getComponent("https4", HttpComponent.class); http4.setHttpClientConfigurer(new HttpClientConfigurer() { @Override public void configureHttpClient(HttpClientBuilder builder) { builder.setSSLSocketFactory(factory); Registry registry = RegistryBuilder.create() .register("https", factory) .build(); HttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager(registry); builder.setConnectionManager(ccm); } }); } catch (IOException e) { e.printStackTrace(); } catch (NoSuchAlgorithmException e) { e.printStackTrace(); } catch (CertificateException e) { e.printStackTrace(); } catch (KeyStoreException e) { e.printStackTrace(); } catch (KeyManagementException e) { e.printStackTrace(); } } After that our route would be able to access the destination securely.

I have just recorded a 5 minute video that demonstrates running an out of stock example from Apache Camel release, the camel-example-servlet packaged as a docker container and running on a kubernetes platform, such as openshift 3. camel-servlet-example scaled up to 3 running containers (pods) which is easy with kubernetes and fabric8 In this video I have already deployed the example and then demonstrates how we can use the fabric8 web console to manage our application. And also connect to the running container and see inside, such as the Camel routes visually as shown above. Then I run a simple bash script from my laptop that sends a HTTP GET to the Camel example and prints the response. The script runs in a endless loop and demonstrates how kubernetes can easily scale up and down multiple Camel containers and load balance across the running containers. And at the end its even self healing when I force killing docker containers. So I suggest to grab a fresh cup of tea or coffee and sit back and play the 5 minutes video. The video is hosted on vimeo and can be seen from this link.

As I get older, I get increasingly to a functionalist position on the world around me. That the social structures and constructs that we see around us are there through a process of evolution and serve some sort of positive benefit because otherwise they would have fallen foul of natural selection. Sometimes this can seem counter-intuitive; crime, for example, has a positive benefit in allowing the vast majority of us to know clearly the boundaries of things that are wrong – a world without law-breaking would probably be a world without laws, not one without crime. When I look at organisations, I seem to see an inexorable move away from structures based on professional expertise. The boundaries by which we define organisations seem to be caught in a previous age, and their utility is becoming less and less. I had the privilege yesterday to spend my time working with a group of Market Insight professionals from a very broad set of organisations. The challenges that they are facing seem in many ways to be linked to a sense of purpose: what the heck is “insight”, and why should a single operational part of an organisation have sole responsibility for it? Organising an organisation by specialism – sales, marketing, operations, production, HR, Finance, IT, legal and so on – is decreasing in function. If that sounds outlandish, remember that most manufacturing organisation moved from specialism-focus to product-focus at the beginning of the last century with the advent of the production line. And today, with matrix management and project-centric management the norm rather than the exception, we’ve implicitly acknowledged the passing of professional specialism. The creation of new divisions like “digital” or “innovation” – at their core, multi-disciplinary activities – again implicitly acknowledges that modern organisations need to have a design based on the things that people are doing, not the skills that they collectively have in common. With the ways in which we can communicate and collaborate (and the breaking down of 9-5 “in the office” through flexible working patterns) the big benefit of specialist operating divisions, communities of practice, becomes lessened dramatically too. Do I gain more professionally (and does my organisation gain more) by surrounding myself with colleagues who have skills rather than necessarily activity in common? This isn’t going to happen overnight. Our professional structures are so ingrained, that it will be at an evolutionary rather than revolutionary pace that change will permeate; some of the memes (like matrix management) are already in place; there are some mutations that will probably wither (looking at you, Holocracy). How to prepare? I’ve heard a lot in recent years about “T-shaped” workers: people with a strong expertise (the down stroke of the T) but with a breadth of knowledge and experience (the horizontal stroke). I personally am hedging more on being “comb-shaped” – many more areas of domain knowledge, and enough understanding to be able to bring in an expert where necessary. The irony that because of follicle challenge I haven’t used a comb in many years isn’t lost…

Written by Craig Wentworth. To understand the furor that’s greeted recent vendor announcements around open source analytics computing engine Spark, and some commentary seemingly setting up a Spark versus Hadoop battle, it’s worth taking a moment to recap on what each actually is (and is not). As I covered in last year’s MWD report on Hadoop and its family of tools, when people talk about Apache Hadoop they’re often referring to a whole framework of tools designed to facilitate distributed parallel processing of large datasets. That processing was traditionally confined to MapReduce batch jobs in Hadoop’s early days, though Hadoop 2 brought the YARN resource scheduler and opened up Hadoop to streaming, real-time querying and a wider array of analytical programming applications (beyond MapReduce). Spark has been designed to run on top of Hadoop’s Distributed File System (amongst other data platforms) as an alternative to MapReduce – tuned for real-time streaming data processing and fast interactive queries, and with multi-genre analytics applicability (machine learning, time series, graph, SQL, streaming out-of-the-box). It gets that speed advantage by caching in-memory (rather than writing interim results to disk, as MapReduce does), but with that approach comes a need for higher-spec physical machines (compared with MapReduce’s tolerance for commodity hardware). So, Spark isn’t about to replace Hadoop -- but it may well supplant MapReduce (especially in growing real-time use cases). Those “Spark vs Hadoop” headlines are about as meaningful as one proclaiming “mushrooms vs pizza." Yes, mushroom might be a more suitable topping than, say, pepperoni (especially in a vegetarian use case), but it’ll still be deployed on the same dough and tomato sauce pizza platform. Nobody’s about to suggest the mushroom should go it alone! But what’s behind the headlines and the hype is a story of enterprise adoption – or at least vendors anticipating that adoption and investing in ‘the weaponization of Spark’ as it faces the more exacting standards of security, scaling performance, consistency, etc. which come with mainstream enterprise deployment. Big names like IBM, Databricks (the company formed by the originators of Spark), and MapR made commitments in and around the Spark Summit earlier this month. MapR has announced three new Quick Start Solutions for its Hadoop distribution to help customers get started with Spark in real-time security log analytics, genome sequencing, and time series analytics; Databricks’ cloud-hosted Spark platform (formerly known as Databricks Cloud) has become generally available; and IBM announced a raft of measures designed to give Spark a significant shot in the arm – it’s open sourcing its SystemML technology to bolster Spark’s machine learning capabilities, integrating Spark into its own analytics platforms, investing in Spark training and education, committing 3,500 of its researchers and developers to work on Spark-related projects, and offering Spark as a service on its Bluemix developer cloud. Given the overlap with Databricks’ business model (of offering development, certification, and support for Spark), IBM’s intentions are likely to tread on some toes before long – but for now, at least, both companies are content to focus on the combined push benefiting the Spark community and its enterprise aspirations overall (though clearly IBM’s betting on all this investment buying it some influence over where Spark goes next). It’s worth bearing in mind that not all its supporters champion Spark wholesale and all the interested parties tend to be interested in particular bits of Spark (as wide-ranging as it is) because of overlaps with their own preferred toolsets. For instance, although Spark supports many analytics genres, Cloudera focuses on its machine learning capabilities (as it has its own SQL-on-Hadoop tool in Impala), and MapR and Hortonworks also promote Drill and Hive as their favoured source of SQL-on-Hadoop. IBM’s support is focused on Spark’s machine learning and in-memory capabilities (hence the SystemML open sourcing news). In the face of such strong vendor preferences, how long before some of Spark’s current features fall away (or at least start to show the effects of being starved of as much care and feeding as is bestowed upon vendors’ favourite Spark components)? The Spark community is at much the same place the Hadoop one was at a while back – it’s showing great promise and suitability in key growth workloads (in Spark’s case, such as real-time IoT applications). However, the product as it stands is too immature for many enterprise tastes. Cue enterprise software vendors stepping up to help grow Spark up fast. Their challenge though is to smooth out the edges without smothering what made it so interesting in the first place.

One mystery in ASP.NET 5 that people are asking me about are ASP.NET 5 console applications. We have web applications running on some web server – what is the point of new type of command-line applications that refer by name to web framework? Here’s my explanation. What we have with ASP.NET 5? CoreCLR – minimal subset of .NET Framework that is ~11MB in size and that supports true side-by-side execution. Yes, your application can specify exact version of CLR it needs to run and it doesn’t conflict with another versions of CLR on same box. DNX runtime – formerly named as K runtime, console based runtime to manage CLR versions, restore packages and run commands that our application defines. Framework level dependency injection – this is something we don’t have with classic console applications that have static entry point but we have it with ASP.NET console applications (they have also method Main but it’s not static). More independence from Visual Studio – it’s easier to build and run applications in build and continuous integration servers as there’s no need (or less need) for Visual Studio and its components. Applications can define their commands for different things like generating EF migrations and running unit tests. Also ASP.NET 5 is more independent from IIS and can be hosted by way smaller servers. Microsoft provides with ASP.NET 5 simple web listener server and new server called Kestrel that is based on libuv and can be used also on Unix-based environments. Application commands Your application can define commands that DNX runtime is able to read from your application configuration file. All these commands are actually ASP.NET console applications that run on command-line with no need for Visual Studio intsalled on your box. When you run command using DNX then DNX is creating instance of class and it looks for method Main(). I come back to those commands in future posts. Framework level dependency injection What we don’t have with classic console applications is framework-level dependency injection. I thinks it’s not easy to implement it when application is actually a class with one static entry point. ASP.NET console applications can be more aware of technical environment around them by supporting dependency injection. Alse we can take our console program to all environments where CoreCLR is running and we don’t have to worry about platform. New environments Speculation alert! All ideas in this little chapter are pure speculation and I have no public or secret sources to refer. This is just what I see that possibly comes in near future. But I’’m not a successful sidekick. CoreCLR can take our ASP.NET applications to different new environments. On Azure cloud we will possibly see that Webjobs can be built as ASP.NET console applications and we can host them with web applications built for CoreCLR. As CoreCLR is very small – remember, just 11MB – I’m almost sure that ASP.NET 5 and console applications will find their way to small devices like RaspberryPi, routers, wearables and so on. It’s possible we don’t need web server support in those environments but we still want use CoreCLR from console. Maybe this market is not big today but it will be huge tomorrow. Wrapping up Although the name “ASP.NET console application” is little confusing we can think of those applications as console applications for DNX. Currently the main usage for those applications are ASP.NET 5 commands but by my speculations we will see much more scenarios for those applications in near future. Related Posts ASP.NET MVC 3: Using controllers scaffolding Visual Studio 2010: Web.config transforms Creating gadget-like blocks for Windows Home Server 2011 web add-in user interface ASP.NET MVC 3: Using multiple view engines in same project Starting with ASP.NET MVC The post What is ASP.NET console application? appeared first on Gunnar Peipman - Programming Blog.

When running a Maven build with many plugins (e.g. the jOOQ or Flyway plugins), you may want to have a closer look under the hood to see what’s going on internally in those plugins, or in your extensions of those plugins. This may not appear obvious when you’re running Maven from the command line, e.g. via: C:\Users\jOOQ\workspace>mvn clean install Luckily, it is rather easy to debug Maven. In order to do so, just create the following batch file on Windows: @ECHO OFF IF "%1" == "off" ( SET MAVEN_OPTS= ) ELSE ( SET MAVEN_OPTS=-Xdebug -Xnoagent -Djava.compile=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005 ) Of course, you can do the same also on a MacOS X or Linux box, by usingexport intead of SET. Now, run the above batch file and proceed again with building: C:\Users\jOOQ\workspace>mvn_debug C:\Users\jOOQ\workspace>mvn clean install Listening for transport dt_socket at address: 5005 Your Maven build will now wait for a debugger client to connect to your JVM on port 5005 (change to any other suitable port). We’ll do that now with Eclipse. Just add a new Remote Java Application that connects on a socket, and hit “Debug”: That’s it. We can now set breakpoints and debug through our Maven process like through any other similar kind of server process. Of course, things work exactly the same way with IntelliJ or NetBeans. Once you’re done debugging your Maven process, simply call the batch again with parameter off: C:\Users\jOOQ\workspace>mvn_debug off C:\Users\jOOQ\workspace>mvn clean install And your Maven builds will no longer be debugged. Happy debugging!

Let’s do a quick review of exception handling with regards to Message Driven Beans. The entry point into a MDB is the overridden onMessage method. It does not provide any scope for throwing checked exceptions and as a result, you will need to propagate unchecked exceptions (subclass of java.lang.RuntimeException) from your code if you want to handle error scenarios. Types of exceptions There are two categories of exceptions defined by the EJB specification and the container differentiates one from the other based on well stated semantics (again, in the EJB specification). Application Exception If you throw a checked exception (not possible for MDB but other EJBs can use this) which is not a java.rmi.RemoteException or it’s subclass, OR a RuntimeException (unchecked) which is annotated with @javax.ejb.ApplicationException, the container treats this as an Application Exception. As a result, it rolls back transaction if specified by the@javax.ejb.ApplicationException rollback attribute and retains the MDB instance for reuse – this is extremely important to note. @ApplicationException(rollback = true) public class InvalidCustomerIDException extends RuntimeException { public InvalidCustomerIDException(){ super(); } } System Exception If you throw a java.rmi.RemoteException (a checked exception) or it’s subclass, OR a RuntimeException (unchecked) which is not annotated with@javax.ejb.ApplicationException, the container treats it as a System Exception. As a result, it executes certain operations like transaction rollback and discards the MDB instance (this is critical). public class SystemExceptionExample extends Exception { public SystemExceptionExample(){ super(); } } What about the critical part ?? It is important to take into account, the discarding of the MDB instance. In case of System Exceptions, the container always discards the instance – so make sure that you are using these exceptions for their intended reason. In case you are using Application Exceptions and they are unchecked ones (they have to be in case of MDBs), make sure you annotate them with @javax.ejb.ApplicationException – this will ensure that the MDB instance itself is not discarded. Under heavy loads, you would want to have as many MDBs in the pool as possible and you would want to avoid MDB instances being moved out of service. Sensible exception handling can help you realize this goal. It’as simple as annotating your exception class with@javax.ejb.ApplicationException and leaving the rest to the container :-) References The EJB (3.2) specification is a 465 page PDF which might look intimidating at the outset, but it’s a great resource nonetheless and not that hard to grasp. In case you want to understand Exception Handling semantics in further detail, please do check out Chapter 9which is dedicated to this topic Cheers!

Spring Integration Kafka 1.2 is out with a major performance overhaul.

Downloading various files (either text or binary) is a bread and butter of every enterprise application. PDF documents, attachments, media, executables, CSV, very large files, etc. Almost every application, sooner or later, will have to provide some form of download. Downloading is implemented in terms of HTTP, so it's important to fully embrace this protocol and take full advantage of it. Especially in Internet facing applications features like caching or user experience are worth considering. This series of articles provides a list of aspects that you might want to consider when implementing all sorts of download servers. Note that I avoid "best practices" term, these are just guidelines that I find useful but are not necessarily always applicable. One of the biggest scalability issues is loading whole file into memory before streaming it. Loading full file into byte[] to later return it e.g. from Spring MVC controller is unpredictable and doesn't scale. The amount of memory your server will consume depends linearly on number of concurrent connections times average file size - factors you don't really want to depend on so much. It's extremely easy to stream contents of a file directly from your server to the client byte-by-byte (with buffering), there are actually many techniques to achieve that. The easiest one is to copy bytes manually: @RequestMapping(method = GET) public void download(OutputStream output) throws IOException { try(final InputStream myFile = openFile()) { IOUtils.copy(myFile, output); } } Your InputStream doesn't even have to be buffered, IOUtils.copy() will take care of that. However this implementation is rather low-level and hard to unit test. Instead I suggest returning Resource: @RestController @RequestMapping("/download") public class DownloadController { private final FileStorage storage; @Autowired public DownloadController(FileStorage storage) { this.storage = storage; } @RequestMapping(method = GET, value = "/{uuid}") public Resource download(@PathVariable UUID uuid) { return storage .findFile(uuid) .map(this::prepareResponse) .orElseGet(this::notFound); } private Resource prepareResponse(FilePointer filePointer) { final InputStream inputStream = filePointer.open(); return new InputStreamResource(inputStream); } private Resource notFound() { throw new NotFoundException(); } } @ResponseStatus(value= HttpStatus.NOT_FOUND) public class NotFoundException extends RuntimeException { } Two abstractions were created to decouple Spring controller from file storage mechanism.FilePointer is a file descriptor, irrespective to where that file was taken. Currently we use one method from it: public interface FilePointer { InputStream open(); //more to come } open() allows reading the actual file, no matter where it comes from (file system, database BLOB, Amazon S3, etc.) We will gradually extend FilePointer to support more advanced features, like file size and MIME type. The process of finding and creatingFilePointers is governed by FileStorage abstraction: public interface FileStorage { Optional findFile(UUID uuid); } Streaming allows us to handle hundreds of concurrent requests without significant impact on memory and GC (only a small buffer is allocated in IOUtils). BTW I am using UUID to identify files rather than names or other form of sequence number. This makes it harder to guess individual resource names, thus more secure (obscure). More on that in next articles. Having this basic setup we can reliably serve lots of concurrent connections with minimal impact on memory. Remember that many components in Spring framework and other libraries (e.g. servlet filters) may buffer full response before returning it. Therefore it's really important to have an integration test trying to download huge file (in tens of GiB) and making sure the application doesn't crash. Writing a download server Part I: Always stream, never keep fully in memory Part II: headers: Last-Modified, ETag and If-None-Match Part III: headers: Content-length and Range Part IV: Implement HEAD operation (efficiently) Part V: Throttle download speed Part VI: Describe what you send (Content-type, et.al.) The sample application developed throughout these articles is available on GitHub.

An effective governance framework is essential for a well-managed intranet. It can be the deciding factor between a good user experience, greatly valued, and a poor user experience with little benefit. Every intranet is different depending on the size, type, and culture of the organisation it supports. However, there are some key governance principles that are common to their success. Recently I spoke at Intranatverk about this based on my book ‘Digital success or digital disaster?‘ which is a practical, experience-based approach to growing and managing a successful intranet. My slides ‘7 principles of good intranet governance’ are avilable for you to share. The alternative to governance can be chaotic anarchy. Posing risks to security and intellectual property provides an awful experience for those who still use your intranet. Where governance can start to get confusing and difficult is in how it is applied. Applying these governance principles leads to a good outcome: Know your organisation Define the scope Put people first Use all resources Compare and benchmark Do what you say you will do Keep it legal Think about how you build a house with the foundations, walls, floors, windows, doors and finally the roof. It would not make sense for you to have windows, doors, and a roof only. The same applies to your governance framework. These principles for good governance are not like a menu that you choose which items to have and leave others alone. You need to follow all of these to build a strong foundation to improve your intranet and implement your strategy. Read the introductory chapter of my new governance book to find out more. A license to share the ebook within your whole organisation is also available.

Spring Data Couchbase provides transparent way to save and load Java classes to and from Couchbase. However, if a loaded class contains a property of unknown class, you will receive org.springframework.data.mapping.model.MappingException: No mapping metadata found for java.lang.Object This may happen if, for example, different versions of your code save and load information. In order to handle situation when we want to load an object, which contains another object on unknown class (in a map or list property) we should override the default SPMappingCouchbaseConverter. Let's see how we do this with Spring XML configuration: I replace my old XML: to the following XML: And create the following class: public class MyMappingCouchbaseConverter extends MappingCouchbaseConverter { public MyMappingCouchbaseConverter(final MappingContext, CouchbasePersistentProperty> mappingContext) { super(mappingContext); } @Override protected R read(final TypeInformation type, final CouchbaseDocument source, final Object parent) { if (Object.class == typeMapper.readType(source, type).type) { return null; } return super.read(type, source, parent); } } Now, if loaded object will contain a property of unknown class or an object of unknown class in a list or map, this property or object will be replaced by null. view source print?

Written by Mark Pollack. Today, we are pleased to announce the general availability of Spring XD 1.2, Spring XD 1.1.3 and the release of Flo for Spring XD Beta. 1.2.0.GA: zip 1.1.3.RELEASE: zip Flo for Spring XD Beta You can also install XD 1.2 using brew and rpm The 1.2 release includes a wide range of new features and improvements. The release journey was an eventful one, mainly due to Spring XD’s popularity with so many different groups, each with their respective request priorities. However the Spring XD team rose to the challenge and it is rewarding to look back and review the amount of innovation delivered to meet our commitments toward simplifying big data complexity. Here is a summary of what we have been busy with for the last 3 months and the value created for the community and our customers. Flo for Spring XD and UI improvements Flo for Spring XD is an HTML5 canvas application that runs on top of the Spring XD runtime, offering a graphical interface for creation, management and monitoring streaming data pipelines. Here is a short screencast showing you how to build an advanced stream definition. You can browse the documentation for additional information and links to additional screen casts of Flo in action. The XD admin screen also includes a new Analytics section that allows you to easily view gauges, counters, field-value counters and aggregate counters. Performance Improvements Anticipating increased high-throughput and low-latency IoT requirements, we’ve made several performance optimizations within the underlying message-bus implementation to deliver several million messages per second transported between Spring XD containers using Kafka as a transport. With these optimizations, we are now on par with the performance from Kafka’s own testing tools. However, we are using the more feature rich Spring Integration Kafka client instead of Kafka’s high level consumer library. For anyone who is interested in reproducing these numbers, please refer to the XD benchmarking blog, which describes the tests performed and infrastructure used in detail. Apache Ambari and Pivotal HD To help automate the deployment of Spring XD on an Apache HadoopⓇ cluster, we added an Apache AmbariⓇ plugin for Spring XD. The plugin is supported on both Pivotal HD 3.0 and Hortonworks HDP 2.2 distributions. We also added support in Spring XD for Pivotal HD 3.0, bringing the total number of Hadoop versions supported to five. New Sources, Processors, Sinks, and Batch Jobs One of Spring XD’s biggest value propositions is its complete set of out-of-the-box data connectivity adapters that can be used to create real-time and batch-based data pipelines, and these require little to no user-code for common use-cases. With the help of community contributions, we now have MongoDB, VideCap, and FTP as source modules, an XSLT-transformer processor, and FTP sink module. The XD team also developed a Cassandra sink and a language-detection processor. Recognizing the important role in the Pivotal Big Data portfolio, we have also added native integration with Pivotal Greenplum Database and Pivotal HAWQ through gpfdist sink for real-time streaming and also support for gpload based batch jobs. Adding to our developer productivity theme and the use of Spring XD in production for high-volume data ingest use-cases, we are delighted to recognize Simon Tao and Yu Cao (EMC² Office of The CTO & Labs China), who have been operationalizing Spring XD data pipelines in production since 2014 and also for the VideCap source module contribution. Their use-case and implementation specifics (in their own words) are below. “There are significant demands to extract insights from large magnitude of unstructured video streams for the video surveillance industry. Prior to being analyzed by data scientists, the video surveillance data needs to be ingested in the first place. To tackle this challenge, we built a highly scalable and extensible video-data ingestion platform using Spring XD. This platform is operationally ready to ingest different kinds of video sources into a centralized Big Data Lake. Given the out-of-the-box features within Spring XD, the platform is designed to allow rich video content processing capabilities such as video transcoding and object detection, etc. The platform also supports various types of video sources—data processors and data exporting destinations (e.g. HDFS, Gemfire XD and Spark)—which are built as custom modules in Spring XD and are highly reusable and composable. With a declarative DSL, a video ingestion stream will be handled by a video ingestion pipeline defined as Directed Acyclic Graph of modules. The pipeline is designed to be deployed in a clustered environment with upstream modules transferring data to downstream ones efficiently via the message bus. The Spring-XD distributed runtime allows each module in the pipeline to have multiple instances that run in parallel on different nodes. By scaling out horizontally, our system is capable of supporting large scale video surveillance deployment with high volume of video data and complex data processing workloads.” Custom Module Registry and HA Support Though we have had the flexibility to configure shared network location for distributed availability of custom modules (via: xd.customModule.home), we also recognized the importance of having the module-registry resilient under failure scenarios—hence, we have an HDFS backed module registry. Having this setup for production deployment provides consistent availability of custom module bits and the flexibility of choices, as needed by the business requirements. Pivotal Cloud Foundry Integration Furthering the Pivotal Cloud Foundry integration efforts, we have made several foundation-level changes to the Spring XD runtime, so we are able to run Spring XD modules as cloud-native Apps in Lattice and Diego. We have aggressive roadmap plans to launch Spring XD on Diego proper. While studying Diego’s Receptor API (written in Go!), we created a Java Receptor API, which is now proposed to Cloud Foundry for incubation. Next Steps We have some very interesting developments on the horizon. Perhaps the most important, we will be launching new projects that focus on message-driven and batch-oriented “data microservices”. These will be built directly on Spring Boot as well as Spring Integration and Spring Batch, respectively. Our main goal is to provide the simplest possible developer experience for creating cloud-native, data-centric microservice apps. In turn, Spring XD 2.0 will be refactored as a layer above those projects, to support the composition of those data microservices into streams and jobs as well as all of the “as a service” aspects that it provides today, but it will have a major focus on deployment to Cloud Foundry and Lattice. We will be posting more on these new projects soon, so stay tuned! Feedback is very important, so please get in touch with questions and comments via * StackOverflowspring-xd tag * Spring JIRA or GitHub Issues Editor’s Note: ©2015 Pivotal Software, Inc. All rights reserved. Pivotal, Pivotal HD, Pivotal Greenplum Database, Pivotal Gemfire and Pivotal Cloud Foundry are trademarks and/or registered trademarks of Pivotal Software, Inc. in the United States and/or other countries. Apache, Apache Hadoop, Hadoop and Apache Ambari are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All Posts Engineering Releases News and Events

Hear the opinion of a Spring and JavaEE developer that wants to share his thoughts on this epic Spring vs JavaEE debate. Covers business and technical aspects.

Learn why 12 Factor Application Patterns, Microservices and CloudFoundry matter when trying to change the way your product is produced.

Spring integration tests allow you to test functionality against a running application. This article shows proper database set- and clean-up with MongoDB.

As developers, we are always looking for a better, faster way of doing things. Whenever I am learning a new language that typically runs in an IDE, then I begin to look for ways to test code snippets through either the Terminal for Mac or the command prompt on Windows. Swift is no exception. As I’ve been working more and more with this language, I’ve uncovered four ways to quickly test Swift code that are not only great for your day-to-day job, but can be used to collaborate and help others learn this new language. #1 : REPL (Read-Eval-Print-Loop) Xcode’s debugger includes an interactive version of the Swift language, known as the REPL (Read-Eval-Print-Loop). This allows you to try out the Swift language within LLDB in Xcode’s console, or from Terminal. If you have at least Xcode 6.1 or higher, then you can simply open your terminal and type: swift You can also invoke it with the following commands on earlier versions of Xcode 6 : xcrun swift lldb --repl It looks like the following: This is great for quick code snippets that you might want to try without launching Xcode. #2 : Swift playgrounds Swift playgrounds are a way to compile and run Swift code live as you type. The results of each line are presented in a timeline as they execute, and variables can be inspected at any point. Playgrounds are typically created as a standalone project (as the image below indicates), but they can be created within an existing Xcode project as well. There are plenty of sample playgrounds out there, and you are free to usemine to get started. Below you will see an example of the timeline in action, providing a visual look of arrays, for loops and more. The obvious reason to use Swift playgrounds is the rich editor that includes syntax highlighting, code completion and more. The disadvantage is that you have to open Xcode in order to do so. #3 : Using an Online Editor SwiftStub has become one of the most popular ways to compile and run Swift code on the fly without requiring a Mac. All you need is a web-browser open to SwiftStub and off you go. It includes the functionality that you would expect, such as a custom URLs and uploading or saving a playground, but it also supports team collaboration. You can easily add people to your current Swift project and even add audio and group chat if neccessary. #4 : Using iTerm2 with Guard-shell This is my preferred environment, but it is geared towards power users that don’t mind spending a few extra minutes setting it up. Don’t worry if you have never done this before as I’ll walk you through the process, step-by-step. I prefer to use iTerm2. Think of it as a replacement for the Terminal app on Mac. In the words of the authors, “iTerm2 brings the terminal into the modern age with features you never knew you always wanted.” I’ve been using it for a couple of months and couldn’t agree more. We are also going to use the help of Guard-shell to automatically run shell commands when watched files are modified. In this case, we’ll be watching files with the .swift extention. Once you have these applications downloaded, you only need to remember a few commands to get started… Within iTerm2, press ⌘D to get a Vertical Split and ⇧⌘D for a horizontal split. Navigate to your home directory and type: vim Guardfile Once you are inside the Guardfile, you will need to switch to “Insert” mode. Simply type the following and when you are finished press “esc” and then type :w to save the file. Type :x to save and exit vim. source 'https://rubygems.org' gem 'guard-shell' You will now have a file named Gemfile and it is time to install the gem. Simply type: bundle install You should then see the following: Fetching gem metadata from https://rubygems.org/............ Fetching version metadata from https://rubygems.org/.. Resolving dependencies... Using hitimes 1.2.2 Using timers 4.0.1 Using celluloid 0.16.0 Installing coderay 1.1.0 Using ffi 1.9.8 Installing formatador 0.2.5 Using rb-fsevent 0.9.4 Using rb-inotify 0.9.5 Using listen 2.9.0 Installing lumberjack 1.0.9 Installing nenv 0.2.0 Installing shellany 0.0.1 Installing notiffany 0.0.6 Installing method_source 0.8.2 Installing slop 3.6.0 Installing pry 0.10.1 Installing thor 0.19.1 Installing guard 2.12.5 Installing guard-compat 1.2.1 Installing guard-shell 0.7.1 Using bundler 1.8.5 Bundle complete! 1 Gemfile dependency, 21 gems now installed. Now would be a good time to create a directory where you want guard-shell to be monitoring for .swift files that have changed. I created a folder called Swift, then ran the following command : bundle exec guard init shell A new file called Guardfile will be created in that folder. Now type vim Guardfile, enter the following lines and save the file the same way you did before. guard :shell do watch(/(.*).swift/) do |m| puts puts puts puts "Running #{m[0]}" puts `swift #{m[0]}` end end Finally type: bundle exec guard If everything worked successfully, then Guard-shell will inform you that it is watching a folder as shown below: Switch over to your left-hand panel and make sure you are in the folder that Guard is watching and type “vim test.swift” and type the following Swift code: var first = "hello" var second = "world" println("\(first) \(second)") Use :w to save the file and see the output in the right-hand panel as shown below. Wrap-up Hopefully you can find a solution that works for your development process out of the four options that I presented today. I assume that, since you are interested in testing Swift code snippets, you are building Swift apps as well. You may be interested in my article on how to build a task app in Swift as well. In addition, Telerik provides several powerful UI componentsfor iOS such as Charts, Calandar, ListView and more. Thanks for reading and sound off in the comments below with your ideal environment.

a few days ago, visual studio 2015 rc was released. among the many updates to .net framework 4.6 with this release, we now have some new utility methods allowing conversion to/from unix timestamps. although these were added primarily to enable more cross-platform support in .net core framework , unix timestamps are also sometimes useful in a windows environment. for instance, unix timestamps are often used to facilitate redis sorted sets where the score is a datetime (since the score can only be a double ). unix timestamp conversion before .net 4.6 until now, you had to implement conversions to/from unix time yourself. that actually isn’t hard to do. by definition , unix time is the number of seconds since 1st january 1970, 00:00:00 utc. thus we can convert from a local datetime to unix time as follows: var datetime = new datetime(2015, 05, 24, 10, 2, 0, datetimekind.local); var epoch = new datetime(1970, 1, 1, 0, 0, 0, datetimekind.utc); var unixdatetime = (datetime.touniversaltime() - epoch).totalseconds; we can convert back to a local datetime as follows: var timespan = timespan.fromseconds(unixdatetime); var localdatetime = new datetime(timespan.ticks).tolocaltime(); unix timestamp conversion in .net 4.6 quoting the visual studio 2015 rc release notes : new methods have been added to support converting datetime to or from unix time. the following apis have been added to datetimeoffset: static datetimeoffset fromunixtimeseconds(long seconds) static datetimeoffset fromunixtimemilliseconds(long milliseconds) long tounixtimeseconds() long tounixtimemilliseconds() so .net 4.6 gives us some new methods, but to use them, you’ll first have to convert from datetime to datetimeoffset. first, make sure you’re targeting the right version of the .net framework: you can then use the new methods: var datetime = new datetime(2015, 05, 24, 10, 2, 0, datetimekind.local); var datetimeoffset = new datetimeoffset(datetime); var unixdatetime = datetimeoffset.tounixtimeseconds(); …and to change back… var localdatetimeoffset = datetimeoffset.fromunixtimeseconds(unixdatetime) .datetime.tolocaltime();

Wondering why the Eclipse Project view might not show all files in the Project Explorer view? For example it shows this: Eclipse Project Explorer View But on disk I have more files? Where are they? :idea: I’m using Eclipse Luna (4.4) in this post, but things are very similar for earlier versions of Eclipse. File System has more Files? Checking the files on my file system (e.g. with the Windows Explorer), I have more files/folders listed: Files on the File System Obviously, files and folders starting with a dot (‘.’) are not shown in the Project Explorer view. Project View Filters The reason is that the Eclipse Project Explorer view has a filter built-in to hide files. There is a setting for this in the View Menu of that view: Project Explorer View Menu There is a ‘Customize View…‘ menu item: Customize Project Explorer View And here I have the filters: Customize Project Explorer View Now it should be clear why the files starting with a dot are ont shown: they are filtered out with the filter for “.*”: Project Explorer View Filters Un-checking that filter will show now as well the dot files: Showing Dot Files in Project Explorer View Defining my own Filters The list of filters in the project view is provided by the plugins. And unfortunately there is no way to define my own filters in the above dialog, unless I would implement my Java plugin. Defining my own filters without programming things is possible, it is just in a different place:-). For example I do not want to see that ‘ProjectInfo.xml’. For this, I select the project and use the ‘Properties’ menu. Inside the project properties, there is Resource > Resource Filters: Resource Filters Use the Add button to add a new filter: Adding Filter To exclude just the ProjectInfo.xml: Exclude all (this will exclude all files matching my filter) Applies to Files (I only want to have it applied to files) [Name] [matches] the file name Excluding Files With that, I can build any kind of filters. Pressing OK, and it gets added to my filter list: Added Filter Now the ProjectInfo.xml is not listed any more in the Project Explorer view: Filtered ProjectView.xml Summary The Eclipse Project Explorer view has a setting to turn on/off filters for files/folders, or in general to configure the view. I can use project resource filters to define my own filters too. Happy Filtering :-)

Besides, Android Studio platform developers also use Eclipse to develop applications, but always thought of Eclipse like a "Student-Project IDE " and learned about it.