The basics of messaging services for Java apps, along with a client example based on ActiveMQ.

This just in, Wordpress is moving to Node.js. Check out what Node.js is, and why you'd want to move from PHP to Node.js.

Docker is an open platform for orchestrating distributed applications.; open source NGINX (and the enhanced and commercially supported version, NGINX Plus) are great use cases for Docker, and this article describes how you can deploy the open source product.

Game developer Nikolaus Gebhardt talks open source game engines, challenges of creating an engine, and working with open source projects.

Hear the case from CUBA why YARG is a solid entry into the already large field of report generators.

Go is awesome and you should learn it. Here's how.

Smart farming is a concept quickly catching on in the agricultural business. Offering high-precision crop control, useful data collection, and automated farming techniques, there are clearly many advantages a networked farm has to offer.

How to create a lwIP project, which is an open source TCP/IP for small systems, using the Kinetis SDK and FreeRTOS on the FRDM-K64F board.

On the heels of the Kinesis Firehose announcement, more people are going to be looking to integrate Kinesis with logging systems. Here is one take on solving that problem that integrates syslog-ng with Kinesis.

What’s good about online schedulers is that you can get an access to your data from any of your devices. Your schedule will be safe if something happens to your laptop or smartphone. But the overall quantity of available schedulers can confuse you. There are JavaScript and .NET scheduling tools. There are third-party plug-ins for big frameworks and built-in parts of these frameworks. And finally, there are stand-alone tools which can be integrated with other components. This article’s aim is to take a look at the four most popular .NET scheduling components to make clear which of them is worthy of attention and good for beginners. Here are the scheduling tools: DayPilot ASP.NET Event Calendar DHTMLX Scheduler .NET Devexpress ASP.NET Scheduler Telerik .NET Scheduler We’ll use Microsoft Visual Studio 2013 to create demo applications. We will check how the basic calendar looks like and behaves. The other important thing is documentation. Good structured documentation pages with understandable step-by-step guides will be a plus. DayPilot ASP.NET Event Calendar DayPilot provides you with different kinds of time management tools: ASP.NET Event Calendar is an event calendar that allows you to plan your day hour by hour: ASP.NET Monthly Calendar will help you create a month schedule: ASP.NET Scheduler is useful if you want to group you events into different categories: ASP.NET Gantt Chart for those who want to build a Gantt Chart: Since Scheduler and Gantt Chart are more specialized tools we’ll take a look at something more commonly used. Let’s say, the Event Calendar. Basic features: Integrated message bar Drag-and-drop Export. Available file formats are BMP, PNG, GIF and JPG Customizable appearance The full list of features is available on this page. If you want to check what you can get from it, check this demo page. The tutorial page is a little bit messy at first sight. There are pages for different kinds of features for all of the DayPilot components. However, it’s not too hard to find the page you want. Which is, in our case, this basic calendar tutorial page. It’s a step-by-step guide which shows how to create a project in Visual Studio, create a database and attach it to your project, etc. It’s a little bit hard to get started with it if you haven’t used Visual Studio before. If you’re lazy enough, you can simply download the source code for this example here. After the job is done, you can check the results: You can’t do much with this simple event calendar. This testing event can be dragged through the calendar grid to change its beginning date. It was created during the calendar creation and by default, there’s no possibility to create a new one. To add this functionality you should spend some more time with tutorials. And you can’t change the scale of the calendar. Therefore, if you want to get a possibility to create long term events as well, you should use both Event Calendar and Monthly Calendar in your project. But here’s some good news. There is an open-source version of Event Calendar available. The cost of DayPilot calendar may vary from $499 to $4999 depending on the license and product version. Well, let’s move along. DHTMLX Scheduler .NET DHTMLX Scheduler .NET is another stand-alone tool that was made to help you create your own online scheduler. Check this demo page to see how it looks like. Basic features: Different views: Decade view, Grid view, Year view Recurring events Drag-and-drop Touchscreen support Easy customizable Integration with Google Maps and Google Calendar You can check all its features on this page. There is more than one possible way to get this scheduler. You can download the package with required files or install them via NuGet or Package Manager Console. This install guide can help you choose the proper way. Let’s talk about documentation. Since DHTMLX Scheduler .NET is basically a .NET wrapper for this JavaScript scheduler, the documentation consists of two parts: server side and client side articles. There are basic calendar creation guides and a bunch of configuration guides. Moreover, there is a video tutorial available. But what we’re interested in right now is how we can create a basic calendar. The developer’s blog contains this tutorial explaining how to build a basic calendar in ASP.NET MVC5. It’s quite detailed and contains a bunch of screenshots. And here’s what we got: This time we can change the scale. There are “Day”, “Week” and “Month” buttons available. The current day is highlighted and there is a “Today” button that will help you in case you lost yourself within the calendar grid. There are several ways of how you can create a new task. You can use drag-and-drop with this calendar. So, select the time period you want and the event property window will appear on your screen: You can also use right click to get access to this window, but in this case you’ll have to set the time period by yourself. Scale changing feature is useful if you want to create a long term event: You can drag the event to change its start date. This is all that you can get by default. The important thing is that, unlike the DayPilot tool, this scheduler can be used to show days, weeks and months. This basic app, as you can see, gives you a possibility to add and change the tasks. Documentation structure is easy to understand, so it’s not too hard to use it to improve your application. The cost of DHTMLX Scheduler .NET is $299 for Commercial License and $979 for Enterprise license. Devexpress ASP.NET Scheduler DevExpress provides you with huge amount of different tools. There are WinForms, iOS, ASP.NET components for desktop and online applications. Very detailed subscription of Devexpress scheduler is available on this page. You can get the installation file on this download page. There’s no possibility to install the scheduler separately from other .NET and MVC components. There’s a lot of documentation. Tons of it. For every single component. The scheduler documentation page is separated into several parts: “Start Here”, Learn More”, “How To” and “Reference”. Each of them consists of chapters that contain a bunch of articles on a specific topic. Everything that you need to know about the every single tool. But if you want to create a basic scheduler fast, you can check two video tutorials available on YouTube: The Project Wizard which will tell you how to create a Devexpress project and Getting Started with the ASPxScheduler which is about the scheduler creation process. This 14 minutes is all that you need to create your first online application. The result is shown below: As you can see, there are a couple of scales available: “Day”, “Work Week”, “Full Week” and “Month”. Moreover, there is a Timeline mode, which can help you trace the current week events. A navigation panel on the right side will help you switch between month and years. Drag-and-drop behavior is available, so you can select the proper area of the calendar grid to create a new event. After the right-click you can choose what you want to do: The properties window will appear. It allows you to set the basic properties (Start time, End time, Subject, etc.) and a couple of additional ones (Label, for example): According to these properties, different events will look differently, which is pretty handy. Here’s the Timeline mode as an example: The cost of Devexpress scheduer is $899.99 for the .NET bundle. Telerik .NET Scheduler Just like the previous developer, Telerik provides you with a wide variety of tools: a mobile application platform for iOS, Android and Windows Mobile; HTML5 and JavaScript widgets for web applications; .NET components for desktop, mobile and online applications and much more. But we’re here for one reason. RadScheduler page contains a detailed list of features with a demo page for each of them. And, like in case of Devexpress, this time we can’t download the scheduler separately. We should get the whole .NET bundle. The documentation page contains comprehensive information about available components. There are tutorials, API documentation, videos, infographics and other useful stuff. It may be a little bit complicated to define where to start from. But there is a YouTube video tutorial that allows you to dive straight into the action instead of reading tons of docs. First 13 minutes are enough to get required basic information about how the things work. As a result you will get something like this: There is something familiar with this one. We can choose between “Day”, “Week”, “Month” and “Timeline” modes. The last one works pretty much the same as the Devexpress Timeline option. This time we have a mini calendar which is useful for navigation. There’s also a “Show full/business hours” toggle button that allows you to separate your work events from the private ones: New event creation is intuitive and similar to other calendars. This time there is a wide variety of properties that allows you to configure the recurrent events. And, as usual, drag-and-drop is available. The cost of Telerik scheduler is $999 for the .NET bundle. Conclusion Described schedulers can be divided into two groups. The first two of them are stand-alone tools. In case of DayPilot there’s no possibility to get the calendar you like. The only possible way to get it is to buy the whole bundle. DHTMLX Scheduler .NET is the only component that can be bought separately from any other tool, which is great if you’re not interested in additional components. The remaining two components are the part of big .NET bundles. And maybe it’s not a good idea to buy them if you want to create a simple online calendar. DayPilot and DHTMLX Scheduler .NET DayPilot components look great. The documentation contains detailed tutorials. The bad thing is that the basic scheduler which was built according to the beginner’s guide is kinda low functional. To make something really useful you should spend some more time with it. There’s also no possibility to change the scale of a calendar, so you have to chose either Event Calendar or Monthly Calendar to plan your events properly. The good news is that, besides the paid components, there is an open-source version available. You can get it here. DHTMLX Scheduler .NET is more attractive, especially for beginners. There are step-by-step guides of different levels of complexity and a video tutorial. The basic app looks pretty useful. It allows you to change the calendar scale, add and configure the tasks. There are Commercial and Enterprise versions available on this page. You can also get the 30-days trial version. As an alternative, you can try the JavaScript version of this scheduler. You have two options with this one: commercial Pro Edition and open-source Standard edition. Devexpress ASP.NET Scheduler and Telerik .NET Scheduler Those two are the components of big and advanced sets of tools. They’re integrated with Visual Studio. As a result, there’s a project wizard for both of them that allows you to configure your application properties. The documentation contains dozens of pages and it may confuse you at the first glance. But you can find an article about the every single property if you spend some time with it. There are video tutorials available as well, so everyone can find the most convenient way of learning. The Devexpress documentation structure looks more comfortable. Every big theme is separated into several parts which are grouped according to the difficulty level. The basic scheduler is pretty functional. For example, there is a possibility to set a proper label for a task. There are a lot of possible combinations of tools that you can buy. But it’s always good to try some before to make a decision and trial versions will help you with that. Check the details on this page. Telerik documentation is not so well grouped which can confuse the beginners. Scheduler creation is intuitive and the basic app is pretty usable. Just like in the previous case, Telerik provides you with a wide variety of possible bundles of tools. And, once again, you can try it before buy it. Trial versions are waiting for you.

[This article was written by Juergen Hoeller.] Spring is well-known to actively support the latest versions of common open source projects out there, e.g. Hibernate and Jackson but also common server engines such as Tomcat and Jetty. We usually do this in a backwards-compatible fashion, supporting older versions at the same time - either through reflective adaptation or through separate support packages. This allows for applications to selectively decide about upgrades, e.g. upgrading to the latest Spring and Jackson versions while preserving an existing Hibernate 3 investment. With the upcoming Spring Framework 4.2, we are taking the opportunity to support quite a list of new open source project versions, including some rather major ones: Hibernate ORM 5.0 Hibernate Validator 5.2 Undertow 1.2 / WildFly 9 Jackson 2.6 Jetty 9.3 Reactor 2.0 SockJS 1.0 final Moneta 1.0 (the JSR-354 Money & Currency reference implementation) While early support for the above is shipping in the Spring Framework 4.2 RCs already, the ultimate point that we’re working towards is of course 4.2 GA - scheduled for July 15th. At this point, we’re eagerly waiting for Hibernate ORM 5.0 and Hibernate Validator 5.2 to GA (both of them are currently at RC1), as well as WildFly 9.0 (currently at RC2) and Jackson 2.6 (currently at RC3). Tight timing… By our own 4.2 GA on July 15th, we’ll keep supporting the latest release candidates, rolling any remaining GA support into our 4.2.1 if necessary. If you’d like to give some of those current release candidates a try with Spring, let us know how it goes. Now is a perfect time for such feedback towards Spring Framework 4.2 GA! P.S.: Note that you may of course keep using e.g. Hibernate ORM 3.6+ and Hibernate Validator 4.3+ even with Spring Framework 4.2. A migration to Hibernate ORM 5.0 in particular is likely to affect quite a bit of your setup, so we only recommend it in a major revision of your application, whereas Spring Framework 4.2 itself is designed as a straightforward upgrade path with no impact on existing code and therefore immediately recommended to all users.

Rework is Hell “Software may be eating the world, but rework is choking software”, tweeted John Jeremiah (@j_jeremiah). To shed more light on what is choking software, new data was released last week in the 2015 State of the Software Supply Chain Report. In its discussion of application quality and integrity, the report revealed that the average application includes 106 open source components. It is clear that the use of these components has benefited development tremendously in helping to speed time to market and improve innovation. While the benefits are undeniable, development teams are also delivering applications that are “insecure by design”. Of the 106 components per application, the report’s analysis revealed an average of 24 (i.e., 23%) have known critical or severe security vulnerabilities. Those same apps also showed an average of 9 restrictive license types (e.g., GPL, AGPL, LGPL). By electively sourcing components with known vulnerabilities and potential license risks, software development teams are building up higher levels of technical and security debt. And in order to reduce some of that debt, unplanned and unscheduled rework is often required. No Developer Intentionally Uses Bad Parts I have never met a developer that intentionally wanted to use “bad” components in their applications. That said, I have also never met a developer that wanted to spend four hours researching the latest versions, open source licenses and known security vulnerabilities for components (including dependencies) they wanted to use in their applications. While 57% of development organizations have internal policies in place that direct “thou shall not use components with poor quality and integrity”, many have stated that the policies are not followed or are simply not enforced. The problem is one of volume and velocity, while people clutch to old processes that simply can’t keep pace. The analysis in the 2015 State of the Software Supply Chain report revealed that billions of open source downloads occurred in 2014, of which 6.2% had known vulnerabilities (1 in 16 downloads). For some large companies, the volume of downloads exceeded 240,000 — where 15,000 (7.5%) had known security vulnerabilities. At these volumes, manual processes and paper-based policies to prevent poor quality or risky components from getting into your software have no possibility of keeping pace. While software development practices aim for higher velocity, through component-based development as well as Agile, continuous and DevOps practices, the processes to keep up with it have not changed enough. The only way to keep pace with this velocity is through automation. And if we want to avoid unplanned rework associated with remediating known security vulnerabilities or selecting alternative components with approved license types, automation has to become the norm rather than the exception within development. At Speed “If I look at the mass, I will never act. If I look at the one, I will.” — Mother Teresa Looking at the massive volume and velocity of open source and other artifact consumption can be discouraging and overwhelming. However, the volume and velocity within our software supply chains will not diminish—and without a new approach, the volume of unchecked quality and integrity of parts being consumed will continue to build up as technical debt. Automation in areas of testing, build, and deployment has provided significant performance benefits. Likewise, investments in software supply chain automation have shown markedly improved efficiency and controlled risk, as the best practices in the 2015 Report illustrate. Automation can unleash the potential of an organization’s development capacity. Rarely is there such an opportunity to simultaneously increase speed, efficiency, and quality. In order to improve the quality and integrity of components used in our applications, we need to release our clutch on old practices. We can further embrace automation across our software supply chains — and improve the quality of our applications — by considering these three steps: 1. Start with creating a software bill of materials for one application. Visibility into one application can help you better understand your current component usage. A number of free and paid services are available to help you create a software bill of materials within a few minutes. The bill of materials will help you to identify the unique component parts used within your application and the suppliers who contributed them. These reports list all components used, and several services also identify component age, popularity, version numbers, licenses, and known vulnerabilities. 2. Design approval processes to be frictionless, scalable, and automated. Manual reviews of components and suppliers cannot keep pace with the current volume and velocity of consumption. Your organization must not only define your policies for supplier and part selection but also find practical ways to enforce them without slowing down development or inadvertently encouraging workarounds. Policies must be agile enough to keep pace with modern development. Strive to automate policy enforcement and minimize drag on developers. 3. Enable developer decision support. Developers are primary consumers of components in software supply chains. They initiate every component request. Help developers by automating the availability of information on component versions, age, popularity, licenses, and known vulnerabilities within their existing development tools so it is easy to pick the best components from the start. By selecting the highest-quality components from the highest-quality suppliers early, you will improve developer productivity and reduce costs. Here is the complete 2015 State of the Software Supply Chain Report. I will also be discussing more of the Report findings and best practices on a webinar scheduled for Wednesday, June 24, 12pm ET (register or view it on-demand). The previous blog in this series was, Fewer and Better Suppliers. Next up, I will be reviewing Visibility and Traceability practices across the software supply chain with more commentary from the 2015 State of the Software Supply Chain Report.

That Supplier is Better For You Since releasing the 2015 State of the Software Supply Chain Report, there has been a lot of great discussion across the industry on best practices for managing the complexity introduced by the volume and velocity of the components used across your software supply chain. Today I want to focus on the huge ecosystem of open source projects (“suppliers”) that feed a steady stream of innovative components into our software supply chains. In the Java ecosystem alone, there are now over 108,000 suppliers of open source components. Across all component types available to developers (e.g., RubyGems, NuGet, npm, Bower, PyPI, etc.), estimates now reach over 650,000 suppliers of open source projects. Source: ModuleCounts.com However, like in traditional manufacturing, not all suppliers deliver parts of comparable quality and integrity. My latest research, the 2015 State of the Software Supply Chain Report, shows that some open source projects use restrictive licenses and vulnerable sub-components, while other projects are far more diligent at updating the overall quality of their components. Choosing the best and fewest suppliers can improve the quality and integrity of the applications we deliver to our customers. While I am hosting a webinar next week to share many of the detailed report findings, I wanted to share a few of the more meaningful stats here. Your 7,600 Suppliers My research for the report revealed many new perspectives on “suppliers” across the software supply chains. First of all, I saw that the average large development organization consumed over 240,000 open source components last year — sourced from over 7,600 open source projects. Average number of open source component downloads per organization in calendar 2014 On the surface, the huge reliance on open source projects is a great thing. Development teams have chosen to not write those pieces themselves, but have sourced the needed components from outside suppliers. This practice speeds development, enables more innovation, and ensures time-to-release goals are achieved. The use of open source is so prolific today, few of us could ever imagine reducing the use of those components and their suppliers in the future. At the same time that we benefit from open source, our high paced, high volume consumption practices don’t allow us the time needed to do the due diligence on the suppliers or open source projects where we source our component parts from. Average number of known vulnerabilities downloaded by organization in calendar 2014. For example, of the 240,000 average component downloads in 2014, the same businesses sourced an average of 15,000 components that included known security vulnerabilities. In many cases, developers were downloading vulnerable component versions, when safer versions of those same components were available from the open source projects. While no one intends to download components with known vulnerabilities, the problem is exacerbated due to the lack the visibility into a better recommended version. Fewer Suppliers, Less Context Switching Choosing an open source project supplier should be considered an important strategic decision in organizations because changing a supplier (“open source project”) used is far more effort than swapping out a specific component. Like traditional suppliers, open source projects have good and bad practices impacting the overall quality of their component parts. Traditional manufacturing supply chains intentionally select specific parts from approved suppliers. They also rely on formalized sourcing and procurement practices. This practice also focuses the organization on using the best and fewest suppliers — an effort that improves quality, reduces context switching, and also accelerates mean time to repair when defects are discovered. One industry example from the report describes how Toyota manages 125 suppliers for their Prius to help sustain competitive advantages over GM who manages over 800 suppliers for the Chevy Volt. By contrast, development teams working with software supply chains often rely on an unchecked variety of supply, where each developer or development team can make their own sourcing and procurement decisions. The effort of managing over 7,600 suppliers introduces a drag on development and is contrary to their need to develop faster as part of agile, continuous delivery and devops practices. Coming to Terms When you come to terms with the volume of consumption and the massive ecosystem of suppliers you can source your components from, you quickly realize it is impossible to address this issue with a manual review process. Any organizations clutching to these outdated manual practices are will continue to be outgunned by the velocity by their software supply chains. Just as traditional manufacturing supply chains have turned to automation, software development teams need to take the same approach by further automating their software supply chains. Information about suppliers and the quality of their projects needs to be made available to developers at the time they are selecting components. Information about the latest versions, features, licenses, known vulnerabilities, popularity of versions being used, and the cadence of new releases should be made available to developers in an automated way. Automating the availability of this information about suppliers can lead to better and fewer suppliers being used. Be sure to read the full 2015 State of the Software Supply Chain Report for more information about open source suppliers and organizations sourcing practices. The report also highlights current and best practices being used in organizations that are managing their use of suppliers that feed their software supply chains.

The use of OpenStack in the private cloud is invaluable for DevOps. It provides engineers the ability to innovate quickly and deal with uncertainty. It also maximizes existing infrastructure and provides a programmable, software-defined IaC. Openstack in the private cloud = agile development OpenStack has emerged as the de facto standard for IaaS in the private cloud. It gives engineers a vital self-service capability to provision (and de-provision) environments, allowing them to act autonomously, in the moment. This helps to eliminate the downstream bottleneck caused by waiting for operations staff to find time to do the provisioning. As OpenStack is open source it is vendor agnostic, allowing you to take advantage of competitive pricing rather than suffering from vendor lock-in. A private cloud means lower cost for the same capacity in a public cloud, which is especially useful for enterprises with high data needs. For security reasons, OpenStack is still mainly used in the private cloud by developers and QA, i.e. in a non-production context. However, OpenStack gives an ability to optimize application performance and/or security by having more control compared to public cloud. The software is increasingly backed by the critical mass of leading IT infrastructure vendors such as IBM, CICSO and HP. Gartner assumes that “by 2019, OpenStack enterprise deployments will grow tenfold, up from just hundreds of production deployments today, due to increased maturity and growing ecosystem support.”1 Challenges to consider OpenStack implementation skills are still rare in the market, so experimentation and self-learning is necessary. Although this takes time, it is offset by the fact the software is free and represents a good opportunity to gain internal expertise. This is particularly valid if you class infrastructure as a core competence. The maturity and functionality of OpenStack projects vary widely - while it covers storage, network and compute, the main adoption currently happens around compute (Nova) and block storage (Cinder), with object storage and network (Neutron) lacking significantly behind. However, without leveraging virtualized network services as part of a private cloud, full-stack environment provisioning is not possible, so don’t forget to add necessary network services to your private cloud. Where to begin Integrating OpenStack clouds with existing infrastructure can be a challenge. It is hardly plug and play. At first, it is best to focus on relatively isolated DevOps environments, such as Gartner’s “mode two”2 applications rather than introducing open stack across the board straight away, (Bimodal IT “refers to having two modes of IT, each designed to develop and deliver information – and technology – intensive services in its own way. Mode 1 is traditional, emphasizing scalability, efficiency, safety and accuracy. Mode 2 is nonsequential, emphasizing agility and speed.”3) As with any open source software, new functions and upgrades are frequently released. This means keeping up with changes in functionality and filling gaps with customizations or third-party products. Upgrades are complex and typically require planned downtime. For these reasons, we recommend choosing a hardened distribution and sticking with it. Openstack is the most complete vendor agnostic solution for storage, network and compute services. The ability for developers to instantly spin up environments at any time is invaluable for a fully agile DevOps environment, and is well worth the effort it takes to acclimatize to Openstack. 1 http://www.prnewswire.com/news-releases/suse-openstack-cloud-5-to-simplify-private-cloud-management-300048721.html 2 http://www.gartner.com/it-glossary/bimodal 3 http://www.gartner.com/it-glossary/bimodal

[This article was written by Ophir Prusak] Guest author Ophir Prusak is chief evangelist atBlazeMeter. To learn more about load and performance testing automation, he invites readers toattend a meetupthis Wednesday, June 24, at New Relic’s San Francisco offices. Performance and load testing are kind of like flossing your teeth. You know you need to do it, but you might not be doing it as much as you should. When your site goes down because it couldn’t handle the load, you look back and realize you might have easily prevented it with a little more testing in advance. That’s why companies are automating their application testing in an effort to lower costs, increase efficiency, and reduce the time needed to release new features. The importance of automated testing in a continuous delivery era Continuous Delivery (CD) is rapidly emerging as the “new normal” in software development, as Perforce discovered in an independent survey, with an estimated 80% of SaaS companies and 51% of non-SaaS companies adopting this practice. Companies that provide Software-as-a-Service know they need to be continuously creating new features, updating their websites, and optimizing their backend. But while software development has adapted nicely in terms of automation, the testing side has moved more slowly. For a fully Continuous Delivery and Integration process to be realized, performance testing must be automated. As the need for testing increases, doing it manually can dramatically increase your time to release. Automating testing throughout the CD process can help detect errors instantly and deliver software faster. Making it work JMeter is the de facto standard in open source load testing. It’s the most widely used open source tool for performance testing for a good reason. There’s virtually nothing it can’t test (websites, native mobile applications, APIs, and Web applications) and it’s extremely powerful and fully featured. Yet there are challenges. JMeter poses a steep learning curve in terms of integration and ease of use. Additionally, it doesn’t integrate easily with APM and Continuous Integration (CI) tools. Many developers have been looking for a way to conduct performance testing with less time and effort—and fewer hiccups along the way. Taurus: An effort to simplify test automation A new open source project called Taurus (Test AUtomation Running Smoothly) is designed to provide exactly that—a way to remove most of the pain of using JMeter on its own. Taurus can give you the ability to Create and define a load test even without using JMeter. Override existing JMeter files or tests configurations. Create human-readable configuration files and testing scripts that are easily added to source control systems like GitHub. Integrate into CI tools like Jenkins. Run multiple tests in parallel. Provide pass/fail criteria back into the CI tool for easier automation of test-results analysis. Make analysis of test results easier and more intuitive. Taurus still uses JMeter under the hood, but is designed to have a much easier learning curve, especially for simple tests. Taurus also offers a built-in result analysis engine that provides both console-based reporting features and result analysis. Performance testing and optimizing your applications is not simple, yet there are solutions available that make the process easier and more successful. I’m looking forward to seeing how the technology evolves even further in the near future. If you want to learn more about Taurus, check out the project on GitHub. Better yet, you are invited to come to a meetup this Wednesday, June 24, at New Relic’s San Francisco offices. You can learn a lot more about Taurus and how you can use it to help scale load and performance testing automation.

What is strus ? The project strus is a collection of libraries and tools written in C++ to build a competitive search engine. Currently it is a single person project that started in September 2014 and therefore the competitiveness in terms of features of the software is more a promise than a fact. It definitely needs more brain to be put into it to catch up with the big players for open source search engines Lucene and Xapian. But strus is not only a me-too-project for search. Strus introduces expression matching and information extraction on a different level than other known open source engines (read more…). Strus simplifies the architecture of a search engine by “outsourcing” of components like the key/value store database storing the data blocks. This componentization (see components of strus) reduces the amount of code drastically and it raises opportunities for experts on a specific topic to contribute (read more…). Strus is not the first attempt to try that, but it is the first attempt as open source project, that has a performance within reach of the big open source search engines. And it does that without a 10 years history of optimization in the back. Strus might not be there at eye level, but let’s see what happens, if more different reasoning and competition is put into it. For who is strus ? People I would primarily like to address with this blog are developers or hackers as potential contributors or for feedback. On the other hand the project could already be interesting for experimental projects that can afford to go along with the development of strus. As stakeholder you can influence the project too. As the demo project, the search on the complete Wikipedia collection (English) shows, it is already possible to build projects, but you have to be aware, that dead lines should not exist, because you might hit a point where a feature you need is not instantaneously available. Project planning gets difficult at the current stage. Furthermore the state of documentation is still quite poor. Programming paradigms All interfaces of strus are pure. No inheritance is used in the main header files. Strus is more a lego thing than a provider of solution classes. If you want for example to build a sequence of terms as feature for your search, you have to build its expression tree with help of a stack, rather than picking a class that implements a sequence query. In PHP this looks as follows: $terms = [ “hello”, “world” ]; $query->pushTerm( “word”/*feature type*/, $term[0] ); $query->pushTerm( “word”/*feature type*/, $term[1] ); $query->pushExpression( “sequence”, 2/*nof terms*/, 2/*position range*/); $query->defineFeature( “docfeat” /*name addressing this feature set*/); The number of interface classes is small (see for example the interface classes of the core), but you have to understand them. If you want to contribute, you should also have a closer look at the programing guidelines. Try it There exist a guide how to fetch, build and install strus. Unfortunately a tutorial is still missing. There will be one soon ! Support I will reply to questions. Please mail me to contact at project dash strus dot net. Thanks I want to thank the authors of LevelDB here. I was looking for some time for a key/value store database that had an upper bound seek function in the interface. The upper bound seek is crucial because it allows you to minimize block accesses on disk when joining sets. A key/value store without upper bound seek would have forced me to create virtual blocks that point to other blocks. This would mean more disk accesses to fetch the data blocks needed. LevelDB has it. Any other alternative candidate to implement the database interface has to have it too. Social Media Github: patrickfrey Twitter: @ProjectStrus

Survey of open source PostgreSQL users found adoption continues to rise with 55% of users deploying it for mission-critical applications Bedford, MA – June 23, 2015 – EnterpriseDB (EDB), the leading provider of enterprise-class Postgres products and database compatibility solutions, today announced the results of its “PostgreSQL Adoption Survey 2015,” a biennial survey of open source PostgreSQL users. Conducted by EnterpriseDB, the survey found PostgreSQL adoption continuing to rise, with 55% of users – up from 40% two years ago – deploying it for mission-critical applications and 77% of users are dedicating all new application deployments to PostgreSQL. These findings give voice to end users and confirm such industry indicators as increasing job listings and monthly rankings on DB-Engines that have pointed to rising interest in and demand for PostgreSQL, also called Postgres. The growing popularity of Postgres also comes as traditional software vendors suffer setbacks in the marketplace. The enterprise-class performance, security and stability of Postgres, on par with traditional database vendors for most corporate workloads, meanwhile have helped position Postgres among the solutions from the world’s largest vendors. The opportunity to transform their data center economics has helped fuel downloads of Postgres as well. End users reported cutting costs with Postgres, with 41% reporting they had first-year cost savings of 50% or more. They’re using Postgres to build web 2.0 applications using unstructured data as evidenced by the 64% of respondents who said they were working with JSON/JSONB and the 47% who said they were using Postgres for collaboration applications. “Postgres is empowering organizations to transform the economics of IT. IT can invest in the customer engagement applications that differentiate their operations from their competition instead of continuing to pay the steep and rising licensing and support fees charged by traditional database vendors,” said Marc Linster, senior vice president of products and services of EnterpriseDB. “With the expanding adoption, EnterpriseDB has experienced dramatic growth year over year, providing the software, services and support that organizations need to be successful with Postgres.” Database Migrations, Replacements The findings also support statements in a recent Gartner report that reflect the widespread acceptance of open source databases. “By 2018, more than 70% of new in-house applications will be developed on an OSDBMS, and 50% of existing commercial RDBMS instances will have been converted or will be in process,” according to the April 2015 Gartner report, The State of Open-Source RDBMs, 2015.* Among Postgres users, the survey findings show migrations are already under way with 37% reporting they had migrated applications from Oracle or Microsoft SQL Server to Postgres. Many users were still planning further migrations, with 37% of PostgreSQL users saying they will gradually replace their legacy systems with Postgres, compared to 29% who said that in the 2013 survey. Further, end users predict their deployments of Postgres will expand significantly, with 32% saying they anticipate production deployments of Postgres to increase by at least 50% over the next year. The survey, conducted by EnterpriseDB using an online tool in May 2015, queried registered users of PostgreSQL and drew 274 respondents worldwide from government organizations and companies ranging in size and industry. *The State of Open-Source RDBMs, 2015, by Donald Feinberg and Merv Adrian, published on April 21, 2015. Connect with EnterpriseDB Read the blog: http://blogs.enterprisedb.com/ Follow us on Twitter: http://www.twitter.com/enterprisedb Become a fan on Facebook: http://www.facebook.com/EnterpriseDB?ref=ts Join us on Google+: https://plus.google.com/108046988421677398468 Connect on LinkedIn: http://www.linkedin.com/company/enterprisedb

[This article was written by Kevin Casey] Welcome to This Week in Modern Software, orTWiMS, New Relic’s weekly roundup of the need-to-know news, stories, and events of interest surrounding software analytics, cloud computing, application monitoring, development methodologies, programming languages, and the myriad of other issues that influence modern software. This week, our top story goes inside President Obama’s secret team of tech geeks, 140 of them and counting: TWiMS Top Story: Inside Obama’s Stealth Startup—Fast Company What it’s about:If the President of the United States walked into the room and personally recruited you to rebuild the country’s technology infrastructure, could you turn him down? He’s serious, and that room is theRoosevelt Room in the West Wing of the White House, by the way. AsLisa Gelobtersays: “What are you going to say that?” Gelobter’s answer was “Yes”—she’s now chief digital officer for the US Department of Education, part of a 140-person-and-counting tech team that’s functioning something like an elite startup embedded inside the federal government. Its business? Only modernizing the technical infrastructure, applications, and processes of just about every federal agency. Why you should care:What was once something of a tech desert—the federal government—is beginning to draw top private-sector talent inside the Beltway. The team, led by Mikey Dickerson (who helped lead the team that rescuedHealthcare.gov) andformer US CTO Todd Park, also includes the likes of former Googler Matthew Weaver, and it hopes to hit 500 people by the end 2016, shortly before President Obama will leave office. Its challenges are immense, from tackling government bureaucracy (to test just how entrenched the suits were, Weaver requested the official title “Rogue Leader”—and he got it) to the fact that its recruiting pitch includes the phrase: “You’ll have to take a pay cut.” But its mission is both noble and necessary, and the appeal of working on major problems with enormous public impacts appears to be working. Recommended reading. Further reading: Mikey Dickerson’s 10 Tips for Dealing with Bureaucracy—New Relic Blog [Video] Airbnb Open Sources Software to Lure Talent Amid ‘Insane’ Competition—CIO Journal What it’s about:Airbnb added three new apps to its open source portfolio earlier this month, but the motivation wasn’t just trying to give employees the best business tools or contribute to the software community at large. Sure, that might have been part of the equation, but the rental booking site hopes open-sourcing some of its toolkit will help recruit the best software talent in the face of what director of engineeringMike Curtiscalls “insane” competition in the Silicon Valley labor market. Why you should care:In the software arms race, any little edge counts. Curtis tellsCIO Journalthat Airbnb will keep the proprietary stuff closely guarded, of course. But it will open source “generic” tools with wider industry use cases, such as its recently releasedAerosolvemachine-learning package and itsAirpalcloud-based data querying tool. The latter, which works with Facebook’s open sourcePrestoDB, aims to simplify SQL queries to the point where you don’t need to be a big data wonk or business intelligence guru to run it. Indeed, one in three Airbnb employees have run a query on it in the year since it launched. Airbnb has contributed a dozen open source tools on its aptly namedNerds site(gotta love that!) to date, something the company hopes both contributes to greater good but also advertises its software innovation to potential hires. Google Is Wielding Its Own Secret Weapon in the Cloud—The New York Times What it’s about:In thecutthroat competitionfor public cloud business, Google may be its own best customer testimonial. In advance of this week’sOpen Network Summit, theTimes’Bits bloglooked at Google’s plan to not only unveil cloud customers such as HTC but reveal much more than ever before about its own infrastructure. Google did just that on Wednesday, offering a look inside itsdata center networking, including its massive-capacity, lightning-fast Jupiter network. Why you should care:As major cloud players continue to zap prices with their shrink-rays, it’s increasingly clear that features and underlying platforms will distinguish one from the other when enterprise users make their pick. Google is taking a big step toward writing its own story in this regard, and the synopsis might read something like: “We’re pretty good at this stuff.” Its Jupiter fabrics deliver 1 petabit per second of bisection bandwidth, according to Google, or “enough for 100,000 servers to exchange information at 10Gb/s each, enough to read the entire scanned contents of the Library of Congress in less than 1/10th of a second.” If it sounds like a bit of bragging, well, yeah—it is. But it’s bragging with a purpose: Attracting devs who want access to the same technology without having to build it themselves.Google’s Amin Vahdat connected the dots in a blog post: “The same networks that power all of Google’s internal infrastructure and services also power Google Cloud Platform.” Move Over, Meeker: Byron Deeter’s State of the Cloud Report—Bessemer Venture Partners What it’s about:With a nod to Mary Meeker’s classicState of the Internet report,Bessemer Venture Partners’Byron Deeterchecks in with his 2015 State of the Cloud Report. Given cloud computing’s relative youth and rampant ascension, it’s no surprise the stats are staggering. Here’s one to start: Cloud revenues have increased tenfold in the last six years, from a scant $5.6 billion in 2008 to more than $56 billion in 2014. And it’s going to double again in the next four years, according to BVP’s projections, to $127.5 billion in 2018. Why you should care:Deeter’s full presentation is worth a weekend watch or read, but it’s the forward-looking slides that may be most compelling for software pros. Deeter notes both the immense risks and opportunities in cloud security, unveiling a 10-point security plan for cloud startups on slide 37. To underscore the security landscape, Deeter quotes an unnamed cloud CEO who says aDDoSattack that took down the firm’s API caused more customer churn in one day than in the rest of its history. Wow. He also addresses the exploding market for cloud services built specifically for developers including, yes, New Relic. And for mobile developers, slide 44 underscores something we’ve talked about before in this space:the real money’s in enterprise apps, and it’s still a largely untapped market. Click through thefull slide deck hereorwatch video of Deeter’s presentation here. Bandwidth: The Next Frontier of Cloud Computing—ZDnet What it’s about:Is networking the next big thing in the everything-as-a-service age? It just might be, as firms likePacnetvie to deliver networking capacity on a pay-for-what-you-use model that some industry folks say better suits cloud environments facing significant but uneven networking needs. Why you should care:As author Drew Turney notes, there’s a common blind spot when it comes to cloud computing’s many shapes and sizes: Moving all that data from points A to Z, and everywhere in between, which can cause both performance problems and undue financial pressures. The promise of Networking-as-a-Service (NaaS), industry execs tell Turney, is that it can provide more efficient, scalable networking for short-term usage bursts such as customer traffic spikes or large cloud backup-and-storage jobs, enabling companies to later dial down their capacity as needed. Combined withSoftware-Defined Networking (SDN),NaaS makes it possible to build intelligent applications that manage their own networking needs, which might be the most significant enterprise potential of NaaS, saysNuage NetworksarchitectMarten Hauville. Page Bloat: Average Web Page Now More Than 2MB—The Performance Beacon (SOASTA) What it’s about:Do you need to put your website on a diet? Apparently so: The average Web page topped 2 MB as of May 2015, according to ongoing tracking atThe Performance Beacon. That’s double the average page weight from just three years ago. The site projects average page weight will exceed 3 MB in late 2017. Why you should care:Performance, performance, performance:Slow speedsare a killerin the modern software era. While author andSOASTAUX evangelistTammy Evertsrightly notes that page weight is not the only factor in Web optimization, we’re simply not paying it enough attention when designing and building Web pages. Images are the big culprit in the Web’s expanding waistline: they comprise nearly two-thirds of the average page’s weight, and video is a growing part of our Web diet, too. But other factors such as custom fonts play a role, adding weight even as the Web sheds previous performance hogs like Flash. The ideal weight? 1 MB, she says, which will save crucial seconds in load times. Sounds like it’s time to hit the virtual treadmill.

Learn why 12 Factor Application Patterns, Microservices and CloudFoundry matter when trying to change the way your product is produced.

hadoop is best known for map reduce and it's distributed file system (hdfs). recently other productivity tools developed on top of these will form a complete ecosystem of hadoop. most of the projects are hosted under apache software foundation . hadoop ecosystem projects are listed below. hadoop common a set of components and interfaces for distributed file system and i/o (serialization, java rpc, persistent data structures) http://hadoop.apache.org/ hadoop ecosystem hdfs a distributed file system that runs on large clusters of commodity hardware. hadoop distributed file system, hdfs renamed form ndfs. scalable data store that stores semi-structured, un-structured and structured data. http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-hdfs/hdfsuserguide.html http://wiki.apache.org/hadoop/hdfs map reduce map reduce is the distributed, parallel computing programming model for hadoop. inspired from google map reduce research paper . hadoop includes implementation of map reduce programming model. in map reduce there are two phases, not surprisingly map and reduce. to be precise in between map and reduce phase, there is another phase called sort and shuffle. job tracker in name node machine manages other cluster nodes. map reduce programming can be written in java. if you like sql or other non- java languages, you are still in luck. you can use utility called hadoop streaming. http://wiki.apache.org/hadoop/hadoopmapreduce hadoop streaming a utility to enable map reduce code in many languages like c, perl, python, c++, bash etc., examples include a python mapper and awk reducer. http://hadoop.apache.org/docs/r1.2.1/streaming.html avro a serialization system for efficient, cross-language rpc and persistent data storage. avro is a framework for performing remote procedure calls and data serialization. in the context of hadoop, it can be used to pass data from one program or language to another, e.g. from c to pig. it is particularly suited for use with scripting languages such as pig, because data is always stored with its schema in avro. http://avro.apache.org/ apache thrift apache thrift allows you to define data types and service interfaces in a simple definition file. taking that file as input, the compiler generates code to be used to easily build rpc clients and servers that communicate seamlessly across programming languages. instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business. http://thrift.apache.org/ hive and hue if you like sql, you would be delighted to hear that you can write sql and hive convert it to a map reduce job. but, you don't get a full ansi-sql environment. hue gives you a browser based graphical interface to do your hive work. hue features a file browser for hdfs, a job browser for map reduce/yarn, an hbase browser, query editors for hive, pig, cloudera impala and sqoop2.it also ships with an oozie application for creating and monitoring workflows, a zookeeper browser and an sdk. pig a high-level programming data flow language and execution environment to do map reduce coding the pig language is called pig latin. you may find naming conventions some what un-conventional, but you get incredible price-performance and high availability. https://pig.apache.org/ jaql jaql is a functional, declarative programming language designed especially for working with large volumes of structured, semi-structured and unstructured data. as its name implies, a primary use of jaql is to handle data stored as json documents, but jaql can work on various types of data. for example, it can support xml, comma-separated values (csv) data and flat files. a "sql within jaql" capability lets programmers work with structured sql data while employing a json data model that's less restrictive than its structured query language counterparts. 1. jaql in google code 2. what is jaql? by ibm sqoop sqoop provides a bi-directional data transfer between hadoop -hdfs and your favorite relational database. for example you might be storing your app data in relational store such as oracle, now you want to scale your application with hadoop so you can migrate oracle database data to hadoop hdfs using sqoop. http://sqoop.apache.org/ oozie manages hadoop workflow. this doesn't replace your scheduler or BPM tooling, but it will provide if-then-else branching and control with hadoop jobs. https://oozie.apache.org/ zookeeper a distributed, highly available coordination service. zookeeper provides primitives such as distributed locks that can be used for building the highly scalable applications. it is used to manage synchronization for cluster. http://zookeeper.apache.org/ hbase based on google's bigtable , hbase "is an open-source, distributed, version, column-oriented store" that sits on top of hdfs. a super scalable key-value store. it works very much like a persistent hash-map (for python developers think like a dictionary). it is not a conventional relational database. it is a distributed, column oriented database. hbase uses hdfs for it's underlying. supports both batch-style computations using map reduce and point queries for random reads. https://hbase.apache.org/ cassandra a column oriented nosql data store which offers scalability, high availability with out compromising on performance. it perfect platform for commodity hardware and cloud infrastructure.cassandra's data model offers the convenience of column indexes with the performance of log-structured updates, strong support for de-normalization and materialized views , and powerful built-in caching. http://cassandra.apache.org/ flume a real time loader for streaming your data into hadoop. it stores data in hdfs and hbase.flume "channels" data between "sources" and "sinks" and its data harvesting can either be scheduled or event-driven. possible sources for flume include avro, files, and system logs, and possible sinks include hdfs and hbase. http://flume.apache.org/ mahout machine learning for hadoop, used for predictive analytics and other advanced analysis. there are currently four main groups of algorithms in mahout: recommendations, a.k.a. collective filtering classification, a.k.a categorization clustering frequent item set mining, a.k.a parallel frequent pattern mining mahout is not simply a collection of pre-existing algorithms; many machine learning algorithms are intrinsically non-scalable; that is, given the types of operations they perform, they cannot be executed as a set of parallel processes. algorithms in the mahout library belong to the subset that can be executed in a distributed fashion. http://en.wikipedia.org/wiki/list_of_machine_learning_algorithms https://www.coursera.org/course/machlearning https://mahout.apache.org/ fuse makes the hdfs system to look like a regular file system so that you can use ls, rm, cd etc., directly on hdfs data. whirr apache whirr is a set of libraries for running cloud services. whirr provides a cloud-neutral way to run services. you don't have to worry about the idiosyncrasies of each provider.a common service api. the details of provisioning are particular to the service. smart defaults for services. you can get a properly configured system running quickly, while still being able to override settings as needed. you can also use whirr as a command line tool for deploying clusters. https://whirr.apache.org/ giraph an open source graph processing api like pregel from google https://giraph.apache.org/ chukwa chukwa, an incubator project on apache, is a data collection and analysis system built on top of hdfs and map reduce. tailored for collecting logs and other data from distributed monitoring systems, chukwa provides a workflow that allows for incremental data collection, processing and storage in hadoop. it is included in the apache hadoop distribution as an independent module. https://chukwa.apache.org/ drill apache drill, an incubator project on apache, is an open-source software framework that supports data-intensive distributed applications for interactive analysis of large-scale datasets. drill is the open source version of google's dremel system which is available as an iaas service called google big query. one explicitly stated design goal is that drill is able to scale to 10,000 servers or more and to be able to process petabytes of data and trillions of records in seconds. http://incubator.apache.org/drill/ impala (cloudera) released by cloudera, impala is an open-source project which, like apache drill, was inspired by google's paper on dremel; the purpose of both is to facilitate real-time querying of data in hdfs or hbase. impala uses an sql-like language that, though similar to hiveql, is currently more limited than hiveql. because impala relies on the hive meta store, hive must be installed on a cluster in order for impala to work. the secret behind impala's speed is that it "circumvents map reduce to directly access the data through a specialized distributed query engine that is very similar to those found in commercial parallel rdbmss." (source: cloudera) http://www.cloudera.com/content/cloudera/en/products-and-services/cdh/impala.html http://training.cloudera.com/elearning/impala/