Common Problems With Open Source

To gather insights on the current and future state of open source software (OSS), we talked to 31 executives. This is nearly double the number we speak to for a research guide and believe this reiterates the popularity of, acceptance of, and demand for OSS.

We began by asking, "What are the most common problems with the Open Source ecosystem today?" Here's what they told us:

Technical Debt

• Failure to address technical debt that is implicit with open source. We had a client who had been using version 2 of JBoss for 10 years. We helped them find a problem and they needed to upgrade to JBoss 7. This took several years. If they had been upgrading to new versions of JBoss as they came out, the upgrade time would have been insignificant. The open source code is going to become vulnerable, it’s just a matter of when. That’s why it’s critical to have an accurate bill of materials for every piece of code. 
• The total cost of ownership (TCO) is higher than people realize. You begin using open source code, find gaps and then need to hire people to fill the gaps and customize for your use case and build your own product. You also need to follow an aggressive upgrade process to avoid incurring massive technical debt.  
• Open Source's biggest challenge is that people forget how important and critical it is and don’t invest in its maintenance. Most of the problems open source faces are problems that the software industry and technology industry face generally. Answering the challenge of IT generally moving into the cloud in a way that maintains "the commons" is probably the biggest question that is open for grabs. Something will happen because the entire technology industry including cloud providers depend on a healthy open source ecosystem. 
• The most important problem with open source ecosystem is the lack of structure around adoption of these tools. Often companies incorporate these tools only because these are free and fail to realize that there is no support infrastructure available. Hence, as we saw in the case of the Equifax hack, open source tools do not get updated for fixing critical vulnerabilities. Additionally, unless the open source tool has a huge community of developers, it does not get updated in time for critical fixes and vulnerabilities. Open source software is successful (and useful) only if its updated regularly, regular contributions from the community add valuable features and fix critical bugs.

Complexity

• Hard to adopt for the mass market, as the ecosystems are geared towards innovation more than usability. It's unclear when a feature is fully ready for production versus in early proof of concept stage. Large open source communities find it hard to share a common vision, and this causes a real slowdown of progress. 
• The speed of change is challenging. The incredible diversity of projects. Good commercial offering throughout entire stack. Longer term challenge back to monetization. Meaningful commercial opportunities around open source. Relying on charity will limit options. 
• Especially in their early years, most open source projects were expert tools from developers for developers. They were not necessarily built so that they can be dropped into a production environment out-of-the-box. Especially for large infrastructure software projects, having an easy and lightweight way to handle continuous operations and support frictionless application development for many teams is a big challenge, in particular for companies that do not have big IT infrastructure teams. Many companies are working to build complete solutions that include open source at the core and make it easier for companies to start running an open source technology in its existing environment with minimal time spent on custom infrastructure or other tooling.
• There is a learning curve in an enterprise environment, so you can’t just adopt open source and expect everything to be smooth from day one. Implementing open source GIS approaches has traditionally demanded DIY capabilities that have placed it beyond the reach of many organizations. In addition, the transition to an open system historically required an abrupt, rip-and-replace effort that is both daunting and risky.
• Adoption of open source has grown rapidly due to two key factors: free license cost and great innovation and creativity from the open source community. The era of proprietary-only IT stacks has ended. However, the innovation happening in open source caused a lot of technologies to overlap and inhibit enterprises from achieving business outcomes. As such, progression towards successfully enabling enterprises to achieve business outcomes slowed down. Enterprises are finding it difficult to operationalize and productize open source to achieve business outcomes, so they need products that will help them move forward and closer to their business goals. The arrival of software that hardens and integrates best of breed technologies has begun to alleviate this problem.
• The first generation of open source software focused on data-at-rest and batch processing as its mainstays, with use cases like search indexing and data warehousing. An ecosystem centered around data lakes has begun to unravel. Data lakes have become data swamps, with low value extraction, and the arrival of enterprise data-in-motion stacks has widened the solution space. Migration of IT stacks from first generation technologies to newer ones that can help enterprises achieve business outcomes will require agility.
• One of the outcomes resulting from the shift away from data-at-rest is the demise of Lambda architecture. With enterprise-ready data-in-motion, an increasing number of businesses no longer run two pipelines for the same product. With IoT and edge processing becoming more scalable and operable, message buses are being used to communicate between various compute and storage grids. Analytics at the edge using high performing data-in-motion stacks are gaining traction. In the future, data lakes will mainly be used for archival and historical analytics.
• 5) Use of open source technologies without a hardened stack requires expertise that is very rare.  Stitching together open source products barely works in engineering heavy areas like Silicon Valley, Bengaluru and Beijing. For open source to be mass market and commoditized for use across the world, we need a new set of products and stack that focus on business outcomes and enables enterprises to use their current expertise. Developing products based on open source technologies needs to be much simpler.
  
  

License Issues

• A lot of organizations do not understand Open Source and the licensing obligation that goes along with it. You must give credit to the OSS you are using. With freedom comes obligation. If you take the code and make changes you are obligated to share those changes with the community. Need to be aware of the potential technical debt which can accrue very quickly. You need to take the time to update tools and code. Spend the time and effort necessary to build changes into the project. There’s more work than just an initial patch. 
• While clear open source licenses such as Apache and MIT are crucial because they provide a certain degree of legal certainty to companies that consume open source, there are still legal matters around open source that are not entirely resolved. Patent trolls can be a bigger threat to companies that produce open source because they are more exposed than those that do not provide transparency about their code. 
• License terms. Apache 2.0 license. Afaro GPL license. We’ve had no major concerns. Have heard concerns from MongoDB forced to switch from open source to enterprise due to Afaro license. Litigious aspects of certain companies like Oracle. API copyright and fair use. May have a chilling on open source projects. 
• As much as the entire world depends on Open Source, there’s still a resource and funding concern, especially for the more invisible infrastructure components. The vast majority of components used as the backbone of every enterprise software product receive little or no funding or support from the billion-dollar industries that depend on them. Additionally, many of these packages are sole proprietorships. By this I mean, while the code is open, and well used, there’s a single author responsible for a vast majority of the work that goes on in these projects. Post-Heartbleed we have seen some good changes in this regard (especially for the more visible pieces of the core infrastructure) but we still have a long way to go. Ease of funding, more awareness of components used, and better collaboration tools are key to fixing this problem. Also, it is still surprising to see the lack of Open Source licensing experience in projects. Many projects lack licenses or lack compliance with their own license. This makes it very difficult for businesses to be in compliance themselves. Education and best practices are still required to make this an easier part of using Open Source.
  
  

Other

• The long-term viability of companies developing OSS. Many people assume OSS is free, which is actually not true. Somewhere, someone is paying for the applications to be developed, and those costs must be recovered somehow. When the vast majority of an OSS’s user base refuses to pay for support (which is the common way OSS is commercialized) then eventually the company can no longer afford to continue development of their product and development stops. This is very common, and this leads to a lot of OSS products that gain traction, only to fail commercially.

• The most obvious challenge is monetizing the business model. Amazon just took the open source tools developed by EKS and added them to AWS. AWS is taking revenue from start-ups without giving anything back to the community. If we do not find a monetization model for open source there could be a churn of frameworks.
• Too many users view open source projects as if they are software vendors. They expect they can just log a defect with the project and someone will have the responsibility to fix it. The reality is, projects don’t have any responsibility to do that. Mostly, you’ll find someone that will because there are people that care deeply about the perception of the project or are worried they may be impacted by the defect themselves. However, at the end of the day if you want to ensure defects that matter get fixed, you need to either have the capability to fix them yourself or some kind of support contract for a third party to take responsibility (and likely contribute the fix to the project).
• No ownership of the code so if you find a bug you have to post to the community and hope someone can fix it. Since the packages are standardized, you have to work around problems that exist in the code, you cannot fix the underlying root cause of the problem.
• Despite the acceptance of open source, the biggest problem is the perception that if it's open source it's free, does not provide adequate support, and that if your source code is out in the world it's easier for people to exploit. From a perception standpoint, open source is still fighting an uphill battle. However, the negative perception is limited to business decision makers. In the tech community, the perception is vastly different. Techies love the fact that things are open source and that they can dig into the source and contribute to it if they want to. In many ways, contributing to open source software is also a means to build your technical reputation and resume.
• Our database is open source. This prevents vendor lock-in. How much benefit you get is based on contributing code, just using code, or paying a vendor. Some companies don’t have big margins. There are more use cases to be handled. Huge range of possibilities can be customized. Dynamically move between clouds, on-prem, and hybrid.
• Community development. All the systems are in place. The Apache Foundation has a “community first” ethos. We need to get strong open source professionals together to achieve common goals – cheerleading or psychology to get people interested in participating and contributing. People feel welcome and important to the project to which they are contributing.
• Release process. Stability of the code you get. Some projects have a very strict release process. A lot of projects don’t have a process or the mindset of enterprise stable software.
• There really aren’t any barriers left today. When looking back 20 years, people said Open Source wasn’t secure or scalable, but now it’s a leading tool. Those old obstacles are pretty much gone, and people now view Open Source as the future of innovative technology.
• The most important problems today are twofold: first, a general lack of contributors, and second, a lack of diversity among the contributors that do exist.  This is driven by the prevalent idea that open source software should be free and the people who work on it should be driven by passion and sheer will (rather than by a salary). This means most open source contributors have to code in their free time.
• As more large companies are contributing to open source, for better or worse it also raises the bar for new open source projects. Even when they are grassroots initiatives, they have significant pressure to “launch as complete” — launch something immediately useful and usable by developers. It means that there are fewer and fewer “grassroots” successes and many more splashy launches and overnight successes like Kubernetes because the product-market fit is already established.
• Security. The idea that being open source makes projects more secure, relies on the idea that the community is actively engaged in reading and improving projects. Without clear, engaged project maintainers and owners, open source projects cannot evolve and improve. These projects suffer from irregular security updates and lack of testing. This is in some ways a downside of the massive proliferation of open source projects: There is not enough time to be active in every open source project used in your stack. This becomes a question of reliability — what projects can you rely on to be maintained and usable in the long run?
• The biggest challenge to open source is finding a way to maintain its legacy and promise for the next 20 years. There seems to be an underlying current of very visible startups that create great technology and use open source purely as a hook to generate buzz. In practice, many of these groups have no intention of creating open source communities or projects – they’re really only in it to attract talent or leads. These can damage the value of the term 'open source' which has become synonymous with good software and fair business practices.

Here’s who shared their insights with us:

• Anthony Calamito, Chief Geospatial Officer, Boundless
• Jakob Freund, CEO, Camunda
• Pete Chestna, Director of Developer Engagement, CA Veracode
• Julian Dunn, Director of Product Marketing, Chef
• Matt Ingenthron, Senior Director of SDK Engineering, Couchbase
• Stephan Ewen, co-founder and CTO, data Artisans
• Amol Kekre, Co-founder and Field CTO, DataTorrent
• OJ Ngo, Co-founder and CTO, DH2i
• Stefano Maffulli, Director of Community, DreamHost
• Kelly Stirman, CMO and VP Strategy, Dremio
• Konstantin Boudnik, CTO Big Data and Open Source Fellow, EPAM
• Tyler McMullen, CTO, Fastly
• Jeff Luszsz, VP of Product Management, Flexera
• Angel Diaz, V.P. Developer Technology and Advocacy, IBM
• Ben Slater, Chief Product Officer, Instaclustr
• Grant Ingersoll, CTO, Lucidworks
• C J Silverio, CTO, npm
• Mark Gamble, Senior Director of Product Marketing, Analytics, OpenText
• Francis Dhina, CEO, OpenVPN
• Sirish Raghuram, CEO and Co-founder, Platform9
• Neil Cresswell, Co-Founder, Portainer.io
• Lars Knoll, CTO, Qt
• Brad Adelberg, Vice President of Engineering, Sauce Labs
• Giorgio Regni, CTO, Scality
• Dor Laor, CEO, ScyllaDB
• Harsh Upreti, Product Marketing Manager, API Products, SmartBear
• Jean-Baptiste Onofre, Technical Fellow and Software Architect, Talend
• Antony Edwards, CTO, Testplant
• Matt Ellis, Architect, TIBCO Software
• Karthik Ranganathan, Co-founder and CTO, YugaByte