This is the eighth edition of the Black Duck Open Source Rookies of the Year awards. Each year we review the world of open source and recognize top new projects launched during the past year. Much has changed since the first awards back in 2008. Open source is no longer just the domain of academic institutions and underground communities of idealistic developers. Open source components are essential building blocks for most enterprise and commercial applications, and open source development is essential to the business strategy of many software development companies.
This trend is reflected in this year’s rookie class. Most of the top projects are sponsored, and contributed to, by for-profit software companies. In some cases, the projects are adjuncts to their sponsors' core products or offshoots of internal development initiatives, but in other cases the projects are drivers of the core products themselves. Companies choose an open source development model for many reasons, but one thing is clear: open source is not viewed merely as a tool for competitive success and profitability, but also as a strategic element in achieving those goals.
Open Source Rookies Reflect Industry Trends
The 2015 Open Source Rookies class also provides a good indication of what's next and many of the new open source projects reflect five industry trends:
The stories behind open source projects are often themselves quite fascinating, and this year is no exception.
Rocket.Chat is an open source web chat platformbuilt for communities and companies wanting to privately host their own chat service. The Rocket.Chat team has based their platform on a modular “package” model, through which Rocket.Chat offers an impressive array of features aimed at making real time communications work for your business: from simple messaging and emoji support, to face-to-face video conferencing, multi-user audio conferences, screen sharing, and more. Rocket.Chat provides native desktop applications for Mac OSX, Windows, and Linux, with mobile apps for iOS and Android.Iteven has a native Firefox OS application for both desktop and mobile - and because the it’s open source, Rocket.Chat is a great option for developers looking to build and evolve their own chat platform.
Another open source alternative to Slack is Mattermost, whose story begins with the company’s origins as an HTML5 game developer. Initially built as a games portal and messaging app to reach gamers outside of Facebook, messaging became a company focus when the team tried to switch internal messaging services only to find their communications archives “held for ransom” by the provider. From this frustration came inspiration for Mattermost, an open source, on-premise Slack alternative, written in Golang and React. Mattermost consolidates all team communication into one place, through an intuitive interface that is searchable and accessible anywhere. Users share messages and files across phones and PCs, keeping vital communications within the safety of a private IT infrastructure. Mattermost interfaces are “Slack-compatible”, and provide access to third-party software created for Slack or Mattermost. They offer two versions, designed to support teams of 1-50 and organizations of hundreds or thousands.
Web-based video conferencing solutions like WebEx and GoToMeeting have become standard tools in many office environments. These solutions have historically relied on proprietary desktop applications, but new web browser capabilities are changing that. The open source software company Linagora strives to bring digital independency to the public and private sectors around the world using these new capabilities. They sponsor a number of open source projects working towards this goal, including OpenPaaS. In addition to this open-API social platform, the team behind OpenPaaS offers Hubl.in, a free and open source video conferencing solution. Hubl.in brings real-time communication to the next level by allowing free communication without requiring any additional plugins. The OpenPaaS team boasts, “if you can read this page, you can probably use Hubl.in right now.” Hubl.in is based on WebRTC which allows decentralized communication between browsers. This means your video conference is not streamed to Hubl.in in any way, increasing security and providing an additional layer of privacy.
Deep learning techniques aim to teach computers to teach themselves by using a system of multiple processing layers to model the neural network of a human brain. Programmers then feed the network a library of data for the computer to learn from. MXNet is a lightweight deep learning library created by DMLC, the people behind CXXNet, Minerva, and Purine2. MXNet combines lessons learned from these previous projects to mix symbolic and imperative programming together. At its core, MXNet utilizes a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer built on top makes MXNet both fast and memory-efficient. The library is portable and lightweight, and readily scales to multiple GPUs and machines. In fact, it can even run tasks such as image recognition on a smartphone. The DMLC group collaborates on open-source machine learning projects to make large-scale machine learning widely available. To that end, MXNet also contains a collection of blue prints and guidelines for building deep learning systems.
Enterprises of all scales are coming to realize that they need to develop code quickly and efficiently to keep up with the competition. Google is very good at this – and now they’ve released Bazel, a subset of their own internal software build system. Bazel aims to buildsoftware quickly and reliably through a shared code repository in which all software is built from source. Bazel automates testing and releases, and uses both parallelism and caching to speed processing.
Bazel is particularly suited for projects that combine a number of characteristics: large codebase, multiple languages, multiple platforms, and extensive tests. Bazel’s core features have undergone thorough testing in the demanding work environment at Google. That said, Google admits Bazel isn’t for everyone: Bazel’s current iteration supports Linux and OS X, but doesn’t yet include Windows support. Moreover, Google says that Bazel is not well suited for running build operations whose outputs should not be cached. Still, the project shows promise and it’s nice to see large companies supporting the greater good of open source.
Docker containers are revolutionizing the way devops teams package and deploy applications, but many organizations still struggle to orchestrate container management at scale. Enter Kontena, an open source container management solution “built to maximize developer happiness.” Kontena features a host of proven technologies and features for accelerating container development and breaking barriers to successful deployment: multi-host, multi-AZ container orchestration, Weave overlay network technology, VPN access to backend containers and an intuitive application deployment workflow. Kontena offers everything a company needs to develop, deploy, and monitor containerized systems. It can be installed into any cloud infrastructure, is fully open source, and will soon expand beyond Docker to support Windows containers, CoreOS rkt, and other container technologies.
Docker’s Open Container Initiative (OCI) may have streamlined application packaging, but it can still be difficult to manage asset dependencies and relationships through every instance of a multi-container application. Nulecule (/NOO-le-kyul/) is a specification for packaging complex multi-container applications while ensuring smooth deployment across all instances. Sponsored by Red Hat, Nulecule offers a holistic system for managing the description and transportation of asset relationships. Most container orchestration systems treat multi-container applications as individual components rather than as a larger entity, an approach that limits flexibility and portability for the whole application. By contrast, Nulecule’s composite container-based application specification creates a standardized way to describe and package multi-container applications while including all dependency references and orchestration metadata within each container image.
The DevOps workflow is often cluttered with tricky, complicated compliance protocols that slow development and frustrate software engineers. InSpec eases the path to release by automating the compliance testing process to make compliance an integral part of the development lifecycle.
InSpec is an open-source compliance testing framework for specifying compliance, security, and policy requirements. InSpec makes it easy to run your tests whenever you need through a number of sophisticated features: automated tests that make compliance a part of the software production line; compliance built into the development cycle; tests specifically targeted to compliance issues; metadata tags for prioritizing controls; and a command-line interface to run tests quickly and efficiently.
Users write controls with InSpec’s specially designed, human- and machine-readable language. InSpec then flags any security, compliance, or policy issues it detects, according to those inputs. And because the inspection framework runs locally on the node being inspected, InSpec can examine any node in an infrastructure system.
It’s not just tech giants investing in open source. This year Capital One went searching for a DevOps dashboard, but found both commercial and open source markets lacking. They built their own: Hygieia, an enterprise DevOps dashboardreleased last year as an open source project on GitHub. Capital One uses Hygieia during software development to give their teams and leaders a simple, accessible snapshot view of the whole DevOps process.
Rather than covering only a portion of the development process, the way traditional dashboards do, Hygieia offers a comprehensive overview through two view methods: widget view and pipeline view. Widget view showcases more detailed information: features in the current sprint, code contribution activities, continuous integration activities, code analysis, security analysis, unit and functional test results, and deployment and environment status. The pipeline view pulls back to show each component’s lifecycle progression through the development, testing, and deployment stages.
Open source can be a very effective way to develop solutions for some of the biggest problems as demonstrated in the Hygieia example. Diabetes affects more than 370 million people worldwide, and hundreds of institutions around the world spend millions each year on research and development of management tools and medication. But when open source contributor Benjamin Kerensa was diagnosed with Type 2 diabetes in May 2015, he realized there were no open source tools for tracking glucose levels and other metrics for people like him. That September, Kerensa and a small team of developers released Glucosio, one of the first open source diabetes monitoring applications.
Glucosio users enter and track blood glucose levels and get helpful tips through an in-app assistant that prompts the user to stay hydrated or check their blood sugar. Users can even contribute to diabetes research by opting to submit anonymous demographic or personal glucose trend information.
Kerensa and the Glucosio team are dedicated to providing free, user-centered apps for diabetes management and research. Glucosio is currently available in 20 languages, has been downloaded in every continent, and there are plans for a free API for researchers to access anonymized diabetes data from users who opt in to the program.
San Francisco-based HashiCorp builds tools to securely manage data centers and they’ve been dreaming up new ways to protect and securely access secrets of all kinds. Their open source Vault project is just one of many products HashiCorp offers for building a secure software development and management environment.
Vault is a tool for securely accessing secrets: API keys, passwords, certificates, employee credentials, and other sensitive resources. Vault encrypts and decrypts data without storing it, allowing security teams to define encryption parameters and giving developers the power to store encrypted data without having to design their own encryption methods. Vault can even generate secrets-on-demand for AWS or SQL databases, then automatically revoke these dynamic secrets after their terms are up. Through a unified access interface, tight access control, and detailed audit logs, users can be confident that their secrets are safe.
Rancher Labs has come up with a novel way to run containers: an entire operating system, in miniature, with everything needed to run containers and nothing more. RancherOS is a 20mb Linux distribution specially designed to be the easiest way to manage Docker containers. In fact, everything within RancherOS runs as Docker containers – even the operating system itself. RancherOS runs Docker directly on top of the Linux Kernel and distributes all user-space Linux services as Docker containers, resulting in two system instances. The “System Docker” instance initiates all system services (udev, DHCP, the console, etc.), each of which run as containers. The “User Docker” instance creates unique user containers within the larger User Docker container. RancherOS delivers updates and features through containers, and can host container management platforms (like Rancher Lab’s Rancher system) at any scale.
The Open Web Application Security Project (OWASP) Foundation is a non-profit community that provides resources and tools for web application security. Despite cybersecurity vulnerabilities regularly making front-page news, developers at OWASP found that many web developers aren’t aware of the risks and vulnerabilities that they are exposed to. To that end, the OWASP Security Knowledge Framework (SKF) provides a free, open source web app security system based on OWASP security standards. The SKF supports software developers throughout the product lifecycle, ensuring security in both pre-development and post-release updates.
OWASP-SKF analyzes the processing techniques that developers use to edit their data, then matches those patterns to known security vulnerabilities. After providing descriptions of linked vulnerabilities and offering feedback on how to implement solutions, the SKF validates that security fixes were implemented correctly. In addition to directly aiding in web-app security, the Security Knowledge Framework also serves as a training tool to teach developers about application security.