News

New release includes a ‘Dockerfile lock’ feature and inclusion of Scancode data in default reports

Tern is a VMware-originated open source container inspection tool. Since Tern’s last release, new features and command line options have been added as a part of Tern 2.0.0, which is now generally available from PyPI. You can also clone the latest changes from GitHub.

One of the most anticipated features in this release is the addition of a new capability we call “Dockerfile lock”.  The Dockerfile lock feature is designed to help users create more reproducible Docker images from their Dockerfiles. Why is this necessary? Dockerfiles are used to automate the assembly and creation of Docker container images. While useful, Dockerfiles are not inherently reproducible the way one might think. This is because Dockerfiles are not declarative of what ultimately gets included in the end product container. 

Take for example the case where you want to build a container on top of a Debian base image. To do this, you might include this line in your Dockerfile: FROM debian:latest. While this FROM line will indeed build a container image based on the latest debian base OS tag, there is no guarantee that the latest base OS will remain the same in the future. The debian:latest base OS with digest abc in January might become debian:latest base OS with digest xyz a few months later. When the base digest changes, it is likely because packages (and their respective versions) which make up the base image have changed. This means that if you were to build a container image in January from your Dockerfile and then build a container image from the same Dockerfile a few months later, the contents of your container would likely be different. This is especially true if you install packages for your container in subsequent lines of your Dockerfile. 

Dockerfiles present unique challenges for building reproducible container images. Tern’s new Dockerfile lock feature addresses some of these issues. When provided a Dockerfile, Tern creates a locked Dockerfile in which the base image is pinned to a digest and the packages installed for each subsequent layer are pinned to their versions, if they are known. Tern will also expand ARG and ENV variables and try to find information about git repositories that may be ADDed within the Dockerfile provided. An example of what can be expected is below.

For more information about this feature, check out Tern’s README on GitHub.

The second headlining feature included in Tern’s 2.0.0 release is the ability to map Scancode’s data into Tern’s data model. Scancode-toolkit is a license scanning tool that finds licenses in source code and binaries. This new feature allows file level licenses found in Scancode’s data collection to be reported in Tern’s default report format when running with the -x, --extend CLI option. Ultimately, this makes it easier to read and consume Scancode output information. This change builds upon the functionality added as a part of Tern’s 1.0.0 release to extend the use other filesystem analysis tools.  For more information about how to utilize Tern and Scancode together, see the Scancode documentation on Tern’s GitHub page. 

Other notable additions to this release include:

• Remove the -l, --logging CLI option and make logging the default behavior.
• Allow user to set the working directory on the command line.
• Expand test coverage.
• Enable tox to run unit tests.
• Use dockerfile-parse to parse Dockerfiles instead of manual parsing.
• Update documentation.

For a more detailed summary of bugfixes and enhancements, see the Release Notes.

While we are excited about the new features Tern has to offer, Tern is still in alpha development mode as a project. If you experience any issues while experimenting with or using Tern, please don’t hesitate to open a bug. If you have any other questions or would like to get in contact with the maintainers, you can reach out to info@tern.dev.

Future work for Tern is focused on adding support for language package managers and adding support for multistage Docker builds. On an administrative note, Tern will move GitHub organizations from VMware to its own organization, tern-tools under the ACT umbrella org. This move coincides with the Linux Foundation’s announcement that founding member commitments had been made for the Automated Compliance Tooling (ACT) workgroup, of which Tern belongs.  

We would like to thank the six community contributors that helped with the delivery of this release, five of them first time contributors. This speaks to Tern’s organic growth as a project and developer community. 

Google, Siemens and VMware commit to the Automated Compliance Tooling project, community accelerating work on Tern, OSS Review Toolkit, FOSSology and Quartermaster

San Francisco, USA – December 12, 2019 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announced founding member commitments from Google, Siemens and VMware for the Automated Compliance Tooling (ACT), as well as key advancements for tools that increase ease and adoption of open source software.

Using open source code comes with a responsibility to comply with the terms of that code’s license. The goal of ACT is to consolidate investments in these efforts and to increase interoperability and usability of open source compliance tooling. Google, Siemens and VMware are among the companies helping to underwrite and lead this collaborative work.

Also announced today is the availability of Tern 1.0. Tern was originally contributed by VMware and is an inspection tool that finds the metadata of the packages installed in a container image. It is now able to generate SPDX. There is also the new FOSSology 3.7 release available today for reading SPDX headers have also been added to more than 75 percent of the source code files in the Linux kernel. And the Google Summer of Code (GSoC) interns have updated the spdx-tools libraries to support translations in Java, Python and Go. This enables other tools to smooth the import and export of SPDX documents.

“One of the most exciting parts of the ACT Project is its integration with pre-existing activities around the Linux Foundation Open Compliance Project,” says Shane Coughlan, OpenChain General Manager. “This includes the OpenChain Reference Tooling Work Group, with its focus on addressing real world challenges as efficiently as possible, an area where targeted investment is critical. The end result of these activities will ensure that open source tooling for open source compliance is more mature, more effective and easier to adopt for entities of all sizes.”

“Open Source tools that support the Open Source compliance process have seen great progress in recent months.” says Mirko Boehm, co-founder of Endocode and the QMSTR project. “With ACT, the efforts of the community, businesses and the funding for QMSTR from the European Commission’s Horizon 2020 program come together under one roof in direct collaboration with related industry projects like OpenChain. We expect an acceleration of the development of Open Source compliance solutions and are excited to collaborate with the partners at ACT, the community and the Linux Foundation”.

“It’s a testament to the community and the importance of automating compliance in software development that ACT membership and tools development and integration are coming together to create open source integrated solutions,” said Kate Stewart, senior director of Strategic Programs at Linux Foundation. “We applaud the contributions coming in from all corners of the community and look forward to what 2020 will bring to the work.”

Community members will be meeting this week at Open Compliance Summit in Tokyo, Japan. ACT is seeking new members, community partners and additional tooling projects. To get involved, contact act@linuxfoundation.org

ACT is composed of five primary projects:

FOSSology An open source license compliance software system and toolkit allowing users to run license, copyright and export control scans from a REST API. As a system, a database and web UI are provided to provide a compliance workflow. License, copyright and export scanners are tools available to help with compliance activities. FOSSology is an existing Linux Foundation project that will move under ACT.

OSS Review Toolkit (ORT) enables highly automated and customizable Open Source compliance checks the source code and dependencies of a project by scanning it, downloading its sources, reporting any errors and violations against user-defined rules, and by creating third-party attribution documentation. ORT is designed for the CI/CD world and supports a wide variety of package managers including Gradle, Go modules, Maven, npm and SBT. The project is being contributed to ACT by HERE Technologies.

Quartermaster(QMSTR), originally contributed by Encode, integrates into the build systems to learn about the software products, their sources and dependencies. Developers can run QMSTR locally to verify outcomes, review problems and produce compliance reports. By integrating into DevOps CI/CD cycles, license compliance can become a quality metric for software development. The project is being contributed to ACT by Endocode.

SPDX Tools Software Package Data Exchange (SPDX) is an open standard for communicating software bill of material information including components, licenses, copyrights and security references. The main SPDX specification will remain separate from, yet complementary to, ACT, while the SPDX tools that meet the spec and help users and producers of SPDX documents will become part of ACT. SPDX is an existing Linux Foundation project.

Tern Tern is an inspection tool to find the metadata of the packages installed in a container image. It provides a deeper understanding of a container’s bill of materials so better decisions can be made about container based infrastructure, integration and deployment strategies. Tern was created by VMware, who are contributing the project to ACT, to help developers meet open source compliance requirements for containers.

Member Quotes

Google, founding member

“To do open source compliance well, at scale, we need to ensure the community has easy access to advanced automation and tooling,” said Will Norris, Open Source Engineering Manager at Google. “Google has invested heavily in our own compliance tooling, and we are proud to be a part of the Automated Compliance Tooling project to share our experience and expertise with the broader community. We look forward to helping make it easier for everyone using open source code to do so respectfully and in accordance with open source licenses.”

New York University’s Secure Systems Lab, affiliate member

“The software compliance ecosystem has long needed an initiative such as ACT, and projects such as SPDX-tools and Tern are key elements in the challenge of automating compliance” said Santiago Torres-Arias, lead of the in-toto project and member of the New York University’s Secure Systems Lab, “We are most excited about the integration of in-toto into SPDX, which will help in providing strong, cryptographically-enforced compliance checks.  Security is not just a matter of protecting against outsiders, but also a matter of ensuring all actors within your supply chain are following the rules.”

Siemens, founding member

“An Open Source license compliance toolchain has to be Open Source itself. ACT is a milestone in building an integrated and automated end to end OSS compliance toolchain consisting of open source. ACT will boost the effort of the OpenChain Reference Tooling Work Group in realizing such a toolchain, which easily can be used free of charge – OSS license compliance for everyone.”

VMware, founding member

“Compliance is at the core of how companies need to engage with open source projects,” said Dirk Hohndel, vice president and chief open source officer, VMware. “The more we automate compliance processing, the better we are able to advance agile development and rapid response to address required changes such as security issues. For years, VMware has worked towards automating compliance tooling and we are committed to helping enterprises better understand what’s inside containers and manage their compliance obligations.”

For more information, please contact: act@linuxfoundation.org

 

About The Linux Foundation
Founded in 2000, The Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see its trademark usage page: www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.