Research on Software Escrow, respectively a framework for that, was part of the work on the EU project TIMBUS. The approach included work on the legal and technical requirements for such a framework, as well as the implementation of a working prototype. With the following simple table of contents you can jump to the single sections of this page:
- Phases of escrow
- Technical Software Escrow Framework
There is an ongoing trend in IT businesses to outsource services. Examples can be found in Software as a Service or cloud computing, which will account for 70% of the overall outsourcing budgets in 2020. Outsourcing represents about 40% of the IT service market, whereas only 16% are Custom Application Development (source: IDC Austria).
Outsourcing is a growing trend, and will be used three times as often in 2020 when compared to 2010:
Outsourcing has many advantages, such as focusing on core competencies in business and giving the outsourcing service to a specialist in the field. However, outsourcing also bears different risks:
- The lost of control over the service
- Financial standing of the vendor, so one cannot be sure how long the service will be available
- The sale of the vendor can occur, due to the aforementioned financial standing or other reasons, which bears uncertainty in future business collaborations
- Maintenance and adjustments of the system are part of the business agreements but could nevertheless lead to disagreement if not implemented in the contract
- Business critical applications that are developed by an outsourced facility, an extern party is responsible
In order to mitigate the risks for the customer mentioned before, Software Escrow offers a mitigation to this as it includes a trustable escrow agent in the
business relationship. The agent is responsible for depositing all material that is needed to develop the software, like source code, documentation, and licenses for software artifacts. If a predefined trigger event occurs, the agent is obliged to hand out the objects to the customer.
Software escrow comprises of three parties:
- The customer, who has a need for a software in his business, wants to ensure that he is able to use the software for a longer time, and secure his investments in it
- The software developer, who makes the compiled object code available to the customer and hands over the sources and all other necessary artifacts to the escrow agent
- The escrow agent, who is responsible for depositing the material and releasing it, and who has to verify that the submitted material meets the requirements as contracted, e.g., that all objects are available, accessible, and fullfill specified quality measurements
Course of action
In the diagram on the right, the relationships between the parties is depicted. The numbers denote the course of action:
- The requirements and specifications for the software in question are discussed and delivered to the developer
- After implementing, the customer receives object code of the software, whereas the source code and all artifacts necessary for the development are committed to the escrow agent, who has to verify them and their quality
- In case of agreed circumstances (see below) the escrow agent releases the source code and all artifacts to the customer
Release events can be for example:
- Failure of the developer to support/maintain the software
- Insufficient maintenance
- Insolvency or bankruptcy of the software developer
Motivation of parties
The customer and developer can have different motivations when agreeing on an escrow contract.
For the customer:
- Escrow is a risk mitigation strategy to guarantee maintenance and support for the long term and thus protect business critical applications from discontinuity
- Investments in the software are ensured
- The source code is evaluated and verified, thus it is ensured that the developer deposited the correct artifacts and the source code provides a quality which ensures that it can be maintained and adjusted in the future
For the software developer:
- By verifying the source code the evidence of the copyright can be proved
- It is a confidence building measure for the customer
- It is payed additionally, as there is usually extra effort involved
The Software Escrow agreement can have different contract schemes. There can be contracts for specific use by one organization, suitable for software developed for one business. In case of a release event, the sources and artifacts will be released to the specified end user. Another type of contract, suitable for software used by more than one customer, involves the registering of the customers beforehand. In case of a release event, the sources the registered licensees can apply for the software source code to be released.
Although escrow mitigates the risk of outsourcing, there are some risks to escrow as it represents a digital preservation strategy:
- Read error storage media: The media (e.g., DVDs, hard disks, …) deposited at the escrow agent can be erroneous when being read again.
- Source code incomplete: When depositing software artifacts, the escrow agents needs to ensure that the deposited material is complete.
- Build environment not deposited: For building the sources artifacts of the build environment are necessary.
- Configuration is not available: Missing configuration of e.g., compiler attributes or config-files settings can increase the effort to redeploy the software again or they can even make it impossible.
- Instructions missing: Instructions on e.g., how to build the software are necessary to understand the software and its development again.
- Test data missing: Test cases, such as unit tests, need to be deposited as well, otherwise the software cannot be tested without tremendous effort.
- Documentation insufficient: Source code needs to be documented sufficiently to guarantee that another programmer is able to understand and extend it.
- Licenses and rights not included: Licenses for development- and build-environment or libraries need to be included, otherwise the usage of them can be illegitimate.
- Deposited material is not up to date: With each deployment of a new version at the customer, the entire escrow life cycle needs to be redone again to guarantee that the current version is deposited with the agent.
Deducted from the risks that come with escrow and that are mentioned above, we come to the following requirements for successful software escrow:
- The source code is only one part of software. Documentation, compiler, test scripts, make files, libraries, … are also needed for software to be developed, compiled, and run eventually. Without additional information source code alone is almost impossible to understand, analyze, use, and change!
- All artifacts that are deposited at the escrow agent need to be up-to-date. It proved to be disadvantageous to only deposit the files once at the beginning of the collaboration and never keeping them updated with the currently deployed software version (see the case of Telecom New Zealand v Aldous for instance). The quality of the deposited artifacts is also of importance, as they have to be understood and developed by other programmers in the future in the case of a release event. Thus maintainability of the source code is crucial.
Three phases of escrow
Software escrow is divided in three phases, which will be discussed in more detail below:
- Planning – including the agreement on a contract specifying measurements for software escrow
- Execution – including the deposit of material with iterations of said deposit
- Redeployment – including the release of the material if a release/trigger event occurs
Phase I – Planning
In this initial phase elementary steps need to be taken in order to provide a foundation for the following phases. The planning phase includes:
- The selection of the escrow agent:
- The choice of the agent needs to be done considering the confidence of both customer and software developer
- The agent has to offer the IT infrastructure needed for the check for completeness and quality of the deposited material
- Data security requirements need to be taken into account
- The information-/notification obligations of the agent need to be agreed on
- Alignment of contracts – Available contracts concerning the software licensing or maintenance of the software need to be checked and aligned accordingly
- Selection of the deposit material – A list of the material that needs to be deposited has to be done in this phase. All relevant artifacts need to be identified and put into the escrow agreement.
- Rights to the artifacts – The rights of the escrow agent as well as the software developer’s rights towards the software artifacts need to be agreed on.
- Legal consequences – The legal consequences of a release event need to be clarified. This includes the determination of the rights transferred to the customer in the case of a release event (e.g., the customer is only allowed to maintain the software not sell it), as well as the obligations to confidentiality.
- Setting up an escrow contract – The outcome of the points listed above (deposit material, rights and legal consequences, information obligations, …) need to be put into a contract all parties of the agreement have to sign.
Phase II – Execution
The second phase includes the execution of the escrow procedure, thus all escrow related processes are done in here, including:
- The deposit procedure, meaning all activities related to the deposit of artifacts at the escrow agent. The following points need to be considered and need to be stated in the escrow contract for this:
- Deadlines for the submission of all artifacts
- Existing verification procedures applied to the artifacts
- Thresholds for a successful deposit (completeness and a certain level of quality)
- Information-/notification obligations of the escrow agent (e.g., notifying the customer of a new version deposited)
- The verification of the deposit material (respectively the artifacts) – for this purpose a Technical Software Escrow Framework was designed and implemented to support the task of an escrow agent
The execution phase has to take place each time a new version of the software is released. Only then it can be guaranteed that the deposited material is up-to-date. A useful software release cycle, including software escrow:
Phase III – Redeployment
The third phase takes action once a release event triggered the hand out of the desposited materials. It thus comprises of:
- The release of the deposited materials – The event that triggered the release (e.g., insolvency, refusal to maintenance,…) needs to be verified by the escrow agent. Only then is he allowed to release the material.
- The agent is obliged to inform or notify the customer of the occurrence of a trigger event, respectively the release of the material.
The verification process is an essential part of the escrow process. It is needed to verify the quality and completeness of the deposited material. In literature, there are three types of verification:
- Standard or basic verification: The material will be tested for readability and it will be verified that it is virus free.
- Full verification: Additionally to the standard verification, the source code will be compiled. The problem with this approach is that for instance, already compiled parts of the software (which cannot be maintained in the future) can be hidden in the sources, or that the licenses of libraries used are not checked.
- Bespoken verification: The extent of the verification will be negotiated when setting up the escrow contract. With this it can be ensured that the specification is met and that a manual review will be done. This review should include the completeness of deposited material and specific quality measurements agreed on (e.g., to measure the quality of documentation, understandability of the source code, ability to test the material, modifyability, …)
Costs and benefits of each verification method have to be taken into account when the decision for one is made.
Technical Software Escrow Framework
Verifying the quality and completeness of deposited material is a complex task because the sources that build the basis of the software can be extensive. A human technical reviewer could need many hours to go through all of the artifacts to evaluate them, which can be an essential cost factor. Thus a framework was proposed that supports the verification by automatically evaluating the artifacts and reporting parts that need to be examined manually.
An ideal framework includes checks for:
- the build environment – compilers, make files, …
- quality metric measurements – complexity, lines of code, …
- test environment – check for test cases, test scripts, …
- source code comments – readability/understandability of comments
- specification verification – consistency of the specification and the implementation to see if the specification documents are usable
- system documentation – user guides, developer guides, …
- data sources – data base, binary files used, …
- external references – web services, libraries, …
- code obfuscation – compiled parts within the sources, complex algorithms, …
The framework that was implemented built upon Sonar, an open source platform for continuous quality inspection. We added checks for:
- completeness of artifacts (by building the software)
- consistency of sources and the binary released to the customer
- quality of documentation (language check, spell check)
- quality of source code (cyclomatic complexity, Halstead’s software metrics, relation of complexity and comment lines density)
- usage of third party resources (check for usage of libraries or web services)
- legal certainty (license checks)
A screenshot of Sonar’s dashboard for aTunes with activated plugins is depicted on the right.
- “A Framework for Automated Verification in Software Escrow” describes the framework introduced in this section and applies it to an open source example.
- The paper “A simpler model of software readability” describes the usage of Halstead’s metrics for the measurement of software readability
The CEN Workshop Agreement CWA 13620 offers guidelines for the different actors in a software agreement:
- Escrowguide for acquirers
- Escrowguide for developers
- Escrowguide for agents
- Escrowguide to the auditing process
- The article Source Code Escrow: Are You Just Following the Herd? which has arguments opposing software escrow.
The prototype implementation for the verification framework is based on Sonar, a framework for continuous quality inspection of software . Sonar determines common software quality metrics and checks the compliance to a set of predefined rules of software projects.
To meet the requirements of escrow agents our prototype extends Sonar using custom plugins that aim to support the process of inspecting the software and detect quality issues. This is done by highlighting parts for which different metrics assume low quality, but also by checking the conformance to predefined rules.
For demonstration purpose such metrics and violations of those rules are shown on an exemplary open source project. In the following screenshot the results that have been calculated by the verification framework are shown:
For most of the issues listed in the custom rules table it is possible to drill down the issue to the actual source file or even line of code. This can be seen in the screenshot of the section “Usage in third party resources”.
The screenshots are taken from the prototype instance in , where it is also possible to browse through the details of the findings.
The following description of the functionality of the framework is taken from :
Consistency of sources and released binary
The software put into escrow has to be the same as the one delivered to the customer. To verify this, the sources at the escrow agent have to be built and compared to the binaries delivered to the customer. Our implementation checks if the output generated by building the software matches a provided set of reference artifacts. This is reflected by the rule violation named “Invalid release”:
Quality of documentation
Documentation about the software development project is required to understand considerations and decisions made during the design and development phase in natural language. It is especially important if the software in question has to be maintained and possibly enhanced at an unknown time in the future because the programmer needs to understand the structure and design of the software. Thus the documentation has to be adequate, easily readable, and easily understandable. The following considerations apply to source code comments as well as additional documentation and specification, like architecture descriptions, requirement documents, manuals, etc.
The rule violation “Contains issue in comment text” refers to a misspelling or other issues with the comments, “Contains unexpected language in comments” refers to the usage of a language that is different to the language specified for this project.
Quality of source code
Software metrics can be used to assess the quality of the software, i.e., maintainability. An adequate level of quality is required for further development of the software. Sonar already implements a number of metrics for quality verification that can be used for our maintainability approach, such as the Cyclomatic Complexity. Halstead’s software metrics are a similar measurement method. As they are not implemented in Sonar, we provided a plugin for calculating the Halstead’s software metrics Difficulty, Effort, Volume, Time to Program, and Bugs Delivered to the best of our knowledge. As mentioned previously besides software metrics also the conformance to specified rules can be used to determine the quality of software, e.g. to verify the adherence to coding standards and bestpractices.
The Halstead metrics are shown on the dashboard that is shown in the above screenshot. Furthermore also a custom metric is shown (“Custom Rating”), which is a configurable formula that makes use of other metrics.
Usage of third party resources
We further extended the framework by some digital preservation concerns that are also useful for escrow such as ensuring the availability of external sources. References to external third party resources, like libraries or Web services used in the software, can affect the functionality of the software. If the provider of the service is not available anymore, this can lead to a non-functioning program. Therefore external references have to be identified and properly inspected when verifying the source code. It has to be ensured that the service they are using is available in the long term. A potential strategy of the escrow agent is to deposit the library or materials needed for a Web service as well. If this is not possible, e.g., in the case of proprietary Web services, it has to be ensured that the executing source code sections are identified and reported as a risk to the customer. The use of external services should be specified in the licenses and escrow contract. Our implementation supports the escrow agent in identifying those external resources by reporting matches of a text-based regex search over source files which looks for Web service calls, system calls, etc. The corresponding rule violations are “Executes binary call” and “Executes external call”, which are keyword searches on the source against a customizeable blacklist, and “Contains non-standard libraries”, which points out libraries that could not be located in the Maven central repository.
Licenses are essential as they specify the legal foundation for the usage of the software. Thus the escrow agent needs to determine the licenses of the software’s artifacts, as it has effects on the allowed usage in case of a
release of the material. Our implementation extracts and identifies license information that is embedded in source files. The “Used Licenses” entry shows which licenses are defined in the source files of the project.
The research prototype is available as Debian package and can be installed on Debian-based operating systems using following commands:
- # add the sonar-escrow repository
sudo sh -c ‘echo “deb http://olymp.ifs.tuwien.ac.at/apt nightly main #IFS/Vienna University of Technology” >>/etc/apt/sources.list.d/ifs-Vienna.list’ && sudo apt-get –quiet update && sudo apt-get –yes –quiet –allow-unauthenticated install olymp-keyring && sudo apt-get –quiet update
- # install sonar-escrow
sudo apt-get install sonar-escrow
The prototype can be started by calling “sudo service sonar start”, which after a few seconds allows using the web UI (http://localhost:9000) and API of Sonar . The configuration can either be done per project in the configuration file of the Sonar client (e.g. sonar-project.properties for the Sonar Java Runner) or globally through the admin UI (Settings / Configuration). Widgets for the web UI are an exception, they can only be added and configured through the web UI (Dashboard / Configure widgets). The documentation for Sonar can be found in .
 Elisabeth Weigl and Johannes Binder and Stephan Strodl and D. Draws and Andreas Rauber, “A Framework for Automated Verification in Software Escrow,” in Proceedings of the 10th International Conference on Preservation of Digital Objects (IPRES 2013), 2013
 http://www.sonarqube.org/, accessed 2014-03-24
 http://kronos.ifs.tuwien.ac.at:49000/dashboard/index/net.sourceforge.atunes:atunes, accessed 2014-02-24
 http://docs.codehaus.org/display/SONAR/Documentation, accessed 2014-04-02