What is Schaapi?
Schaapi is a tool for early detection of breaking changes based on API usages!
It also contains the Dutch word for sheep ("schaap") 🐑
But why?
Library developers are often unaware of how their library is used exactly in practice. When a library developer changes the internals of a library, this may unintentionally affect or even break the working of the library users' code. While it is possible to detect when a syntactic breaking change occurs, it is not as easy to detect semantic breaking changes, where the implicit contract of a functionality changes, sometimes unbeknownst to the library developer. Because library users rarely test the behaviour they expect of the library, neither the library developer nor the library user will be aware of the new behaviour.
As a library developer, you want to be able to see how a change in your library will affect your users before a new version of the library is deployed. More specifically, you want to gain insight into how users use the library, and want to see if and how changes affect users. This will allow you to determine whether the new version of the library is backwards compatible. Finally, after deploying the breaking changes, you want to notify the affected users of the changes and of a solution to the issue.
Schaapi's approach
Schaapi has two pipelines: a mining and a validation pipeline.
In the mining pipeline, we first mine open-source repositories that use a certain library version. We then build graph-based models from their program structures to understand how the library is being used. These library usages are then mined for common patterns to generate tests that capture the usage of the library.
In the validation pipeline, the generated tests are run against each new library revision as a continuous integration service. Library developers can then be informed directly on the effects of their proposed changes.