About APISnoop
APISnoop tracks the testing and conformance coverage of Kubernetes by analyzing the audit logs created by e2e test runs.
It updates thrice-weekly. It was last updated on
How We gather our coverage
For our front page sunburst, we use two data sources:
- The open api spec for Kubernetes
- Audit logs from an e2e test run that was run for this release(example test)
We spin up a postgres database integrated with some custom data processing tools, and add each audit event to our database, along with every endpoint as defined in the spec. The audit events show every time an endpoint was hit and the useragent that hit it. Since 1.12, all tests include their test name in their useragent. This means we can see if an endoint was hit by seeing if its hit by a useragent with 'e2e.test' in its text string. Then we can take every endpoint and see whether its hit by e2e tests and e2e tests with '[Conformance]' in their name. We output these results to json, which we use to build up the sunburst.
Our Conformance Progress page works the same, but we add an additional data source: The list of all conformance tests, as outlined in test/conformance/testdata/conformance.yaml
This file includes the release in which the test was promoted to conformance. We take the api specs from 1.9 onward, and see when an endpoint first appeared in them. Then we sort all the conformance tests that hit it by their release date and use the first test to determine when the endpoint was first conformance tested.
As Conformance is concerned with stable(or generally available) endpoints, we focus our progress reports to this same set. Specifically, we only look at endpoints that are stable as of this last release and that are eligible for Conformance. Because of this, there may be some difference in numbers from this page and the sunburst, as the Sunburst shows all endpoints for a release, including those that have since been deprecated.