Bomly CLI v0.28.1

What this shows

• The binary users actually download produces the same dependency graphs as the source tree it was built from.

What it does not

• A representative subset of ecosystems runs against the released binary; the full ecosystem matrix runs on the source tree before the release is tagged.

What this shows

• For every supported ecosystem, the command line produces exactly the dependency graph, policy result, and output shape recorded in the golden files.

What it does not

• Golden files pin one revision of one example project per ecosystem. They do not cover every project layout or every version of a package manager.
• Cases that reach live advisory services can change when upstream data changes.

What this shows

• A detector fallback preserves its origin and produces a typed degraded-coverage warning.

What it does not

• The synthetic detector case proves orchestration behavior, not the fidelity of every concrete fallback.

What this shows

• The native Bun detector preserves the package inventory and dependency placement represented by the pinned lockfile.

What it does not

• This is one Bun lockfile shape and does not prove behavior for every historical Bun format.

What this shows

• The Go detector resolves a build-tool-backed module graph for the pinned repository.

What it does not

• The result depends on a compatible Go toolchain and the evidence available from that toolchain.

What this shows

• The Maven detector resolves a build-tool-backed dependency graph for the pinned repository.

What it does not

• Artifact resolution depends on Maven repositories and a compatible Java and Maven installation.

What this shows

• The npm detector preserves lockfile package inventory, versions, scopes, and dependency placement.

What it does not

• This case covers the lockfile versions present in one pinned repository.

What this shows

• The pnpm detector preserves lockfile package inventory and dependency placement.

What it does not

• This case does not cover every pnpm lockfile generation.

What this shows

• The pip detector reads the pinned requirements lock and preserves the resolved Python package graph.

What it does not

• Unpinned requirements and environment inspection have different fidelity and are not proven by this case.

What this shows

• The Yarn detector preserves package inventory and dependency placement represented by the pinned lockfile.

What it does not

• This case covers one Yarn lockfile family; fallback behavior is documented separately.

What this shows

• A project baseline can keep a denied package finding visible with suppressed policy status.

What it does not

• The local matcher fixture is deterministic but does not measure advisory freshness.

What this shows

• The license auditor evaluates AND, OR, nested, exception, custom-reference, and invalid SPDX expressions under allow and deny policy.

What it does not

• The tests prove policy evaluation after license data is present; they do not prove the accuracy of every upstream license source.

What this shows

• Diff marks a same-version registry-to-Git move for review and the package auditor can fail it when source-change policy is enabled.

What it does not

• A source change is a review signal, not proof of malicious intent, and unknown source evidence cannot be classified.

What this shows

• The vulnerability auditor composes severity, reachability, known exploitation, and advisory allowlists with the documented repeated-constraint behavior.

What it does not

• Policy can only evaluate vulnerability, reachability, and exploitation evidence that is present in the registry.

What this shows

• The Go analyzer attaches named analyzer, tier, status, and reason evidence to matched vulnerabilities.

What it does not

• Advisories come from live services and may change; an unreachable result is not proof that a package is safe.

What this shows

• The Java analyzer separates package-reachable and package-unreachable vulnerability evidence.

What it does not

• This is package-tier evidence, not symbol-level proof, and live advisory results may change.

What this shows

• The JavaScript analyzer separates package-reachable and package-unreachable vulnerability evidence.

What it does not

• This is package-tier evidence, dynamic loading can reduce confidence, and live advisory results may change.

What this shows

• The Python analyzer separates package-reachable and package-unreachable vulnerability evidence.

What it does not

• This is package-tier evidence, reflective imports can reduce confidence, and live advisory results may change.

What this shows

• One central component derives fix status, recommended versions, occurrence actions, and detector advice without applying changes.

What it does not

• Suggestions are read-only evidence and are not guaranteed to work in every checkout.

What this shows

• A vulnerability or license finding that remains across a package version change is classified as persisted instead of one resolved and one introduced finding.

What it does not

• Persistence uses the canonical package-finding identity; a genuinely different advisory or rule remains a distinct finding.

What this shows

• Bomly can inventory the operating-system packages in the checked Alpine image.

What it does not

• The upstream image tag can move; the checked-in golden is a snapshot, not an immutable image claim.

What this shows

• The SBOM documents this release writes are accepted by the official SPDX and CycloneDX validators, at the exact validator versions recorded in the result.
• A document merged from two sources states its own identity and links both sources, rather than adopting one of them.

What it does not

• Validators check that a document is well formed against its specification. They do not check that every receiving tool interprets it the same way.
• The merge case uses two checked-in fixtures; it does not cover every combination of source formats.
• The documents are generated by a binary built from the release commit, before it is tagged, not by the published archive. The published files are verified separately.

What this shows

• Bomly ingests the checked CycloneDX 1.6 dependency graph.

What it does not

• The fixture covers one document shape and does not imply lossless conversion from every producer.

What this shows

• Bomly ingests the checked SPDX 2.3 dependency graph.

What it does not

• The fixture covers one document shape and does not imply lossless conversion from every producer.

What this shows

• The source tree compiles into both binaries for every Linux, macOS, and Windows target a release ships.

What it does not

• A successful build is not a test: it does not run the binary or check what it produces.

What this shows

• Every unit test passes on all three supported operating systems, twice in a row, so a pass is not a one-off.

What it does not

• Unit tests run in process against fixtures. They do not download repositories or call external services.

What this shows

• The build-tool-backed Java detector suites passed ten consecutive times.

What it does not

• This exercises the detectors against local fixtures with the toolchain installed on the runner.

What this shows

• Each registered parser handled its seed corpus and newly generated inputs without crashing during the time budget of this run.

What it does not

• Fuzzing explores a limited time budget. A clean run is evidence of robustness, not proof that no malformed input can cause a failure.
• Parsers that are command-backed or handled entirely by the standard library are excluded and listed in test/assurance/PARSER_FUZZING.md.

What this shows

• Repeated runs of the same scan produced identical normalized output, and the recorded timings show how long that scan took on this machine.

What it does not

• Timing and memory are observations from one CI machine, not limits. Comparing them across releases is only meaningful because the case and the machine class stay the same.

What this shows

• Every file a user downloads from the public release page is present, reachable without credentials, and matches a checksum list signed by this repository's release workflow.

What it does not

• This checks availability and integrity at one moment from one network location.
• It confirms the published files match the signed list; it does not re-check the provenance attestation, which is verified before publication.
• Whether each archive contains a working binary is checked separately, against the draft release and by the install scripts.

What this shows

• The release carries an archive for every supported platform, both binaries, every Linux package format, the checksum list, its signature, and the provenance file.

What it does not

• This counts files. Whether each archive contains a working binary is checked separately.

What this shows

• Both the full and lite binaries start on Linux, macOS, and Windows and report the version that is being released.

What it does not

• Only the architecture of the CI runner is executed for each operating system; other architectures are verified by checksum and build only.

What this shows

• Every file the release ships hashes to exactly the value recorded in the checksum list, and the list covers every file.

What it does not

• Checksums prove the files were not altered after they were built; the signature and provenance checks prove who built them.

What this shows

• The release archives were produced by this repository's tagged release workflow, as recorded in signed build provenance.

What it does not

• Provenance describes how and where the files were built. It does not review what the source code does.

What this shows

• The checksum list carries a valid Sigstore signature issued to this repository's release workflow, so the checksums themselves can be trusted.

What it does not

• The signature covers the checksum list. Individual files are tied to it through their checksums.

What this shows

• The documented one-line install works on Linux, macOS, and Windows for this exact release, and the installed binary reports this version.

What it does not

• This installs into a temporary directory on a clean runner. It does not cover every shell, proxy, or locked-down corporate environment.

What this shows

• Every claim in this report points at a real check and at files that are present in this release's source.

What it does not

• A valid catalog says the paperwork is consistent; the checks themselves decide whether the software works.