How we strengthened Bomly CLI's quality and security

We recently spent two weeks reviewing the quality and security of Bomly CLI. The resulting changes are included in Bomly CLI v0.21.0.

The work improved how Bomly builds dependency graphs, handles uncertain evidence, protects trust boundaries, and stays consistent across workflows and platforms. If you use scan, explain, or diff, the results now carry more accurate detail—and when the evidence is uncertain, Bomly says so.

The goal was to make the features already in Bomly more dependable.

We started by asking harder questions

A dependency scanner can produce polished output while still getting important details wrong. Is a package direct or transitive? Does it belong to the project, or is it a published registry package? Did two data sources describe the same vulnerability differently? What happens when the evidence is incomplete?

We turned questions like these into tests and small public examples. That process exposed real problems in dependency graphs, workspace handling, package identity, cross-platform parsing, and security boundaries. Each fix made the next result easier to trust.

The central lesson was simple: uncertain evidence should stay uncertain. Bomly no longer guesses a parent relationship when a detector cannot prove one. Local projects and workspace packages remain visible in the graph without being treated as public registry releases. Duplicate versions, aliases, scopes, and manifests keep their own occurrence details instead of being flattened away.

You can read more about the model in the architecture guide and see current ecosystem coverage in the support matrix.

We made one result serve every workflow

Bomly uses the same dependency graph for scanning, auditing, explaining why a package is present, comparing two targets, producing SBOMs, and answering MCP requests.

During this work, we strengthened that shared foundation:

• Native npm, pnpm, Yarn, and Bun handling received broader graph and workspace coverage.
• Vulnerability records from different matchers are consolidated without throwing away richer evidence.
• License policy now has systematic coverage for complex SPDX expressions and invalid input.
• Finding baselines, dependency diffs, and source-change review preserve the same package identity across output formats.
• Vulnerability remediation guidance now comes from one central, read-only model instead of separate surface-specific logic.

That last point also led to an important product decision. We experimented with automatically changing manifests and lockfiles, then removed the feature before release. Applying dependency updates safely requires more value and confidence than the early design could provide. Bomly now reports fix availability, recommended versions, and package-manager guidance where the evidence supports them—but leaves the change and validation in your hands.

We tightened the safety boundaries

Security tooling needs a clear trust model of its own.

Repository-local configuration is no longer loaded automatically; selecting it is an explicit decision. Network enrichment remains opt-in through --enrich. Input parsers, plugin downloads, archives, baselines, caches, and remote responses have defined size limits. Remote Git operations have time, cancellation, checkout, and symlink protections.

We also changed what troubleshooting output shows. At normal verbosity, Bomly avoids exposing arbitrary subprocess output. At debug verbosity, it can show the underlying tool diagnostics you need to understand a failed package-manager or matcher command. Because those tools may print sensitive values, the security guide explains why debug logs should be handled as sensitive data.

These controls reduce risk, but they are not a sandbox. Explicitly selected tools and native plugins still run with the user's privileges. We think stating this limit clearly is part of the security work.

We made the evidence reproducible

Tests should prove behavior, not just pass.

Bomly now has focused fuzz tests for parsers, repeated cold and warm performance samples, official SPDX and CycloneDX validation, and portable unit-test and build runs across Linux, macOS, and Windows. One of those portable runs found a Windows-only line-ending bug that ordinary development had missed.

We also published a catalog of 24 evidence cases covering dependency graphs, policy, read-only remediation guidance, SBOMs, targets, fallback behavior, and operational stability. Each case says what it proves, how to reproduce it, and what it does not prove.

The catalog is available in the reproducible evidence guide. From a Bomly CLI checkout, you can verify it with:

make evidence

What this means for users

The practical outcome is not a new command. It is greater confidence in the commands already there:

• scan keeps more accurate occurrence and relationship detail.
• explain has a stronger graph for answering why a package is present.
• diff keeps package, finding, source, and relationship changes aligned.
• JSON, text, Markdown, SARIF, TUI, SBOM, and MCP views derive from shared models.
• Degraded, unknown, or incomplete evidence is reported as uncertain instead of being shown as certain.
• Tests and public evidence make important behavior easier to inspect independently.

There are still limits. Advisory data can change, detector fidelity varies by ecosystem, package-level reachability is triage evidence rather than proof of safety, and read-only remediation suggestions are not guaranteed fixes. We would rather state those limits clearly than hide them behind a broad promise.

This is the standard we want Bomly to keep as it grows: test assumptions, state limits, and make important behavior easy to verify.

Try Bomly CLI on your project