Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 33 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,8 @@ type Dependency struct {
Integrity string // Opaque verification value, when available
Direct bool // True if declared directly, false if transitive
PURL string // Package URL (pkg:ecosystem/name@version)
Ecosystem string // Ecosystem name, in the same form as ParseResult.Ecosystem
NoPURL bool // Source-only entry with no known package identity
RegistryURL string // Source registry URL (if non-default)
Source Source // Explicit Git, path, or ecosystem source override
}
Expand All @@ -257,17 +259,38 @@ convert the value with knowledge of the source format.

When a dependency comes from a non-default registry, the PURL includes a `repository_url` qualifier (e.g., `pkg:npm/foo@1.0.0?repository_url=https://npm.mycompany.com/`). Default registries like registry.npmjs.org, pypi.org, and rubygems.org are not included in the PURL.

A file can refer to packages from more than one ecosystem, such as `npm:`
imports in `deno.json`. A parser marks those entries by setting `Ecosystem` to
an ecosystem name in the same form as `ParseResult.Ecosystem`. `Parse` sets
`Ecosystem` on every dependency, using the file's ecosystem when the parser did
not set one, so consumers never need to fall back to `ParseResult.Ecosystem`.
Entries with `NoPURL` keep an empty `Ecosystem` because they have no package
identity.

`Parse` resolves each dependency's PURL in this order:

1. `NoPURL` leaves the PURL empty. Source-only entries, such as direct download
URLs, keep their location in `Source` without an invented package identity.
2. A PURL supplied by the parser is preserved as written. A parser that
supplies a PURL from another ecosystem should also set `Ecosystem`.
3. Otherwise the PURL is built from `Ecosystem`. Manifest PURLs omit the
version. Lockfile and supplement PURLs include it.

`Declaration` follows the same rules and its PURL always omits the version.

### Declaration

```go
type Declaration struct {
Name string // Package name
Version string // Version requirement as written in the manifest
Scope Scope // runtime, development, test, build, optional
Direct bool // Direct rather than generated or transitive
PURL string // Versionless Package URL
Location string // Opaque parser-defined identity within the manifest
Source Source // Explicit source override as written
Name string // Package name
Version string // Version requirement as written in the manifest
Scope Scope // runtime, development, test, build, optional
Direct bool // Direct rather than generated or transitive
PURL string // Versionless Package URL
Ecosystem string // Ecosystem name, in the same form as ParseResult.Ecosystem
NoPURL bool // Source-only entry with no known package identity
Location string // Opaque parser-defined identity within the manifest
Source Source // Explicit source override as written
}
```

Expand All @@ -276,9 +299,9 @@ merging, interpolation, or other effective-model resolution. Consumers can use
`Location` to match the same logical entry across edits, but should not parse
its ecosystem-specific value. A declaration PURL omits the version because the
raw requirement may be a range or property expression. When a parser supplies
its own PURL, `Parse` preserves it so one manifest can refer to packages from
different ecosystems. Otherwise `Parse` builds the PURL from the parser's
ecosystem.
its own PURL or `Ecosystem`, `Parse` uses it so one manifest can refer to
packages from different ecosystems. Otherwise `Parse` builds the PURL from the
file's ecosystem and records that ecosystem in `Ecosystem`.

`Direct` distinguishes explicit requirements from generated or transitive
entries when the source format records that distinction, such as `go.mod`.
Expand Down
28 changes: 24 additions & 4 deletions internal/core/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,21 @@ type Dependency struct {
Scope Scope
// Integrity is an opaque verification value derived from the manifest or
// lockfile. Its digest encoding is ecosystem-specific.
Integrity string
Direct bool
PURL string
Integrity string
Direct bool
// PURL identifies the package. Parse preserves a parser-supplied value and
// otherwise builds it from Ecosystem, Name and, for lockfiles and
// supplements, Version.
PURL string
// Ecosystem is the dependency's ecosystem name, in the same form as
// ParseResult.Ecosystem. A parser sets it when the package belongs to a
// different ecosystem than the file, such as an npm package imported from
// deno.json. Parse fills in the file's ecosystem otherwise and leaves it
// empty when NoPURL is set.
Ecosystem string
// NoPURL marks a source-only entry, such as a direct download URL, whose
// package identity is unknown. Parse leaves its PURL empty.
NoPURL bool
RegistryURL string
// Source is set only when this dependency has an explicit source override.
// It is intentionally separate from RegistryURL because paths and Git
Expand All @@ -76,7 +88,15 @@ type Declaration struct {
Direct bool
// PURL identifies the declared package without a version. Parsers may set
// it when a file can contain references from more than one ecosystem.
PURL string
PURL string
// Ecosystem is the declaration's ecosystem name, in the same form as
// ParseResult.Ecosystem. A parser sets it when the package belongs to a
// different ecosystem than the file. Parse fills in the file's ecosystem
// otherwise and leaves it empty when NoPURL is set.
Ecosystem string
// NoPURL marks a source-only entry whose package identity is unknown.
// Parse leaves its PURL empty.
NoPURL bool
Location string
Source Source
}
Expand Down
19 changes: 15 additions & 4 deletions internal/npm/deno.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,11 @@ func (p *denoJSONParser) Parse(filename string, content []byte) (*core.Result, e
name, version := parseDenoSpec(spec)
if name != "" {
deps = append(deps, core.Dependency{
Name: name,
Version: version,
Scope: core.Runtime,
Direct: true,
Name: name,
Version: version,
Scope: core.Runtime,
Direct: true,
Ecosystem: denoSpecEcosystem(spec),
})
}
}
Expand Down Expand Up @@ -95,6 +96,7 @@ func (p *denoLockParser) Parse(filename string, content []byte) (*core.Result, e
Scope: core.Runtime,
Integrity: pkg.Integrity,
Direct: false,
Ecosystem: "npm",
})
}

Expand Down Expand Up @@ -123,6 +125,15 @@ func parseDenoSpec(spec string) (name, version string) {
return spec, ""
}

// denoSpecEcosystem returns the package ecosystem of an npm: import, or empty
// so the dependency keeps the deno ecosystem.
func denoSpecEcosystem(spec string) string {
if strings.HasPrefix(spec, "npm:") {
return "npm"
}
return ""
}

// parseNPMSpec parses an npm spec like "chalk@5.3.0" or "chalk".
func parseNPMSpec(spec string) (name, version string) {
// Handle scoped packages
Expand Down
20 changes: 10 additions & 10 deletions lock_capacity_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,27 +16,27 @@ func TestLockDependencyCapacity(t *testing.T) {
want []Dependency
}{
{"composer.lock", `{"packages":[{"name":"vendor/runtime","version":"1.2.3","dist":{"shasum":"abc","sha256":"def"}}],"packages-dev":[{"name":"vendor/dev","version":"2.0.0","dist":{"shasum":"abc"}}]}`, []Dependency{
{Name: "vendor/runtime", Version: "1.2.3", Scope: Runtime, Integrity: "sha256-def", PURL: "pkg:composer/vendor/runtime@1.2.3"},
{Name: "vendor/dev", Version: "2.0.0", Scope: Development, Integrity: "sha1-abc", PURL: "pkg:composer/vendor/dev@2.0.0"},
{Name: "vendor/runtime", Version: "1.2.3", Scope: Runtime, Integrity: "sha256-def", PURL: "pkg:composer/vendor/runtime@1.2.3", Ecosystem: "composer"},
{Name: "vendor/dev", Version: "2.0.0", Scope: Development, Integrity: "sha1-abc", PURL: "pkg:composer/vendor/dev@2.0.0", Ecosystem: "composer"},
}},
{"packages.lock.json", `{"version":1,"dependencies":{"net8.0":{"Alpha":{"type":"Direct","resolved":"1.2.3","contentHash":"abc"}},"net9.0":{"Alpha":{"type":"Direct","resolved":"1.2.3","contentHash":"abc"},"Beta":{"type":"Transitive","resolved":"2.0.0"}},"net10.0":{"Gamma":{"type":"Transitive","resolved":"3.0.0"}}}}`, []Dependency{
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", Direct: true, PURL: "pkg:nuget/Alpha@1.2.3"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0"},
{Name: "Gamma", Version: "3.0.0", Scope: Runtime, PURL: "pkg:nuget/Gamma@3.0.0"},
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", Direct: true, PURL: "pkg:nuget/Alpha@1.2.3", Ecosystem: "nuget"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0", Ecosystem: "nuget"},
{Name: "Gamma", Version: "3.0.0", Scope: Runtime, PURL: "pkg:nuget/Gamma@3.0.0", Ecosystem: "nuget"},
}},
{"project.assets.json", `{"targets":{"net8.0":{"Alpha/1.2.3":{"type":"package"},"App/1.0":{"type":"project"},"invalid":{"type":"package"}},"net9.0":{"Alpha/1.2.3":{"type":"package"},"Beta/2.0.0":{"type":"package"}},"net10.0":{"Gamma/3.0.0":{"type":"package"}}},"libraries":{"Alpha/1.2.3":{"sha512":"abc"}}}`, []Dependency{
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", PURL: "pkg:nuget/Alpha@1.2.3"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0"},
{Name: "Gamma", Version: "3.0.0", Scope: Runtime, PURL: "pkg:nuget/Gamma@3.0.0"},
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", PURL: "pkg:nuget/Alpha@1.2.3", Ecosystem: "nuget"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0", Ecosystem: "nuget"},
{Name: "Gamma", Version: "3.0.0", Scope: Runtime, PURL: "pkg:nuget/Gamma@3.0.0", Ecosystem: "nuget"},
}},
}
for _, name := range []string{"example.deps.json", "Project.lock.json"} {
tests = append(tests, struct {
name, content string
want []Dependency
}{name, `{"libraries":{"Alpha/1.2.3":{"type":"package","sha512":"abc"},"Beta/2.0.0":{"type":"package"},"App/1.0":{"type":"project"},"invalid":{"type":"package"}}}`, []Dependency{
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", PURL: "pkg:nuget/Alpha@1.2.3"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0"},
{Name: "Alpha", Version: "1.2.3", Scope: Runtime, Integrity: "sha512-abc", PURL: "pkg:nuget/Alpha@1.2.3", Ecosystem: "nuget"},
{Name: "Beta", Version: "2.0.0", Scope: Runtime, PURL: "pkg:nuget/Beta@2.0.0", Ecosystem: "nuget"},
}})
}
for _, tt := range tests {
Expand Down
53 changes: 43 additions & 10 deletions manifests.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
package manifests

import (
"cmp"

"github.com/git-pkgs/manifests/internal/core"
"github.com/git-pkgs/purl"
)
Expand Down Expand Up @@ -138,16 +140,16 @@ func Parse(filename string, content []byte, opts ...Options) (*ParseResult, erro
res = &core.Result{}
}

// Generate PURLs for all dependencies
// Resolve the ecosystem and PURL of every dependency and declaration
for i := range res.Dependencies {
version := ""
if kind == Lockfile || kind == Supplement {
version = res.Dependencies[i].Version
}
res.Dependencies[i].PURL = makePURL(eco, res.Dependencies[i].Name, version, res.Dependencies[i].RegistryURL)
dep := &res.Dependencies[i]
dep.PURL = dependencyPURL(eco, kind, *dep)
dep.Ecosystem = entryEcosystem(eco, dep.Ecosystem, dep.NoPURL)
}
for i := range res.Declarations {
res.Declarations[i].PURL = declarationPURL(eco, res.Declarations[i])
declaration := &res.Declarations[i]
declaration.PURL = declarationPURL(eco, *declaration)
declaration.Ecosystem = entryEcosystem(eco, declaration.Ecosystem, declaration.NoPURL)
}

return &ParseResult{
Expand All @@ -165,13 +167,44 @@ func Parse(filename string, content []byte, opts ...Options) (*ParseResult, erro
}, nil
}

// declarationPURL preserves a parser-supplied package identity or builds one
// from the parser's ecosystem.
// entryEcosystem returns the ecosystem a parser set on a dependency or
// declaration, falling back to the parser's. Entries without a package
// identity have no ecosystem.
func entryEcosystem(ecosystem, entry string, noPURL bool) string {
if noPURL {
return ""
}
return cmp.Or(entry, ecosystem)
}

// dependencyPURL preserves a parser-supplied package identity or builds one
// from the dependency's ecosystem, falling back to the parser's. Only
// lockfiles and supplements include the version.
func dependencyPURL(ecosystem string, kind Kind, dep core.Dependency) string {
if dep.NoPURL {
return ""
}
if dep.PURL != "" {
return dep.PURL
}
version := ""
if kind == Lockfile || kind == Supplement {
version = dep.Version
}
return makePURL(entryEcosystem(ecosystem, dep.Ecosystem, false), dep.Name, version, dep.RegistryURL)
}

// declarationPURL preserves a parser-supplied package identity or builds a
// versionless one from the declaration's ecosystem, falling back to the
// parser's.
func declarationPURL(ecosystem string, declaration core.Declaration) string {
if declaration.NoPURL {
return ""
}
if declaration.PURL != "" {
return declaration.PURL
}
return makePURL(ecosystem, declaration.Name, "", "")
return makePURL(entryEcosystem(ecosystem, declaration.Ecosystem, false), declaration.Name, "", "")
}

// makePURL creates a Package URL for a dependency.
Expand Down
22 changes: 11 additions & 11 deletions npm_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,10 @@ func TestNPMLockfileNestedDependencies(t *testing.T) {
t.Fatal(err)
}
want := []Dependency{
{Name: "@scope/leaf", Version: "3.0.0", Scope: Development, Integrity: "sha512-leaf", PURL: "pkg:npm/%40scope/leaf@3.0.0"},
{Name: "parent", Version: "1.0.0", Scope: Development, PURL: "pkg:npm/parent@1.0.0"},
{Name: "shared", Version: "2.0.0", Scope: Optional, PURL: "pkg:npm/shared@2.0.0"},
{Name: "shared", Version: "4.0.0", Scope: Runtime, PURL: "pkg:npm/shared@4.0.0"},
{Name: "@scope/leaf", Version: "3.0.0", Scope: Development, Integrity: "sha512-leaf", PURL: "pkg:npm/%40scope/leaf@3.0.0", Ecosystem: "npm"},
{Name: "parent", Version: "1.0.0", Scope: Development, PURL: "pkg:npm/parent@1.0.0", Ecosystem: "npm"},
{Name: "shared", Version: "2.0.0", Scope: Optional, PURL: "pkg:npm/shared@2.0.0", Ecosystem: "npm"},
{Name: "shared", Version: "4.0.0", Scope: Runtime, PURL: "pkg:npm/shared@4.0.0", Ecosystem: "npm"},
}
sort.Slice(result.Dependencies, func(i, j int) bool {
a, b := result.Dependencies[i], result.Dependencies[j]
Expand Down Expand Up @@ -159,9 +159,9 @@ func TestNPMLockfileLineIteration(t *testing.T) {
t.Fatal(err)
}
want := []Dependency{
{Name: "direct", Version: "1.0.0", Scope: Runtime, Direct: true, Integrity: integrity, PURL: "pkg:npm/direct@1.0.0"},
{Name: "@scope/nested", Version: "2.0.0", Scope: Development, PURL: "pkg:npm/%40scope/nested@2.0.0"},
{Name: "last", Version: "3.0.0", Scope: Optional, PURL: "pkg:npm/last@3.0.0"},
{Name: "direct", Version: "1.0.0", Scope: Runtime, Direct: true, Integrity: integrity, PURL: "pkg:npm/direct@1.0.0", Ecosystem: "npm"},
{Name: "@scope/nested", Version: "2.0.0", Scope: Development, PURL: "pkg:npm/%40scope/nested@2.0.0", Ecosystem: "npm"},
{Name: "last", Version: "3.0.0", Scope: Optional, PURL: "pkg:npm/last@3.0.0", Ecosystem: "npm"},
}
if !reflect.DeepEqual(result.Dependencies, want) {
t.Fatal("dependency fields or order differ")
Expand Down Expand Up @@ -281,8 +281,8 @@ func TestNPMLockfileRootEntryLast(t *testing.T) {
t.Fatal(err)
}
want := []Dependency{
{Name: "direct", Version: "1.0.0", Scope: Runtime, Direct: true, PURL: "pkg:npm/direct@1.0.0"},
{Name: "transitive", Version: "2.0.0", Scope: Runtime, PURL: "pkg:npm/transitive@2.0.0"},
{Name: "direct", Version: "1.0.0", Scope: Runtime, Direct: true, PURL: "pkg:npm/direct@1.0.0", Ecosystem: "npm"},
{Name: "transitive", Version: "2.0.0", Scope: Runtime, PURL: "pkg:npm/transitive@2.0.0", Ecosystem: "npm"},
}
if !reflect.DeepEqual(result.Dependencies, want) {
t.Fatalf("got %+v, want %+v", result.Dependencies, want)
Expand All @@ -302,8 +302,8 @@ func TestNPMV1LockfilePackagesDependency(t *testing.T) {
}
}`)
want := []Dependency{
{Name: "is-number", Version: "7.0.0", Scope: Runtime, PURL: "pkg:npm/is-number@7.0.0"},
{Name: "packages", Version: "0.0.8", Scope: Runtime, PURL: "pkg:npm/packages@0.0.8"},
{Name: "is-number", Version: "7.0.0", Scope: Runtime, PURL: "pkg:npm/is-number@7.0.0", Ecosystem: "npm"},
{Name: "packages", Version: "0.0.8", Scope: Runtime, PURL: "pkg:npm/packages@0.0.8", Ecosystem: "npm"},
}
for _, filename := range []string{"package-lock.json", "npm-shrinkwrap.json"} {
t.Run(filename, func(t *testing.T) {
Expand Down
Loading
Loading