Skip to main content

Export schema

This page is the schema for the full data export: what the bundle contains and the shape of each record. How to start an export, verify it and what limits apply are on the data export page. The Data Processing Agreement §9 points here.

Bundle layout

export-<your-org-slug>-<YYYY-MM-DDTHH-MM-SS-mmmZ>.tar.gz
├── VERIFY.md                          — step-by-step verification recipe
├── manifest.json                      — every file below, SHA-256 and size
├── manifest.json.intoto.jsonl         — DSSE-signed in-toto Statement over manifest.json
├── audit.ndjson                       — your org's audit events, last 90 days
├── access-log.ndjson                  — your registry access log (every package and container download, upload and delete)
├── bundles/
│   └── <repo>.bundle                  — git bundle: full history, every ref
└── metadata/
    ├── packages.ndjson                — package inventory, see "Packages" below
    ├── packages-files.ndjson          — every file of every package version
    ├── packages-container-tags.ndjson — container image tag → manifest digest
    ├── components.ndjson              — third-party component inventory, see "Component inventory" below
    └── <repo>/
        ├── repo.ndjson                — repository settings, description, topics
        ├── issues.ndjson              — every issue, open and closed
        ├── pulls.ndjson               — every pull request, open and closed
        ├── comments.ndjson            — issue and pull-request comments
        ├── labels.ndjson
        ├── milestones.ndjson          — open and closed
        ├── releases.ndjson            — release metadata; see "Release assets" below
        └── lfs-manifest.ndjson        — LFS object inventory, see "LFS content" below

Repository names containing characters outside a-z 0-9 . _ - appear with those characters replaced by _.

Standards used:

  • Repos: Git’s native git bundle format. git clone <bundle> reconstructs the repo end-to-end, including history, branches and tags.
  • Issues, pull requests and the other metadata/ files: NDJSON, one JSON object per line, in the same shape the Forgejo API returns for each object.
  • Audit log: NDJSON, one event per line (id, action, actor, fields_json, created_at). Same shape as fremforge’s internal audit stream.
  • Registry access log: NDJSON, oldest first, one event per line in the shape GET /orgs/{slug}/access-log returns, with client IPs and logins as recorded. It covers everything still held under your audit log retention. See Registry access log.
  • Integrity: manifest.json lists every file with its SHA-256 hash and size. The manifest is DSSE-signed with fremforge’s Ed25519 builder key, the signature lives in manifest.json.intoto.jsonl and verifies against the public trust root at https://www.frem.sh/.well-known/slsa-trust-root.json using openssl, jq, python3, and curl, no fremforge tooling required. The full recipe is in VERIFY.md inside the bundle and in §“Verifying the signed manifest” on the data export page. This is the same trust root published for SLSA build attestations, one key, two predicate types, zero Sigstore dependency.

No proprietary formats. Every artifact in the bundle reads with standard OSS tooling. No fremforge CLI required.

Packages

metadata/packages.ndjson is the inventory of your org’s package registry: one line per package version, with its type (container, npm, maven, pypi, generic, …), name and version, as the Forgejo package API lists them.

metadata/packages-files.ndjson lists every file of every one of those versions, one line per file:

FieldMeaning
owner, type, name, versionthe package version the file belongs to
file_id, file_namethe file’s identifier and name in the registry
sizesize in bytes
sha256SHA-256 of the file; sha512 as well where the registry records one
download_pathpath on frem.sh that downloads exactly this file with an API token (Authorization: token …); the registry may answer with a redirect to object storage. null for registry types whose file URL can’t be derived from the inventory alone (for example Maven); use web_path for those
web_paththe file’s download link on its package page, for a signed-in browser

For container images, a package version is a tag (or, for an untagged manifest such as one platform of a multi-arch image, the digest itself). metadata/packages-container-tags.ndjson maps each tag to its manifest digest (owner, name, tag, manifest_digest), which is what docker pull <image>@sha256:… and other OCI tools use. The image’s layers and config appear in packages-files.ndjson as files named by their digest, with download_path pointing at the registry’s /v2/…/blobs/… and /v2/…/manifests/… endpoints.

Check a downloaded file against the inventory:

row=$(jq -c 'select(.name=="@acme/lib" and .version=="1.2.0")' metadata/packages-files.ndjson)
curl -sSL -H "Authorization: token $TOKEN" -o pkg.bin "https://frem.sh$(jq -r .download_path <<<"$row")"
echo "$(jq -r .sha256 <<<"$row")  pkg.bin" | sha256sum -c -

The inventory is always in the bundle. The package files themselves come only on request, as a separate archive: see §“Package contents”.

Component inventory

metadata/components.ndjson is your org’s third-party component inventory: one line per component and place it is or was used, the same rows GET /orgs/{slug}/components/usages answers. Uses that have ended are kept for 180 days and are included, so the history leaves with you.

FieldMeaning
purl, ecosystem, name, versionthe component, as a package URL and its parts
licence, licence_sourcethe licence expression, and where it was read from
subject_kind, subject_name, subject_refwhere it is used: the repository, package or image, and the ref or digest
image_tagsfor an image, the tags pointing at it; otherwise empty
source_repothe repository that built it, where known
is_directtrue for a direct dependency, false for a transitive one, null where the SBOM does not say
environment, environment_sourcethe environment class (sandbox, development, test or production) and how it was set
is_current, first_seen_at, last_seen_at, removed_atwhether the use is current, and when it was first and last seen and removed