Skip to main content

Security and supply chain

fremforge ships a layered supply-chain security stack included in every seat plan. Controls run at four choke points: the push (secret scanning), the PR (dependency scanning, SAST), the artifact (container scanning, SBOM, SLSA provenance), and the commit (SSH-key or GPG signing).

For the contractual posture (BSI C5, ISO 27001/27017/27018, TISAX, GDPR, the Schrems II / CLOUD Act analysis) see frem.sh/trust.

Where findings are listed

Security › Findings opens on All: every open finding of every scanner in one list, most important first (known exploited, then severity, then the oldest). Filter it by type and severity; each chip shows how many findings it would list. A row links to the finding on its own tab (SAST, Secrets, Dependencies, Container images, Licences, Malware), where its details and actions are. The same list is in the API as GET /orgs/{org}/security/findings; see Findings API.

Secret scanning (push protection)

Gitleaks runs as a pre-receive hook on every push, before the commit is accepted into the repository (on a new repository, from about a minute after it is created). Push protection is always on: no organisation or repository can turn the scan off. The one setting is the scanner failure mode under Security › Policies › Push protection: when the scanner itself is unavailable, a push is either rejected (the default) or let through and recorded as an allow-on-error event in the audit trail; secrets that get through in that window are caught later by the weekly history sweep. See Push protection.

What gets blocked

Gitleaks matches 120+ high-confidence patterns including:

CategoryExample patterns
Cloud provider credentialsAWS access/secret keys, GCP service account JSON, Azure storage connection strings
Source control tokensGitHub PATs, GitLab PATs, Bitbucket app passwords
Payment keysStripe secret keys, Stripe webhook signing secrets
Communication tokensSlack bot tokens, Slack webhook URLs, Twilio auth tokens
Private keysRSA/EC/Ed25519 private keys, PEM-encoded certificates
Generic secretsHigh-entropy strings matching the generic-api-key pattern, JWT signing secrets
Database URLsPostgreSQL, MySQL, MongoDB, Redis connection strings with embedded credentials

When a push is blocked

The push is rejected at the remote with an error message that includes the matched pattern type and the file + line number. The commit is not accepted.

Do not just delete the file and re-commit. The secret is in git history and will be blocked again. The correct remediation:

  1. Rotate the secret immediately at the issuing service (AWS IAM, Stripe dashboard, etc.). Assume it is compromised.
  2. Rewrite history to remove the secret from every affected commit:
# Install git-filter-repo (https://github.com/newren/git-filter-repo)
pip install git-filter-repo

# Create a replacements file
echo '<secret-value>==>REDACTED' > replacements.txt

# Rewrite history
git filter-repo --replace-text replacements.txt

# Force-push the cleaned branch
git push --force-with-lease origin <branch>
  1. Notify anyone who may have cloned the repo before the rewrite.

Override flow

If a blocked secret is intentional (internal tooling token, published test credential), an org owner can add a scoped override:

  1. Go to Security › Policies › Push protection › Active overrides → New override.
  2. Specify: Repository (or select “All repos”), secret pattern (from the Gitleaks rule ID), and justification (free text, logged in the audit trail).
  3. Save. The override takes effect on the next push.

All overrides are visible to other org owners and appear in the audit log with actor, timestamp, and justification. Overrides can be revoked at any time from the same page.

Overriding a push rejection

When a push is rejected by the secret scanning gate, the developer sees the rejection reason in the terminal output, the matched pattern type, the file path, and the line number.

An org owner can approve an override at Security › Policies › Push protection › Recent rejections. Each rejection entry shows: the repository, the committer, the timestamp, the secret type detected, and the affected file path.

Override scopes (select when approving):

  • This commit only, approves the specific commit SHA. The same secret in a future commit will be rejected again.
  • This file path, approves the secret at that path in any future commit. Use for test fixtures or deliberately non-sensitive placeholder values.
  • This repository, approves any occurrence of this secret type in the repository. Use sparingly.
  • Org-wide, approves the secret type across all repositories. Only for declared false positives (e.g., a custom token format that matches a rule pattern but is not a real secret).

Override reason is required (free text). It is written to the audit log.

The developer must re-push after an override is granted. The override does not retroactively accept the rejected push.

Custom allowlist

To silence known false positives in a specific repository (test fixtures, example keys in documentation, intentional public credentials), add a .gitleaks.toml at the repo root:

[extend]
useDefault = true

[[allowlists]]
description = "Test fixture AWS keys"
paths = ["tests/fixtures/.*", "docs/examples/.*"]

[[allowlists]]
description = "Example key in README"
regexes = ["AKIAIOSFODNN7EXAMPLE"]

The allowlist is evaluated per-repo during the pre-receive hook. It does not override org-level push protection for patterns not in the allowlist.

Pull request checks: only new findings block

The dependency (fremforge/dep-scan), SAST (fremforge/sast), licence (fremforge/license-scan), secret (fremforge/secret-scan) and image (fremforge/image-scan) checks scan the whole repository, but each fails a pull request only on findings the pull request introduces. A finding the branch it merges into already has does not block, so a pull request that changes nothing vulnerable can merge in a repository that has open findings. The status then reads, for example, No new findings at or above critical; 8 already on main.

fremforge compares the pull request’s findings with the latest scan of that branch by the same scanner. A finding is the same when:

CheckSame finding when
Dependenciesthe ecosystem, package and advisory match. Bumping a package to a version that still has the same advisory is not new; a new advisory is
SASTthe rule, the file and the matched code match. A finding that only moved to another line is not new
Licencesthe ecosystem, package and licence match
Secretsthe rule and the matched lines match, compared by a keyed hash computed by the scanner. The secret itself is never stored or sent. A secret that only moved to another line is not new
Imagesthe advisory, package and file match
  • Copies count: a second occurrence of a finding the branch has once is new.
  • Everything that already decides whether a finding blocks still applies first: the block severity, dismissals, exceptions and the licence policy. A new malicious package or known-exploited vulnerability always blocks, whether or not your organisation chose that unreachable findings do not block.
  • Existing findings are not hidden. They are listed in the pull request comment and under Security › Findings, count towards your SLA, and alert as before.

The branch is scanned after a push to your default branch, at most once an hour, with the newest push winning. A finding that reached the branch since its last scan therefore still counts as the pull request’s until the next scan; a new finding is never mistaken for an existing one. When there is no scan of the branch yet, every finding blocks, as before, and the pull request’s scan asks for one. Once the branch’s first scan is recorded, fremforge scans again, by itself, the open pull requests into it whose check failed meanwhile (at most 20 per repository, the most recently updated, per check), so they turn green, or stay red only on what they add, without a re-run or a new push. A pull request into a branch other than your default branch always counts every finding.

Dependency scanning

osv-scanner checks every pull request, and the default branch after each push, against the OSV advisory database. It reads lockfiles and manifests offline, without resolving dependencies, and posts the fremforge/dep-scan commit status and a pull request comment. Findings are listed under Security › Findings › Dependencies.

Settings and what blocks a merge

Under Security › Findings › Dependencies an owner can turn dependency scanning off for the organisation and set the block severity: the lowest severity that fails fremforge/dep-scan. The default is critical. On a pull request, only findings the pull request introduces block. Whether a failing check actually blocks the merge depends on the check’s mode (enforce, audit or off) under Security › Prioritise › Pull request checks, and on the check being required by branch protection.

To keep a finding from blocking, fix it or record an exception. An organisation can also choose that findings which are not reachable do not block. That choice never covers malicious packages or known-exploited vulnerabilities.

Fix pull requests

Two things open fix pull requests for vulnerable dependencies:

  • Security fixes: Open fix PR on a finding (or the automatic policy) opens one pull request per package at the lowest version that fixes every open advisory. See Security fixes.
  • Dependency updates: Renovate’s own vulnerability alerts, for organisations enrolled in Dependency updates. Each enrolled organisation’s repositories are run every night, and Run now starts a run out of cycle.

Supported manifests

The lockfiles and manifests read, and which ecosystems get fix pull requests and reachability, are listed once, in Supported ecosystems.

Container image scanning

Trivy scans every OCI image pushed to the fremforge package registry. Results appear at Security › Findings › Container images.

Trivy runs centrally, with nothing to install: trivy fs on every pull request (the fremforge/image-scan status, which blocks only on findings the pull request introduces), trivy config as its own check (fremforge/iac-scan), and trivy image on every image in your registry and every image pulled through the dependency proxy, and rescanned regularly. Misconfiguration findings from trivy config have their own tab, Security › Findings › IaC; see IaC scanning.

Each finding includes:

FieldDescription
CVE IDe.g. CVE-2024-12345 with link to NVD
SeverityCRITICAL / HIGH / MEDIUM / LOW / UNKNOWN
Affected packagePackage name and installed version
Fix versionFirst version that resolves the CVE, if available
EPSS scoreExploit Prediction Scoring System probability

Scanning in your own workflow (optional)

There is nothing to enable: the scans above run without a workflow file. If you also want your own build to fail before it pushes an image, you can run Trivy in a workflow yourself, for example:

name: Image scan
on:
  push:
    branches: [main]
  pull_request:

jobs:
  trivy:
    runs-on: fremforge
    steps:
      - uses: actions/checkout@<full-40-char-sha>   # pin by SHA, never a tag or branch
      - name: Build image
        run: docker build -t ${{ github.repository }}:${{ github.sha }} .
      - name: Scan image
        run: |
          docker run --rm \
            -v /var/run/docker.sock:/var/run/docker.sock \
            aquasec/trivy:0.74.0@sha256:<digest> \
            image --exit-code 1 --severity HIGH,CRITICAL \
            ${{ github.repository }}:${{ github.sha }}

Set --severity to the threshold you want the build to fail at.

Why this example pins by SHA and by digest

An earlier version of this page used aquasecurity/trivy-action@master and actions/checkout@v4. Both are mutable references, and on 19 March 2026 that distinction stopped being theoretical: an attacker rewrote the trivy-action v0.69.4 tag to point at a malicious release. Pipelines referencing the action by tag or branch pulled the attacker’s code on their next run, and the credentials it harvested were used to publish two backdoored releases of another project to PyPI.

A tag is a pointer somebody else can move. A commit SHA and an image digest are not.

  • Actions — reference the full 40-character commit SHA, not @v4 and never @master. Renovate still proposes upgrades; it rewrites the SHA and its comment together.
  • Scanners — prefer the tool’s container image pinned by digest over a wrapper action. Fewer moving parts, and the thing you pinned is the thing that runs.
  • Mirror what you depend on — fremforge reaches no upstream registry from a production path; every image comes from our own registry, pinned by digest.

That incident required no action on the fremforge side: we have never used trivy-action, and our own Trivy is a digest-pinned image. The project that was compromised had been installing Trivy from apt with no pinned version. Pinning is unglamorous, and it is the control that would have stopped it.

SBOM generation

Syft generates CycloneDX 1.6 SBOMs for every container image pushed to the registry and for every release tag, with an SPDX 2.3 twin from the same run (best effort; see SBOMs). Uploads are accepted as CycloneDX 1.5–1.7 or SPDX 2.x/3.x. SBOM generation is on by default for every org, there is no enable step.

Download

From the package registry UI: open the image → Attestations tab → Download SBOM.

Via API:

curl -H "Authorization: token <your-pat>" \
  https://frem.sh/_app/api/v1/orgs/<org>/findings/attestations \
  -o sbom.json

The SBOM is CycloneDX JSON and includes all transitive dependencies with their PURL identifiers, license expressions, and version ranges.

SAST

OpenGrep runs static analysis centrally on every pull request and on the default branch. It is on by default for every organisation. Findings post a fremforge/sast status and a pull request comment, and are listed under Security › Findings › SAST. Languages, rules and settings are on the SAST page. Custom rules are not supported.

Signed commits

fremforge supports two commit-signing paths, both verified natively by Forgejo’s “Verified” badge in the web UI:

  • SSH-key signing (recommended default), reuses the SSH key you already have registered for git push. No new key material to manage, no third-party sub-processor, no transparency-log dependency, no US-jurisdiction transit. This is the canonical fremforge path. (For new orgs created after 2026-05-22, SSH transport is disabled by default; SSH-key signing still works regardless of whether you use SSH or HTTPS for git push.)
  • GPG signing, classical OpenPGP key. Use this if your organisation already has a GPG key-management policy in place (smartcard, HSM, central keyserver).

Keyless commit signing IS available at fremforge via a self-hosted Sigstore stack: Fulcio CA at sign.frem.sh and a TSA at tsa.frem.sh, both running on T Cloud Public in eu-de with the root CA in DEW KMS. The public Linux Foundation Sigstore instance is NOT used — fremforge does not route any signing or verification traffic through sigstore.dev / rekor.sigstore.dev, so the no-US-sub-processor posture holds. There is deliberately no Rekor / transparency log for commit signing; TSA-anchored timestamping is the integrity primitive instead. (fremforge does run a transparency log, at log.frem.sh, but it covers build-provenance attestations only and is not Rekor — see transparency log. The two are independent: a signed commit is not entered into it.) See keyless commit signing for the customer-side gitsign setup. (Note: build/release provenance is a separate path — fremforge SLSA provenance is signed with a platform-held Ed25519 key, NOT through Fulcio. See the SLSA section below.)

SSH-key signing, configure

# Tell Git to use SSH for signing (Git ≥ 2.34)
git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/id_ed25519.pub
git config --global commit.gpgsign true
git config --global tag.gpgsign true

# (Optional) point Git at your local allowedSignersFile for verify
git config --global gpg.ssh.allowedSignersFile ~/.ssh/allowed_signers

Add the matching SSH key to your fremforge user profile under Settings → SSH and GPG keys → Add signing key (Forgejo accepts the same key for both push and signing, or a separate signing-only key).

On the next git commit, Git signs with your SSH key. Forgejo verifies the signature against the org’s allowedSignersFile (auto-derived from members’ registered signing keys) and renders the “Verified” badge in the web UI.

Verify

git log --show-signature HEAD~5..HEAD

Each commit shows a Good "git" signature for <user>@<org> line. The Forgejo web UI shows a green “Verified” badge next to the commit hash.

GPG signing, configure

If you prefer GPG, the standard git config --global commit.gpgsign true + user.signingkey <gpg-key-id> flow works unchanged; upload your public key under Settings → SSH and GPG keys → Add GPG key. Forgejo verifies and renders the “Verified” badge.

Require signed commits (org policy)

Enforce signed commits on protected branches with Commit signing under Security › Policies › Repository rules (see Branch protection), or per-repo at Repository → Settings → Branches → <branch> → Require signed commits. Unsigned commits are rejected at push time. Both SSH-key and GPG signatures satisfy the requirement.

SLSA provenance

Build/release provenance is generated server-side by fremforge — there is no workflow file in your repos and no runner minutes are spent. On every Forgejo release.published and package event, the fremforge api hashes each published asset, builds an in-toto Statement wrapped in a DSSE envelope, signs it with the platform builder key, persists the attestation to EU-resident storage scoped to your tenant, and uploads the envelope back alongside the asset as <asset>.intoto.jsonl. It is also listed in the console under Security › Components › Build attestations.

The standalone slsa-provenance.yaml workflow template that customers were previously expected to call from their build pipelines was retired on 2026-06-16 — a fleet audit found zero callers, and provenance now fires automatically server-side with no customer wiring.

EU-sovereign by design. The signing keys, the trust root, the certificate authority and the storage all live inside the fremforge T Cloud Public account (eu-de). There is no public Sigstore, no Rekor, and no tuf-repo-cdn.sigstore.dev lookup anywhere in the verification path.

Two signatures, both on every attestation. The primary is a platform-held Ed25519 key, and it is the one the Verify recipe below checks — plain curl / jq / openssl / python3, nothing else needed. Additionally, each attestation is signed keyless against fremforge’s self-hosted Sigstore: a short-lived Fulcio certificate whose SAN is the build source identity, timestamped by the fremforge RFC3161 TSA. Both are described in the trust root, which carries fulcio_root_ca_pem, tsa_cert_chain_pem, certificate_oidc_issuer and certificate_identity_pattern so the keyless signature is verifiable offline too.

There is still no transparency log — the trust root says "transparency_log": "none" — so the keyless signature is validated against the RFC3161 timestamp rather than Rekor (cosign --use-signed-timestamps --insecure-ignore-tlog). What that does and does not buy you is set out in §Independent verifiability.

(fremforge’s commit signing is a separate path on the same self-hosted CA — see keyless commit signing.)

Trust root: https://www.frem.sh/.well-known/slsa-trust-root.json, a small JSON file listing the builder public keys. Its shape is fremforge-specific:

{
  "trusted_keys": [
    {
      "kid": "fremforge-slsa-builder-v1",
      "alg": "ed25519",
      "pem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----",
      "valid_from": "2026-05-17T00:00:00Z",
      "valid_to": "2026-09-12T00:00:00Z",
      "builder_id": "https://frem.sh/runner-controller/v1"
    },
    {
      "kid": "fremforge-slsa-builder-v2",
      "alg": "ed25519",
      "pem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----",
      "valid_from": "2026-09-12T00:00:00Z",
      "builder_id": "https://frem.sh/runner-controller/v1"
    }
  ]
}

fremforge-slsa-builder-v2 is the active signing key since the rollover of 2026-09-12. fremforge-slsa-builder-v1 signed everything before that and is kept for verification only: its valid_to is the instant it stopped signing, not an expiry for the signatures it made. Always verify against the key named by the keyid in the envelope you hold, never a fixed name.

Pin the file (or a copy of it) in your CI to lock-in trust over the verification window. Key rotations are announced 30 days in advance per the trust page §Sub-processors notification rule; a signature stops verifying only if its key is removed from trusted_keys.

Envelope + statement shape (DSSE envelope wrapping an in-toto Statement):

FieldValue
envelope payloadTypeapplication/vnd.in-toto+json
envelope signatures[0].keyidfremforge-slsa-builder-v2 for anything signed since 2026-09-12, fremforge-slsa-builder-v1 before (matches a trusted_keys[].kid in the trust root)
Statement _typehttps://in-toto.io/Statement/v1
Statement predicateTypehttps://slsa.dev/provenance/v1
predicate.buildDefinition.buildTypeWhich path produced the statement — see §What each buildType means. One of …/buildtypes/forgejo-action-run/v1, …/container-image/v1, …/forgejo-release/v1, …/forgejo-package/v1
predicate.buildDefinition.internalParameters.run_idThe Actions run that produced the artifact. Present on build provenance; absent on a publication statement
predicate.buildDefinition.internalParameters.attestation_kindpublication when the statement observes published bytes rather than a build. Absent on build provenance
predicate.buildDefinition.externalParameters.repository<org>/<repo>
predicate.buildDefinition.externalParameters.refThe git ref (e.g. refs/tags/v1.2.3)
predicate.runDetails.builder.idhttps://frem.sh/runner-controller/v1 (server-controlled and force-overridden at sign time — a compromised runner cannot forge this)
subject[].name / subject[].digest.sha256The asset filename and the SHA-256 fremforge computed for it

What this is, precisely. Until 2026-09-11 this section claimed SLSA Build L2. We re-derived it against the v1.0 spec, the claim did not hold, and we withdrew it. The gap that made it untrue — provenance that described no build — was closed on 2026-09-12. We are still not quoting a level, for a reason given at the end of this section.

Two different statements are now signed with the same key, and you can tell them apart from the envelope alone.

What each buildType means

buildType ends withWhat it assertsNames a build run?
forgejo-action-run/v1fremforge’s control plane read this run’s uploaded artifacts from the build platform’s own records, fetched the bytes, and hashed themYes — run_id
container-image/v1An image pushed by a run, attested with the run identity taken from a short-lived OIDC token fremforge minted for that jobYes — run_id, cryptographically bound
forgejo-release/v1A published release asset. Inherits a build’s commit when the identical digest was already attested as built — see belowOnly when joined
forgejo-package/v1A package version that appeared in the registryNo

Build provenance (the first two) describes how the artifact was produced: it names the run, the workflow and the commit, and the run identity comes from the platform rather than from anything the build asserted about itself.

Publication attestation (the last two, unjoined) says something narrower and says it explicitly: attestation_kind: publication. It means fremforge observed these byte-exact contents published here, and signs that observation with a key the build cannot reach. It is a real, checkable property. It is not a statement about how the artifact was built, and it no longer pretends to be: where no source commit is known, commit_sha is the literal no-source-commit rather than a value shaped like a git commit. If you see 40 hex characters there, it is a real commit.

The digest join. A release asset whose sha256 already has a build attestation inherits that build’s commit, and the statement points at it (source_commit_from: digest-join, plus the build attestation’s id and object key). The link is the artifact digest, so “same bytes” is a cryptographic fact rather than a naming convention — publishing an artifact later does not change what produced it. An explicit target_commitish on the release always wins over the inference.

The platform properties behind all of this:

  • Runs cannot influence one another. One ephemeral ECS VM per job, destroyed when the job ends, with a per-VM ENI and a per-runner credential that dies with the job — see §Kernel isolation model.
  • The signing key is out of reach of the build. It lives in the api, never on a runner, and the builder identity is overridden server-side at signing time, so a compromised build cannot forge it.
  • No customer wiring for the webhook-driven paths. Nothing to call from a workflow, and no runner minutes spent.

Why we still do not quote a SLSA level. Those two isolation properties are what SLSA v1.0 Build L3 asks for, and build provenance now describes an actual build. We would rather you read the table above and decide what it is worth than accept a number from the party being audited. (Hermetic builds, reproducible artifacts and transparency logs are sometimes quoted as L3 requirements. They are not: those were SLSA v0.1 L4 concepts and appear at no level in v1.0, which lists them under future directions.)

What is still missing is a transparency log, which is a different axis from any level — see §Independent verifiability.

Verify

Do not use upstream slsa-verifier verify-artifact --trusted-root. That tool has no --trusted-root flag on verify-artifact and cannot consume fremforge’s custom trust-root JSON or a bare-Ed25519 DSSE envelope — it expects a Sigstore-style bundle. fremforge’s provenance is a plain Ed25519-signed DSSE envelope, so you verify it directly with curl / jq / openssl / python3 — all distribution-default, no fremforge tooling required.

The verification has three independent checks: the DSSE signature is valid for the published builder key, the asset’s SHA-256 matches what was signed, and the statement names the right repo/builder.

1. Fetch the trust root and extract the builder public key

curl -sSfL https://www.frem.sh/.well-known/slsa-trust-root.json -o trust-root.json
jq -r '.trusted_keys[] | select(.kid=="fremforge-slsa-builder-v2") | .pem' \
  trust-root.json > builder.pub

For an attestation signed before 2026-09-12, select fremforge-slsa-builder-v1 instead. jq -r '.signatures[0].keyid' <asset>.intoto.jsonl tells you which one.

2. Verify the DSSE signature, the asset digest, and the statement fields

Run from a directory containing the downloaded asset and its <asset>.intoto.jsonl envelope. Set ASSET to the asset filename:

ASSET=myapp python3 - <<'PY'
import base64, json, hashlib, os, subprocess, sys, pathlib

asset = os.environ["ASSET"]
env = json.loads(pathlib.Path(f"{asset}.intoto.jsonl").read_text().strip())

payload_type = env["payloadType"]                 # application/vnd.in-toto+json
payload      = base64.b64decode(env["payload"])

# Reconstruct the DSSE pre-authentication encoding (DSSEv1):
#   "DSSEv1 " + LEN(type) + " " + type + " " + LEN(payload) + " " + payload
# where LEN is the utf-8 byte length and separators are single ASCII spaces.
pae = (
    b"DSSEv1 "
    + str(len(payload_type.encode())).encode() + b" "
    + payload_type.encode() + b" "
    + str(len(payload)).encode() + b" "
    + payload
)
pathlib.Path("pae.bin").write_bytes(pae)
pathlib.Path("sig.bin").write_bytes(base64.b64decode(env["signatures"][0]["sig"]))

# Ed25519 ("pure" EdDSA — no pre-hash) verify with openssl. Feed the raw PAE.
r = subprocess.run(
    ["openssl", "pkeyutl", "-verify", "-pubin", "-inkey", "builder.pub",
     "-rawin", "-in", "pae.bin", "-sigfile", "sig.bin"],
    capture_output=True, text=True,
)
if r.returncode != 0 or "Signature Verified Successfully" not in r.stdout:
    print("DSSE signature FAILED:", r.stdout, r.stderr, file=sys.stderr); sys.exit(1)
print("DSSE signature: OK")

stmt = json.loads(payload)

# The signed subject digest must match the asset bytes on disk.
want = next((s["digest"]["sha256"] for s in stmt["subject"] if s["name"] == asset), None)
got  = hashlib.sha256(pathlib.Path(asset).read_bytes()).hexdigest()
if want is None:
    print(f"asset {asset} not found among signed subjects", file=sys.stderr); sys.exit(1)
if want != got:
    print(f"digest mismatch: signed={want} actual={got}", file=sys.stderr); sys.exit(1)
print(f"asset digest: OK ({got})")

# Sanity-check the builder identity and predicate type.
builder = stmt["predicate"]["runDetails"]["builder"]["id"]
if builder != "https://frem.sh/runner-controller/v1":
    print(f"unexpected builder.id: {builder}", file=sys.stderr); sys.exit(1)
if stmt["predicateType"] != "https://slsa.dev/provenance/v1":
    print(f"unexpected predicateType: {stmt['predicateType']}", file=sys.stderr); sys.exit(1)
print("builder.id + predicateType: OK")
PY

If all three checks print OK, the asset is the exact bytes fremforge attested, the envelope was signed by the published builder key, and the statement names the fremforge builder. Verification fails (non-zero exit) if the signature doesn’t validate against builder.pub (tampered envelope or wrong/rotated key), the asset’s SHA-256 doesn’t match the signed subject (tampered asset), or the builder.id / predicateType don’t match.

The keyid in the envelope (fremforge-slsa-builder-v2 for anything signed since 2026-09-12) must match the kid you selected from the trust root; an envelope checked against the other key fails with DSSE signature FAILED. If fremforge rotates the signing key, re-fetch the trust root and select the kid the envelope names.

Why not Sigstore at launch

Sigstore’s public Fulcio + Rekor + Trillian stack is operated by the OpenSSF (Linux Foundation), a US-incorporated body. Using it would send every build’s metadata to US-hosted infrastructure, incompatible with fremforge’s EU-only sub-processor commitment in DPA Annex B. fremforge runs its own self-hosted Fulcio + TSA at sign.frem.sh / tsa.frem.sh on T Cloud Public eu-de, used for both commit signing (see keyless commit signing) and, since 2026-06-21, as the second signature on build provenance. What is deliberately absent is the public Sigstore deployment and Rekor.

Every attestation’s SHA-256 is also hash-chained into the platform’s audit log, whose anchors sit in a 3-year COMPLIANCE-mode OBS Object Lock. That is a useful internal control and it is not a substitute for a transparency log, so we no longer describe it as equivalent tamper-evidence. Two reasons, both concrete: the anchor is not readable by you, so it cannot be part of your verification; and OBS Object Lock protects each object version — measured 2026-08-14, a newer version can shadow a locked key and a delete marker hides it from ordinary reads, both of which are actions available to the operator. See §Independent verifiability for what the trust model does and does not give you.

Immutable releases

Org owners can make every release frozen at publish: its tag, target commit and assets can never change again (title and notes stay editable), a deleted release’s tag name can never be reused, and container tags can be frozen too (v1.2.3-style tags, or all but a list). Each immutable release gets one signed in-toto release statement naming every asset by SHA-256 and the tag by commit, checkable with fremforge release verify. See Immutable releases and release attestation.

Independent verifiability

Relocating a repo, a runner and an artifact store to Germany answers where the data sits. It does not by itself answer whether the audit trail can be trusted without trusting the operator who produced it. Those are different properties and we would rather be explicit about which one we deliver today.

What you get without trusting us at verification time. The attestation travels with the artifact — <asset>.intoto.jsonl, and alongside it <asset>.sigstore.json — so you hold both halves. The builder public key, the Fulcio root CA and the TSA chain are all in a static trust root you are encouraged to pin a copy of in your own CI. Verification recomputes the artifact digest and checks signatures locally, with curl / jq / openssl / python3. No fremforge API is contacted, and no fremforge tooling is required. Once you have pinned the root, a later change on our side cannot retroactively alter what your pipeline accepts.

What you do not get, stated plainly. There is now an append-only transparency log for provenance attestations at log.frem.sh, but it does not yet give you the property people usually mean by the word. Concretely:

  • The log makes us unable to rewrite history quietly: if you hold an inclusion proof and we later alter or drop your attestation, your proof stops verifying and you can demonstrate the contradiction. That is real and it is new.
  • It is partly split-view resistant, as of 16 September 2026. One independent witness — witness.stagemole.eu, operated by Mullvad VPN AB — co-signs every checkpoint, so there is now a third party who would notice us signing something unusual. That is the property this bullet has always been about, and it is no longer absent. One witness is not a quorum: it raises the cost of showing you a different tree from everybody else, but cannot detect collusion between the log and its only witness, and its operator calls the instance staging. Note also that the only production-grade EU witnesses — Glasklar Teknik, Mullvad VPN and Tillitis — share a parent company, Amagicom AB; more of them would buy availability, not independence.
  • The RFC3161 timestamp that partly substitutes for a log is issued by our TSA, and the trust root is distributed from our domain. Pinning addresses the second for anyone who pins at onboarding; it does not address first-contact trust.
  • Our internal audit-chain anchor is not readable by you, so it cannot form part of your verification.

If the operator is the party you need to hold accountable — which is the right question for defence and critical infrastructure — then what closes the gap is a transparency log with external witnesses, an independent timestamp authority, and reproducible builds you can rebuild and compare without our involvement. The first of those three now exists and is described above, with its limits; the timestamp authority and reproducible builds do not. Note that a self-hosted log would not close it: a log we operate is a log we control. Independent verifiability requires an outside party, and the honest trade is that doing it via the public Sigstore log would publish artifact digests, repo identities and release timing outside the EU.

We would rather scope that against a real requirement than guess at one. If this is a procurement constraint for you, tell us which of those properties you need and on what timeline, and we will come back with a dated plan rather than a claim.

Kernel isolation model

Each job gets its own ephemeral ECS virtual machine on T Cloud Public in eu-de, and it is destroyed once that job finishes. VMs are single-task: the runner is deleted after its one task and its per-runner credential dies with the job, so a VM cannot serve a second job even within the same workflow run. (Before 2026-07-30 successive jobs over one commit could share a VM; they no longer can.) The isolation properties:

PropertyDetail
Kernel sharingNo kernel is shared between commits or between orgs — each commit gets a dedicated VM. Jobs of the same commit may share that VM, so treat a pipeline as one trust domain
PersistenceThe VM is destroyed once the commit’s jobs complete; no filesystem state survives to another commit or another org. State can carry between jobs of the same commit — a build step running third-party dependency code shares the machine with a later deploy step, so scope any credential a deploy needs to that pipeline
Platform-API accessThe VM has no IAM agency attached; the metadata-derived identity has no roles, so the OTC API is unreachable from inside
NetworkPer-VM Security Group with egress-only allowlist; runners cannot receive inbound connections
T Cloud Public metadataBlocked by SSRF outbound proxy; runner code cannot reach the instance metadata endpoint with privileged scope

See CI runners for the full isolation specification and BYO runner registration.

Tenant isolation

Every fremforge tenant is a separately-keyed namespace in the platform. The isolation chain runs from the URL through middleware, the database query layer, the storage layer, and the audit chain.

LayerMechanismWhat stops cross-tenant access
URL routing/<slug>/_admin/* middleware re-loads the tenant from the slug on every requestA user with a session for org A who pastes org B’s URL is rejected at the membership check, not at a stale cookie
Membership checkForgejo Owners-team lookup against the slug, verified on every requestThe api never trusts an inbound claim about which orgs the user belongs to
DatabaseEvery tenant-keyed table has a tenant_id foreign key; every read AND every write filters on itSELECT / UPDATE / DELETE without tenant_id is structurally impossible in the audit-flagged paths
Signed tokensEvery short-lived signed URL (billing magic link, undo cancellation, OIDC state) carries the tenant id in its payload; the verify-side cross-checks against the URL slugA token minted for tenant A is rejected if replayed against tenant B’s URL
Storage (OBS)SBOMs, attestations, audit-log objects all keyed <artifact>/<tenant_id>/...; signed-URL minting re-validates tenant.id === row.tenant_id before issueA direct OBS-presigned URL bound to tenant A cannot be rewritten to read tenant B
Audit chainTamper-evident hash chain keyed per-tenant; one tenant’s appends never modify another’s chain headA break in tenant A’s chain (e.g. an attempted retroactive delete) does not affect tenant B’s chain integrity
Operator accessStaff (fremverk) viewing customer data is logged to the customer’s audit chain as an operator action; tenant admins can see operator visits in their own audit logNo silent staff access to customer data

Owners-team revocation lag

The api caches a customer’s Forgejo Owners-team membership for up to 5 minutes to keep page-loads under 50ms (the alternative is a Forgejo API round-trip on every request, which would be 100-200ms slower per page). The trade-off:

  • When you add someone to Owners on the Forgejo side, they gain admin access within 5 minutes.
  • When you remove someone from Owners on the Forgejo side, they lose admin access within 5 minutes, not instantly.

For most operations this lag is acceptable: a former owner cannot create new Forgejo resources (Forgejo’s own check is immediate), and any privileged action via the fremforge admin UI is recorded in the per-tenant audit log with the actor’s username, so post-revocation activity remains attributable.

If you need immediate revocation (e.g. as part of an incident response after a credential compromise), the operator on-call can pin the cache to zero TTL for your tenant on request, open a ticket at support@frem.sh with subject prefix [urgent-revocation] and the username + tenant slug. Standard SLA: under 15 minutes during business hours, under 60 minutes outside. The same effect is achieved by revoking the user’s Forgejo session (Forgejo Owner → User profile → Sign out), which invalidates the session cookie the api parses, no fremforge-side action needed.

Platform security commitments

CVE severityPatch SLA (from upstream fixed release)
Critical (CVSS ≥ 9.0)48 hours
High (7.0-8.9)72 hours
Medium (4.0-6.9)7 days
LowNext scheduled maintenance window

The patch SLA is published contractually and cited verbatim in the DPA security annex.

Audit-log integrity: tamper-evident hash chain anchored to T Cloud Public OBS WORM storage every 2 minutes; hourly FunctionGraph integrity check; chain breaks page on-call.

Troubleshooting

Push blocked but I cannot find the secret in the diff.

The secret may be in a file that was modified but whose full content is not visible in the standard diff. Run Gitleaks locally to identify the exact location:

pip install gitleaks   # or brew install gitleaks
gitleaks detect --source . --verbose

No fix pull request for a known CVE.

Check that the package’s ecosystem and manifest get fix pull requests (Supported ecosystems) and that the advisory has a fixed version. Renovate runs every night for an enrolled organisation; Security › Fix › Dependency updates → Run now starts one out of cycle, and Open fix PR on the finding asks for a security fix at once. See When no pull request is opened.

SBOM download returns 404.

SBOMs are generated only for images pushed after SBOM generation was enabled. Images pushed before enabling do not have an SBOM. Push a new tag to generate one.

SSH-signing fails with ssh-keygen: gpg failed to sign the data.

Make sure git config gpg.format is set to ssh (not the default openpgp) and user.signingkey points at a public-key file. On macOS, also confirm ssh-keygen is on $PATH (Apple’s bundled ssh-keygen works; if you’ve installed Homebrew OpenSSH, ensure it shadows correctly).

Cross-references