Design international public participation into SAFE now — lessons from ASRS and the 1998 Internet governance process

[gab16]

I contribute here in a personal capacity. Earlier in my career, I flew commercially and submitted reports to NASA’s Aviation Safety Reporting System—the system this RFC appropriately takes as a model. I later spent much of my career in the United Nations system working on technology and development. Both experiences point to the same design lesson: legitimacy, representation and public reach should be built into a reporting system at the beginning, rather than added after the system and its accumulated knowledge have become established.

1. The ASRS analogy has a missing institutional element

ASRS works in significant part because reports are received, processed and de-identified by NASA, a public, non-regulatory and non-commercial third party—not by the FAA as regulator and not by the airlines whose operations are being reported.

NASA’s lack of enforcement authority and commercial interest makes confidentiality credible, while the FAA separately defines the limited enforcement protections available to qualifying reporters. The institutional separation is therefore not incidental to ASRS; it is a central part of why people trust the system enough to report.

The current SAFE draft correctly separates learning from enforcement. However, the closest international analogue to this public-interest role is absent. The Scope and Membership section includes governments and standards bodies as “non-controlling observers,” but does not identify multilateral or regional public-interest institutions as a participant class.

I propose adding to Scope and Membership:

Multilateral and regional public-interest institutions may participate in a non-controlling capacity, with standing to contribute to governance design, incident taxonomies and schemas, geographically inclusive pilots, capacity-building and dissemination of de-identified lessons.

Their contribution would be practical rather than ceremonial: broad international reach, public-interest mandates, access to countries and institutions outside the principal AI markets, and the ability to connect SAFE with emerging international work on interoperable incident reporting, shared taxonomies and a possible common machine-readable incident schema.

“Standing” should not mean governmental control, veto power or access to identifiable reports. It should mean an established right to participate in relevant working groups, propose agenda items, contribute evidence and help shape the mechanisms through which SAFE’s learning reaches the wider international community.

2. Make geographically inclusive participation a guiding principle

The draft understandably does not yet define pilots or early-access arrangements. When implementation begins, however, pilot participation should be geographically and institutionally diverse from the outset rather than determined principally by proximity to the founding organizations.

The system’s accumulated evidence, operational experience and trust will be among its most valuable assets. If access to the learning process begins unevenly, that advantage will compound from the first day. Representation should therefore be established as a design principle before implementation choices make it difficult to achieve.

I propose adding to Guiding Principles:

Inclusive participation: SAFE should seek geographically and institutionally diverse participation in pilots, working groups and early implementation activities, including organizations from regions currently underrepresented in AI security, safety and assurance initiatives.

This does not require national quotas or equal representation from every jurisdiction. It establishes a presumption that the system should not develop solely around the institutions, markets and risk perceptions of its founding network.

3. The Internet governance transition offers a relevant precedent

The evolution from the 1998 Internet Green Paper and White Paper through ICANN, the two phases of the World Summit on the Information Society, and the 2016 IANA stewardship transition illustrates how long the international legitimacy, accountability and stewardship of critical coordination infrastructure can take to mature.

The lesson is not that industry should wait for an intergovernmental process. Industry’s ability to move quickly is precisely one of SAFE’s strengths. The lesson is that speed and international legitimacy should be designed together.

The institutions created around Internet coordination became functional well before the international questions surrounding stewardship, accountability and legitimacy were fully resolved. Their formal globalization evolved over nearly two decades. Providing a defined avenue for international public-interest participation in SAFE would require only a paragraph now.

4. Define when withholding is justified—and safe for whom

The draft already anticipates a public safety report and, where appropriate, publication of reusable tests, machine-readable policies, detection rules, reference configurations and incident-response guidance.

However, publication is conditioned on whether it would expose “sensitive evidence” or create “additional risk.” That formulation raises a prior governance question: safe for whom, determined by whom, and according to what standard?

Safety should primarily mean protection against concrete harm to affected people, vulnerable systems, reporter confidentiality and legitimate investigations. It should not silently expand to include avoiding embarrassment, commercial disadvantage, reputational damage or regulatory attention merely because publication is inconvenient to a member.

AI development cannot be contained within a membership roster. It occurs wherever models, weights, tools and computing capacity are available. If the most useful outputs of the Disclosure Model and the catalog under From Lessons to Controls remain accessible principally to members, the open development community will be excluded from the learning loop precisely where substantial experimentation and deployment occur.

ASRS again offers the useful principle: individual reports and identifying information remain protected, while de-identified data and aggregated safety learning are made available to the broader aviation community.

I propose adding to the Disclosure Model:

De-identified systemic lessons, defensive recommendations, verification methods, tests, machine-readable policies, detection rules, reference configurations and catalog entries should be public by default.

Publication may be restricted only where disclosure of specific information presents a concrete, foreseeable and material risk to affected individuals, vulnerable systems, reporter confidentiality, legitimate legal rights or an active investigation. Reputational harm, commercial disadvantage, embarrassment or the possibility of regulatory scrutiny should not, by themselves, justify withholding.

Restrictions should use the least restrictive available measure, including redaction, aggregation or delayed publication rather than complete suppression. The decision and its evidentiary basis should be documented, subject to independent review, time-limited and reconsidered at defined intervals.

Where possible, SAFE should publish notice that material has been withheld, the general category of risk involved and the date of the next review.

The organization involved in an incident should be allowed to identify factual errors and concrete disclosure risks, but it should not be the sole judge of whether publication is safe. That determination should involve an independent disclosure function with appropriate public-interest and affected-community representation.

This would extend a principle the draft already recognizes: an affected organization may correct factual errors but should not have veto power over the resulting lessons or recommendations. The same principle should apply to decisions about whether those lessons reach the public.

Reports confidential; lessons public; exceptions narrow, reasoned and reviewable.

5. Reciprocity from international institutions

The corresponding responsibility does not rest only with SAFE. UN bodies and other international institutions should move beyond general language about the adoption of responsible AI and take a concrete step: formally request structured access to SAFE’s aggregated learning and offer useful contributions in return.

Those contributions could include schema and taxonomy alignment, international distribution, translation, institutional outreach, capacity-building and the connection of SAFE’s work with incident-reporting initiatives developing elsewhere.

I will make that argument to UN interlocutors as I make this one here. International participation should not be understood as a request for institutional control, but as an offer of reach, interoperability and reciprocal contribution.

The appropriate institutional response to a request for comments is comments.

Whoever builds this system will influence what counts as an incident, what evidence is preserved, which failures become visible and which remain outside the field of view. Getting the participation and disclosure architecture right at the beginning will be far less costly than retrofitting legitimacy, representation and access after SAFE has become established.

[safal207]

Strong direction overall. One implementation gap in the new disclosure rule may be worth tightening: it requires a restriction decision to have an evidentiary basis, independent review, a time limit, and defined reconsideration, but it does not specify a minimum record that would make those properties auditable.

Without a structured decision record, “time-limited” and “subject to independent review” can remain narrative properties, and a withholding exception can persist without a mechanically visible state transition.

Would you consider requiring a small machine-readable disclosure restriction record, for example:

• decision_id / finding or incident reference
• material or fields being restricted
• risk category and evidence references supporting the risk claim
• least-restrictive measure chosen (redaction | aggregation | delay | suppression)
• decision-maker and independent reviewer
• decided_at, review_at and/or expires_at
• current status and any superseded decision

A useful invariant could be roughly:

restriction_active -> evidence_basis_present && independent_review_recorded && review_or_expiry_time_present

and the lifecycle could be represented as explicit transitions such as:

restricted -> partially_published -> published

with reason, timestamp, and evidence attached to each transition.

That would make the proposal’s “least restrictive, time-limited and reviewable” requirement mechanically auditable, and it would fit well with SAFE’s existing machine-readable update direction. It also preserves the author’s policy intent without prescribing who the independent reviewer must be.