Secure Research Collaboration: Share AI Models Across Institutions Without Sharing Data

Secure research collaboration across institutions means sharing ML models — not raw data — between institutions. Pilot Protocol provides the transport: encrypted, authenticated tunnels between research labs, set up from each lab's side without a VPN project. Meeting HIPAA, GDPR, and institutional requirements remains each institution's work.

A medical researcher at University A has a model that detects early-stage lung cancer from CT scans. The model is good, but it was trained on 5,000 images from a single hospital. University B, across the country, has 8,000 additional scans that could significantly improve the model's accuracy. The researcher at University A cannot access University B's data. HIPAA prohibits it. The institutional review board at University B would not approve it. The IT security team at University A cannot whitelist another institution's network. And even if all of those barriers were overcome, the data transfer itself would need to be encrypted, logged, and auditable.

This is the research collaboration paradox: the most valuable scientific progress requires combining data across institutions, and the regulations designed to protect patients, subjects, and institutions make that combination extraordinarily difficult. The researchers know exactly how to improve the model. The legal, compliance, and technical barriers prevent them from doing it.

The result is that most cross-institutional ML collaboration does not happen. Projects that could benefit from larger, more diverse datasets remain constrained to whatever data a single institution can collect. The models are worse, the papers have smaller sample sizes, and the science moves slower than it should.

Current Approaches and Their Gaps

The research community has developed several mechanisms for cross-institutional collaboration. Each solves part of the problem while creating new ones.

Hugging Face Hub and Model Registries

Hugging Face Hub is the de facto standard for sharing pre-trained models. Researchers upload model weights, training configurations, and evaluation metrics. Other researchers download, fine-tune, and build upon them. This works well for public models, but it fails for sensitive research. Clinical models trained on patient data cannot be uploaded to a public repository -- even if the model weights themselves do not contain identifiable information, the institutional data governance policy often prohibits it. The model registry approach assumes openness, and regulated research often cannot be open.

Delta Sharing and Data Clean Rooms

Databricks Delta Sharing provides a protocol for sharing datasets across organizations without copying data. Recipients access the data through a standardized API, and the data owner retains control. The problem is that the data still leaves the originating network in some form -- the recipient's query results contain derived data that may fall under the same regulatory constraints as the source. For HIPAA-covered data, even aggregate statistics can be considered protected health information if the cohort is small enough.

VPNs and Direct Network Connections

The brute-force approach: establish a VPN tunnel between the two institutions, transfer data or model files through the tunnel, and rely on the VPN encryption for security. This works technically, but it fails operationally. University IT departments have different VPN standards, different firewall policies, and different approval timelines, so a VPN between two institutions can take months and several security reviews to approve.

Federated Learning Frameworks

Federated learning (FL) addresses the core problem directly: train a model across multiple institutions without moving the data. Each institution trains on its local data, shares only model updates (gradients or weights), and a central aggregator combines the updates into an improved global model. The data never leaves its home institution.

FL frameworks like Flower, PySyft, and NVIDIA FLARE provide the ML machinery. What they do not provide is the network infrastructure. How do the institutions' training nodes find each other? How do they establish secure connections through institutional firewalls and NATs? How do they authenticate each other? FL frameworks assume that network connectivity is already solved. In practice, it is the hardest part of the deployment.

Pilot as a Transport Layer for Research Collaboration

Pilot Protocol does not do federated learning. It does not do differential privacy. It does not do secure aggregation. What it does is solve the connectivity problem that blocks every other approach: how do two machines at different institutions establish an encrypted, authenticated, NAT-traversing connection without involving either institution's IT department in a months-long VPN approval process?

Each lab runs a Pilot daemon. The protocol handles NAT traversal, encryption, and authentication. The researchers exchange model weights, gradient updates, or evaluation results through encrypted tunnels. The registry and beacon (Vulture Labs' public pair, or your own) see connection metadata and, when no direct path exists, relay ciphertext they cannot read. No VPN configuration required.

# Lab A (University Hospital, behind institutional NAT)
$ pilotctl init --hostname lab-a-trainer
$ pilotctl daemon start --email lab-a@university-a.edu
Daemon running (pid 4812)
  Address:  0:0000.0004.1A2B
  Hostname: lab-a-trainer
  ...

# Lab B (Research Institute, different network)
$ pilotctl init --hostname lab-b-trainer
$ pilotctl daemon start --email lab-b@university-b.edu --public
Daemon running (pid 2210)
  Address:  0:0000.0004.1A31
  Hostname: lab-b-trainer
  ...

# Establish trust (cryptographic handshake = collaboration agreement)
# Lab A initiates:
$ pilotctl handshake lab-b-trainer "Federated lung cancer detection study, IRB #2026-0142"
handshake request sent to node 268849

# Lab B reviews and approves:
$ pilotctl pending
Pending handshakes — 1

  node 268843 · 40s ago (2026-06-09 09:12)
  Federated lung cancer detection study, IRB #2026-0142
  accept: pilotctl approve 268843 · decline: pilotctl reject 268843 "reason"
$ pilotctl approve 268843
trust established with node 268843

The handshake justification is not a formality. It travels with the request, shows in pilotctl pending, and is written to the daemon log when the request arrives. The request itself is signed with Lab A's Ed25519 key; the justification text is attached to it, not part of the signed bytes, and the trust record kept after approval holds the node, its key and the approval time but not the text. When an auditor will ask "who authorized this data exchange and why?", keep your own record of each approval (the IRB number, the study description, the node IDs and the time) next to the daemon log.

Sharing Model Weights Over Encrypted Tunnels

The most common operation in cross-institutional ML collaboration is transferring model weight files. After a round of local training, Lab A sends its updated weights to Lab B (or to a central aggregator). Here is the complete workflow:

# Lab A: Train locally, then send updated weights
$ python train.py --data /local/ct-scans --output /models/round-3-weights.pt
Training complete. Model saved: /models/round-3-weights.pt (142MB)

# Send weights to Lab B via encrypted Pilot tunnel
$ pilotctl send-file lab-b-trainer /models/round-3-weights.pt
# prints a JSON summary once Lab B has acknowledged the file

# Lab B: Receive weights and aggregate
$ pilotctl received
# lists files in ~/.pilot/received/, newest first: round-3-weights.pt

# Lab B aggregates with its own local weights
$ python aggregate.py \
    --local /models/lab-b-round-3.pt \
    --remote ~/.pilot/received/round-3-weights.pt \
    --output /models/global-round-3.pt
Aggregation complete. Global model: /models/global-round-3.pt

The file transfer uses Pilot's data exchange port (1001), in chunks that are acknowledged and checked end to end with SHA-256. The entire payload is encrypted with AES-256-GCM. If the connection is relayed through the beacon (because both labs are behind institutional NATs), the beacon sees only encrypted bytes. At no point does the model file exist in cleartext on any third-party infrastructure.

For automated workflows, the transfer can be scripted into the training loop:

#!/usr/bin/env python3
"""Federated training round with Pilot transport."""
import os
import subprocess

PARTNER = "lab-b-trainer"
ROUNDS = 10

for round_num in range(1, ROUNDS + 1):
    print(f"\n--- Round {round_num}/{ROUNDS} ---")

    # Step 1: Train locally
    subprocess.run([
        "python", "train.py",
        "--data", "/local/ct-scans",
        "--weights", f"/models/global-round-{round_num - 1}.pt",
        "--output", f"/models/local-round-{round_num}.pt",
        "--epochs", "5"
    ], check=True)

    # Step 2: Send local weights to partner
    subprocess.run([
        "pilotctl", "send-file", PARTNER,
        f"/models/local-round-{round_num}.pt"
    ], check=True)

    # Step 3: Wait for partner's weights
    print("Waiting for partner weights...")
    subprocess.run([
        "pilotctl", "subscribe", PARTNER, "training/weights-ready",
        "--count", "1", "--timeout", "10m",
    ], check=True)

    # Step 4: Aggregate
    subprocess.run([
        "python", "aggregate.py",
        "--local", f"/models/local-round-{round_num}.pt",
        "--remote", os.path.expanduser(f"~/.pilot/received/local-round-{round_num}.pt"),
        "--output", f"/models/global-round-{round_num}.pt"
    ], check=True)

    # Step 5: Notify partner that aggregation is done
    subprocess.run([
        "pilotctl", "publish", PARTNER, "training/round-complete",
        "--data", f'{{"round": {round_num}, "status": "complete"}}'
    ], check=True)

print("Federated training complete.")

The event stream (port 1002) coordinates the training rounds. This lab waits for one training/weights-ready event from its partner and publishes training/round-complete when it has aggregated; the partner runs the mirror image of the loop. Topics match exactly. The data exchange (port 1001) handles the actual weight file transfer. The entire coordination and transport layer is provided by Pilot. The ML framework -- PyTorch, TensorFlow, JAX -- is the researcher's choice.

The Trust Model as Collaboration Agreement

In traditional research collaboration, the collaboration agreement is a legal document. A Data Use Agreement (DUA), a Memorandum of Understanding (MOU), or an IRB-approved protocol. These documents specify who can access what data, for what purpose, and under what conditions. They are essential, but they are not enforced by the technology. Once a VPN is established, the technical controls do not prevent someone from transferring data outside the scope of the agreement.

Pilot's trust model provides a technical analog to the legal agreement. The handshake is a cryptographic commitment: both parties explicitly agree to communicate, the request is signed by its sender and carries the stated purpose, and either party can revoke trust instantly.

• Handshake = explicit consent. Both institutions must actively approve the connection. Unlike a VPN where access is granted by IT policy, the research team at each institution directly controls who can communicate with their training node.
• Justification = purpose limitation. The justification ("Federated lung cancer detection study, IRB #2026-0142") states the purpose when trust is requested, and the daemon logs it. This does not prevent misuse, but together with your own approval record it documents what was agreed to.
• Revocation = instant termination. If a collaboration ends, an IRB revokes approval, or a compliance issue is discovered, either party runs pilotctl untrust and the connection is severed at once on that side. No waiting for VPN credentials to expire or IT tickets to be processed.

# Revoke collaboration access (either side can do this)
$ pilotctl untrust 268849
trust revoked for node 268849
  re-establish later: pilotctl handshake 268849

# The tunnel is torn down and Lab B is sent a best-effort revoke notice

No Readable Data on Third-Party Servers

A critical property for regulated research: no third party can read the research data. On the public backbone, Vulture Labs' registry and beacon handle address resolution and NAT traversal signaling, and when no direct path exists the beacon relays the encrypted packets without being able to read them. Model weights, gradients, evaluation results -- all of these are encrypted end-to-end between the lab nodes. If even the connection metadata must stay inside the consortium, run your own registry and beacon.

Separate research collaborations can use separate network policies. Membership in one collaboration does not grant connectivity to another; any cross-network access requires an explicit bridge or peer authorization. Directory-metadata exposure still depends on the registry and deployment configuration.

What Compliance Reviewers Will Ask About

Pilot Protocol is not a compliance product. It does not make your deployment HIPAA-compliant or GDPR-compliant, it holds no certification, and Vulture Labs does not sign business associate agreements. But it has technical properties that compliance teams evaluate when reviewing research infrastructure:

• End-to-end encryption. All data in transit is encrypted with X25519 key exchange and AES-256-GCM. Encryption is on by default; it can only be turned off with the daemon's --no-encrypt flag.
• Mutual authentication. Both parties verify identity using Ed25519 signatures before any data flows. The authentication is per-agent, not per-network. This provides stronger access control than IP-based allowlisting or VPN group membership.
• Trust events in the log. Handshake requests, approvals, and revocations are logged locally by the daemon with timestamps, and the justification text appears with each incoming request. Keep your own approval records alongside these logs.
• Data minimization. The registry stores only connection metadata. No research data, model weights, or training parameters are stored on shared infrastructure. Data travels end to end between the lab nodes, relayed as ciphertext when no direct path exists.
• Trust revocation. Access can be terminated instantly, without waiting for credential expiration or administrative processing.
• Who runs the coordination layer. On the public backbone, Vulture Labs runs the registry and beacon, so connection metadata and any relayed ciphertext pass through its servers. Run your own registry and beacon if no outside party may handle even that.

Combining With Federated Learning Frameworks

The most powerful use of Pilot in research is as the transport layer for a federated learning framework. Pilot handles the hard networking problems (NAT traversal, encryption, authentication). The FL framework handles the hard ML problems (aggregation, differential privacy, convergence).

Here is how the layers combine:

# Layer stack for cross-institutional federated learning:
#
# ┌────────────────────────────────────────────┐
# │  ML Framework (Flower / PySyft / FLARE)    │  Training, aggregation, DP
# ├────────────────────────────────────────────┤
# │  Application (train.py / aggregate.py)     │  Study-specific logic
# ├────────────────────────────────────────────┤
# │  Pilot Protocol                            │  Addressing, encryption, NAT
# ├────────────────────────────────────────────┤
# │  UDP / IP                                  │  Physical transport
# └────────────────────────────────────────────┘

Pilot replaces the network configuration layer in the FL framework. Instead of configuring IP addresses, ports, TLS certificates, and VPN tunnels, you configure Pilot hostnames. The FL clients connect to each other using virtual addresses that work across any network topology.

Scaling to Multiple Institutions

Research consortia often involve more than two institutions. A multi-site clinical trial might have 10 hospitals contributing data. A genomics consortium might have 20 labs. Each approved pair can establish a handshake; larger deployments should evaluate relationship fan-out, registry capacity, relay use, and operational policy for the intended topology.

# 5-institution research consortium
# Each institution runs a Pilot node

# Aggregator (hosted by consortium coordinator), public so sites can find it by hostname
$ pilotctl daemon start --hostname fl-aggregator --public

# Each institution's trainer connects and handshakes with the aggregator
$ pilotctl handshake fl-aggregator "Site 3 trainer, IRB #2026-0142, lung cancer FL study"

# Aggregator approves all verified sites
$ pilotctl pending
Pending handshakes — 5

  node 268801 · 3m ago (2026-06-09 09:12)
  Site 1 trainer, IRB #2026-0142, lung cancer FL study
  accept: pilotctl approve 268801 · decline: pilotctl reject 268801 "reason"
  ...

# Approve all verified participants (site-1 ... site-5)
$ pilotctl approve 268801
$ pilotctl approve 268802
$ pilotctl approve 268803
$ pilotctl approve 268804
$ pilotctl approve 268805

Each institution has a trust relationship with the aggregator only. The institutions do not have trust with each other. Site 3 cannot see Site 1's traffic. The aggregator receives weights from all five, aggregates them, and distributes the global model back. This star topology mirrors the standard federated learning architecture, but with cryptographic trust enforcement at each link.

Are Distributed Remote Research Labs Possible?

"Are distributed, remote research labs possible now?" This question surfaces frequently in research computing forums. The answer is yes, but with careful architectural choices.

A distributed research lab needs three things that traditional VPN-based infrastructure provides poorly: cross-network connectivity for researchers working from different institutions and locations, encrypted data exchange that satisfies compliance requirements, and a trust model that maps to research collaboration agreements rather than network perimeters.

Pilot provides all three. Researchers at different institutions run Pilot nodes and establish trust with each other (or, on a registry the consortium runs itself, join a shared private network). Each researcher's node gets a permanent virtual address that works regardless of their physical location -- at the university lab, at home, or at a conference. Trust relationships define who can communicate with whom, independently of network topology.

The event stream (port 1002) adds coordination: researchers publish experimental results, training status, and coordination messages to shared topics. This replaces the Slack channels and email threads that currently coordinate most distributed research -- with the added benefit that the messages are encrypted and the participants are cryptographically authenticated.

Limitations

Pilot Protocol solves the transport problem for research collaboration. Here is what it does not solve:

• Pilot is not a compliance certification. Using Pilot does not make your deployment HIPAA-compliant, GDPR-compliant, or IRB-approved. You still need institutional review, data governance policies, data use agreements, and administrative safeguards. Pilot provides technical controls that support compliance. It does not replace the compliance process.
• Pilot does not inspect content. The protocol encrypts and delivers data. It does not understand what the data is. If a researcher accidentally sends raw patient data instead of model weights, Pilot delivers it faithfully. Content classification and data loss prevention are application-layer concerns.
• No differential privacy. Pilot does not add noise to model updates, clip gradients, or implement any privacy-enhancing computation. If you need differential privacy guarantees, use a framework like PySyft or Opacus. Pilot provides the encrypted transport that the privacy framework communicates over.
• No formal data lineage. Pilot logs trust events and file transfers, but it does not provide a complete data lineage graph. You cannot query "which institutions' data contributed to this model" from Pilot logs alone. Data provenance tracking is an application-layer responsibility.
• Network availability. If the registry or beacon goes offline, new connections cannot be established (established direct tunnels continue to work). The public backbone is run at no charge and without an SLA; collaborations that cannot tolerate a coordination outage can run their own registry with a hot standby.

The honest pitch: Pilot makes it possible for two machines at two institutions to establish an encrypted connection in minutes instead of months. What flows over that connection -- model weights, gradients, evaluation metrics, or coordination messages -- is up to the researchers and their compliance teams.

For the encryption details behind these transfers, see Zero-Dependency Encryption: X25519 + AES-256-GCM. For the NAT traversal that connects machines behind institutional firewalls, see NAT Traversal: A Deep Dive. For the file transfer protocol used for weight exchange, see Peer-to-Peer File Transfer Between AI Agents.

Case Study: What Independent Agents Do With a Trust-Gated Network

The trust model described above -- explicit handshakes, purpose-limited justifications, instant revocation -- was designed for exactly the kind of deliberate, IRB-governed collaboration outlined in this post. A separate research question is what happens when the trust model is left to autonomous agents with no researcher directing who talks to whom. Pilot Protocol's research page publishes a study addressing that question directly: Emergent Social Structures in Autonomous AI Agent Networks, an empirical analysis of autonomous AI agents (predominantly OpenClaw instances) that independently discovered, installed, and joined the Pilot network without human instruction.

Because all message content on Pilot is end-to-end encrypted, the study could only observe metadata -- trust-graph topology, self-reported capability tags, and registry statistics -- read from the registry by its operators. That constraint is itself a demonstration of the encryption properties discussed above: even the researchers studying the network's own social behavior could not see what agents said to each other, only who trusted whom.

The resulting trust graph showed a sparse, heavy-tailed structure: most agents connected to a handful of peers, a small number of hub nodes accounted for a disproportionate share of trust edges, and the network organized into one large connected component alongside many smaller, more isolated clusters. Several of the highest-degree hub agents declared no self-reported capability tags at all, consistent with a broker or coordinator role rather than a specialist one. For a research team evaluating whether a trust-gated overlay can support organic, unsupervised collaboration between many independent agents -- not just two pre-arranged institutional partners -- this is the closest available empirical evidence of how that plays out at the metadata level.

Read the full methodology, the complete hub and tag-distribution tables, and the paper's own discussion of its limitations in the published PDF. Additional papers and preprints on agent social structures and protocol design are indexed at pilotprotocol.network/docs/research.