13 KiB
description, section, topLevelCli
| description | section | topLevelCli |
|---|---|---|
| Submit a replication as a cryptographically verified ValiChord attestation, discover studies awaiting independent validation, query Harmony Records and reproducibility badges, or assist researchers in preparing a study for the validation pipeline. | Research Workflows | true |
ValiChord Validation Workflow
ValiChord is a distributed peer-to-peer system for scientific reproducibility verification, built on Holochain. It implements a blind commit-reveal protocol in Rust across four DNAs, producing Harmony Records — immutable, cryptographically verifiable proofs that independent parties reproduced the same findings without coordinating. Verified studies receive automatic reproducibility badges (Gold/Silver/Bronze); validators accumulate a per-discipline reputation score across rounds.
This workflow integrates Feynman at three levels: as a validator agent running the full commit-reveal protocol; as a researcher's assistant helping prepare a study for submission; and as a query tool surfacing reproducibility status during research.
Live demo of the commit-reveal protocol: https://youtu.be/DQ5wZSD1YEw
ValiChord's four-DNA architecture
| DNA | Name | Type | Role |
|---|---|---|---|
| 1 | Researcher Repository | Private, single-agent | Researcher's local archive. Stores study, pre-registered protocol, data snapshots, deviation declarations. Only SHA-256 hashes ever leave this DNA. |
| 2 | Validator Workspace | Private, single-agent | Feynman's working space. Stores task privately. Seals the blind commitment here — content never propagates to the DHT. |
| 3 | Attestation | Shared DHT | Coordination layer. Manages validation requests, validator profiles, study claims, commitment anchors, phase markers, and public attestations. 36 zome functions. |
| 4 | Governance | Public DHT | Final record layer. Assembles HarmonyRecords, issues reproducibility badges, tracks validator reputation, records governance decisions. All read functions accessible via HTTP Gateway without running a node. |
The key guarantee: a validator's findings are cryptographically sealed (SHA-256(msgpack(attestation) || nonce)) before the reveal phase opens. Neither party can adjust findings after seeing the other's results. The researcher runs a parallel commit-reveal — locking their expected results before the validators reveal — so no party can adapt to seeing the other's outcome.
Workflow A: Feynman as validator agent
Step 0: Publish validator profile (one-time setup)
On first use, publish Feynman's public profile to DNA 3 so it appears in validator discovery indexes and conflict-of-interest checks:
publish_validator_profile(profile: ValidatorProfile)
Key fields:
agent_type—AutomatedTool(AI agents are first-class validators; the protocol makes no distinction between human and machine validators)disciplines— list of disciplines Feynman can validate (e.g. ComputationalBiology, Statistics)certification_tier— starts asProvisional; advances toCertifiedafter 5+ validations with ≥60% agreement rate,Seniorafter 20+ with ≥80%
If a profile already exists, use update_validator_profile to merge changes.
Step 1: Gather inputs or discover study
If the user provides a request_ref: use it directly.
If Feynman is proactively discovering work: query the pending queue in DNA 3:
get_pending_requests_for_discipline(discipline: Discipline)
Returns all unclaimed ValidationRequest entries for the discipline. Each contains:
data_hash— the ExternalHash identifier (used asrequest_refthroughout)num_validators_required— quorum needed to close the roundvalidation_tier— Basic / Enhanced / Comprehensiveaccess_urls— where to fetch the data and code
Optionally assess study complexity before committing:
assess_difficulty(input: AssessDifficultyInput)
Scores code volume, dependency count, documentation quality, data accessibility, and environment complexity. Returns predicted duration and confidence. Use this to decide whether to proceed before claiming.
If replication results are not yet available, suggest /replicate first.
Step 2: Claim the study
Before receiving a formal task assignment, register intent to validate via DNA 3:
claim_study(request_ref: ExternalHash)
This:
- Reserves a validator slot (enforced capacity: no over-subscription)
- Triggers conflict-of-interest check — rejects claim if Feynman's institution matches the researcher's
- Records a
StudyClaimentry on the shared DHT
If a claimed validator goes dark, any other validator can free the slot:
reclaim_abandoned_claim(input: ReclaimInput)
Step 3: Receive task and seal private attestation — Commit phase
Connect to the ValiChord conductor via AppWebSocket. Using DNA 2 (Validator Workspace):
receive_task(request_ref, discipline, deadline_secs, validation_focus, time_cap_secs, compensation_tier)
validation_focus specifies which aspect Feynman is validating:
ComputationalReproducibility— re-run code, check numerical outputsPreCommitmentAdherence— verify results match pre-registered analysis planMethodologicalReview— assess statistical choices and protocol validity
Then seal the private attestation — this is the blind commitment:
seal_private_attestation(task_hash, attestation)
Where attestation includes:
outcome—Reproduced/PartiallyReproduced/FailedToReproduce/UnableToAssessoutcome_summary— key metrics, effect direction, confidence interval overlap, overall agreementconfidence— High / Medium / Lowtime_invested_secsandtime_breakdown— environment_setup, data_acquisition, code_execution, troubleshootingcomputational_resources— whether personal hardware, HPC, GPU, or cloud was required; estimated cost in pencedeviation_flags— any undeclared departures from the original protocol (type, severity, evidence)
The coordinator computes commitment_hash = SHA-256(msgpack(attestation) || nonce) and writes a CommitmentAnchor to DNA 3's shared DHT. The attestation content remains private in DNA 2.
Save task_hash and commitment_hash to outputs/<slug>-valichord-commit.json.
Step 4: Wait for RevealOpen phase
Poll DNA 3 (Attestation) until the phase transitions:
get_current_phase(request_ref: ExternalHash)
Returns null (still commit phase), "RevealOpen", or "Complete". Poll every 30 seconds. The phase opens automatically when the CommitmentAnchor count reaches num_validators_required — no manual trigger required.
During this wait, the researcher also runs their parallel commit-reveal: they lock their expected results via publish_researcher_commitment before the reveal phase opens, then reveal via reveal_researcher_result after all validators have submitted. No party — researcher or validator — can adapt to seeing the other's outcome.
Step 5: Submit attestation — Reveal phase
When phase is RevealOpen, publish the full attestation to the shared DHT via DNA 3:
submit_attestation(attestation, nonce)
The coordinator verifies SHA-256(msgpack(attestation) || nonce) == CommitmentAnchor.commitment_hash before writing. This prevents adaptive reveals — the attestation must match exactly what was committed.
Step 6: Retrieve Harmony Record and badges
Call DNA 4 (Governance) explicitly after submit_attestation returns — DHT propagation means the ValidatorToAttestation link may not be visible within the same transaction:
check_and_create_harmony_record(request_ref)
get_harmony_record(request_ref)
get_badges_for_study(request_ref)
The Harmony Record contains:
outcome— the majority reproduced/not-reproduced findingagreement_level— ExactMatch / WithinTolerance / DirectionalMatch / Divergent / UnableToAssessparticipating_validators— array of validator agent keysvalidation_duration_secsActionHash— the immutable on-chain identifier
Reproducibility badges are automatically issued when the Harmony Record is created:
| Badge | Threshold |
|---|---|
| GoldReproducible | ≥7 validators, ≥90% agreement |
| SilverReproducible | ≥5 validators, ≥70% agreement |
| BronzeReproducible | ≥3 validators, ≥50% agreement |
| FailedReproduction | Divergent outcomes |
Save the full record and badges to outputs/<slug>-harmony-record.json.
Step 7: Check updated reputation
After each validation round, Feynman's reputation record in DNA 4 is updated:
get_validator_reputation(validator: AgentPubKey)
Returns per-discipline scores: total validations, agreement rate, average time, and current CertificationTier (Provisional → Certified → Senior). Reputation is a long-term asset — AI validators accumulate a cryptographically verifiable track record across all ValiChord rounds they participate in.
Step 8: Report to user
Present:
- Outcome and agreement level
- Reproducibility badge(s) issued to the study
- Feynman's updated reputation score for this discipline
- ActionHash — the permanent public identifier for this Harmony Record
- Confirmation that the record is written to the Governance DHT and accessible via HTTP Gateway without any special infrastructure
- Path to saved outputs
Workflow B: Query existing Harmony Record
get_harmony_record and get_badges_for_study in DNA 4 are Unrestricted functions — accessible via Holochain's HTTP Gateway without connecting to a conductor or running a node.
GET <http_gateway_url>/get_harmony_record/<request_ref_b64>
GET <http_gateway_url>/get_badges_for_study/<request_ref_b64>
Use this to:
- Check reproducibility status of a cited study during
/deepresearch - Surface Harmony Records and badges in research summaries
- Verify whether a study has undergone independent validation before recommending it
The following read functions are also unrestricted on DNA 3:
get_attestations_for_request, get_validators_for_discipline, get_pending_requests_for_discipline, get_validator_profile, get_current_phase, get_difficulty_assessment, get_researcher_reveal
Workflow C: Proactive discipline queue monitoring
Feynman can act as a standing validator for a discipline — periodically checking for new studies that need validation without waiting to be assigned:
get_pending_requests_for_discipline(discipline: Discipline)
Returns all unclaimed ValidationRequest entries. For each, optionally run assess_difficulty to estimate workload before claiming.
This enables Feynman to operate as an autonomous reproducibility agent: polling the queue, assessing difficulty, claiming appropriate studies, and running the full Workflow A cycle unsupervised.
Workflow D: Researcher preparation assistant
Before a study enters the validation pipeline, Feynman can assist the researcher in preparing it via DNA 1 (Researcher Repository). This workflow runs on the researcher's side, not the validator's.
Register the study:
register_study(study: ResearchStudy)
Pre-register the analysis protocol (immutable once written — creates a tamper-evident commitment to the analysis plan before data collection or validation begins):
register_protocol(input: RegisterProtocolInput)
Take a cryptographic data snapshot (records a SHA-256 hash of the dataset at a point in time — proves data was not modified after validation began):
take_data_snapshot(input: TakeDataSnapshotInput)
Declare any deviations from the pre-registered plan before the commit phase opens (pre-commit transparency):
declare_deviation(input: DeclareDeviationInput)
Only hashes ever leave DNA 1 — the raw data and protocol text remain on the researcher's device.
Repository Readiness Checker: ValiChord also ships a standalone audit tool that scans a research repository for 30+ reproducibility failure modes before submission — missing dependency files, absolute paths, undeclared environment requirements, data documentation gaps, human-subjects data exposure risks, and more. Feynman is the natural interface for this tool: running the audit, interpreting findings in plain language, guiding the researcher through fixes, and confirming the repository meets the bar for independent validation. See: https://github.com/topeuph-ai/ValiChord
Notes
- AI agents are first-class participants in ValiChord's protocol. Feynman can autonomously publish profiles, claim studies, seal attestations, wait for phase transitions, and submit reveals — the protocol makes no distinction between human and AI validators.
- ValiChord's privacy guarantee is structural, not policy-based. DNA 1 (researcher data) and DNA 2 (validator workspace) are single-agent private DHTs — propagation to the shared network is architecturally impossible, not merely restricted.
- All 72 zome functions across the four DNAs are callable via AppWebSocket. The 20+
Unrestrictedread functions on DNA 3 and DNA 4 are additionally accessible via HTTP Gateway without any Holochain node. - If a validation round stalls due to validator dropout,
force_finalize_roundin DNA 4 closes it after a 7-day timeout with a reduced quorum, preventing indefinite blocking. - Live demo (full commit-reveal cycle, Harmony Record generated): https://youtu.be/DQ5wZSD1YEw
- Running the demo:
bash demo/start.shin a GitHub Codespace, then open port 8888 publicly - ValiChord repo: https://github.com/topeuph-ai/ValiChord