The method

How a simulation works, start to finish

Language-model agents, each playing a faction inside a real actor, act over a multi-year horizon in a world your experts designed. The same design runs many times against different draws of disruptive events, and findings are read as frequencies across those runs.

The serviceWhat a simulation can hold
SYG Consulting mark, reversed

One design pack, three surfaces

All domain content lives in one versioned design pack: actors, factions, state descriptors, options, events, sources, and the disagreement register. The authoring surface, the engine, and the analysis surface read from that pack, and none of them changes between engagements, so a design can be reviewed, frozen, hashed, and cited independently of the software.

Architecture: three surfaces on one design packARCHITECTUREThree surfaces, one design packAuthoring surfaceWeb application for the workshopPre-loaded design contentSilent review per entityDisagreement registerFreeze validator, hash, tagSimulation enginePython serviceGraph-structured tick loopDecision schema and validatorReferee and ensemble adjudicatorOrchestrator, gates, spend capAnalysis surfaceTrajectory database and toolingBrowsable simulation recordPilot checks and bias batteryOutcome-question readerSix analytical lensesEngine configurationmodel pins, tiers, concurrency, spend ceilingDESIGN PACKOne versioned JSON document carries all domain content; the platform code is engagement-agnostic.Actors · factions · state variables · dimensions · options · events · sources · disagreement register · glossaryFrozen at the workshop: validator pass, SHA-256 content hash, tagged version. Stable identifiers carry into the record.Three layers (inputs, engine, outputs); the same three surfaces run every engagement. Frozen pack flows in; trajectories flow out.
Figure 1.Three surfaces, one design pack; only the pack changes between engagements.

One quarter inside a run

Tick loop: one quarter inside a runTHE ENGINEOne quarter inside a run1ReadState, own memory,messages received2Round oneAll agents decidesimultaneously3MessagesPrivate and broadcastdelivery4Round twoResponse cycle within thequarter5Events sampledBase probability ×conditional modifiers6AdjudicationReferee rules andensemble judgement7PersistState, decisions,messages, events,adjudications8Memory updateCurated summary per agentnext quarter: quarterly steps over the horizon set per engagementTHREE FEEDBACK PATHSAgent to worldActions change state that every agent reads next round.Agent to agentPrivate messages to named counterparts and broadcast signals.World to publicOpinion proxies move with visible actions and events; decision-making factions read them.Moves are simultaneous within a round and visible from the next; every state change traces to a rule or a logged adjudication.
Figure 2.Read, round one, messages, round two, events sampled, adjudication, persist, memory update.

Time advances in ticks, typically quarters, each with two decision rounds in which every agent acts simultaneously. Agents read the state, their memory, and their messages; they decide, messages are delivered, and they decide again. Events are sampled, actions and event effects are adjudicated into the next state, and everything is written to the record before memory is updated.

Agents and red lines

Actors are grouping units and do not act. Factions act, one agent per faction, which is how internal politics are played rather than assumed. Each persona carries documented positions, red lines written as acts the faction takes, behavioral notes, relationships, and grounding sources; its memory is explicit, curated, and inspectable at any quarter of any run. A decision is a structured object: situation read, reasoning, action, posture, intensity, expected reaction, and messages.

Options as starting boards

An option is a starting configuration of the state plus constraints on what each faction may do, marked permitted, prohibited, or conditional. Constraints are enforced softly, so an attempt at a forbidden act is observable and counted rather than filtered away. A status-quo option runs as a full arm, and every option runs against matched random seeds, so a difference between two options is attributable to the option and not to the event draw.

Shocks, organic and scripted

Each event carries a base probability per quarter, conditional modifiers, effects on descriptors, an occurrence cap, and a calibration note for your experts. On the organic path an event is sampled each quarter, conditioned on state and history. On the scripted path it is forced at a fixed quarter in a fixed share of runs, which measures sensitivity to an event without waiting for chance to supply it. Events with known timing and unknown resolution run as timed branching events.

Adjudication: rule, ensemble, mixed

Adjudication turns a round's actions and any fired events into the next state, and the mode is declared per descriptor at the workshop. Rule mode handles mechanical descriptors; in ensemble mode, models from two or three vendor families judge contextual ones in parallel, with disagreement logged and routed to your experts. The referee carries inertia, decay toward baseline, and expert-set plausibility bounds, because a referee without them ratchets the state one way and produces a storyline that repeats under every option and looks like a finding. The ensemble protects against single-vendor idiosyncrasy; it does not correct bias shared across models.

Referee correction: a one-way ratchet versus a referee with inertia, decay and boundsMETHOD · REFEREE CORRECTIONA one-way ratchet, and its correctionUncorrected referee: a one-way ratchetstep = sign(net pressure)maxmintime (quarters)one descriptorreaches the ceiling and staysevery step is one notch up;nothing brings it backCorrected refereeinertia · decay · bounds · magnitudemaxmintime (quarters)one descriptorplausibility bounds, set by expertsbaselinemoves both waysdecays toward baseline when nothing pushesStep scaled to intensityDecay toward baselineBounds set by experts per optionA one-way rule produces a storyline that repeatsin every option. The corrected referee lets adescriptor rise, fall and settle.What our testing taught us, now standard. Conceptual trajectories.
Figure 3.The state layer before and after correction: a one-way ratchet against a referee with inertia, decay, and bounds.

Design elicitation with your experts

The protocol is adapted from the IDEA method in structured expert judgment. SYG pre-loads the full design in your own terminology; every expert reviews every entity in silence and answers confirm, not my area, revise, or flag; a workshop resolves what the review surfaced; every change enters a register with its evidence; and the pack passes a validator, is hashed, and is signed.

Elicitation protocol: pre-load to freezeDESIGN ELICITATIONThe workshop protocol, adapted from IDEA1Pre-loadDesign team drafts actors, factions with sources and anchors, variables, options, events withprovisional probabilities, glossary.2Silent reviewINVESTIGATEEach expert answers per entity: confirm · not my area · revise · flag, plus free-text suggestions.Every response is logged per expert and per field3WorkshopDISCUSSFour hours: table sessions by domain, then a clinic on the questions the tables surfaced. Splits goto the register, anonymised.4Change registerESTIMATEEvery revise, flag and suggestion becomes a change item with evidence, current content, proposedchange, decision required.Rule: adding agents adds noise5Decisions fileOpen questions answered by the design owner, recorded against the pack version they govern.6FreezeAGGREGATEValidator pass, SHA-256 hash, tagged version; the design confirmation document is generated forsignature.Gate: design frozenThe register of expert splits is the distinctive output: recorded splits are legitimate outcomes and become sweep parameters.Rests on the expert-elicitation literature (IDEA, Cooke, SSHAC centre-body-range, ICD 203), not on structured analytic techniques.
Figure 4.Pre-load, silent review, workshop, change register, decisions, freeze.
The disagreement register

A recorded expert split is a legitimate outcome of the workshop. Positions are anonymized, and where a split defines a range it becomes a sweep parameter, so your experts' own unresolved disagreements define the uncertainty space the sensitivity analysis explores.

The four gates

Build does not start until the design is frozen, and production does not start until calibration clears. If your experts cannot accept that agents behave recognizably across the pilot trajectories, production does not proceed.

Gate Condition Who decides
Design frozen Validator passes; pack hashed; confirmation signed Expert group
Pilot passed Mechanics reliable; plausibility reviewed on sampled trajectories; bias battery answered Design owner and expert group
Calibration cleared Corrections applied and verified on fresh runs; historical-anchor check reviewed Expert group
Validation review Sample of production trajectories read; residual bias measured and disclosed Expert group
Engagement lifecycle with four expert gatesENGAGEMENT LIFECYCLEEight phases, four expert gates0FramingQuestion-class fit, alternatives, horizon, configuration, deliverables1DesignPre-load, silent review, workshop, change register, decisions fileGATE · DESIGN FROZENValidator passes; hash computed; design confirmation signed — expert group2BuildEngine adapted to the pack; prompts generated and checked for parity3PilotStaged runs with automatic gates and a spend ceiling; bias batteryGATE · PILOT PASSEDMechanics reliable; plausibility reviewed; battery responded to — design owner and experts4Client confirmationDesign-and-pilot report, decision form, player and event book, record5Corrections and verificationContent validator first, then referee and ensemble; four-run verificationGATE · CALIBRATION CLEAREDVerification gates pass, including mobility; anchors reviewed — production does not start before this6ProductionHundreds to thousands of runs per option; extended where not convergedGATE · VALIDATION REVIEWSampled production trajectories read by experts; residual bias measured and disclosed7HandoverDatabase, record, frozen pack, method note, replication kit; 90-day supportBuild does not start until the design is frozen; production does not start until calibration clears. Experts re-confirmed at every gate.
Figure 5.The engagement lifecycle with its four expert gates.

Human-in-the-loop modes

No established typology exists for the human role in a multi-agent language-model simulation; the four modes are offered as an adaptation.

Mode Description Status
Fully autonomous Agents and adjudication run without intervention between gates Production batches
Expert gates Design frozen, pilot passed, calibration cleared, validation review Default
Adjudicator override Experts rule on flagged disagreements; each ruling is recorded as a design change Calibration; available in production
Expert plays a faction A human seated against the full agent board Least evidenced; offered as untested
Four human-in-the-loop configurationsHUMAN IN THE LOOPFour configurations, presented as an adaptationless human intervention between gatesmoreMODE 0Fully autonomousAgents and adjudicationrun without interventionbetween gates.Production batchesMODE 1Expert gatesDesign freeze, pilotreview, calibrationsign-off, validation review.The defaultMODE 2Adjudicator overrideExperts rule on flaggedensemble disagreements;rulings become designchanges.Used in calibrationMODE 3Human on the boardOne faction played bya person against a fullagent board.Least evidencedBeyond the gates, humans are decisive at three points: plausibility bounds, decay rates and fatigue modelsare signed by experts; red-line flags are routed for expert reading; the outcomequestions are the experts' to define.No established typology exists for this application; no controlled study seats humans and language-model agents together.
Figure 6.Four configurations for the human role; expert gates are the default.

The pre-registered bias battery

Design-time mitigation reduces bias without showing that it did. The battery shows it: ten families are fixed before the pilot data is opened, each traced to a design mechanism, and the same scripts re-run after every correction. The residual that survives calibration is stated in the method note.

Ten pre-registered bias families, mechanism and statusMETHOD · BIASThe pre-registered bias batteryFAMILYMECHANISM EXAMINEDSTATUSState-direction ratchetReferee ruleFound · correctedVariable mobilityReferee rule · ensemble anchorsFound · correctedPopulation-proxy behaviourProxy design · one-signal ruleFound · correctedTier gradientModel tieringFound · correctedText integrityStructured-output handlingFound · correctedAttention concentrationMessage quotaReporting ruleRecord compositionRoster and quotaReporting ruleOption differentiationOption constraintsClearedEvent behaviourEvent library and samplingClearedRepertoire parityAction-space constraintsClearedFamilies are fixed before the data is opened, each traced to a design mechanism; runs of earlier designversions serve as controls. The same scripts re-run after each correction, which is how corrections are shown to work.Fixed before the data is opened; re-run after every correction; residual bias disclosed in the method note.
Figure 7.Ten families, each traced to the mechanism it examines.

Nine sources of bias, and what reaches each

Model priors on the domain; escalation propensity; persona drafting; prompt framing; model-strength asymmetry between agents; state-transition rules; message quotas; free-text corruption that survives schema validation; and the Western, English-language normalization of the evidence base. Each has a named design response except the last, which only disclosure and a heavier expert-calibration burden address, and we say so.

Nine bias sources mapped to mitigationsMETHOD · BIASNine sources of bias, and what addresses each1 Substrate priors2 Escalation propensity3 Persona drafting4 Prompt framing5 Tier asymmetry6 State-transition rule7 Quota artefacts8 Record integrity9 Cultural normalisationPersona grounding in registered sourcesEnsemble adjudication (diagnostic, notcorrective)Restraint as a named, costed option; everyrewrite measured on fixed seedsPaired uniform-model testCorrected referee: inertia, decay, boundsReporting rule: in-degree is not salienceContent validator on free-text fieldsDisclosure and a heavier expert-calibrationburdenOnly the first source is the one usually meant by 'model bias'. Sources 6 and 3 produced the largest distortions in our testing, not source 1.Source 9 (dashed) is addressed only by disclosure; the evidence base is Western-normed and English-language.
Figure 8.Nine sources mapped to their mitigations; one is addressed by disclosure alone.

Models, tested before they are selected

Candidate models are compared on pre-registered metrics, on fixed seeds, before the fleet is set; a uniform-model configuration is tested the same way, because tier and role are confounded and only a paired test settles it. Versions are pinned per batch and recorded in every call, and the provider allow-list runs with fallbacks disabled, so a call reaches the pinned model or fails visibly.

Model selection: candidates, experiments on pre-registered metrics, selected fleet by tierMODEL SELECTIONModels tested before they are selectedCANDIDATE MODELSseveral vendor familiesVendor family AVendor family BVendor family CVendor family DVendor family EEXPERIMENTSsame design, fixed seedsPre-registered metricsPosture entropyRestraint shareText integritySchema validityInter-judge agreementEvery model runs the same battery.SELECTED FLEETby tier · versions pinned per batchTop tierprincipal decision nodesMid tiersub-factions, secondary actorsBase tierbackground, opinion proxiesPinned model, or a visible failure.testedselectedCross-vendor adjudication ensemblea diagnostic on contextual judgments: two or three vendor families judge the same case;disagreement is measured and routed to experts, not averaged awaySelection is experiment-based and repeated when a model version changes; the fleet in use is stated in the method note.
Figure 9.A multi-vendor fleet feeding three agent tiers.

Traceability

Traceability chain: claim to run to tick to decision to persona to sourceTRACEABILITYFrom any claim to its sourceClaimA sentence in the findings brief, with the runs it rests onRunRun identifier, option, random seed, pack version and hashTickQuarter and round; the state the agents readDecisionStructured object: situation read, reasoning, action, posture, messagesPersonaThe faction brief and the agent's memory at that tickSourceA registered grounding document: internal assessment, public position or historical recordStable identifiers propagate from the design pack through the trajectory database and the audit log.Every model call is logged with model identifier, full input, full output and reasoning.Client documents strip the internal codes; the database keeps them.
Figure 10.From a claim in the report to the source that grounded the persona.

Every claim traces along one chain: claim, run, quarter, decision, persona, source. Every model call is logged with its full input and output, and every state change carries a causal field naming the actions and events that produced it. Reproducibility of process rests on the frozen pack with its hash, the audit log, the trajectory database, and a replication kit with pinned model versions.

Six lenses on one database

Six analytical lenses on one trajectory databaseANALYSISSix analytical lenses on one databaseTrajectorydatabaseone self-contained fileOutcome frequenciesper option, against the outcome questionsConditional probabilitiesof an outcome given an event sequenceScenario treehow futures diverge from a shared startTipping pointswhere small input differences drive large shiftsSensitivity rankingwhich assumptions the findings depend onTraceabilityfrom any claim to runs, decisions, sourcesAll six read the same record; a change of question or weighting is an analytical choice and needs no re-run.
Figure 11.Six lenses on one trajectory database.

Outcome frequencies per option, conditional probabilities given an event sequence, the scenario tree, tipping points, a sensitivity ranking of assumptions, and traceability. All six read the same record, so a new question needs no re-run.

What we claim, and what we do not
  • Reproducibility of process, never of outcome. No run is a forecast.
  • The method is a measurement instrument that follows expert elicitation; it does not replace expert judgment.
  • Resettability is its one categorical advantage over the human wargame; the rest are differences of degree.
  • Scale improves reliability, not validity.
  • Agent bias and expert bias are symmetrical in kind; the simulation's bias can be measured, which makes it auditable.
  • The evidence base is Western-normed and English-language, which argues for a heavier expert-calibration burden.
  • Known gaps are stated: no published simulation of this kind has made a calibrated out-of-sample forecast, and no controlled study seats humans and language-model agents together.

See what a simulation can hold.

Actors, options, shocks, scenarios, and the probability layer on top, in capability ranges.

CapabilitiesThe service