From Traditional Chinese Medicine to AI Harness: How to Wrap a Probabilistic Core in a Deterministic Shell

Background:

1. Lately I have been studying traditional Chinese medicine and drinking herbal decoctions to build up my health. I see the doctor once every two weeks; after pulse diagnosis, tongue inspection, and a consultation, he fine-tunes the proportions and ingredients of the herbs. I find this process quite interesting.
2. Today I was studying AI Harness: "
   Why The Harness Matters More Than The Model | YC Paper Club"
3. Suddenly a flash of insight hit me — it felt like these two things overlap, so I asked AI several questions. What follows is a technical article that AI itself summarized from its own answers.
   Background ends.

One-sentence summary:
Traditional software engineering deals with deterministic logic that can be specified, reproduced, and verified; modern AI deals with probabilistic generalization induced from data. TCM syndrome differentiation and treatment is structurally similar to AI: knowledge is not explicit rules but implicit experiential patterns; output is not logical necessity but a high-probability choice under a conditional distribution. What the AI Harness should learn most is not to make the model deterministic, but to wrap the probabilistic core in a deterministic shell, compressing variance and tail risk into an acceptable range.

---

0. Summary

This article organizes a discussion about “the biggest difference between AI and traditional software engineering,” and extends it to an analogy between traditional Chinese medicine (TCM) and the AI Harness.

The core thread is as follows:

1. The biggest difference between AI and traditional software engineering: traditional software pursues a deterministic closed loop of specification, code, execution, and verification; AI is data, training, weights, and probabilistic generation.
2. TCM is a typical traditional system of “probabilistic generalization”: syndrome differentiation and treatment, formula–syndrome correspondence, and acupuncture point selection all induce implicit patterns from a large number of experiential samples, and then make contextual judgments about new situations.
3. TCM uses a set of experiential engineering practices to control risk: strong priors, syndrome-differentiation guidelines, the four diagnostic methods used in combination, safety contraindications, treatment tailored to time, place, and person, follow-up visits and feedback, mentorship and consultation, herbal processing, institutional regulation, and modern evidence-based practice.
4. These measures map almost one-to-one onto the AI Harness: versioned policy packages, task routing, rule engines, RAG, multi-source evidence, sandboxes, approvals, rollback, monitoring, refusal with handoff to a human, and case feedback loops.
5. Maturity is uneven: prompting, routing, RAG, guardrails, fine-tuning, monitoring, and human handoff are already fairly widespread; but versioning linked with eval gates, hard constraints for risk tiers, multi-source evidence arbitration, tool transaction rollback, OOD-calibrated refusal, and online closed-loop case feedback are still immature.
6. Priorities going forward: compress tail risk first, then raise the ceiling. Prioritize versioning + eval gates, risk routing + hard rules, tool sandboxing + approval + rollback, evidence grading + conflict arbitration + calibration, and OOD refusal + human handoff + case feedback.

---

1. Starting Point: The Biggest Difference Between AI and Traditional Software Engineering

The core pipeline of traditional software engineering is:

Specification → Code → Execution → Verification

You write down the rules, and the system runs according to those rules. Given the same input and the same state, it should in principle produce the same output. A bug can be traced to a specific line of code, a specific state, or a specific dependency; correctness can be approached as a Boolean-style guarantee through unit tests, contracts, and formal verification.

For modern AI, especially large models, the core pipeline is:

Data → Training → Weights → Probabilistic Generation

It does not write out an “if A then B” rule explicitly; instead it compresses statistical regularities into parameters. Its output is not the result of logical deduction but a sample of the “most likely appropriate answer.”

Therefore:

• The same question phrased differently may get a different answer;
• The same input may produce different output depending on the sampling settings;
• The error is not “a line written incorrectly,” but may be data bias, distribution shift, prompt sensitivity, or hallucination;
• Correctness is hard to judge as “satisfies / does not satisfy the specification”; it can only be judged statistically through evaluation sets, metrics, human feedback, and online monitoring.

So, the biggest difference is not whether AI makes mistakes, but whether correctness can be deterministically specified and reproduced.

Traditional software engineering pursues: within clear boundaries, behavior that is predictable, auditable, and verifiable.
AI engineering pursues: in an open distribution, behavior that is statistically usable, evaluable, alignable, and monitorable.

Engineering methods therefore differ as well:

Dimension	Traditional Software	AI Systems
Core assets	Code, specs, architecture	Data, weights, prompts, context
Correctness	Boolean, provable	Statistical, evaluable
Reproducibility	Strong	Weak, affected by sampling and distribution
Error localization	Line-level, state-level	Data, distribution, prompt, model, toolchain
Engineering focus	Testing, contracts, versioning	Evaluation, alignment, monitoring, rollback

Traditional software is more like “building a machine that runs according to a blueprint”; modern AI is more like “training an expert system that performs well in a statistical sense.”

---

2. TCM: Another Kind of “Probabilistic Generalization Induced from Data”

If you had to pick just one analogy, the most typical one is:

TCM’s system of syndrome differentiation and treatment, especially classical formula–syndrome correspondences and acupuncture point selection.

2.1 Why They Are Structurally Isomorphic to AI

This is not to say “TCM is AI,” but that their forms of knowledge and their modes of decision-making are structurally similar:

Layer	TCM	AI
Data layer	Medical case records, pulse records, tongue presentations, properties and flavors of herbs	Corpora, annotations, logs, feedback
Model layer	Six meridians, zang-fu organs, meridians and collaterals, formula–syndrome, herb–syndrome	Weights, hidden representations, probability distributions
Output layer	Same disease treated differently, different diseases treated the same, treat according to the syndrome	Conditional generation, sampling, multiple candidates
Validation layer	Follow-up visits, feedback, adjustment	Evaluation sets, metrics, online monitoring
Transmission layer	Mentorship, mnemonic verses, feel, tacit knowledge	Fine-tuning, few-shot, human feedback

When a TCM practitioner sees a set of symptoms, pulse presentations, tongue presentations, constitution, and solar term, they are not looking up a fixed table but performing pattern recognition: which “syndrome” does this presentation resemble most? With which formulas, which acupoints, and which additions and subtractions has this syndrome historically co-occurred at high frequency?

This is very much like the statistical associations that a large model compresses into its weights from a corpus, except that the TCM “weights” are hidden in mentorship mnemonics, analogies from medical case records, and bodily memory.

2.2 Typical Features: Same Disease, Different Treatments; Different Diseases, Same Treatment; Treat According to the Syndrome

For the same common cold, one person is given Mahuang Tang (Ephedra Decoction), another Yinqiao San (Lonicera and Forsythia Powder), and another needs support for the healthy qi; for the same person, summer and winter differ, and the south and the north differ. A prescription is not the single deterministic solution, but the plan most likely to suit the current context.

Acupuncture is the same: for the same illness, point selection, tonification and reduction, the time of day, and the deqi response may all differ.

This is very much like probabilistic generation: the output is not a logical necessity but a high-probability choice under a conditional distribution.

2.3 The Differences Must Also Be Stated Clearly

AI relies on mathematical optimization and large-scale compute; TCM relies on bodily senses, master–apprentice transmission, and cultural context. TCM does not “guarantee” good outcomes, nor does it have calibration in the modern statistical sense. Misdiagnosis, mistreatment, herbal harm, and “deteriorated disease” have always existed.

So a more accurate statement is:

TCM is a pre-modern body of experiential engineering that uses strong priors, structured syndrome differentiation, safety contraindications, follow-up feedback, and institutional regulation to keep the variance and tail risk of probabilistic generalization as small as possible.

---

3. How TCM Controls the Quality and Risk of Probabilistic Generalization

TCM does not guarantee a high score every time, but it tries to avoid the worst outcome and keeps correcting itself through feedback. Its specific measures can be divided into ten layers.

3.1 Strong Priors and Classical Constraints: Equivalent to a Foundation Model Plus Version Management

The Huangdi Neijing (Yellow Emperor’s Inner Canon), Shanghan Lun (Treatise on Cold Damage), Jingui Yaolue (Essential Prescriptions of the Golden Cabinet), Shennong Bencao Jing (Classic of the Materia Medica), Bencao Gangmu (Compendium of Materia Medica), and other works constitute a set of strong priors. Later commentaries, collations, and medical case records resemble continuously updated training corpora and version revisions.

In particular, “formula corresponding to syndrome” and “where there is this syndrome, use this formula” compress many decisions into fairly hard correspondences, reducing arbitrary generalization.

3.2 Syndrome-Differentiation Guidelines: Equivalent to Dimensionality Reduction, Regularization, and Latent Variables

Eight-principle differentiation, six-meridian differentiation, zang-fu differentiation, wei-qi-ying-xue (defense, qi, nutritive, and blood) differentiation, and triple-burner differentiation map high-dimensional, noisy symptoms onto low-dimensional “syndrome patterns.”

This is both like feature engineering and like regularization: it prevents over-inference from just one or two symptoms. A syndrome pattern is not a deterministic rule, but it gives generalization a framework and boundaries.

3.3 The Four Diagnostic Methods Used in Combination and Confidence Weighting: Equivalent to Multimodal Fusion

Inspection, auscultation and olfaction, inquiry, and palpation, along with the “Song of Ten Questions,” require multiple sources of information to corroborate one another; one cannot rely on a single pulse or tongue sign alone. When contradictions arise, there are weighing principles such as “discard the pulse and follow the syndrome” or “discard the syndrome and follow the pulse,” similar to confidence weighting across different modalities.

The goal is to reduce misjudgment caused by noise at a single point.

3.4 Formula–Syndrome Correspondence: Equivalent to a Rule Engine and Constrained Decoding

Where there is this syndrome, use this formula. Key actions must satisfy hard rules. For example, the Mahuang Tang syndrome requires absence of sweating and a floating, tight pulse; if there is sweating and a weak pulse, it is contraindicated, to prevent mistaken diaphoresis.

This resembles outputting a JSON schema, a grammar, or precondition checks.

3.5 Safety Guardrails and Contraindications: Equivalent to Constrained Decoding and Rejection Sampling

The Eighteen Incompatibilities, the Nineteen Antagonisms, pregnancy contraindications, processing of toxic herbs, pre-decoction and prolonged decoction, dosage rules, “stop as soon as the disease is hit,” and “stop once purging occurs” are all hard constraints.

For Dachengqi Tang (Major Order the Qi Decoction), “stop once purging occurs” is similar to early stopping. These measures do not guarantee optimality, but they prioritize avoiding catastrophic errors.

3.6 Treatment Tailored to Time, Place, and Person: Equivalent to Conditional Generation

Adapting to the time, the place, and the individual emphasizes treating the same disease differently. It acknowledges that the output is not a globally unique solution but a contextual solution under a conditional distribution.

This improves individual fit, but it also requires the practitioner to judge the context accurately.

3.7 Follow-Up Visits and Treating According to the Syndrome: Equivalent to Online Learning and a Feedback Loop

The Shanghan Lun says, “observe the pulse and symptoms, know what error has been committed, and treat according to the syndrome” — a classic error-correction mechanism. There are also sayings such as “if effective, do not change the formula; if ineffective, change the formula,” along with small-dose trial administration, exploratory treatment, and adjusting the formula at follow-up visits.

TCM treats a prescription as an intervention with feedback, not as a final, irrevocable decision.

3.8 Mentorship, Medical Case Records, and Consultation: Equivalent to Fine-Tuning, Human Feedback, and Ensemble Voting

Master–apprentice teaching is a typical transmission of tacit knowledge, similar to domain fine-tuning. Medical case records document successes and failures, similar to labeled experiential data. Consultations, case discussions, and cross-referencing among schools are similar to multi-model ensembles, reducing the bias of any single doctor “model.”

But medical case records also suffer from publication bias and survivorship bias.

3.9 Authentic Regional Herbs and Processing Standards: Equivalent to Data Governance and Input Standardization

“An orange grown south of the Huai River is an orange; grown north of it, it is a trifoliate orange.” Origin, harvest, processing, and the method of decoction and administration all change the medicinal properties.

Authentic regional herbs, processing according to the prescribed method, and pharmacopoeia standards control the input distribution and reduce data drift. Without this layer, even the best syndrome differentiation will be dragged down by poor-quality herbs.

3.10 Institutions and Ethics: Equivalent to Regulation and Market Access

Medical licensing, pharmacy management, pharmacopoeias, mentorship examinations, case-record assessments, and refusal-to-treat principles such as the “six not treated” are all institutional guardrails.

Referral of critical cases and treating the not-yet-arisen disease rather than the already-arisen one are similar to triage and refusal mechanisms.

3.11 Modern Reinforcement: Evidence-Based Medicine, RCTs, Real-World Research, Pharmacovigilance, and AI Assistance

These traditional TCM measures can improve stability and safety, but they cannot provide deterministic guarantees the way formal verification does.

So modern TCM must add randomized controlled trials, meta-analyses, real-world data, toxicology and pharmacovigilance, standardized electronic medical records, and AI-assisted decision-making and monitoring.

---

4. Mapping to the AI Harness: A Deterministic Shell Wrapping a Probabilistic Core

4.1 What Is an AI Harness

The AI Harness discussed in this article is defined broadly:

The scaffolding around a model for operation, evaluation, feedback, and governance: prompts and policies, tool calls, state machines, guardrails, evaluation sets, monitoring, rollback, and auditing.

The core principle in one sentence:

Wrap the probabilistic core in a deterministic shell.

TCM’s “syndrome differentiation and treatment” is the probabilistic core; contraindications, herb processing, follow-up visits, mentorship, pharmacopoeias, and institutions are the deterministic shell. The AI Harness should do the same.

4.2 One-to-One Mapping Table

TCM Measure	AI Harness Counterpart	How to Do It Concretely
Classical priors, versioned commentaries	System prompts, policy packages, model/data versions	Version-control the system prompt, tool schemas, safety policies, and knowledge base; changes must pass an eval gate and be rollback-capable
Eight-principle, six-meridian, and zang-fu differentiation	Task routing, state machines, ontologies	First classify intent, domain, and risk level, then choose the model, prompt, tools, and workflow; high-risk cases take the slow path
Formula–syndrome correspondence	Rule engines, constrained decoding	Output JSON schema, grammar, preconditions; key actions must satisfy hard rules
The four diagnostic methods used in combination	Multi-source evidence fusion, RAG	User input, retrieval, tool results, and memory all carry provenance and confidence; conflicts are resolved by authority, recency, and voting; citations are mandatory
Eighteen Incompatibilities, Nineteen Antagonisms, pregnancy contraindications	Safety guardrails, access control	Tool allowlists, least privilege, sandboxes, forbidden combinations, budgets, timeouts, maximum steps, kill switch
Treatment tailored to time, place, and person	Context conditioning	Inject user profile, region, time, and compliance zone; personalized but constrained by policy
Follow-up visits, treating according to the syndrome	Online feedback loop	Verify after execution; on failure, retry, switch strategy, or roll back; A/B testing, monitoring, alerting
Mentorship, medical case records, consultation	Fine-tuning, case libraries, multi-agent systems	few-shot, LoRA, expert feedback, similar-case retrieval; for high risk, multi-model voting / debate / arbitration
Authentic regional herbs, processing	Data governance, preprocessing	Authoritative sources, recency, deduplication, cleaning, de-identification, format normalization, embedding model version management
Pharmacopoeias, medical licensing	Governance, auditing, compliance	Logs, traces, audits, access assessment, chain of responsibility, SLOs, model cards, data cards
The six not treated	Refusal, human handoff	Hand off to a human or refuse when confidence is low, the input is out of distribution, risk is high, or the request exceeds capability
Modern reinforcement	Statistical evaluation, red teaming, formal verification	eval harness, regression gates, calibration, OOD detection, red teaming, formal verification of critical paths

4.3 A Concrete Architecture Pipeline

It can be built as a pipeline like this:

1. Ingress layer: input normalization, de-identification, intent classification, risk scoring.
2. Routing layer: select the model, prompt, tools, and retrieval sources by task type and risk.
3. Context layer: RAG retrieval, tool queries, memory injection; each piece of evidence carries provenance, time, and confidence.
4. Reasoning layer: generate plans, multiple candidate answers, self-checks; high-risk tasks use multi-model voting or debate.
5. Constraint layer: schema validation, rule engine, permission checks, constrained decoding.
6. Execution layer: tool sandbox, dry-run, approval, transactions, idempotency, timeouts, rollback.
7. Verification layer: fact-checking, citation checks, unit tests, simulated execution, output filtering.
8. Feedback layer: logs, metrics, user feedback, online evaluation, case ingestion, periodic fine-tuning.
9. Governance layer: version management, auditing, compliance, SLOs, canary rollout, rollback, chain of responsibility.

4.4 Example: A Refund Agent

• Ingress: recognize the “refund” intent; risk is medium-high.
• Routing: use the refund-specific prompt and tools.
• Context: retrieve the refund policy and the order history.
• Reasoning: the model generates a refund plan.
• Constraints: the rule engine checks that the order exists, that the amount ≤ the original amount, the time window, and user permissions.
• Execution: sandbox dry-run, high-risk cases go to human approval, then the payment tool is actually called.
• Verification: confirm that the refund succeeded, the amount is correct, and the logs are complete.
• Feedback: on failure, roll back or retry; add the case to the eval set; monitor the refund anomaly rate.

This is essentially the TCM structure of “syndrome differentiation — formula–syndrome — contraindications — follow-up visit.”

---

5. A Maturity Inventory: What Is Already Used and What Is Not

The following assessment is based on the maturity of leading production systems versus ordinary teams. Many measures are “present, but shallow.”

5.1 Widely Adopted or Partially Widely Adopted

TCM Measure	AI Harness Counterpart	Maturity
Classical priors, versioned commentaries	Versioning of system prompts, models, knowledge bases, and tool schemas	Widespread at the top, weak in small teams
Eight-principle / six-meridian differentiation	Intent classification, task routing, state machines, agent frameworks	Widespread
Formula–syndrome correspondence	JSON schema, function calling, constrained decoding, rule engines	Widespread
The four diagnostic methods used in combination	RAG, multi-source retrieval, cited sources	Widespread, but conflict arbitration is weak
Eighteen Incompatibilities, pregnancy contraindications	Content filtering, permissions, sandboxes, tool allowlists	Widespread
Treatment tailored to time, place, and person	Injection of user profile, region, time, and compliance context	Widespread but crude
Follow-up visits, treating according to the syndrome	Logs, monitoring, A/B testing, retries	Partially widespread, automatic rollback is rare
Mentorship, medical case records, consultation	few-shot, fine-tuning, case libraries	Widespread; multi-agent debate is rare in production
Authentic regional herbs, processing	Data cleaning, deduplication, de-identification, format normalization	Widespread but uneven in quality
Pharmacopoeias, medical licensing	Auditing, compliance, model cards	Widespread at the top, rare in small teams
The six not treated	Refusal, human handoff	Widespread
Modern reinforcement	Offline eval, red teaming	Widespread at the top, formal verification is rare

In one sentence: “prompting, routing, RAG, guardrails, fine-tuning, monitoring, and human handoff” are already widespread; but very few teams string them together into a strict closed loop with gates.

5.2 Not Yet Widely Adopted or Highly Immature

1. Versioned policy packages + eval gate integration: many teams change prompts, swap models, or replace knowledge bases without regression testing, and without mandatory canary rollout and rollback.
2. Risk-tiered routing + full coverage of hard rules: intent classification exists, but high-risk tasks are not forced onto the slow path, the rule engine, and human approval.
3. Authority grading, recency weighting, conflict arbitration, and confidence calibration for multi-source evidence: RAG has citations, but which source is more authoritative, whom to believe in a conflict, and whether confidence is calibrated are generally weak.
4. Tool combination safety, dry-run, and transaction rollback: per-tool permissions exist, but the risk of tool combinations, pre-execution, idempotency, and rollback are not systematic.
5. OOD detection and calibration-driven refusal with human handoff: refusal exists, but much of it relies on keywords or thresholds rather than calibrated out-of-distribution detection.
6. Online closed-loop automatic diagnosis, strategy switching, and case feedback: monitoring exists, but automatically locating failures, switching strategies, rolling back, and feeding failure cases back into the eval or training set is rare.
7. Multi-model ensembles / debate / arbitration for high-risk cases: much research, little production use.
8. Data drift, embedding model versions, and input distribution monitoring: data governance exists, but drift monitoring and version linkage are not common.
9. Causal evaluation, real-world studies, and pharmacovigilance-style online safety monitoring: A/B testing exists, but causal inference and long-term safety signals are rare.
10. Chain of responsibility, auditing, and standardized model cards / data cards: big companies have them; the industry is not standardized.
11. Formal verification of critical paths: almost only in a few domains such as finance and aerospace.

---

6. Priorities Going Forward: Compress Tail Risk First, Then Raise the Ceiling

Following the order of “compress tail risk first, then raise the ceiling,” the recommendations are as follows.

P0: Do Immediately

1. Versioned policy packages + eval gates
   Version everything: all prompts, tool schemas, safety policies, knowledge bases, and model versions; changes must pass offline evals, red teaming, and regression tests, and must support canary rollout and rollback.
   Corresponds to TCM’s “classical priors + pharmacopoeia + versioned commentaries.”

2. Risk-tiered routing + hard constraints
   First determine the risk level and the domain, then choose the model, prompt, tools, and workflow. Key actions are backstopped by conventional code, rule engines, and constrained decoding.
   Corresponds to “syndrome-differentiation guidelines + formula–syndrome correspondence + contraindications.”

3. Tool sandbox + approval + rollback
   Least privilege, dry-run, transactions, idempotency, timeouts, maximum steps, kill switch; high-risk actions go to a human.
   Corresponds to “Eighteen Incompatibilities + follow-up visits + stop as soon as the disease is hit.”

P1: Do Next

4. Authority grading for multi-source evidence + conflict arbitration + confidence calibration
   Source grading, recency weighting, mandatory citations, voting / arbitration, calibration.
   Corresponds to “the four diagnostic methods used in combination.”

5. OOD detection + refusal with human handoff + online monitoring
   Hand off to a human when confidence is low, the input is out of distribution, risk is high, or the request exceeds capability; monitor and alert.
   Corresponds to “the six not treated + follow-up visits.”

6. Case feedback + regular updates
   Failure cases go into the eval set, are annotated by experts, and update few-shot examples, RAG, and fine-tuning.
   Corresponds to “mentorship + medical case records.”

P2: Do in the Medium Term

7. Multi-model ensembles / debate / arbitration for high-risk cases. Corresponds to “consultation.”
8. Data drift and embedding model version monitoring. Corresponds to “authentic regional herbs + processing.”
9. Audit chain of responsibility, model cards, data cards, and compliance. Corresponds to “pharmacopoeia + medical licensing.”
10. Formal verification of critical paths, causal evaluation, real-world studies. Corresponds to “modern reinforcement.”

If You Only Do Five Things

1. Versioning + eval gates;
2. Risk routing + hard rules;
3. Tool sandbox + approval + rollback;
4. Evidence grading + conflict arbitration + calibration;
5. OOD refusal + human handoff + case feedback.

These five correspond exactly to TCM’s “classical priors, syndrome differentiation, formula–syndrome, contraindications, the four diagnostic methods used in combination, the six not treated, and follow-up visits.”

---

7. Conclusion

The AI Harness cannot make a model deterministic, but it can keep the variance and tail risk of probabilistic generalization within an acceptable range.

TCM’s experiential engineering is a pre-modern version of governing a probabilistic system:

Strong priors, typed routing, multiple sources of evidence, hard contraindications, follow-up feedback, mentorship ensembles, data governance, institutional guardrails.

What the AI Harness should learn most is not to mystify TCM, but to translate this way of thinking into modern engineering language:

Deterministic shell + probabilistic core + feedback loop + governance and auditing.

Traditional software is like a machine that runs according to a blueprint; modern AI is like an expert system that has been trained.
And the job of the AI Harness is to make this probabilistic expert system, in the real world:
make fewer major mistakes, be monitorable, be correctable, be accountable, and be rollback-capable.

📚 References

Why The Harness Matters More Than The Model | YC Paper Club