engine: hybrid · llm proposes → user reviews → math decides

Deterministic verdicts on top of a skeptical model.

MakeOrKillIt wraps every non-deterministic LLM call in a knockout gate, banded scoring, and a fixed aggregation rule. Same input plus same review yields the same verdict — the model reads and argues, the math decides.

> start_audit read the spec

engine.specmake/hold/kill

scoring_dimensions

6 weighted, Σ weights = 1.0

knockout_criteria

5 binary, must-meet

score_type

band [low, high] — width ∝ (1 − confidence)

thresholds

make_t = 65 · kill_t = 40 · width_t = 20

output

verdict + binding_constraints + experiments

the failure mode

mean() hides the fatal flaw.

Averaging one number per dimension lets a strong score cancel a disqualifying one. The engine separates fatal from weak instead of summing them on one scale.

[ naive ]

score → mean → verdict

market=9 cancels legality=1. One collapsed number, real blind spots.

[ engine ]

gate → band → aggregate

Fatal flaw caught pre-scoring. Survivors scored in confidence-weighted ranges.

the pipeline

Six functions, text in, verdict out.

LLM calls are non-deterministic; everything they touch is wrapped in a user-override plus deterministic band/aggregation layer.

extract_attributes

(text) → attrs, confidence

Pull structured attributes from free text. Never fabricates numbers; thin evidence lowers confidence.

knockout_gate

(attrs) → {killed, criteria}

Five binary must-meet checks. Any clear fail → kill, no scoring. null → asked, not guessed.

score_dimensions

(attrs) → [point, conf, band]

Per-dimension point + confidence. band = f(point, conf). Low conf falls to user review.

aggregate

(scores) → [low, high]

Conservative weighted sum of bounds → 0..100. Correlated uncertainty widens the band.

decide

(low, high) → verdict

Two thresholds + band width map to make / hold / kill.

explain

() → constraints, experiments

Binding constraints, cheapest uncertainty-reducing tests, and what flips the verdict.

banded scoring & the rule

Confidence is encoded as band width.

Guessing instead of reading drops confidence, which widens the band, which pushes the verdict toward hold. Uncertainty is priced in, not hidden behind a false point estimate.

derive_band(point, confidence):
    half = (1 - confidence) * MAX_HALFWIDTH   # 3.0
    return clamp(point - half, 0, 10),
           clamp(point + half, 0, 10)

decide(low, high):
    if low  >= 65:  # worst case still clears
        return "make"
    if high <= 40:  # best case still too weak
        return "kill"
    if (high - low) > 20:
        return "hold", "reduce_uncertainty"   # wide = unsure
    else:
        return "hold", "reshape"               # narrow = mediocre

DIMENSIONS[]

Six weighted axes, Σ = 1.0.

Weights are the initial config; the calibration loop re-fits them from real outcomes.

keyweightw

market_attractiveness0.22

differentiation0.20

financial_reward0.18

strategic_fit0.15

risk_inverse0.13

team_capability0.12

KNOCKOUTS[]

Five must-meet checks, evaluated first.

Any passed == false → instant kill. passed == null → asked, never guessed.

legal_ethical_feasibility

Lawful and ethical to build at all.

minimum_market

A meaningful floor of users exists.

technical_feasibility

Core build is reasonably possible.

strategic_alignment

Fits the stated mission and strategy.

resource_availability

Minimum resources are reachable.

adversarial: on

Every score ships with the strongest evidence against it.

The scoring model runs at low temperature and skeptical by default. Per dimension it emits disconfirming_evidence; for the whole idea it emits a kill_case — the most convincing argument for killing it, believed or not. Both are surfaced to you as the case against.

decide()

Two thresholds, a width, three regions.

make

low >= 65

Worst realistic case still clears the bar. Build it.

hold

band crosses a line

width > 20 → reduce_uncertainty. else → reshape.

kill

high <= 40

Best realistic case still too weak. Move on.

recalibrate()

The loop that keeps it honest.

Each verdict is stored as a PredictionRecord. Real results land as OutcomeRecords. Past MIN_N_RECAL = 30 paired outcomes, weights and thresholds are re-fit against reality — human-approved, versioned, reversible. Without it the engine is an opinion generator; with it, it earns the verdict.

Run an idea through the engine.

Paste the problem, the user, and the model. Gate, bands, and verdict resolve live.

> start_audit