TIES RAG Filter

›

Trust — S

S ∈ [0, 1]

Sensitivity — I

I ∈ (0, 1]

Environment — E

E ∈ [0, 1]

Minimum — R

R ∈ (0, 1]

0.95

0.45

— — —

TIES Evaluation Suite

Benchmark — Test TIES on your own data

Compare BM25 and TF-IDF retrieval with and without TIES filtering. Paste your text chunks and questions, then run. Everything runs locally in your browser.

BM25

A ranking algorithm used in search engines. It counts how often a word appears in a chunk and how rare that word is across the whole corpus. Rare words that appear many times in a chunk score higher. BM25 is practical and battle-tested for keyword-based retrieval.

Best Practices · Production Standard

TF-IDF

Term Frequency × Inverse Document Frequency. Similar idea to BM25: a word scores higher if it appears often in a chunk but rarely elsewhere. TF-IDF is simpler and older, but still widely used as a solid baseline for comparison.

Classic Baseline · Simple & Transparent

Corpus chunks

One chunk per line. Format: [id] text — IDs are assigned automatically if omitted.

Queries JSON

JSON array. Fields: id, text. Optional: relevant (array of chunk IDs) — required for recall & precision.

What do the metrics mean?

Recall ↑

relevant found ÷ total relevant

What fraction of the correct chunks were actually retrieved? 100% means nothing was missed. Higher is better.

Precision ↑

correct chunks ÷ retrieved chunks

Of everything retrieved, how much was actually useful? High precision means fewer irrelevant chunks reaching the AI.

F1 ↑

2 × (Recall × Precision) ÷ (Recall + Precision)

A single balanced score combining recall and precision. Useful when you care about both equally.

Efficiency ↑

F1 ÷ log₂(chunks + 2)

How much quality do you get per chunk sent to the AI? A method that achieves 80% F1 with 2 chunks is more efficient than one needing 10.

Chunks ↓

avg number retrieved

Fewer chunks means less noise for the AI and lower token cost. Lower is better — as long as recall stays acceptable.

Tokens ↓

estimated prompt tokens

Approximate cost per query sent to the language model. Calculated from word count × 1.3. Lower means cheaper API calls.

Token savings

Results by category

All methods

★ = best for this metric

TIES Model — Source Code

// TIES decision function — browser / JavaScript

function computeTrust(scores, idx, I, R, LAM = 0.30) {
  const mu = scores.reduce((a, b) => a + b, 0) / scores.length;
  const THR = 0.18;
  const seen = scores.slice(0, idx + 1);
  const win  = seen.map(s => Math.abs(s - mu) > THR ? 'D' : 'C');

  let wD = 0, wT = 0;
  win.forEach((v, i) => {
    const age = win.length - 1 - i;
    const w   = Math.exp(-LAM * age);
    if (v === 'D') wD += w;
    wT += w;
  });

  const S    = Math.max(0, 1 - wD / Math.max(wT, 1e-9));
  const E    = win.filter(x => x === 'C').length / win.length;
  const prod = S * I * E;

  return {
    S, E, prod,
    verdict: (win.length >= 2 && prod < R) ? 'REMOVE' : 'KEEP'
  };
}

# TIES decision function — Python

import math

def ties_decision(opp: list[float],
                   I: float = 0.80,
                   R: float = 0.40,
                   lam: float = 0.30) -> str:
    """
    Return 'C' (cooperate / keep passage) or 'D' (defect / remove passage).
    Mirrors the IPD-inspired trust scoring.
    """
    if not opp:
        return 'C'

    total   = len(opp)
    defects = opp.count('D')          

    if isinstance(opp[0], float):
        mu  = sum(opp) / total
        THR = 0.18
        win = ['D' if abs(s - mu) > THR else 'C' for s in opp]

        wD = wT = 0.0
        for i, v in enumerate(win):
            age = (total - 1) - i
            w   = math.exp(-lam * age)
            if v == 'D':
                wD += w
            wT += w

        S       = max(0.0, 1.0 - wD / max(wT, 1e-9))
        E       = win.count('C') / total
        prod    = S * I * E
        return 'D' if (total >= 2 and prod < R) else 'C'

    S    = 0.99 ** defects
    E    = (total - defects) / total
    if R <= (S * I * E):
        return 'C'
    return 'D'


# ── Example ─────────────────────────────────────────────────────
if __name__ == '__main__':
    scores = [0.88, 0.85, 0.82, 0.28, 0.80]  
    for idx in range(len(scores)):
        verdict = ties_decision(scores[:idx + 1], I=0.80, R=0.40)
        print(f"idx={idx}  score={scores[idx]:.2f}  → {verdict}")

Usage: pass a growing list of scores. Returns 'D' (remove) or 'C' (keep).

// BM25 retriever
function buildBM25(docs, k1=1.5, b=0.75) {
  const N=docs.length, tok=docs.map(d=>tokenize(d.text));
  const avgdl=tok.reduce((s,t)=>s+t.length,0)/N, df={};
  tok.forEach(ts=>[...new Set(ts)].forEach(w=>{df[w]=(df[w]||0)+1;}));
  const idf={};
  Object.entries(df).forEach(([w,f])=>{idf[w]=Math.log(((N-f+.5)/(f+.5))+1);});
  return q => tok.map((ts,i)=>{
    const tf={};ts.forEach(w=>{tf[w]=(tf[w]||0)+1;});
    const dl=ts.length;let sc=0;
    tokenize(q).forEach(w=>{if(!idf[w])return;
      const fw=tf[w]||0;sc+=idf[w]*(fw*(k1+1))/(fw+k1*(1-b+b*dl/avgdl));});
    return {id:docs[i].id,score:sc};
  }).sort((a,b)=>b.score-a.score);
}
// TF-IDF retriever
function buildTFIDF(docs) {
  const N=docs.length, tok=docs.map(d=>tokenize(d.text));
  const df={}; tok.forEach(ts=>[...new Set(ts)].forEach(w=>{df[w]=(df[w]||0)+1;}));
  const idf={}; Object.keys(df).forEach(w=>{idf[w]=Math.log((N+1)/(df[w]+1))+1;});
  function vec(ts){const tf={},len=ts.length||1;ts.forEach(w=>{tf[w]=(tf[w]||0)+1;});
    const v={};Object.keys(df).forEach(w=>{if(tf[w])v[w]=(tf[w]/len)*idf[w];});return v;}
  function cos(a,b){const na=norm(a),nb=norm(b);if(!na||!nb)return 0;
    let d=0;Object.keys(a).forEach(w=>{if(b[w])d+=a[w]*b[w];});return d/(na*nb);}
  const dvs=tok.map(t=>vec(t));
  return q=>{const qv=vec(tokenize(q));
    return dvs.map((dv,i)=>({id:docs[i].id,score:cos(qv,dv)})).sort((a,b)=>b.score-a.score);}
}