{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b66c7871",
   "metadata": {},
   "source": [
    "# 02 · Búsqueda semántica y recuperación híbrida\n",
    "\n",
    "El módulo 01 terminó con BM25 fallando en preguntas que *significan* lo mismo que un pasaje pero usan palabras\n",
    "distintas. La solución es la **búsqueda semántica**: convertir cada texto en un vector de números (un **embedding**) de\n",
    "modo que los textos de significado parecido queden cerca, y buscar por distancia. Este módulo la construye, la mide\n",
    "frente a BM25 en tres tipos de pregunta y muestra un resultado que sorprende a muchos: **ninguno gana en todo, y una\n",
    "combinación ingenua 50/50 tampoco gana automáticamente.** Combinarlos (*recuperación híbrida*) es lo que usa el\n",
    "asistente de chat supervisado que se construye en este proyecto, pero los pesos importan, y solo se descubre midiendo.\n",
    "\n",
    "**Aprenderás**\n",
    "1. Qué es un embedding y cómo funciona la búsqueda por similitud (similitud del coseno)\n",
    "2. Ejecutar un modelo de embeddings pequeño en la CPU de un portátil\n",
    "3. Dónde la búsqueda semántica supera a BM25 (paráfrasis) y dónde pierde (códigos y nombres exactos)\n",
    "4. Reciprocal Rank Fusion, la forma simple y robusta de fusionar dos ordenaciones\n",
    "5. Calibrar cuándo negarse a responder con puntuaciones de embeddings\n",
    "6. Cuándo empieza a importar un *índice* vectorial (FAISS y similares), medido"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "976435e6",
   "metadata": {},
   "source": [
    "## 1. El corpus y la línea base BM25\n",
    "\n",
    "El mismo manual de Harbourview del módulo 01 (copiado aquí para que el notebook se baste solo) y el mismo BM25 que\n",
    "escribimos allí. Además de las 14 preguntas con respuesta del módulo 01, añadimos dos conjuntos nuevos con los que\n",
    "ningún método se ajustó:\n",
    "\n",
    "* **Paráfrasis**: diez preguntas escritas como las haría de verdad un compañero, que comparten pocas palabras con el\n",
    "  manual (\"Are boats cancelled when the sea is too rough?\").\n",
    "* **Identificadores**: cinco preguntas sobre códigos y nombres exactos (`PF-203`, `YW-110`, `OSS1`...)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1ba9fbfe",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:30.507143Z",
     "iopub.status.busy": "2026-11-17T21:22:30.506728Z",
     "iopub.status.idle": "2026-11-17T21:22:30.523395Z",
     "shell.execute_reply": "2026-11-17T21:22:30.521794Z"
    }
   },
   "outputs": [],
   "source": [
    "# El manual de Harbourview: 22 pasajes, ficticios, escritos para este curso\n",
    "PASSAGES = [\n",
    "    [\"p01\", \"Safety / Access\", \"Technicians must hold a valid GWO Basic Safety Training certificate before boarding a crew transfer vessel. Certificates are checked at the harbour gate and expire after two years.\"],\n",
    "    [\"p02\", \"Safety / Weather limits\", \"Crew transfer vessels do not sail when the significant wave height exceeds 1.5 m or the wind speed at the harbour exceeds 20 m/s. The duty coordinator decides at 06:00 and may cancel later.\"],\n",
    "    [\"p03\", \"Safety / Lone working\", \"Lone working inside a nacelle is forbidden. Every task inside a turbine needs at least two people, and both must carry a personal locator beacon.\"],\n",
    "    [\"p04\", \"Turbines / Specification\", \"Each of the 40 turbines is rated at 8.0 MW with a rotor diameter of 164 m. Cut-in wind speed is 3 m/s, rated speed is 12.5 m/s and cut-out speed is 25 m/s.\"],\n",
    "    [\"p05\", \"Turbines / Naming\", \"Turbines are named HV01 to HV40, numbered from the western edge of the array. The offshore substation is OSS1, in the centre.\"],\n",
    "    [\"p06\", \"Maintenance / Schedule\", \"Every turbine gets a planned service once a year between April and September. The service takes two technicians about two days, plus weather delays.\"],\n",
    "    [\"p07\", \"Maintenance / Gearbox oil\", \"Gearbox oil is sampled every six months. If the particle count exceeds ISO 4406 code 18/16/13, a filter change is scheduled within 14 days.\"],\n",
    "    [\"p08\", \"Maintenance / Blade inspection\", \"Blades are inspected by drone every 12 months and after any lightning alarm. Leading-edge erosion above grade 3 is repaired in the next planned service.\"],\n",
    "    [\"p09\", \"Faults / Alarm levels\", \"Alarms have three levels. Level 1 is informational. Level 2 needs a remote check within 4 hours. Level 3 stops the turbine and needs a site visit within 24 hours when the weather allows.\"],\n",
    "    [\"p10\", \"Faults / Pitch system\", \"A pitch system fault (code PF-203) usually means a low accumulator pressure. Reset it remotely once. If it returns within 48 hours, send a crew to replace the hydraulic accumulator.\"],\n",
    "    [\"p11\", \"Faults / Yaw misalignment\", \"A yaw error above 8 degrees for more than 10 minutes raises alarm YW-110. Persistent errors point to a faulty wind vane: the vane is recalibrated, then replaced if the error remains.\"],\n",
    "    [\"p12\", \"Faults / Grid loss\", \"On loss of grid connection the turbines stop and the converters isolate. Restart is allowed only when the grid operator confirms voltage within 5% of nominal for 10 minutes.\"],\n",
    "    [\"p13\", \"Environment / Marine mammals\", \"Piling and heavy cable work pause when a marine mammal is sighted within 500 m. Work resumes after 30 minutes without a sighting.\"],\n",
    "    [\"p14\", \"Environment / Bird collision\", \"Between October and March, turbines HV01 to HV08 are curtailed at night during the peak migration periods announced by the ornithology team.\"],\n",
    "    [\"p15\", \"Environment / Waste\", \"Used oil, filters and composite offcuts are brought ashore and logged in the waste register. Nothing is discharged at sea.\"],\n",
    "    [\"p16\", \"Logistics / Spare parts\", \"The harbour warehouse keeps the critical spares: two pitch accumulators, four wind vanes and a spare converter module. Reorder when stock falls below one unit.\"],\n",
    "    [\"p17\", \"Logistics / Helicopter\", \"A helicopter is used only for medical evacuation and for urgent Level 3 visits when vessels cannot sail. Booking goes through the duty coordinator.\"],\n",
    "    [\"p18\", \"Contacts / Duty coordinator\", \"The duty coordinator is reachable 24 hours a day on the control room line. Out-of-hours safety incidents are reported there first.\"],\n",
    "    [\"p19\", \"Reporting / Incidents\", \"Any injury or near miss is reported within one hour and a written report is filed within 24 hours. Environmental incidents follow the same deadlines.\"],\n",
    "    [\"p20\", \"Reporting / Performance\", \"Availability is reported monthly. The target is 97% time-based availability, counting weather downtime as available.\"],\n",
    "    [\"p21\", \"Grid / Export\", \"Power is exported at 66 kV through two cables to the onshore substation. The export limit agreed with the grid operator is 300 MW.\"],\n",
    "    [\"p22\", \"Training / Induction\", \"New staff complete a two-day induction covering the handbook, emergency drills and the sea survival course before their first transfer.\"],\n",
    "]\n",
    "\n",
    "# (id, pregunta, pasaje correcto, dato que debe contener la respuesta); None = el manual no lo dice\n",
    "QUESTIONS = [\n",
    "    [\"q01\", \"What wave height stops the crew boats from sailing?\", \"p02\", \"1.5 m\"],\n",
    "    [\"q02\", \"How often is the gearbox oil sampled?\", \"p07\", \"six months\"],\n",
    "    [\"q03\", \"Who may work alone inside a nacelle?\", \"p03\", \"forbidden\"],\n",
    "    [\"q04\", \"What is the rotor diameter of a turbine?\", \"p04\", \"164 m\"],\n",
    "    [\"q05\", \"What do I do if the pitch fault comes back after a reset?\", \"p10\", \"replace the hydraulic accumulator\"],\n",
    "    [\"q06\", \"When must a Level 3 alarm be attended on site?\", \"p09\", \"24 hours\"],\n",
    "    [\"q07\", \"How soon must an injury be reported?\", \"p19\", \"one hour\"],\n",
    "    [\"q08\", \"What is the largest amount of power the farm is allowed to send ashore?\", \"p21\", \"300 MW\"],\n",
    "    [\"q09\", \"Which turbines are switched off at night in the autumn to protect birds?\", \"p14\", \"HV01 to HV08\"],\n",
    "    [\"q10\", \"How many spare wind vanes are kept in stock?\", \"p16\", \"four wind vanes\"],\n",
    "    [\"q11\", \"What qualification do I need before getting on the boat?\", \"p01\", \"GWO Basic Safety Training\"],\n",
    "    [\"q12\", \"When can the turbines restart after a blackout?\", \"p12\", \"10 minutes\"],\n",
    "    [\"q13\", \"When can a helicopter be used?\", \"p17\", \"medical evacuation\"],\n",
    "    [\"q14\", \"What happens when a dolphin is seen near the piling work?\", \"p13\", \"pause\"],\n",
    "    [\"q15\", \"What is the price of electricity per MWh?\", None, None],\n",
    "    [\"q16\", \"Who is the chief executive of the operating company?\", None, None],\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "eaa640b7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:30.528087Z",
     "iopub.status.busy": "2026-11-17T21:22:30.527643Z",
     "iopub.status.idle": "2026-11-17T21:22:31.476234Z",
     "shell.execute_reply": "2026-11-17T21:22:31.473648Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22 passages; {'original (14)': 14, 'paraphrases (10)': 10, 'identifiers (5)': 5}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/patito/.local/lib/python3.12/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.\n",
      "  warnings.warn(\"Unable to import Axes3D. This may be due to multiple versions of \"\n"
     ]
    }
   ],
   "source": [
    "import re, math, time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "PARAPHRASES = [\n",
    "    (\"r01\", \"Are boats cancelled when the sea is too rough?\", \"p02\"),\n",
    "    (\"r02\", \"Can I go up the tower alone to fix something?\", \"p03\"),\n",
    "    (\"r03\", \"How big are the blades?\", \"p04\"),\n",
    "    (\"r04\", \"Who do I call in the middle of the night about an accident?\", \"p18\"),\n",
    "    (\"r05\", \"What if the oil in the gearbox gets dirty?\", \"p07\"),\n",
    "    (\"r06\", \"Do you stop work when whales or seals swim close?\", \"p13\"),\n",
    "    (\"r07\", \"How much spare equipment is kept on shore?\", \"p16\"),\n",
    "    (\"r08\", \"Is the farm allowed to produce at full output?\", \"p21\"),\n",
    "    (\"r09\", \"What should new colleagues do before their first trip?\", \"p22\"),\n",
    "    (\"r10\", \"How should rubbish and old oil be handled?\", \"p15\"),\n",
    "]\n",
    "IDENTIFIERS = [\n",
    "    (\"e01\", \"What does code PF-203 mean?\", \"p10\"),\n",
    "    (\"e02\", \"alarm YW-110\", \"p11\"),\n",
    "    (\"e03\", \"ISO 4406 18/16/13\", \"p07\"),\n",
    "    (\"e04\", \"Where is OSS1?\", \"p05\"),\n",
    "    (\"e05\", \"66 kV cables\", \"p21\"),\n",
    "]\n",
    "SETS = {\n",
    "    \"original (14)\": [(q[0], q[1], q[2]) for q in QUESTIONS if q[2]],\n",
    "    \"paraphrases (10)\": PARAPHRASES,\n",
    "    \"identifiers (5)\": IDENTIFIERS,\n",
    "}\n",
    "\n",
    "ids = [p[0] for p in PASSAGES]\n",
    "chunks = [f\"{section}. {text}\" for _, section, text in PASSAGES]\n",
    "\n",
    "STOP = set(\"a an and are as at be by can do does for from how i if in is it of on or that the this to what when which who with you\".split())\n",
    "def tokens(text):\n",
    "    return [w for w in re.findall(r\"[a-z0-9]+\", text.lower()) if w not in STOP]\n",
    "\n",
    "class BM25:\n",
    "    def __init__(self, docs, k1=1.5, b=0.75):\n",
    "        self.k1, self.b = k1, b\n",
    "        self.tf = [{} for _ in docs]\n",
    "        for i, d in enumerate(docs):\n",
    "            for t in tokens(d):\n",
    "                self.tf[i][t] = self.tf[i].get(t, 0) + 1\n",
    "        self.len = np.array([sum(tf.values()) for tf in self.tf], dtype=float)\n",
    "        self.avg = self.len.mean()\n",
    "        df = {}\n",
    "        for tf in self.tf:\n",
    "            for t in tf:\n",
    "                df[t] = df.get(t, 0) + 1\n",
    "        n = len(docs)\n",
    "        self.idf = {t: math.log(1 + (n - c + 0.5) / (c + 0.5)) for t, c in df.items()}\n",
    "    def scores(self, query):\n",
    "        s = np.zeros(len(self.tf))\n",
    "        for t in set(tokens(query)):\n",
    "            if t in self.idf:\n",
    "                for i, tf in enumerate(self.tf):\n",
    "                    f = tf.get(t, 0)\n",
    "                    if f:\n",
    "                        s[i] += self.idf[t] * f * (self.k1 + 1) / (f + self.k1 * (1 - self.b + self.b * self.len[i] / self.avg))\n",
    "        return s\n",
    "\n",
    "bm25 = BM25(chunks)\n",
    "print(len(PASSAGES), \"passages;\", {k: len(v) for k, v in SETS.items()})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24b520f2",
   "metadata": {},
   "source": [
    "## 2. Embeddings: el significado como geometría\n",
    "\n",
    "Un modelo de embeddings lee un texto y devuelve un vector de longitud fija (aquí, 384 números). Se entrenó para que las\n",
    "frases que significan lo mismo obtengan vectores que apuntan en la misma dirección. La similitud estándar es el\n",
    "**coseno** del ángulo entre dos vectores: 1 significa la misma dirección, 0 sin relación.\n",
    "\n",
    "Usamos `all-MiniLM-L6-v2`, un modelo de 22 millones de parámetros que corre cómodamente en una CPU y es lo bastante\n",
    "pequeño para incluirlo dentro de una aplicación. (Los modelos alojados más grandes, como Amazon Titan Text Embeddings,\n",
    "usados en otros proyectos de este portfolio, dan mejor calidad a un precio por token; el código que los rodea es\n",
    "idéntico.) La primera ejecución descarga unos 90 MB."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34c57fa9",
   "metadata": {},
   "source": [
    "### Imagen: el significado como geometría\n",
    "\n",
    "La misma pregunta se encuentra con dos pasajes. Una búsqueda por palabras clave no halla ninguna palabra en común\n",
    "con el correcto, así que lo coloca muy abajo. Un modelo de embeddings convierte cada texto en 384 números, y la\n",
    "pregunta queda mucho más cerca del pasaje que la responde que de uno que no tiene relación. Los valores de coseno\n",
    "los imprime la celda de más abajo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "eedfd280",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:31.480789Z",
     "iopub.status.busy": "2026-11-17T21:22:31.480237Z",
     "iopub.status.idle": "2026-11-17T21:22:31.512357Z",
     "shell.execute_reply": "2026-11-17T21:22:31.511110Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 800 330\" width=\"100%\" font-family=\"Helvetica, Arial, sans-serif\"><defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs><rect width=\"800\" height=\"330\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/><text x=\"400.0\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">Mismo significado, distintas palabras</text><rect x=\"20\" y=\"112\" width=\"200\" height=\"64\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"120.0\" y=\"131.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Pregunta</text><text x=\"120.0\" y=\"148.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">Are boats cancelled when</text><text x=\"120.0\" y=\"165.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">the sea is too rough?</text><line x1=\"220\" y1=\"144\" x2=\"256\" y2=\"144\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"256\" y=\"118\" width=\"150\" height=\"52\" rx=\"9\" fill=\"#dbeafe\" stroke=\"#3b82f6\" stroke-width=\"1.6\"/><text x=\"331.0\" y=\"140.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">all-MiniLM-L6-v2</text><text x=\"331.0\" y=\"157.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">texto a 384 números</text><line x1=\"406\" y1=\"138\" x2=\"470\" y2=\"76\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><text x=\"436\" y=\"100\" text-anchor=\"end\" font-size=\"12\" fill=\"#333\">coseno 0.45</text><line x1=\"406\" y1=\"150\" x2=\"470\" y2=\"212\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><text x=\"436\" y=\"204\" text-anchor=\"end\" font-size=\"12\" fill=\"#333\">coseno 0.07</text><rect x=\"470\" y=\"44\" width=\"310\" height=\"62\" rx=\"9\" fill=\"#dcfce7\" stroke=\"#16a34a\" stroke-width=\"1.6\"/><text x=\"625.0\" y=\"71.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">[p02] Safety / Weather limits</text><text x=\"625.0\" y=\"88.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">Crew transfer vessels do not sail...</text><rect x=\"470\" y=\"182\" width=\"310\" height=\"62\" rx=\"9\" fill=\"#f1f1f4\" stroke=\"#8a8aa0\" stroke-width=\"1.6\"/><text x=\"625.0\" y=\"209.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">[p07] Maintenance / Gearbox oil</text><text x=\"625.0\" y=\"226.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">Gearbox oil is sampled every six months</text><rect x=\"20\" y=\"262\" width=\"370\" height=\"52\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/><text x=\"30\" y=\"282\" font-size=\"12\" fill=\"#444\">Visión por palabras (BM25): la pregunta no comparte</text><text x=\"30\" y=\"298\" font-size=\"12\" fill=\"#444\">ninguna palabra con p02, así que p02 es 8.º de 22.</text><rect x=\"410\" y=\"262\" width=\"370\" height=\"52\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/><text x=\"420\" y=\"282\" font-size=\"12\" fill=\"#444\">Visión por embeddings: mismo significado, misma</text><text x=\"420\" y=\"298\" font-size=\"12\" fill=\"#444\">dirección, así que p02 es 1.º de 22.</text></svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import SVG, display\n",
    "import html as _h\n",
    "\n",
    "_COL = {  # fill, stroke  (light cards with dark text: readable in light and dark themes)\n",
    "    \"gold\": (\"#fff4c2\", \"#b89d28\"), \"blue\": (\"#dbeafe\", \"#3b82f6\"), \"green\": (\"#dcfce7\", \"#16a34a\"),\n",
    "    \"red\": (\"#fee2e2\", \"#dc2626\"), \"grey\": (\"#f1f1f4\", \"#8a8aa0\"), \"violet\": (\"#ede9fe\", \"#7c3aed\"),\n",
    "}\n",
    "\n",
    "def diagram(width, height, items, title=None):\n",
    "    # items: (\"box\", x, y, w, h, \"text\\nmore text\", colour) | (\"arrow\", x1, y1, x2, y2, \"label\") |\n",
    "    #        (\"text\", x, y, \"text\", size, \"start|middle|end\") | (\"note\", x, y, w, h, \"text\")\n",
    "    out = [f'<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 {width} {height}\" width=\"100%\" '\n",
    "           f'font-family=\"Helvetica, Arial, sans-serif\">',\n",
    "           '<defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" '\n",
    "           'orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs>',\n",
    "           f'<rect width=\"{width}\" height=\"{height}\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/>']\n",
    "    if title:\n",
    "        out.append(f'<text x=\"{width / 2}\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">{_h.escape(title)}</text>')\n",
    "    for it in items:\n",
    "        kind = it[0]\n",
    "        if kind == \"box\":\n",
    "            _, x, y, w, h, text, col = it\n",
    "            fill, stroke = _COL[col]\n",
    "            out.append(f'<rect x=\"{x}\" y=\"{y}\" width=\"{w}\" height=\"{h}\" rx=\"9\" fill=\"{fill}\" stroke=\"{stroke}\" stroke-width=\"1.6\"/>')\n",
    "            lines = text.split(\"\\n\")\n",
    "            y0 = y + h / 2 - (len(lines) - 1) * 8.5 + 4.5\n",
    "            for k, ln in enumerate(lines):\n",
    "                bold = ' font-weight=\"700\"' if k == 0 and len(lines) > 1 else \"\"\n",
    "                out.append(f'<text x=\"{x + w / 2}\" y=\"{y0 + k * 17}\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\"{bold}>{_h.escape(ln)}</text>')\n",
    "        elif kind == \"arrow\":\n",
    "            _, x1, y1, x2, y2, label = it\n",
    "            out.append(f'<line x1=\"{x1}\" y1=\"{y1}\" x2=\"{x2}\" y2=\"{y2}\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/>')\n",
    "            if label:\n",
    "                out.append(f'<text x=\"{(x1 + x2) / 2}\" y=\"{(y1 + y2) / 2 - 6}\" text-anchor=\"middle\" font-size=\"11.5\" fill=\"#555\">{_h.escape(label)}</text>')\n",
    "        elif kind == \"text\":\n",
    "            _, x, y, text, size, anchor = it\n",
    "            for k, ln in enumerate(text.split(\"\\n\")):\n",
    "                out.append(f'<text x=\"{x}\" y=\"{y + k * (size + 4)}\" text-anchor=\"{anchor}\" font-size=\"{size}\" fill=\"#333\">{_h.escape(ln)}</text>')\n",
    "        elif kind == \"note\":\n",
    "            _, x, y, w, h, text = it\n",
    "            out.append(f'<rect x=\"{x}\" y=\"{y}\" width=\"{w}\" height=\"{h}\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/>')\n",
    "            for k, ln in enumerate(text.split(\"\\n\")):\n",
    "                out.append(f'<text x=\"{x + 10}\" y=\"{y + 20 + k * 16}\" font-size=\"12\" fill=\"#444\">{_h.escape(ln)}</text>')\n",
    "    out.append(\"</svg>\")\n",
    "    display(SVG(\"\".join(out)))\n",
    "\n",
    "\n",
    "diagram(800, 330, [\n",
    "    (\"box\", 20, 112, 200, 64, \"Pregunta\\nAre boats cancelled when\\nthe sea is too rough?\", \"gold\"),\n",
    "    (\"arrow\", 220, 144, 256, 144, \"\"),\n",
    "    (\"box\", 256, 118, 150, 52, \"all-MiniLM-L6-v2\\ntexto a 384 números\", \"blue\"),\n",
    "    (\"arrow\", 406, 138, 470, 76, \"\"),\n",
    "    (\"text\", 436, 100, \"coseno 0.45\", 12, \"end\"),\n",
    "    (\"arrow\", 406, 150, 470, 212, \"\"),\n",
    "    (\"text\", 436, 204, \"coseno 0.07\", 12, \"end\"),\n",
    "    (\"box\", 470, 44, 310, 62, \"[p02] Safety / Weather limits\\nCrew transfer vessels do not sail...\", \"green\"),\n",
    "    (\"box\", 470, 182, 310, 62, \"[p07] Maintenance / Gearbox oil\\nGearbox oil is sampled every six months\", \"grey\"),\n",
    "    (\"note\", 20, 262, 370, 52, \"Visión por palabras (BM25): la pregunta no comparte\\nninguna palabra con p02, así que p02 es 8.º de 22.\"),\n",
    "    (\"note\", 410, 262, 370, 52, \"Visión por embeddings: mismo significado, misma\\ndirección, así que p02 es 1.º de 22.\"),\n",
    "], \"Mismo significado, distintas palabras\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "18ca25dc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:31.517713Z",
     "iopub.status.busy": "2026-11-17T21:22:31.517346Z",
     "iopub.status.idle": "2026-11-17T21:22:50.541491Z",
     "shell.execute_reply": "2026-11-17T21:22:50.539746Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/patito/.local/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "Loading weights: 100%|██████████| 103/103 [00:00<00:00, 3567.65it/s]"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_300328/4019971189.py:4: FutureWarning: The `get_sentence_embedding_dimension` method has been renamed to `get_embedding_dimension`.\n",
      "  print(f\"model loaded in {time.time() - t0:.1f} s, {sum(p.numel() for p in model.parameters()) / 1e6:.0f} M parameters, dimension {model.get_sentence_embedding_dimension()}\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "model loaded in 5.8 s, 23 M parameters, dimension 384\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "embedded 22 passages in 0.87 s -> matrix (22, 384), 16 KiB as float16\n",
      "cosine  0.45   'Are boats cancelled when the sea is too rough?' vs 'Crew transfer vessels do not sail when the significant wave '\n",
      "cosine  0.07   'Are boats cancelled when the sea is too rough?' vs 'Gearbox oil is sampled every six months'\n",
      "cosine  0.55   'What does code PF-203 mean?' vs 'A pitch system fault (code PF-203) usually means a low accum'\n"
     ]
    }
   ],
   "source": [
    "from sentence_transformers import SentenceTransformer\n",
    "t0 = time.time()\n",
    "model = SentenceTransformer(\"sentence-transformers/all-MiniLM-L6-v2\", device=\"cpu\")\n",
    "print(f\"model loaded in {time.time() - t0:.1f} s, {sum(p.numel() for p in model.parameters()) / 1e6:.0f} M parameters, dimension {model.get_sentence_embedding_dimension()}\")\n",
    "\n",
    "t0 = time.time()\n",
    "E = model.encode(chunks, normalize_embeddings=True)          # longitud unitaria, así que un producto escalar ES el coseno\n",
    "print(f\"embedded {len(chunks)} passages in {time.time() - t0:.2f} s -> matrix {E.shape}, {E.astype(np.float16).nbytes / 1024:.0f} KiB as float16\")\n",
    "\n",
    "def semantic_scores(query):\n",
    "    return E @ model.encode(query, normalize_embeddings=True)\n",
    "\n",
    "for a, b in [(\"Are boats cancelled when the sea is too rough?\", \"Crew transfer vessels do not sail when the significant wave height exceeds 1.5 m\"),\n",
    "             (\"Are boats cancelled when the sea is too rough?\", \"Gearbox oil is sampled every six months\"),\n",
    "             (\"What does code PF-203 mean?\", \"A pitch system fault (code PF-203) usually means a low accumulator pressure\")]:\n",
    "    va, vb = model.encode([a, b], normalize_embeddings=True)\n",
    "    print(f\"cosine {va @ vb:5.2f}   {a!r} vs {b[:60]!r}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da6c26c4",
   "metadata": {},
   "source": [
    "La paráfrasis y el pasaje que la responde puntúan claramente más alto que la paráfrasis y un pasaje sin relación, aunque\n",
    "casi no comparten palabras. Esa es toda la idea. Ahora, la prueba justa."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4225731e",
   "metadata": {},
   "source": [
    "## 3. Léxica frente a semántica, en tres tipos de pregunta\n",
    "\n",
    "Ejecutamos la misma evaluación del módulo 01 (hit@1, hit@3, MRR), ahora sobre tres conjuntos de preguntas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ff814d9d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:50.547200Z",
     "iopub.status.busy": "2026-11-17T21:22:50.546433Z",
     "iopub.status.idle": "2026-11-17T21:22:51.333226Z",
     "shell.execute_reply": "2026-11-17T21:22:51.331484Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "method                                     original (14)                  paraphrases (10)                   identifiers (5)\n",
      "                                    hit@1   hit@3    MRR              hit@1   hit@3    MRR              hit@1   hit@3    MRR\n",
      "BM25 (lexical)                  0.86    1.00    0.93          0.40    0.60    0.53          1.00    1.00    1.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MiniLM (semantic)               0.93    1.00    0.96          0.90    1.00    0.95          0.80    1.00    0.87\n"
     ]
    }
   ],
   "source": [
    "def ranks_of(scorer, questions):\n",
    "    out = []\n",
    "    for _, question, gold in questions:\n",
    "        order = [ids[i] for i in np.argsort(-scorer(question))]\n",
    "        out.append(order.index(gold) + 1)\n",
    "    return np.array(out)\n",
    "\n",
    "def report(scorers):\n",
    "    print(f\"{'method':22s}\" + \"\".join(f\"{name:>34s}\" for name in SETS))\n",
    "    print(f\"{'':22s}\" + f\"{'hit@1   hit@3    MRR':>34s}\" * len(SETS))\n",
    "    for label, scorer in scorers.items():\n",
    "        row = f\"{label:22s}\"\n",
    "        for questions in SETS.values():\n",
    "            r = ranks_of(scorer, questions)\n",
    "            row += f\"{(r == 1).mean():>14.2f}{(r <= 3).mean():>8.2f}{(1 / r).mean():>8.2f}\"\n",
    "        print(row)\n",
    "\n",
    "report({\"BM25 (lexical)\": bm25.scores, \"MiniLM (semantic)\": semantic_scores})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c01a836c",
   "metadata": {},
   "source": [
    "Mira las columnas, no el promedio. La búsqueda semántica debería ganar en las **paráfrasis**: para eso se construyó.\n",
    "Pero mira los **identificadores**: un modelo que aprendió *significado* no tiene motivo especial para tratar `PF-203` o\n",
    "`YW-110` como claves exactas, mientras que BM25 trata un token raro como la señal más fuerte posible. Veamos qué\n",
    "preguntas falla cada uno."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "87470ede",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:51.337264Z",
     "iopub.status.busy": "2026-11-17T21:22:51.336908Z",
     "iopub.status.idle": "2026-11-17T21:22:51.742669Z",
     "shell.execute_reply": "2026-11-17T21:22:51.741001Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "paraphrases (10)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  r01  BM25 rank  8 | semantic rank  1   Are boats cancelled when the sea is too rough?\n",
      "  r02  BM25 rank  3 | semantic rank  1   Can I go up the tower alone to fix something?\n",
      "  r03  BM25 rank  5 | semantic rank  1   How big are the blades?\n",
      "  r04  BM25 rank 18 | semantic rank  1   Who do I call in the middle of the night about an accident?\n",
      "  r08  BM25 rank 21 | semantic rank  1   Is the farm allowed to produce at full output?\n",
      "  r10  BM25 rank  2 | semantic rank  2   How should rubbish and old oil be handled?\n",
      "identifiers (5)\n",
      "  e03  BM25 rank  1 | semantic rank  3   ISO 4406 18/16/13\n"
     ]
    }
   ],
   "source": [
    "for name, questions in list(SETS.items())[1:]:\n",
    "    print(name)\n",
    "    for (qid, question, gold), rb, rs in zip(questions, ranks_of(bm25.scores, questions), ranks_of(semantic_scores, questions)):\n",
    "        if rb > 1 or rs > 1:\n",
    "            print(f\"  {qid}  BM25 rank {rb:2d} | semantic rank {rs:2d}   {question}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af4869eb",
   "metadata": {},
   "source": [
    "## 4. Recuperación híbrida con Reciprocal Rank Fusion\n",
    "\n",
    "Cada método falla en preguntas distintas, así que pregunta a ambos y fusiona las dos ordenaciones. La trampa es que las\n",
    "puntuaciones **no son comparables**: las de BM25 van de 0 a quizá 15, los cosenos de 0 a 1. Normalizarlas y sumarlas es\n",
    "frágil. **Reciprocal Rank Fusion (RRF)** esquiva las puntuaciones por completo y usa solo las *posiciones*:\n",
    "\n",
    "$$\\text{RRF}(d)=\\sum_{\\text{rankers}}\\frac{1}{k+\\text{rank}(d)}\\qquad(k=60\\text{ es la constante estándar})$$\n",
    "\n",
    "Un pasaje cercano a la cabeza de cualquiera de las listas obtiene una puntuación fusionada alta, y uno cercano a la\n",
    "cabeza de *ambas* obtiene la más alta. No necesita ajuste, ni entrenamiento, ni calibrar puntuaciones, y por eso es la\n",
    "receta híbrida por defecto en la mayoría de los sistemas de búsqueda."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e00e83dc",
   "metadata": {},
   "source": [
    "### Imagen: cómo dos clasificaciones se convierten en una\n",
    "\n",
    "La primera pregunta parafraseada va a los dos clasificadores. Cada uno informa del puesto del pasaje correcto (p02) y\n",
    "RRF convierte los dos puestos en una sola puntuación, con un peso para BM25. Las cuatro cajas de abajo son el\n",
    "resultado medido de cambiar ese peso, sobre las diez preguntas parafraseadas de la tabla que verás a continuación."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e2f165ef",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:51.747403Z",
     "iopub.status.busy": "2026-11-17T21:22:51.747087Z",
     "iopub.status.idle": "2026-11-17T21:22:51.762192Z",
     "shell.execute_reply": "2026-11-17T21:22:51.760292Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 800 340\" width=\"100%\" font-family=\"Helvetica, Arial, sans-serif\"><defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs><rect width=\"800\" height=\"340\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/><text x=\"400.0\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">Flujo de datos de la recuperación híbrida</text><rect x=\"20\" y=\"76\" width=\"190\" height=\"66\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"115.0\" y=\"96.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Pregunta</text><text x=\"115.0\" y=\"113.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">Are boats cancelled when</text><text x=\"115.0\" y=\"130.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">the sea is too rough?</text><line x1=\"210\" y1=\"98\" x2=\"270\" y2=\"66\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><line x1=\"210\" y1=\"122\" x2=\"270\" y2=\"154\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"270\" y=\"40\" width=\"200\" height=\"54\" rx=\"9\" fill=\"#dbeafe\" stroke=\"#3b82f6\" stroke-width=\"1.6\"/><text x=\"370.0\" y=\"63.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Clasificador BM25</text><text x=\"370.0\" y=\"80.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">pasaje correcto p02: puesto 8</text><rect x=\"270\" y=\"128\" width=\"200\" height=\"54\" rx=\"9\" fill=\"#dbeafe\" stroke=\"#3b82f6\" stroke-width=\"1.6\"/><text x=\"370.0\" y=\"151.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Clasificador semántico</text><text x=\"370.0\" y=\"168.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">pasaje correcto p02: puesto 1</text><line x1=\"470\" y1=\"67\" x2=\"540\" y2=\"100\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><text x=\"484\" y=\"60\" text-anchor=\"start\" font-size=\"12\" fill=\"#333\">w / (60+8)</text><line x1=\"470\" y1=\"155\" x2=\"540\" y2=\"122\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><text x=\"484\" y=\"178\" text-anchor=\"start\" font-size=\"12\" fill=\"#333\">1 / (60+1)</text><rect x=\"540\" y=\"70\" width=\"240\" height=\"76\" rx=\"9\" fill=\"#ede9fe\" stroke=\"#7c3aed\" stroke-width=\"1.6\"/><text x=\"660.0\" y=\"95.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Reciprocal Rank Fusion</text><text x=\"660.0\" y=\"112.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">puntuación = w / (60 + puesto)</text><text x=\"660.0\" y=\"129.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">sumada sobre los dos clasificadores</text><text x=\"400\" y=\"214\" text-anchor=\"middle\" font-size=\"13\" fill=\"#333\">Preguntas parafraseadas, hit@1, al cambiar el peso w de BM25</text><rect x=\"20\" y=\"228\" width=\"180\" height=\"62\" rx=\"9\" fill=\"#dcfce7\" stroke=\"#16a34a\" stroke-width=\"1.6\"/><text x=\"110.0\" y=\"246.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">w = 0</text><text x=\"110.0\" y=\"263.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">hit@1 0.90</text><text x=\"110.0\" y=\"280.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">(solo semántico)</text><rect x=\"220\" y=\"228\" width=\"180\" height=\"62\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"310.0\" y=\"255.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">w = 0.25</text><text x=\"310.0\" y=\"272.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">hit@1 0.60</text><rect x=\"420\" y=\"228\" width=\"180\" height=\"62\" rx=\"9\" fill=\"#fee2e2\" stroke=\"#dc2626\" stroke-width=\"1.6\"/><text x=\"510.0\" y=\"255.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">w = 0.5</text><text x=\"510.0\" y=\"272.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">hit@1 0.50</text><rect x=\"620\" y=\"228\" width=\"160\" height=\"62\" rx=\"9\" fill=\"#fee2e2\" stroke=\"#dc2626\" stroke-width=\"1.6\"/><text x=\"700.0\" y=\"255.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">w = 1 (igual)</text><text x=\"700.0\" y=\"272.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">hit@1 0.50</text><rect x=\"20\" y=\"302\" width=\"760\" height=\"30\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/><text x=\"30\" y=\"322\" font-size=\"12\" fill=\"#444\">Cada clasificador vota con sus puestos, así que el más débil en este tipo de pregunta puede arrastrar la fusión.</text></svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "diagram(800, 340, [\n",
    "    (\"box\", 20, 76, 190, 66, \"Pregunta\\nAre boats cancelled when\\nthe sea is too rough?\", \"gold\"),\n",
    "    (\"arrow\", 210, 98, 270, 66, \"\"),\n",
    "    (\"arrow\", 210, 122, 270, 154, \"\"),\n",
    "    (\"box\", 270, 40, 200, 54, \"Clasificador BM25\\npasaje correcto p02: puesto 8\", \"blue\"),\n",
    "    (\"box\", 270, 128, 200, 54, \"Clasificador semántico\\npasaje correcto p02: puesto 1\", \"blue\"),\n",
    "    (\"arrow\", 470, 67, 540, 100, \"\"),\n",
    "    (\"text\", 484, 60, \"w / (60+8)\", 12, \"start\"),\n",
    "    (\"arrow\", 470, 155, 540, 122, \"\"),\n",
    "    (\"text\", 484, 178, \"1 / (60+1)\", 12, \"start\"),\n",
    "    (\"box\", 540, 70, 240, 76, \"Reciprocal Rank Fusion\\npuntuación = w / (60 + puesto)\\nsumada sobre los dos clasificadores\", \"violet\"),\n",
    "    (\"text\", 400, 214, \"Preguntas parafraseadas, hit@1, al cambiar el peso w de BM25\", 13, \"middle\"),\n",
    "    (\"box\", 20, 228, 180, 62, \"w = 0\\nhit@1 0.90\\n(solo semántico)\", \"green\"),\n",
    "    (\"box\", 220, 228, 180, 62, \"w = 0.25\\nhit@1 0.60\", \"gold\"),\n",
    "    (\"box\", 420, 228, 180, 62, \"w = 0.5\\nhit@1 0.50\", \"red\"),\n",
    "    (\"box\", 620, 228, 160, 62, \"w = 1 (igual)\\nhit@1 0.50\", \"red\"),\n",
    "    (\"note\", 20, 302, 760, 30, \"Cada clasificador vota con sus puestos, así que el más débil en este tipo de pregunta puede arrastrar la fusión.\"),\n",
    "], \"Flujo de datos de la recuperación híbrida\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "99dfdb7f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:51.767249Z",
     "iopub.status.busy": "2026-11-17T21:22:51.766876Z",
     "iopub.status.idle": "2026-11-17T21:22:53.427062Z",
     "shell.execute_reply": "2026-11-17T21:22:53.425548Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "method                                     original (14)                  paraphrases (10)                   identifiers (5)\n",
      "                                    hit@1   hit@3    MRR              hit@1   hit@3    MRR              hit@1   hit@3    MRR\n",
      "BM25 (lexical)                  0.86    1.00    0.93          0.40    0.60    0.53          1.00    1.00    1.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MiniLM (semantic)               0.93    1.00    0.96          0.90    1.00    0.95          0.80    1.00    0.87\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hybrid (RRF)                    0.86    1.00    0.93          0.50    0.80    0.66          1.00    1.00    1.00\n"
     ]
    }
   ],
   "source": [
    "def rrf(scorers, k=60, weights=None):\n",
    "    weights = weights or [1.0] * len(scorers)\n",
    "    def fused(query):\n",
    "        total = np.zeros(len(ids))\n",
    "        for scorer, w in zip(scorers, weights):\n",
    "            order = np.argsort(-scorer(query))\n",
    "            for rank, i in enumerate(order, start=1):\n",
    "                total[i] += w / (k + rank)\n",
    "        return total\n",
    "    return fused\n",
    "\n",
    "hybrid = rrf([bm25.scores, semantic_scores])\n",
    "report({\"BM25 (lexical)\": bm25.scores, \"MiniLM (semantic)\": semantic_scores, \"Hybrid (RRF)\": hybrid})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7338423",
   "metadata": {},
   "source": [
    "Lee esta tabla con cuidado, porque no dice lo que dicen los manuales. En este corpus el híbrido 50/50 simple arregla los\n",
    "fallos de identificadores de la búsqueda semántica, pero en las paráfrasis **lo arrastra el ordenador más débil**: BM25\n",
    "pone pasajes equivocados arriba en esas preguntas y la fusión con pesos iguales le cree. La fusión ayuda cuando ambos\n",
    "ordenadores son aceptables y se equivocan en sitios *distintos*; no rescata a un ordenador que es simplemente malo en un\n",
    "tipo entero de pregunta. El mando que importa es el **peso**. Barrido:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "aa2eef33",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:53.431204Z",
     "iopub.status.busy": "2026-11-17T21:22:53.430786Z",
     "iopub.status.idle": "2026-11-17T21:22:55.835422Z",
     "shell.execute_reply": "2026-11-17T21:22:55.833724Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "weight of BM25 relative to the semantic ranker (semantic weight = 1):\n",
      "method                                     original (14)                  paraphrases (10)                   identifiers (5)\n",
      "                                    hit@1   hit@3    MRR              hit@1   hit@3    MRR              hit@1   hit@3    MRR\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BM25 weight 0                   0.93    1.00    0.96          0.90    1.00    0.95          0.80    1.00    0.87\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BM25 weight 0.25                0.93    1.00    0.96          0.60    1.00    0.78          1.00    1.00    1.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BM25 weight 0.5                 0.93    1.00    0.96          0.50    0.90    0.69          1.00    1.00    1.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BM25 weight 1                   0.86    1.00    0.93          0.50    0.80    0.66          1.00    1.00    1.00\n"
     ]
    }
   ],
   "source": [
    "print(\"weight of BM25 relative to the semantic ranker (semantic weight = 1):\")\n",
    "report({f\"BM25 weight {w:g}\": rrf([bm25.scores, semantic_scores], weights=[w, 1.0]) for w in (0, 0.25, 0.5, 1.0)})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00408c74",
   "metadata": {},
   "source": [
    "Bajar el peso de BM25 cambia una debilidad por la otra: con 0.25 las preguntas de identificadores vuelven a salir bien y\n",
    "el MRR de las paráfrasis se recupera en parte (0.78, frente a 0.66 con pesos iguales y 0.95 del ordenador semántico\n",
    "solo), pero ningún ajuste de aquí es el mejor en todos los conjuntos. Con solo 29 preguntas las diferencias entre\n",
    "números cercanos están dentro del ruido, así que no copies un peso de esta tabla: **ajústalo con tus propias preguntas**\n",
    "(el módulo 07 muestra cómo construir y dimensionar esa evaluación) y vuelve a comprobarlo cada vez que cambie el corpus\n",
    "o el modelo de embeddings."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5ebbea",
   "metadata": {},
   "source": [
    "## 5. Negarse a responder con puntuaciones de embeddings\n",
    "\n",
    "Con BM25, una puntuación de cero significaba \"ninguna palabra en común\". Las puntuaciones de coseno nunca llegan a cero\n",
    "(todo se parece un poco a todo), así que el umbral de rechazo hay que leerlo de los datos otra vez, usando las dos\n",
    "preguntas sin respuesta más algunas ajenas al tema."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ca89ba47",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:55.838883Z",
     "iopub.status.busy": "2026-11-17T21:22:55.838542Z",
     "iopub.status.idle": "2026-11-17T21:22:56.887658Z",
     "shell.execute_reply": "2026-11-17T21:22:56.885664Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAENCAYAAACSMPPWAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAANrZJREFUeJzt3Xl4VOX99/HPZA9JJgmRJUAgRIgEZBMKAipRoaJCxaICtYgWsCqIiChaF1aFKhSs268gJdaloCzWVgsoJdACIgpRljxJCEtEwioQwp6Z+/mDZspAtpmTzCTx/bquuWDOnDnne3/Pkvlk5kxsxhgjAAAAALAgwN8FAAAAAKj5CBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsCzI3wXgf5xOp/bt26eoqCjZbDZ/lwMAAICfOGOMTpw4oUaNGikgoOz3JAgW1ci+ffuUkJDg7zIAAAAAN99//72aNGlS5jwEi2okKipK0oUNZ7fb/VxN5XA6nTp06JDq1atXbspF5aDn/kHffY+e+wd99z167h/0/YKCggIlJCS4XqeWhWBRjRR//Mlut9eqYHHmzBnZ7faf9EHpS/TcP+i779Fz/6DvvkfP/YO+u6vIx/TpEgAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwLMjfBQCoPNn7C/TF1h/kPH1cAeFH1evqxkpuaPd3WR7L3l+gL7btU8GZ87KHBatXm0ZKbmgvdXp1VtNq9rZeT5/nyfy+6GFN206euHRsLRvYlXOgoNSxZu8v0MINu7Q9/7hsklLiozWwa3Ov+1G8vMz84zKSWnu5PF9vo6pcX2Ut++LlnC9ySpKCgwJ0vsipY6fOKf/4aUs9r8raK4un+7c3cg4U6MOv9ljehyuipP5KqlY9L4vNGGP8XQQuKCgoUHR0tI4fPy67vXruMJ5yOp06ePCg6tevr4AA3iCrKnuOFOp3izYpI++oggOk5Bgp+5h03il1bFpXL97VUc3iIv1dZrkuHkdggE02m2SM5HAaRYQE6eS5osumV5fxXbqvlzWW6lLzxbyt19PneTJ/efNOHdBeYUWnLJ1fatp28sRlY5NU5Pzfj/zAAEmyucY68uarNHPZNmXmF5S4vNaNojVjUGclxNap0Hl9z5FCPfHXr5WZf7zk5cVHa8bgzuX219fbqCrX5+2yyzq/BNgkZwVfyVW055VZe1W5tB7JyOH83+NBATYZWatv9+ECzf54g1buOiVnCY9b6eelSuqv02lc2zbAJgUE2PzSc09enxIsSmGz2bR06VL179+/xMd3796t5s2ba/PmzerQoUOlrJNgAW/sOVKowW+tUeGZIjmMUYD+FyyckgJtNkWGBemvD99QrV8gXTqOiqou47t4X//+6Kkyx1Jdai5WXu9Lq9fT53kyv6Ry540KC9Rb97TW1S2aeXV+8XbcNYGnx1OAVOILp0tFhgVp4cPXlxvo9hwp1D1vrlbhmaKylxcapA9H9iy1v77eRlW5PivL9uT8Up7yel7ZtVcFT/dvb+rbc6RQA99MV+Mwh+vnaUm86WdJ6/J0m/qy5568PuWVHlDD/W7RpjJPRg5jVHimSM8u2uzjyjxT3jhKUx3HV9O2ibf1evo8T+avyLwnzxTp3XU7KzrMy9S07eQJT4+nioQKSSo8U6TnF2dUeP3lLu9s2f319TaqyvVV1rK9PVcWK6/n3qzT18eKpz3wpr7fLdqkk2cc5c7nTT9LWpen27S6np9+UsHC4XDI6azo6ROo/rL3Fygj72i5JyOHMdqc96Oy95f8EQd/q+g4SlOdxpdzoGZtE2/3IU+f9/nWfR7NX7F5pZ2HCpVzwPMe1pZjpyRWj6fyZOQd1Q9HT5W7/ooqrb++3kZVub7KWnZFzy/l8aT+6naseLt/e1JfZe3DFWHleK2O5yePg8WyZct03XXXKSYmRnFxcerbt69yc3MlXfh4kM1m05IlS3TjjTeqTp06at++vdavX+96/p49e9SvXz/FxsYqIiJCbdq00WeffSZJ6ty5s2bMmOGat3///goODlZhYaEkae/evbLZbNqxY4ck6ezZsxo3bpwaN26siIgIde3aVenp6a7np6WlKSYmRp988olat26t0NBQ5eXlaePGjerdu7euuOIKRUdHq2fPntq0adNlY83Pz9ett96q8PBwJSUladGiRWX2ZuvWrbr11lsVGRmpBg0aaMiQITp8+HCp8589e1YFBQVuN+nCW5616WaM8XsNtfX2xdYfFBxw4UC++GbT5dOCAy5c/OXvmj0Zhye36jA+Y4xWbt1X4bFUh5o96f3F9Xr6vA++3Fnh+QNtUmAFt3ugTVq5Ld9n464Jt8o4nsrsuaRv846Uuf6Kbr/ibVhSf329japyfZWxbE/PL970vCYcK1b274rWd/E+XNLPUyv9rOzj1Vfnp4ry+FuhTp48qbFjx6pdu3YqLCzUCy+8oDvvvFMZGRmueZ599lnNmDFDLVu21LPPPqvBgwdrx44dCgoK0siRI3Xu3DmtWbNGERER2r59uyIjL3w2rGfPnkpPT9e4ceNkjNG///1vxcTE6D//+Y/69Omj1atXq3HjxmrRooUkadSoUdq+fbsWLFigRo0aaenSperTp4+2bNmili1bSpJOnTql3//+93r77bcVFxen+vXra+fOnRo6dKhee+01GWM0c+ZM3XbbbcrJyVFUVJRrHM8//7ymT5+uV199Ve+++64GDRqkLVu2KCUl5bK+HDt2TDfddJOGDx+uWbNm6fTp0xo/frzuuece/etf/yqxl9OmTdOkSZMum37o0CGdOXPG001TLTmdTh0/flzGGK6xqALO08eVHCO3C9ZskhpHXPj/xb//CLRJzlPHdPDgQR9WWDEljcNT/h5f8b7uPFNQ4bH4u2bJs95fXK+nz7PrVIXnt/333/J+f1e8rztPH/e4h96OuyaojOOpLIGSnGdP6eDBgyWe14vXX9Hfv9pUcn99vY2qcn1Wl+3N+aUspfW8JNXtWLGyf1e0vuJ1SCX/PL2UJ/0sbV3eblNf9PzEiRMVntfjYDFgwAC3+3/+859Vr149t4Awbtw43X777ZKkSZMmqU2bNtqxY4datWqlvLw8DRgwQG3btpUkJSUluZaVmpqqefPmyeFwaOvWrQoJCdHAgQOVnp6uPn36KD09XT179pQk5eXlaf78+crLy1OjRo1c6122bJnmz5+vl156SZJ0/vx5vfnmm2rfvr1rPTfddJPbGObMmaOYmBitXr1affv2dU2/++67NXz4cEnSlClT9Pnnn+u1117Tm2++eVlfXn/9dXXs2NG13uLeJCQkKDs7W8nJyZc955lnntHYsWNd9wsKCpSQkKB69erVqou3bTab6tWrR7CoAgHhR5V97IDbCam4yznH3D83HRggpdaJUf369X1YYcWUNA5P+Xt8xft6QJip8Fj8XbPkWe8vrtfT50XE1FH2sdMVCxYXvuCl3BemAZICbFLL8GiPe+jtuGuCyjieyhIoqU9onVIv3i5ef4WDhU26qYT++nobVeX6rC7bm/NLWUrreUmq27FiZf+uaH3F6yj+JcelP08v5Uk/S1uX18HCBz0PCwur8Lwev9LLycnR4MGDlZSUJLvdrsTEREkXXugXa9eunev/8fHxkuRKUqNHj9bUqVPVo0cPTZgwQd99951r3uuvv14nTpzQ5s2btXr1avXs2VOpqamujzetXr1aqampkqQtW7bI4XAoOTlZkZGRrtvq1atdH82SpJCQELd6JOnAgQMaMWKEWrZsqejoaNntdhUWFrqNQZK6det22f3MzMwS+/Ltt99q1apVbrW0atVKktzquVhoaKjsdrvbTZICAgJq1c1ms/m9htp663V1Y513XjjhXXwzunzaeafU++pGfq/Zk3F4cqsO47PZbLr56kYVHkt1qNmT3l9cr6fP+1W3pArP7zCSo4Lb3WGkXlfH+2zcNeFWGcdTmT2X1KFZXJnrr+j2K96GJfXX19uoKtdXGcv29PziTc9rwrFiZf+uaH0X78Ml/Ty10s/KPl59dX6qKI+DRb9+/fTjjz9q7ty52rBhgzZs2CBJOnfunGue4OBg1/9ttgt5r/jzWcOHD9fOnTs1ZMgQbdmyRZ07d9Zrr70mSYqJiVH79u2Vnp7uChE33HCDNm/erOzsbOXk5LjesSgsLFRgYKC++eYbZWRkuG6ZmZl69dVXXesPDw931VBs6NChysjI0Kuvvqp169YpIyNDcXFxbmPwVGFhofr16+dWS0ZGhnJycnTDDTd4vVygLMkN7erQNFaBl+zjlwoMsKlj07pq2cDuo8o8U9FxlKY6ja9lg5q1Tbzdhzx9Xu82jTyav2LzSkn1ItWivuc9rC3HTkmsHk/l6dA0Vo1i6pS7/ooqrb++3kZVub7KWnZFzy/l8aT+6naseLt/e1JfZe3DFWHleK2O5yePgsWRI0eUlZWl5557TjfffLNSUlJ09GjFr5ovlpCQoIceekhLlizRE088oblz57oe69mzp1atWqU1a9YoNTVVdevWVUpKil588UXFx8e7PlLUsWNHORwOHTx4UC1atHC7NWzYsMz1r127VqNHj9Ztt92mNm3aKDQ0tMSLrL/88svL7pd0fYUkXXPNNdq2bZsSExMvqyciIsLTFgEV9tJd1ygyLKjUk1JggE2RoUF68a6OPq7MM+WNozTVcXw1bZt4W6+nz/Nk/orMGxEapCHdk0p8vCJq2nbyhKfHU0AFD7vIsCBNGdChwusvd3nl9NfX26gq11dZy/b2XFnMm/qr27HiaQ+8qe+lu65RRFhgufNVxri92abV9fzkUbCIjY1VXFyc5syZox07duhf//qX2zUCFTFmzBgtX75cu3bt0qZNm7Rq1Sq3F+upqalavny5goKCXB8lSk1N1fvvv+96t0KSkpOTde+99+q+++7TkiVLtGvXLn311VeaNm2aPv300zJraNmypd59911lZmZqw4YNuvfeexUeHn7ZfB999JH+/Oc/Kzs7WxMmTNBXX32lUaNGlbjMkSNH6scff9TgwYO1ceNG5ebmavny5XrggQfkcJT/PciAt5rFReqvD9+gdgkXfrsSGPDfb9X579HdrklsjfgDX5ePw6agANt//6KqFBESVOL06ji+8sZS3Wr2tl5Pn+fJ/BWZ9/2Hrld9++Xn7qoed01Q2tguduE8cWFa+4S6evuBbmodH13qMls3itaHj/RU07rl96NZXKQ+fKSnWjcqY3nx0eX+YTFfb6OqXF9lLfvS5VQ0FEoV63lV1l5ZSqon8JI+WK2vWVykFjx8gxLqlv7unLf9LGldJfX34m0bYLM+Jl/w+C9vf/HFFxo9erR27typq666Sn/84x+VmpqqpUuXqkOHDpf9Nepjx44pNjZWq1atUmpqqh599FH985//1N69e2W329WnTx/NmjVLcXFxkqQff/xRV1xxhe655x4tWLBAkvTxxx/rzjvv1P/93//pt7/9rauW8+fPa+rUqfrLX/6iH374QVdccYWuvfZaTZo0SW3btlVaWprGjBmjY8eOuY1h8+bNevDBB7V161YlJCTopZde0rhx4zRmzBiNGTPmQmNsNr3xxhv6+OOPtWbNGsXHx+v3v/+97rnnHkkl/+XtnJwcjR8/XqtWrdLZs2fVrFkz9enTR3/4wx8u+zhWSfjL27Aqe3/Bha+dO3VMAXVi1PvqRtXqLdKKyt5foC+256vg9DnZw0PUu028Wjawlzq9OihtX6/ONZfE23o9fZ4n85c2b2WeX2radvLEpWNr2SBKOQdOlDrW7P0FWvjVLmXuOy7pQqAY2LW5ax5P+37p8lIaxWhQ10SP++vrbVSV6/N02RU5v5x3OGWTFBQYoPMOp46dOqf8Y6cled/zyqi9qnm6f3uiuO/HTZg+3LjH8j5cESX11xj5teeevD71OFig6hAsUBnouX/Qd9+j5/5B332PnvsHfb/Ak9enP90uAQAAAKg0BAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGAZwQIAAACAZQQLAAAAAJYRLAAAAABYRrAAAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWEawAAAAAGBZkL8LgP8V5mbrcPoXKio8oaDIKF2R2kuRVyb7u6wy+bLmmtif2oC+AxwHAGoWgsVP2Om9efp/U36ngi0ZUmCgbDabjDHaM+9N2dt2VKvnX1R4k6b+LtONL2uuif2pDeg7wHEAoGaqUR+FWrt2rdq2bavg4GD179+/1Gm+ZLPZ9PHHH/t8vVad3punTcMGqWDblgsTHA6ZoiLJ4ZAkFWz7TpuGDdLpvXl+rNKdL2uuif2pDeg7wHEAoOaqUcFi7Nix6tChg3bt2qW0tLRSp10qMTFRs2fPrpKa8vPzdeutt1bJsqvS/5vyOxWdPCk5HSXP4HSo6ORJ/b8pz/q2sDL4suaa2J/agL4DHAcAaq4aFSxyc3N10003qUmTJoqJiSl1mi81bNhQoaGhPl+vFYW52RfeXi/th1Yxp0MFWzarMDfbJ3WVxZc118T+1Ab0HeA4AFCzVZtgcfbsWY0ePVr169dXWFiYrrvuOm3cuFGStHv3btlsNh05ckS/+c1vZLPZlJaWVuK0S6WmpmrPnj16/PHHZbPZZLPZXI8tXrxYbdq0UWhoqBITEzVz5ky35yYmJmrKlCkaPHiwIiIi1LhxY73xxhtu81z6Uai9e/dq8ODBqlu3riIiItS5c2dt2LCh1DEXFBS43STJ6XRW6e1Q+hcyQcEytoDyb0HBOrR6paX1GWNqVM2+7k9V3Cqj576+0Xdu9LzmHAe1re814UbP6bs/bxVVbS7efuqpp7R48WK98847atasmV5++WXdcsst2rFjhxISEpSfn6+rrrpKkydP1sCBAxUVFaU+ffq4TYuOjr5suUuWLFH79u314IMPasSIEa7p33zzje655x5NnDhRAwcO1Lp16/TII48oLi5O999/v2u+V155Rb/73e80adIkLV++XI899piSk5PVu3fvy9ZVWFionj17qnHjxvrkk0/UsGFDbdq0qdQNMm3aNE2aNOmy6YcOHdKZM2e86GLFHHMYFSW1lBwV2FECA3SsyKnwgwe9WpfT6dTx48dljFFAgPc51pc1+3JdVaGyeu5r9B2eqo09rwnHQW3se3VHz/2Dvl9w4sSJCs9bLYLFyZMn9dZbbyktLc11vcLcuXP1+eefa968eXryySfVsGFD2Ww2RUdHq2HDhpKkiIiIy6Zdqm7dugoMDFRUVJTbPH/4wx9088036/nnn5ckJScna/v27XrllVfcgkWPHj309NNPu+ZZu3atZs2aVWKw+OCDD3To0CFt3LhRdevWlSS1aNGi1HE/88wzGjt2rOt+QUGBEhISVK9ePdnt9oq0ziunA206vjPHdSFgmQIDFXPDjapfv75X63I6nbLZbKpXr56lg9KXNftyXVWhsnrua/QdnqqNPa8Jx0Ft7Ht1R8/9g75fEBYWVuF5q0WwyM3N1fnz59WjRw/XtODgYHXp0kWZmZlVss7MzEzdcccdbtN69Oih2bNny+FwKDAwUJLUrVs3t3m6detW6oXgGRkZ6tixoytUlCc0NLTE6zMCAgKqdAeul9pLefPerNjMRU7V79nLUj02m83ymHxZs6/7UxUqo+e+Rt/hjdrW85pyHNS2vtcE9Nw/6Ls8GvtPt0tVIDw83N8lVEjklcmyt+0gBQSWPWNgoOxtOyriypY+qassvqy5JvanNqDvAMcBgJqtWgSLK6+8UiEhIVq7dq1r2vnz57Vx40a1bt3a8vJDQkLkuORt5ZSUFLf1SRf+JkZycrLr3QpJ+vLLL93m+fLLL5WSklLietq1a6eMjAz9+OOPlmuuaq2ef0lBERGl//AKDFRQnQi1ev5F3xZWBl/WXBP7UxvQd4DjAEDNVS2CRUREhB5++GE9+eSTWrZsmbZv364RI0bo1KlTGjZsmOXlJyYmas2aNfrhhx90+PBhSdITTzyhlStXasqUKcrOztY777yj119/XePGjXN77tq1a/Xyyy8rOztbb7zxhj766CM99thjJa5n8ODBatiwofr376+1a9dq586dWrx4sdavX295DJUtvElTXTNvgext2l2YEBgoW1CQ9N9QZW/dTtfMW1Ct/rKrL2uuif2pDeg7wHEAoOaqFtdYSNL06dPldDo1ZMgQnThxQp07d9by5csVGxtredmTJ0/Wb3/7W1155ZU6e/asjDG65ppr9OGHH+qFF17QlClTFB8fr8mTJ7tduC1dCCBff/21Jk2aJLvdrj/84Q+65ZZbSlxPSEiIVqxYoSeeeEK33XabioqK1Lp168u+ora6CG/SVB3/9K4Kc7N1ePVKFZ0oUFCUXfV69qq2b6/7suaa2J/agL4DHAcAaiabMcb4u4jqKjExUWPGjNGYMWN8sr6CggJFR0fr+PHjVfqtUL7kdDp18OBB1a9f/yd94ZMv0XP/oO++R8/9g777Hj33D/p+gSevT3+6XQIAAABQaQgWAAAAACyrNtdYVEe7d+/2dwkAAABAjcA7FgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAAAAACwjWAAAAACwjGABAAAAwDKCBQAAAADLCBYAAAAALAvydwH4H2OMJKmgoMDPlVQep9OpEydOKCwsTAEB5FhfoOf+Qd99j577B333PXruH/T9guLXpcWvU8tCsKhGTpw4IUlKSEjwcyUAAADA/5w4cULR0dFlzmMzFYkf8Amn06l9+/YpKipKNpvN3+VUioKCAiUkJOj777+X3W73dzk/CfTcP+i779Fz/6DvvkfP/YO+X2CM0YkTJ9SoUaNy37nhHYtqJCAgQE2aNPF3GVXCbrf/pA9Kf6Dn/kHffY+e+wd99z167h/0XeW+U1Hsp/uBMQAAAACVhmABAAAAwDKCBapUaGioJkyYoNDQUH+X8pNBz/2DvvsePfcP+u579Nw/6LvnuHgbAAAAgGW8YwEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAUveeOMNJSYmKiwsTF27dtVXX31V6rzbtm3TgAEDlJiYKJvNptmzZ/uu0FrGk77PnTtX119/vWJjYxUbG6tevXqVOT9K50nflyxZos6dOysmJkYRERHq0KGD3n33XR9WWzt40vOLLViwQDabTf3796/aAmspT/qelpYmm83mdgsLC/NhtbWDp/v6sWPHNHLkSMXHxys0NFTJycn67LPPfFRt7eFJ31NTUy/b1202m26//XYfVly9ESzgtYULF2rs2LGaMGGCNm3apPbt2+uWW27RwYMHS5z/1KlTSkpK0vTp09WwYUMfV1t7eNr39PR0DR48WKtWrdL69euVkJCgn//85/rhhx98XHnN5mnf69atq2effVbr16/Xd999pwceeEAPPPCAli9f7uPKay5Pe15s9+7dGjdunK6//nofVVq7eNN3u92u/Px8123Pnj0+rLjm87Tn586dU+/evbV7924tWrRIWVlZmjt3rho3buzjyms2T/u+ZMkSt/1869atCgwM1N133+3jyqsxA3ipS5cuZuTIka77DofDNGrUyEybNq3c5zZr1szMmjWrCqurvaz03RhjioqKTFRUlHnnnXeqqsRayWrfjTGmY8eO5rnnnquK8molb3peVFRkunfvbt5++20zdOhQc8cdd/ig0trF077Pnz/fREdH+6i62snTnr/11lsmKSnJnDt3zlcl1kpWz+uzZs0yUVFRprCwsKpKrHF4xwJeOXfunL755hv16tXLNS0gIEC9evXS+vXr/VhZ7VYZfT916pTOnz+vunXrVlWZtY7VvhtjtHLlSmVlZemGG26oylJrDW97PnnyZNWvX1/Dhg3zRZm1jrd9LywsVLNmzZSQkKA77rhD27Zt80W5tYI3Pf/kk0/UrVs3jRw5Ug0aNNDVV1+tl156SQ6Hw1dl13iV8fN03rx5GjRokCIiIqqqzBqHYAGvHD58WA6HQw0aNHCb3qBBA+3fv99PVdV+ldH38ePHq1GjRm4nU5TN274fP35ckZGRCgkJ0e23367XXntNvXv3rupyawVvev6f//xH8+bN09y5c31RYq3kTd+vuuoq/fnPf9bf/vY3vffee3I6nerevbv27t3ri5JrPG96vnPnTi1atEgOh0OfffaZnn/+ec2cOVNTp071Rcm1gtWfp1999ZW2bt2q4cOHV1WJNVKQvwsA4DvTp0/XggULlJ6ezsWVPhAVFaWMjAwVFhZq5cqVGjt2rJKSkpSamurv0mqdEydOaMiQIZo7d66uuOIKf5fzk9KtWzd169bNdb979+5KSUnRn/70J02ZMsWPldVeTqdT9evX15w5cxQYGKhOnTrphx9+0CuvvKIJEyb4u7yfhHnz5qlt27bq0qWLv0upVggW8MoVV1yhwMBAHThwwG36gQMHuDC7Clnp+4wZMzR9+nR98cUXateuXVWWWet42/eAgAC1aNFCktShQwdlZmZq2rRpBIsK8LTnubm52r17t/r16+ea5nQ6JUlBQUHKysrSlVdeWbVF1wKVcW4PDg5Wx44dtWPHjqoosdbxpufx8fEKDg5WYGCga1pKSor279+vc+fOKSQkpEprrg2s7OsnT57UggULNHny5KossUbio1DwSkhIiDp16qSVK1e6pjmdTq1cudLtN1eoXN72/eWXX9aUKVO0bNkyde7c2Rel1iqVtb87nU6dPXu2KkqsdTzteatWrbRlyxZlZGS4br/4xS904403KiMjQwkJCb4sv8aqjH3d4XBoy5Ytio+Pr6oyaxVvet6jRw/t2LHDFZ4lKTs7W/Hx8YSKCrKyr3/00Uc6e/asfv3rX1d1mTWPv68eR821YMECExoaatLS0sz27dvNgw8+aGJiYsz+/fuNMcYMGTLEPP300675z549azZv3mw2b95s4uPjzbhx48zmzZtNTk6Ov4ZQI3na9+nTp5uQkBCzaNEik5+f77qdOHHCX0OokTzt+0svvWRWrFhhcnNzzfbt282MGTNMUFCQmTt3rr+GUON42vNL8a1Q3vG075MmTTLLly83ubm55ptvvjGDBg0yYWFhZtu2bf4aQo3jac/z8vJMVFSUGTVqlMnKyjL/+Mc/TP369c3UqVP9NYQaydtzzHXXXWcGDhzo63JrBD4KBa8NHDhQhw4d0gsvvKD9+/erQ4cOWrZsmetCqLy8PAUE/O9NsX379qljx46u+zNmzNCMGTPUs2dPpaen+7r8GsvTvr/11ls6d+6c7rrrLrflTJgwQRMnTvRl6TWap30/efKkHnnkEe3du1fh4eFq1aqV3nvvPQ0cONBfQ6hxPO05KoenfT969KhGjBih/fv3KzY2Vp06ddK6devUunVrfw2hxvG05wkJCVq+fLkef/xxtWvXTo0bN9Zjjz2m8ePH+2sINZI355isrCz95z//0YoVK/xRcrVnM8YYfxcBAAAAoGbjVz0AAAAALCNYAAAAALCMYAEAAADAMoIFAAAAAMsIFgAAAAAsI1gAAAAAsIxgAQAAAMAyggUAAAAAywgWAGDBxIkT1aFDB7/WkJiYqNmzZ1taxqXjuP/++9W/f39Ly5SktLQ0xcTEWF6ON06dOqUBAwbIbrfLZrPp2LFjFX6uzWbTxx9/XKn1+LMXAOALBAsAsGDcuHFauXKlX2vYuHGjHnzwQUvLqKpxDBw4UNnZ2a77vgxi77zzjv79739r3bp1ys/PV3R09GXzVIdgWJkqI2QCgLeC/F0AANRkkZGRioyM9GsN9erVs7yMqhjH+fPnFR4ervDw8EpdbkXl5uYqJSVFV199tV/WDwA/NbxjAaDWczqdevnll9WiRQuFhoaqadOmevHFF12Pb9myRTfddJPCw8MVFxenBx98UIWFha7H09PT1aVLF0VERCgmJkY9evTQnj17JJX+EaIZM2YoPj5ecXFxGjlypM6fP++a5+zZsxo3bpwaN26siIgIde3aVenp6aXWb4zRxIkT1bRpU4WGhqpRo0YaPXq06/FLf0tts9n0pz/9SX379lWdOnWUkpKi9evXa8eOHUpNTVVERIS6d++u3Nxc13PK+839smXLdN111ykmJkZxcXHq27ev2/N3794tm82mhQsXqmfPngoLC9P777/v9vGftLQ0TZo0Sd9++61sNptsNpvS0tL0m9/8Rn379nVb3/nz51W/fn3Nmzev1JoWL16sNm3aKDQ0VImJiZo5c6brsdTUVM2cOVNr1qyRzWZTamrqZc8vrZ5ihw8f1p133qk6deqoZcuW+uSTT9yev3XrVt16662KjIxUgwYNNGTIEB0+fLjUeot9/PHHatmypcLCwnTLLbfo+++/d3v8b3/7m6655hqFhYUpKSlJkyZNUlFRkaSy94XU1FTt2bNHjz/+uGs8pbHZbHrrrbd06623Kjw8XElJSVq0aJHbPOPHj1dycrLq1KmjpKQkPf/882778bfffqsbb7xRUVFRstvt6tSpk77++mtJ0p49e9SvXz/FxsYqIiJCbdq00WeffSZJcjgcGjZsmJo3b67w8HBdddVVevXVV93WXVRUpNGjR7v2t/Hjx2vo0KFuH89zOp2aNm2aaznt27e/bAwAfMwAQC331FNPmdjYWJOWlmZ27Nhh/v3vf5u5c+caY4wpLCw08fHx5pe//KXZsmWLWblypWnevLkZOnSoMcaY8+fPm+joaDNu3DizY8cOs337dpOWlmb27NljjDFmwoQJpn379q51DR061NjtdvPQQw+ZzMxM8/e//93UqVPHzJkzxzXP8OHDTffu3c2aNWvMjh07zCuvvGJCQ0NNdnZ2ifV/9NFHxm63m88++8zs2bPHbNiwwW15zZo1M7NmzXLdl2QaN25sFi5caLKyskz//v1NYmKiuemmm8yyZcvM9u3bzbXXXmv69Onjek5J47jjjjtc9xctWmQWL15scnJyzObNm02/fv1M27ZtjcPhMMYYs2vXLiPJJCYmmsWLF5udO3eaffv2mfnz55vo6GhjjDGnTp0yTzzxhGnTpo3Jz883+fn55tSpU2bt2rUmMDDQ7Nu3z7W+JUuWmIiICHPixIkSe/L111+bgIAAM3nyZJOVlWXmz59vwsPDzfz5840xxhw5csSMGDHCdOvWzeTn55sjR45ctozS6inuYZMmTcwHH3xgcnJyzOjRo01kZKRrOUePHjX16tUzzzzzjMnMzDSbNm0yvXv3NjfeeGOJ9RpjzPz5801wcLDp3LmzWbdunfn6669Nly5dTPfu3V3zrFmzxtjtdpOWlmZyc3PNihUrTGJiopk4caIxpux94ciRI6ZJkyZm8uTJrvGURpKJi4szc+fONVlZWea5554zgYGBZvv27a55pkyZYtauXWt27dplPvnkE9OgQQPz+9//3vV4mzZtzK9//WuTmZlpsrOzzYcffmgyMjKMMcbcfvvtpnfv3ua7774zubm55u9//7tZvXq1McaYc+fOmRdeeMFs3LjR7Ny507z33numTp06ZuHCha5lT5061dStW9csWbLEZGZmmoceesjY7Xa3fXLq1KmmVatWZtmyZSY3N9fMnz/fhIaGmvT09FLHDaBqESwA1GoFBQUmNDTUFSQuNWfOHBMbG2sKCwtd0z799FMTEBBg9u/fb44cOWIklfpipaQX5M2aNTNFRUWuaXfffbcZOHCgMcaYPXv2mMDAQPPDDz+4Lefmm282zzzzTInrmDlzpklOTjbnzp0r8fGSgsVzzz3nur9+/XojycybN8817a9//asJCwsrcxwXv4i71KFDh4wks2XLFmPM/4LF7Nmz3ea7OFiUtJ5irVu3dnvR2q9fP3P//feXuv5f/epXpnfv3m7TnnzySdO6dWvX/ccee8z07Nmz1GWUVc+lPSwsLDSSzD//+U9jzIUX3T//+c/dnvP9998bSSYrK6vEdc2fP99IMl9++aVrWmZmppFkNmzYYIy5sB+89NJLbs979913TXx8vDHG832hNJLMQw895Data9eu5uGHHy71Oa+88orp1KmT635UVJRJS0srcd62bdu6wlBFjBw50gwYMMB1v0GDBuaVV15x3S8qKjJNmzZ17ZNnzpwxderUMevWrXNbzrBhw8zgwYMrvF4AlYuPQgGo1TIzM3X27FndfPPNpT7evn17RUREuKb16NFDTqdTWVlZqlu3ru6//37dcsst6tevn1599VXl5+eXuc42bdooMDDQdT8+Pl4HDx6UdOFjVw6HQ8nJya7rGiIjI7V69Wq3jxZd7O6779bp06eVlJSkESNGaOnSpa6PxpSmXbt2rv83aNBAktS2bVu3aWfOnFFBQUGZyymWk5OjwYMHKykpSXa7XYmJiZKkvLw8t/k6d+5coeVdavjw4Zo/f74k6cCBA/rnP/+p3/zmN6XOn5mZqR49erhN69Gjh3JycuRwOLyq4VIX9zAiIkJ2u921Hb/99lutWrXKbRu2atVKkkrdjpIUFBSkn/3sZ677rVq1UkxMjDIzM13LnTx5sttyR4wYofz8fJ06dcqrfaE03bp1u+x+cR2StHDhQvXo0UMNGzZUZGSknnvuObftPXbsWA0fPly9evXS9OnT3cY9evRoTZ06VT169NCECRP03Xffua3rjTfeUKdOnVSvXj1FRkZqzpw5rmUfP35cBw4cUJcuXVzzBwYGqlOnTq77O3bs0KlTp9S7d2+3Xv3lL38ps/8AqhbBAkCtVhkXDs+fP1/r169X9+7dtXDhQiUnJ+vLL78sdf7g4GC3+zabTU6nU5JUWFiowMBAffPNN8rIyHDdMjMzL/ucebGEhARlZWXpzTffVHh4uB555BHdcMMNbp93L6uG4s/alzStuK7y9OvXTz/++KPmzp2rDRs2aMOGDZKkc+fOuc13cUDzxH333aedO3dq/fr1eu+999S8eXNdf/31Xi2rspS3Hfv16+e2DTMyMpSTk6MbbrjB63UWFhZq0qRJbsvcsmWLcnJyFBYW5tW+4I3169fr3nvv1W233aZ//OMf2rx5s5599lm37T1x4kRt27ZNt99+u/71r3+pdevWWrp0qaQLQXHnzp0aMmSItmzZos6dO+u1116TJC1YsEDjxo3TsGHDtGLFCmVkZOiBBx64bF8qr0+S9Omnn7r1avv27VxnAfgRwQJArdayZUuFh4eX+lWqKSkp+vbbb3Xy5EnXtLVr1yogIEBXXXWVa1rHjh31zDPPaN26dbr66qv1wQcfeFVPx44d5XA4dPDgQbVo0cLt1rBhw1KfFx4ern79+umPf/yj0tPTtX79em3ZssWrGjx15MgRZWVl6bnnntPNN9+slJQUHT161KtlhYSElPiOQlxcnPr376/58+crLS1NDzzwQJnLSUlJ0dq1a92mrV27VsnJyW7vFnlbT3muueYabdu2TYmJiZdtx7LCVVFRkesCZ0nKysrSsWPHlJKS4lpuVlbWZcts0aKFAgIu/Mgua1/wZDyXhuMvv/zSVce6devUrFkzPfvss+rcubNatmzp+sKCiyUnJ+vxxx/XihUr9Mtf/tL1rpN0IRA/9NBDWrJkiZ544gnNnTtX0oXt1L17dz3yyCPq2LGjWrRo4fYuQ3R0tBo0aKCNGze6pjkcDm3atMl1v3Xr1goNDVVeXt5lfUpISKjQ+AFUPr5uFkCtFhYWpvHjx+upp55SSEiIevTooUOHDmnbtm0aNmyY7r33Xk2YMEFDhw7VxIkTdejQIT366KMaMmSIGjRooF27dmnOnDn6xS9+oUaNGikrK0s5OTm67777vKonOTlZ9957r+677z7NnDlTHTt21KFDh7Ry5Uq1a9dOt99++2XPSUtLk8PhUNeuXVWnTh299957Cg8PV7Nmzay2p0JiY2MVFxenOXPmKD4+Xnl5eXr66ae9WlZiYqJ27dqljIwMNWnSRFFRUQoNDZV04bfcffv2lcPh0NChQ8tczhNPPKGf/exnmjJligYOHKj169fr9ddf15tvvllp9ZRl5MiRmjt3rgYPHqynnnpKdevW1Y4dO7RgwQK9/fbbpYab4OBgPfroo/rjH/+ooKAgjRo1Stdee63rYz8vvPCC+vbtq6ZNm+quu+5SQECAvv32W23dulVTp04td19ITEzUmjVrNGjQIIWGhuqKK64odQwfffSROnfurOuuu07vv/++vvrqK9e3cLVs2VJ5eXlasGCBfvazn+nTTz91vRshSadPn9aTTz6pu+66S82bN9fevXu1ceNGDRgwQJI0ZswY3XrrrUpOTtbRo0e1atUqV2hp2bKl/vKXv2j58uVq3ry53n33XW3cuFHNmzd3Lf/RRx/VtGnT1KJFC7Vq1Uqvvfaajh496nqnLSoqSuPGjdPjjz8up9Op6667TsePH9fatWtlt9vL3X8AVBF/X+QBAFXN4XCYqVOnmmbNmpng4GDTtGlTtwtkv/vuO3PjjTeasLAwU7duXTNixAjXtxHt37/f9O/f38THx5uQkBDTrFkz88ILL7i+DakiFz1fehFx8bfiJCYmmuDgYBMfH2/uvPNO891335VY/9KlS03Xrl2N3W43ERER5tprrzVffPGF6/GSLt5eunSp637xhdWbN292TVu1apWRZI4ePVqhcXz++ecmJSXFhIaGmnbt2pn09HS39ZS0DmMuv3j7zJkzZsCAASYmJsZIcn2LkzHGOJ1O06xZM3PbbbeV2IdLLVq0yLRu3dq1TS++2NeYil28XVo9l/bQGGOio6Pd6s3OzjZ33nmniYmJMeHh4aZVq1ZmzJgxxul0lriu4l4sXrzYJCUlmdDQUNOrVy/XN4wVW7ZsmenevbsJDw83drvddOnSxfXNT+XtC+vXrzft2rUzoaGhpqwf8ZLMG2+8YXr37m1CQ0NNYmKi27cyGXPhYvi4uDgTGRlpBg4caGbNmuXalmfPnjWDBg0yCQkJJiQkxDRq1MiMGjXKnD592hhjzKhRo8yVV15pQkNDTb169cyQIUPM4cOHXT2///77TXR0tImJiTEPP/ywefrpp932v/Pnz5tRo0YZu91uYmNjzfjx483dd99tBg0a5JrH6XSa2bNnm6uuusoEBwebevXqmVtuucX17VMAfM9mjDH+CjUAABQrLCxU48aNNX/+fP3yl7/0dzm1ms1m09KlS93+LkR15nQ6lZKSonvuuUdTpkzxdzkASsFHoQAAfuV0OnX48GHNnDlTMTEx+sUvfuHvkuBne/bs0YoVK9SzZ0+dPXtWr7/+unbt2qVf/epX/i4NQBkIFgAAv8rLy1Pz5s3VpEkTpaWlKSiIH00/dQEBAUpLS9O4ceNkjNHVV1+tL774wnWdBoDqiY9CAQAAALCMr5sFAAAAYBnBAgAAAIBlBAsAAAAAlhEsAAAAAFhGsAAAAABgGcECAAAAgGUECwAAAACWESwAAAAAWPb/Aa7faPtR/YoKAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x280 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "answerable: lowest 0.20   off-topic: highest 0.46\n"
     ]
    }
   ],
   "source": [
    "off_topic = [\"What is the price of electricity per MWh?\", \"Who is the chief executive of the operating company?\",\n",
    "             \"What is the best pasta recipe?\", \"How do I reset my email password?\", \"Who won the football match yesterday?\"]\n",
    "best_off = [semantic_scores(q).max() for q in off_topic]\n",
    "best_on = [semantic_scores(q[1]).max() for qs in SETS.values() for q in qs]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8, 2.8))\n",
    "ax.scatter(best_on, np.ones(len(best_on)), s=60, color=\"#2a7ab0\", label=\"answerable\")\n",
    "ax.scatter(best_off, np.zeros(len(best_off)), s=60, color=\"#c0392b\", label=\"not in the handbook\")\n",
    "ax.set_yticks([0, 1], [\"off topic\", \"answerable\"]); ax.set_xlabel(\"cosine similarity of the best passage\"); ax.set_ylim(-.6, 1.6); ax.grid(alpha=.3)\n",
    "plt.tight_layout(); plt.show()\n",
    "print(f\"answerable: lowest {min(best_on):.2f}   off-topic: highest {max(best_off):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6204b32",
   "metadata": {},
   "source": [
    "Los dos grupos **se solapan** aquí (la puntuación más baja con respuesta es menor que la más alta fuera de tema), así\n",
    "que ningún umbral único es correcto: subirlo rechaza preguntas reales, bajarlo responde disparates. Una respuesta\n",
    "habitual en producción es *no decidir aquí*: mantener un umbral permisivo, dejar que el generador vea los pasajes y que\n",
    "el generador (y más tarde el supervisor del módulo 08) diga \"los documentos no contienen esto\". La recuperación\n",
    "propone; la etapa de respuesta y el evaluador deciden."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4160e84a",
   "metadata": {},
   "source": [
    "## 6. ¿Cuándo hace falta un índice vectorial?\n",
    "\n",
    "Comparamos el vector de la pregunta con los 22 vectores de pasajes, una búsqueda de *fuerza bruta*. Es exacta y, a este\n",
    "tamaño, instantánea. Existen bibliotecas como **FAISS** (usada por otro proyecto de este portfolio), pgvector o bases\n",
    "de datos vectoriales gestionadas para corpus donde la fuerza bruta se queda lenta: cambian un poco de exactitud por una\n",
    "búsqueda de vecinos más cercanos *aproximada* mucho más rápida. Mide antes de adoptar una:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2848452b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:22:56.892013Z",
     "iopub.status.busy": "2026-11-17T21:22:56.891631Z",
     "iopub.status.idle": "2026-11-17T21:22:58.282216Z",
     "shell.execute_reply": "2026-11-17T21:22:58.280808Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    1,000 vectors:    0.09 ms per brute-force query,      2 MB in float32\n",
      "   10,000 vectors:    1.22 ms per brute-force query,     15 MB in float32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  100,000 vectors:    7.64 ms per brute-force query,    154 MB in float32\n"
     ]
    }
   ],
   "source": [
    "rng = np.random.default_rng(0)\n",
    "dim = E.shape[1]\n",
    "for n in (1_000, 10_000, 100_000):\n",
    "    M = rng.standard_normal((n, dim)).astype(np.float32)\n",
    "    M /= np.linalg.norm(M, axis=1, keepdims=True)\n",
    "    q = M[0]\n",
    "    t0 = time.perf_counter()\n",
    "    for _ in range(20):\n",
    "        top = np.argpartition(-(M @ q), 5)[:5]\n",
    "    ms = (time.perf_counter() - t0) / 20 * 1000\n",
    "    print(f\"{n:>9,d} vectors: {ms:7.2f} ms per brute-force query, {M.nbytes / 1e6:6.0f} MB in float32\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c88244bf",
   "metadata": {},
   "source": [
    "La fuerza bruta escala de forma lineal: diez veces más vectores, diez veces más tiempo. Unos pocos miles de trozos son\n",
    "milisegundos, y una aplicación que sirva unos cientos de miles de trozos en una CPU sigue dentro de un presupuesto\n",
    "interactivo. La decisión de adoptar un índice aproximado depende, por tanto, del **tamaño del corpus y del volumen de\n",
    "consultas**, no de la moda; por debajo de unos cien mil trozos, una matriz de NumPy enviada como un solo archivo (en\n",
    "`float16` ocupa la mitad) es más simple de desplegar, probar y mantener correcta. Esa misma lógica hizo que dos\n",
    "componentes de recuperación de este portfolio eligieran FAISS en proceso o BM25 sin más, en vez de un servidor de base\n",
    "de datos siempre encendido.\n",
    "\n",
    "## Ejercicios\n",
    "\n",
    "1. Añade cinco paráfrasis tuyas y una pregunta de identificador. ¿Dónde falla cada método?\n",
    "2. Cambia la constante `k` de RRF a 5 y a 500. ¿Qué tan sensible es el híbrido?\n",
    "3. Dale a BM25 un peso de 2 en `rrf`. ¿Qué conjunto gana, cuál pierde, y cómo sabrías en producción hacia qué lado\n",
    "   girar el mando sin una batería etiquetada?\n",
    "4. Calcula el embedding de `section + text` frente a `text` solo. ¿Ayuda poner el título de la sección en el texto\n",
    "   incrustado?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
