{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9cdb8504",
   "metadata": {},
   "source": [
    "# 09 · Observabilidad: Langfuse y LangSmith\n",
    "\n",
    "Un sistema RAG que funciona el día que lo despliegas se desviará con el tiempo: los documentos cambian, los\n",
    "usuarios preguntan cosas nuevas, un proveedor cambia un modelo, alguien edita un prompt. Los módulos 07 y 08\n",
    "juzgan las respuestas por lotes. La **observabilidad** es la capa que está debajo y que hace posibles esos\n",
    "juicios: una **traza** de cada petición que registra qué se recuperó, qué prompt vio el modelo, qué respondió,\n",
    "cuántos tokens usó, cuánto tardó cada paso, cuánto costó y qué puntuaciones se le asociaron después.\n",
    "\n",
    "Este módulo construye desde cero un trazador diminuto que funciona sin conexión (para que veas cada pieza), lo\n",
    "equipara uno a uno con el vocabulario de **Langfuse** (la herramienta de código abierto para trazar LLM que se\n",
    "usa en este portafolio), muestra el código real de Langfuse en una celda opcional que **aquí no se ejecuta**, y\n",
    "después examina **LangSmith**, la otra herramienta por la que todo el mundo pregunta, y por qué no la necesitamos.\n",
    "\n",
    "**Aprenderás**\n",
    "1. Qué registrar en una aplicación con LLM, y el vocabulario: traza, tramo (span), generación, puntuación, sesión\n",
    "2. Un trazador basado en decoradores, con conteo de tokens y coste a partir de una tabla de precios\n",
    "3. Asociar a las trazas puntuaciones de estilo supervisor\n",
    "4. Leer un panel: coste por respuesta, percentiles de latencia, tokens, tasa de rechazo\n",
    "5. Encontrar las peores trazas, la verdadera recompensa de trazar\n",
    "6. El SDK real de Langfuse (con LiteLLM y LangChain) y cómo se compara LangSmith"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf69759a",
   "metadata": {},
   "source": [
    "## 1. Qué es una traza\n",
    "\n",
    "Piensa en una traza como la caja negra de una petición:\n",
    "\n",
    "| Término de Langfuse | Significado | En nuestro pipeline RAG |\n",
    "|---|---|---|\n",
    "| **Traza** (trace) | una petición de principio a fin | toda la llamada \"responde esta pregunta\" |\n",
    "| **Tramo** (span) | un paso cronometrado dentro de la traza | `retrieve`, `build_prompt` |\n",
    "| **Generación** (generation) | un tramo que es una llamada a un modelo: nombre del modelo, entrada, salida, uso de tokens, coste | `generate` |\n",
    "| **Puntuación** (score) | un número o etiqueta asociado a una traza o a un tramo | `correct`, `faithfulness`, `refused` |\n",
    "| **Sesión** (session) | un grupo de trazas de una misma conversación | cuatro preguntas seguidas de un usuario |\n",
    "\n",
    "Los tramos se anidan: la traza contiene el tramo de recuperación y la generación. Con esa estructura puedes\n",
    "responder preguntas que ninguna línea de log permite: *¿qué paso es lento?, ¿qué preguntas cuestan más?, ¿qué\n",
    "respuestas estaban mal respaldadas?, ¿qué vio exactamente el modelo cuando se equivocó en esta?*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "834663eb",
   "metadata": {},
   "source": [
    "### Imagen: anatomía de una traza\n",
    "\n",
    "Una petición es una **traza**. Dentro de ella, los pasos cronometrados son **tramos** (spans), y la llamada al\n",
    "modelo es un tramo especial llamado **generación**. Las puntuaciones las añade después otro proceso, y varias\n",
    "trazas de una conversación forman una **sesión**. Los números son los de la primera traza que registra este notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b6a07c8b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:00.660099Z",
     "iopub.status.busy": "2026-11-17T21:26:00.659615Z",
     "iopub.status.idle": "2026-11-17T21:26:00.708472Z",
     "shell.execute_reply": "2026-11-17T21:26:00.706607Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 840 360\" width=\"100%\" font-family=\"Helvetica, Arial, sans-serif\"><defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs><rect width=\"840\" height=\"360\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/><text x=\"420.0\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">Traza, tramo, generación, puntuación, sesión</text><rect x=\"20\" y=\"50\" width=\"560\" height=\"290\" rx=\"9\" fill=\"#f1f1f4\" stroke=\"#8a8aa0\" stroke-width=\"1.6\"/><text x=\"300.0\" y=\"199.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\"/><text x=\"40\" y=\"78\" text-anchor=\"start\" font-size=\"14\" fill=\"#333\">Traza: answer_question (una petición, 385 ms)</text><rect x=\"50\" y=\"100\" width=\"200\" height=\"70\" rx=\"9\" fill=\"#dbeafe\" stroke=\"#3b82f6\" stroke-width=\"1.6\"/><text x=\"150.0\" y=\"131.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Tramo: retrieve</text><text x=\"150.0\" y=\"148.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">3 ms</text><rect x=\"50\" y=\"200\" width=\"200\" height=\"110\" rx=\"9\" fill=\"#dcfce7\" stroke=\"#16a34a\" stroke-width=\"1.6\"/><text x=\"150.0\" y=\"234.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Generación: generate</text><text x=\"150.0\" y=\"251.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">modelo, entrada, salida</text><text x=\"150.0\" y=\"268.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">tokens ent./sal. 122/32</text><text x=\"150.0\" y=\"285.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">coste $0.0000150</text><rect x=\"290\" y=\"100\" width=\"270\" height=\"210\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/><text x=\"300\" y=\"120\" font-size=\"12\" fill=\"#444\">Los tramos se anidan en la traza.</text><text x=\"300\" y=\"136\" font-size=\"12\" fill=\"#444\">Con esta forma puedes preguntar:</text><text x=\"300\" y=\"152\" font-size=\"12\" fill=\"#444\">¿qué paso es lento?</text><text x=\"300\" y=\"168\" font-size=\"12\" fill=\"#444\">¿qué preguntas cuestan más?</text><text x=\"300\" y=\"184\" font-size=\"12\" fill=\"#444\">¿qué respuestas están mal</text><text x=\"300\" y=\"200\" font-size=\"12\" fill=\"#444\">respaldadas?</text><text x=\"300\" y=\"216\" font-size=\"12\" fill=\"#444\">¿qué vio el modelo cuando</text><text x=\"300\" y=\"232\" font-size=\"12\" fill=\"#444\">responde mal?</text><line x1=\"580\" y1=\"160\" x2=\"620\" y2=\"160\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><text x=\"600.0\" y=\"154.0\" text-anchor=\"middle\" font-size=\"11.5\" fill=\"#555\">luego</text><rect x=\"620\" y=\"90\" width=\"200\" height=\"140\" rx=\"9\" fill=\"#ede9fe\" stroke=\"#7c3aed\" stroke-width=\"1.6\"/><text x=\"720.0\" y=\"130.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Puntuaciones</text><text x=\"720.0\" y=\"147.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">añadidas tras la</text><text x=\"720.0\" y=\"164.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">petición por un juez:</text><text x=\"720.0\" y=\"181.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">correct, faithfulness,</text><text x=\"720.0\" y=\"198.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">refused</text><line x1=\"580\" y1=\"290\" x2=\"620\" y2=\"290\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"620\" y=\"260\" width=\"200\" height=\"70\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"720.0\" y=\"282.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Sesión</text><text x=\"720.0\" y=\"299.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">4 trazas de una</text><text x=\"720.0\" y=\"316.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">conversación de usuario</text></svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import SVG, display\n",
    "import html as _h\n",
    "\n",
    "_COL = {  # fill, stroke  (light cards with dark text: readable in light and dark themes)\n",
    "    \"gold\": (\"#fff4c2\", \"#b89d28\"), \"blue\": (\"#dbeafe\", \"#3b82f6\"), \"green\": (\"#dcfce7\", \"#16a34a\"),\n",
    "    \"red\": (\"#fee2e2\", \"#dc2626\"), \"grey\": (\"#f1f1f4\", \"#8a8aa0\"), \"violet\": (\"#ede9fe\", \"#7c3aed\"),\n",
    "}\n",
    "\n",
    "def diagram(width, height, items, title=None):\n",
    "    # items: (\"box\", x, y, w, h, \"text\\nmore text\", colour) | (\"arrow\", x1, y1, x2, y2, \"label\") |\n",
    "    #        (\"text\", x, y, \"text\", size, \"start|middle|end\") | (\"note\", x, y, w, h, \"text\")\n",
    "    out = [f'<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 {width} {height}\" width=\"100%\" '\n",
    "           f'font-family=\"Helvetica, Arial, sans-serif\">',\n",
    "           '<defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" '\n",
    "           'orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs>',\n",
    "           f'<rect width=\"{width}\" height=\"{height}\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/>']\n",
    "    if title:\n",
    "        out.append(f'<text x=\"{width / 2}\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">{_h.escape(title)}</text>')\n",
    "    for it in items:\n",
    "        kind = it[0]\n",
    "        if kind == \"box\":\n",
    "            _, x, y, w, h, text, col = it\n",
    "            fill, stroke = _COL[col]\n",
    "            out.append(f'<rect x=\"{x}\" y=\"{y}\" width=\"{w}\" height=\"{h}\" rx=\"9\" fill=\"{fill}\" stroke=\"{stroke}\" stroke-width=\"1.6\"/>')\n",
    "            lines = text.split(\"\\n\")\n",
    "            y0 = y + h / 2 - (len(lines) - 1) * 8.5 + 4.5\n",
    "            for k, ln in enumerate(lines):\n",
    "                bold = ' font-weight=\"700\"' if k == 0 and len(lines) > 1 else \"\"\n",
    "                out.append(f'<text x=\"{x + w / 2}\" y=\"{y0 + k * 17}\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\"{bold}>{_h.escape(ln)}</text>')\n",
    "        elif kind == \"arrow\":\n",
    "            _, x1, y1, x2, y2, label = it\n",
    "            out.append(f'<line x1=\"{x1}\" y1=\"{y1}\" x2=\"{x2}\" y2=\"{y2}\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/>')\n",
    "            if label:\n",
    "                out.append(f'<text x=\"{(x1 + x2) / 2}\" y=\"{(y1 + y2) / 2 - 6}\" text-anchor=\"middle\" font-size=\"11.5\" fill=\"#555\">{_h.escape(label)}</text>')\n",
    "        elif kind == \"text\":\n",
    "            _, x, y, text, size, anchor = it\n",
    "            for k, ln in enumerate(text.split(\"\\n\")):\n",
    "                out.append(f'<text x=\"{x}\" y=\"{y + k * (size + 4)}\" text-anchor=\"{anchor}\" font-size=\"{size}\" fill=\"#333\">{_h.escape(ln)}</text>')\n",
    "        elif kind == \"note\":\n",
    "            _, x, y, w, h, text = it\n",
    "            out.append(f'<rect x=\"{x}\" y=\"{y}\" width=\"{w}\" height=\"{h}\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/>')\n",
    "            for k, ln in enumerate(text.split(\"\\n\")):\n",
    "                out.append(f'<text x=\"{x + 10}\" y=\"{y + 20 + k * 16}\" font-size=\"12\" fill=\"#444\">{_h.escape(ln)}</text>')\n",
    "    out.append(\"</svg>\")\n",
    "    display(SVG(\"\".join(out)))\n",
    "\n",
    "\n",
    "diagram(840, 360, [\n",
    "    (\"box\", 20, 50, 560, 290, \"\", \"grey\"),\n",
    "    (\"text\", 40, 78, \"Traza: answer_question (una petición, 385 ms)\", 14, \"start\"),\n",
    "    (\"box\", 50, 100, 200, 70, \"Tramo: retrieve\\n3 ms\", \"blue\"),\n",
    "    (\"box\", 50, 200, 200, 110, \"Generación: generate\\nmodelo, entrada, salida\\ntokens ent./sal. 122/32\\ncoste $0.0000150\", \"green\"),\n",
    "    (\"note\", 290, 100, 270, 210, \"Los tramos se anidan en la traza.\\nCon esta forma puedes preguntar:\\n¿qué paso es lento?\\n¿qué preguntas cuestan más?\\n¿qué respuestas están mal\\nrespaldadas?\\n¿qué vio el modelo cuando\\nresponde mal?\"),\n",
    "    (\"arrow\", 580, 160, 620, 160, \"luego\"),\n",
    "    (\"box\", 620, 90, 200, 140, \"Puntuaciones\\nañadidas tras la\\npetición por un juez:\\ncorrect, faithfulness,\\nrefused\", \"violet\"),\n",
    "    (\"arrow\", 580, 290, 620, 290, \"\"),\n",
    "    (\"box\", 620, 260, 200, 70, \"Sesión\\n4 trazas de una\\nconversación de usuario\", \"gold\"),\n",
    "], \"Traza, tramo, generación, puntuación, sesión\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3e76449",
   "metadata": {},
   "source": [
    "## 2. Un trazador en cuarenta líneas\n",
    "\n",
    "Primero el pipeline que queremos observar: el manual de Harbourview, la recuperación BM25 y el mismo generador\n",
    "extractivo de sustitución del módulo 01 (en este notebook no se llama a ningún modelo). Después, el trazador.\n",
    "\n",
    "Como el generador es un sustituto, **la latencia aquí es simulada** con un reloj virtual (a una llamada a un\n",
    "modelo se le carga una sobrecarga fija más un tiempo por token de salida, con ruido con semilla). Lo que importa\n",
    "es la contabilidad, no la velocidad de un proveedor real, y un reloj virtual además hace que cada ejecución del\n",
    "notebook sea idéntica. Los tokens se estiman a 1,3 por palabra, la regla aproximada habitual para el inglés; un\n",
    "proveedor real devuelve los recuentos exactos en su respuesta y deberías registrar esos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "cb4ba2d1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:00.713425Z",
     "iopub.status.busy": "2026-11-17T21:26:00.712959Z",
     "iopub.status.idle": "2026-11-17T21:26:00.730721Z",
     "shell.execute_reply": "2026-11-17T21:26:00.728896Z"
    }
   },
   "outputs": [],
   "source": [
    "# The Harbourview handbook: 22 passages, fictional, written for this course\n",
    "PASSAGES = [\n",
    "    [\"p01\", \"Safety / Access\", \"Technicians must hold a valid GWO Basic Safety Training certificate before boarding a crew transfer vessel. Certificates are checked at the harbour gate and expire after two years.\"],\n",
    "    [\"p02\", \"Safety / Weather limits\", \"Crew transfer vessels do not sail when the significant wave height exceeds 1.5 m or the wind speed at the harbour exceeds 20 m/s. The duty coordinator decides at 06:00 and may cancel later.\"],\n",
    "    [\"p03\", \"Safety / Lone working\", \"Lone working inside a nacelle is forbidden. Every task inside a turbine needs at least two people, and both must carry a personal locator beacon.\"],\n",
    "    [\"p04\", \"Turbines / Specification\", \"Each of the 40 turbines is rated at 8.0 MW with a rotor diameter of 164 m. Cut-in wind speed is 3 m/s, rated speed is 12.5 m/s and cut-out speed is 25 m/s.\"],\n",
    "    [\"p05\", \"Turbines / Naming\", \"Turbines are named HV01 to HV40, numbered from the western edge of the array. The offshore substation is OSS1, in the centre.\"],\n",
    "    [\"p06\", \"Maintenance / Schedule\", \"Every turbine gets a planned service once a year between April and September. The service takes two technicians about two days, plus weather delays.\"],\n",
    "    [\"p07\", \"Maintenance / Gearbox oil\", \"Gearbox oil is sampled every six months. If the particle count exceeds ISO 4406 code 18/16/13, a filter change is scheduled within 14 days.\"],\n",
    "    [\"p08\", \"Maintenance / Blade inspection\", \"Blades are inspected by drone every 12 months and after any lightning alarm. Leading-edge erosion above grade 3 is repaired in the next planned service.\"],\n",
    "    [\"p09\", \"Faults / Alarm levels\", \"Alarms have three levels. Level 1 is informational. Level 2 needs a remote check within 4 hours. Level 3 stops the turbine and needs a site visit within 24 hours when the weather allows.\"],\n",
    "    [\"p10\", \"Faults / Pitch system\", \"A pitch system fault (code PF-203) usually means a low accumulator pressure. Reset it remotely once. If it returns within 48 hours, send a crew to replace the hydraulic accumulator.\"],\n",
    "    [\"p11\", \"Faults / Yaw misalignment\", \"A yaw error above 8 degrees for more than 10 minutes raises alarm YW-110. Persistent errors point to a faulty wind vane: the vane is recalibrated, then replaced if the error remains.\"],\n",
    "    [\"p12\", \"Faults / Grid loss\", \"On loss of grid connection the turbines stop and the converters isolate. Restart is allowed only when the grid operator confirms voltage within 5% of nominal for 10 minutes.\"],\n",
    "    [\"p13\", \"Environment / Marine mammals\", \"Piling and heavy cable work pause when a marine mammal is sighted within 500 m. Work resumes after 30 minutes without a sighting.\"],\n",
    "    [\"p14\", \"Environment / Bird collision\", \"Between October and March, turbines HV01 to HV08 are curtailed at night during the peak migration periods announced by the ornithology team.\"],\n",
    "    [\"p15\", \"Environment / Waste\", \"Used oil, filters and composite offcuts are brought ashore and logged in the waste register. Nothing is discharged at sea.\"],\n",
    "    [\"p16\", \"Logistics / Spare parts\", \"The harbour warehouse keeps the critical spares: two pitch accumulators, four wind vanes and a spare converter module. Reorder when stock falls below one unit.\"],\n",
    "    [\"p17\", \"Logistics / Helicopter\", \"A helicopter is used only for medical evacuation and for urgent Level 3 visits when vessels cannot sail. Booking goes through the duty coordinator.\"],\n",
    "    [\"p18\", \"Contacts / Duty coordinator\", \"The duty coordinator is reachable 24 hours a day on the control room line. Out-of-hours safety incidents are reported there first.\"],\n",
    "    [\"p19\", \"Reporting / Incidents\", \"Any injury or near miss is reported within one hour and a written report is filed within 24 hours. Environmental incidents follow the same deadlines.\"],\n",
    "    [\"p20\", \"Reporting / Performance\", \"Availability is reported monthly. The target is 97% time-based availability, counting weather downtime as available.\"],\n",
    "    [\"p21\", \"Grid / Export\", \"Power is exported at 66 kV through two cables to the onshore substation. The export limit agreed with the grid operator is 300 MW.\"],\n",
    "    [\"p22\", \"Training / Induction\", \"New staff complete a two-day induction covering the handbook, emergency drills and the sea survival course before their first transfer.\"],\n",
    "]\n",
    "\n",
    "# (id, question, gold passage, fact the answer must contain); None = the handbook does not say\n",
    "QUESTIONS = [\n",
    "    [\"q01\", \"What wave height stops the crew boats from sailing?\", \"p02\", \"1.5 m\"],\n",
    "    [\"q02\", \"How often is the gearbox oil sampled?\", \"p07\", \"six months\"],\n",
    "    [\"q03\", \"Who may work alone inside a nacelle?\", \"p03\", \"forbidden\"],\n",
    "    [\"q04\", \"What is the rotor diameter of a turbine?\", \"p04\", \"164 m\"],\n",
    "    [\"q05\", \"What do I do if the pitch fault comes back after a reset?\", \"p10\", \"replace the hydraulic accumulator\"],\n",
    "    [\"q06\", \"When must a Level 3 alarm be attended on site?\", \"p09\", \"24 hours\"],\n",
    "    [\"q07\", \"How soon must an injury be reported?\", \"p19\", \"one hour\"],\n",
    "    [\"q08\", \"What is the largest amount of power the farm is allowed to send ashore?\", \"p21\", \"300 MW\"],\n",
    "    [\"q09\", \"Which turbines are switched off at night in the autumn to protect birds?\", \"p14\", \"HV01 to HV08\"],\n",
    "    [\"q10\", \"How many spare wind vanes are kept in stock?\", \"p16\", \"four wind vanes\"],\n",
    "    [\"q11\", \"What qualification do I need before getting on the boat?\", \"p01\", \"GWO Basic Safety Training\"],\n",
    "    [\"q12\", \"When can the turbines restart after a blackout?\", \"p12\", \"10 minutes\"],\n",
    "    [\"q13\", \"When can a helicopter be used?\", \"p17\", \"medical evacuation\"],\n",
    "    [\"q14\", \"What happens when a dolphin is seen near the piling work?\", \"p13\", \"pause\"],\n",
    "    [\"q15\", \"What is the price of electricity per MWh?\", None, None],\n",
    "    [\"q16\", \"Who is the chief executive of the operating company?\", None, None],\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7f918011",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:00.735004Z",
     "iopub.status.busy": "2026-11-17T21:26:00.734599Z",
     "iopub.status.idle": "2026-11-17T21:26:01.664404Z",
     "shell.execute_reply": "2026-11-17T21:26:01.662355Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/patito/.local/lib/python3.12/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.\n",
      "  warnings.warn(\"Unable to import Axes3D. This may be due to multiple versions of \"\n"
     ]
    }
   ],
   "source": [
    "import re, math, json, random, contextvars, itertools, statistics\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# ---------- the pipeline to observe: BM25 retrieval + an extractive stand-in generator ----------\n",
    "STOP = set(\"a an and are as at be by can do does for from how i if in is it of on or that the this to what when which who with you\".split())\n",
    "tokens = lambda text: [w for w in re.findall(r\"[a-z0-9]+\", text.lower()) if w not in STOP]\n",
    "ids = [p[0] for p in PASSAGES]\n",
    "chunks = [f\"{section}. {text}\" for _, section, text in PASSAGES]\n",
    "tfs = [{} for _ in chunks]\n",
    "for i, c in enumerate(chunks):\n",
    "    for t in tokens(c):\n",
    "        tfs[i][t] = tfs[i].get(t, 0) + 1\n",
    "lens = np.array([sum(tf.values()) for tf in tfs], float)\n",
    "df = {}\n",
    "for tf in tfs:\n",
    "    for t in tf:\n",
    "        df[t] = df.get(t, 0) + 1\n",
    "idf = {t: math.log(1 + (len(chunks) - c + 0.5) / (c + 0.5)) for t, c in df.items()}\n",
    "\n",
    "def bm25_scores(query, k1=1.5, b=0.75):\n",
    "    s = np.zeros(len(chunks))\n",
    "    for t in set(tokens(query)):\n",
    "        for i, tf in enumerate(tfs):\n",
    "            f = tf.get(t, 0)\n",
    "            if f and t in idf:\n",
    "                s[i] += idf[t] * f * (k1 + 1) / (f + k1 * (1 - b + b * lens[i] / lens.mean()))\n",
    "    return s\n",
    "\n",
    "# ---------- a virtual clock, so latencies are reproducible ----------\n",
    "class Clock:\n",
    "    ms = 0.0\n",
    "    @classmethod\n",
    "    def advance(cls, ms):\n",
    "        cls.ms += ms\n",
    "rng = random.Random(7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "da64e41f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.668512Z",
     "iopub.status.busy": "2026-11-17T21:26:01.668040Z",
     "iopub.status.idle": "2026-11-17T21:26:01.685735Z",
     "shell.execute_reply": "2026-11-17T21:26:01.683291Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trazador listo\n"
     ]
    }
   ],
   "source": [
    "# ---------- the tracer ----------\n",
    "PRICES = {\"stand-in-small\": (0.06, 0.24)}      # ASSUMED USD per million input / output tokens: check your provider's price list\n",
    "OBS, TRACES = [], []                           # every observation, flat; every trace root\n",
    "_current = contextvars.ContextVar(\"current\", default=None)\n",
    "_ids = itertools.count(1)\n",
    "\n",
    "def observe(name=None, as_type=\"span\"):\n",
    "    # decorator: record a timed observation nested under whichever observation is running\n",
    "    def deco(fn):\n",
    "        def wrapper(*args, **kwargs):\n",
    "            parent = _current.get()\n",
    "            ob = dict(id=next(_ids), name=name or fn.__name__, type=as_type, parent=parent[\"id\"] if parent else None,\n",
    "                      start=Clock.ms, input=args[0] if args else None, usage=None, scores={}, session=None)\n",
    "            if parent is None:\n",
    "                TRACES.append(ob)\n",
    "            OBS.append(ob)\n",
    "            token = _current.set(ob)\n",
    "            try:\n",
    "                out = fn(*args, **kwargs)\n",
    "                ob[\"output\"] = out\n",
    "                return out\n",
    "            finally:\n",
    "                ob[\"end\"] = Clock.ms\n",
    "                _current.reset(token)\n",
    "        return wrapper\n",
    "    return deco\n",
    "\n",
    "def record_usage(model, text_in, text_out):\n",
    "    # attach token usage and cost to the running generation\n",
    "    ob = _current.get()\n",
    "    n_in, n_out = round(len(text_in.split()) * 1.3), round(len(text_out.split()) * 1.3)\n",
    "    price_in, price_out = PRICES[model]\n",
    "    ob[\"usage\"] = dict(model=model, input=n_in, output=n_out, cost=(n_in * price_in + n_out * price_out) / 1e6)\n",
    "    return n_out\n",
    "\n",
    "def score(trace, name, value):\n",
    "    trace[\"scores\"][name] = value\n",
    "\n",
    "def children(trace):\n",
    "    return [o for o in OBS if o[\"parent\"] == trace[\"id\"]]\n",
    "print(\"trazador listo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7d3da1d3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.689619Z",
     "iopub.status.busy": "2026-11-17T21:26:01.689225Z",
     "iopub.status.idle": "2026-11-17T21:26:01.707194Z",
     "shell.execute_reply": "2026-11-17T21:26:01.705698Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "traza 1 answer_question 385 ms\n",
      "  tramo retrieve  [span]      3 ms\n",
      "  tramo generate  [generation]    382 ms  model=stand-in-small tokens in/out=122/32 cost=$0.0000150\n"
     ]
    }
   ],
   "source": [
    "THRESHOLD = 1.0\n",
    "\n",
    "@observe(\"retrieve\")\n",
    "def retrieve(question, k=2):\n",
    "    Clock.advance(2 + rng.random() * 3)                       # a lexical search takes a few ms\n",
    "    s = bm25_scores(question)\n",
    "    return [(ids[i], float(s[i])) for i in np.argsort(-s)[:k]]\n",
    "\n",
    "@observe(\"generate\", as_type=\"generation\")\n",
    "def generate(question, hits):\n",
    "    if hits[0][1] < THRESHOLD:\n",
    "        text, prompt = \"I cannot find that in the handbook.\", question\n",
    "    else:\n",
    "        passages = [chunks[ids.index(pid)] for pid, _ in hits]\n",
    "        prompt = \"Answer only from these passages and cite them.\\n\" + \"\\n\".join(passages) + \"\\nQuestion: \" + question\n",
    "        section, body = passages[0].split(\". \", 1)\n",
    "        q = set(tokens(question))\n",
    "        text = max(re.split(r\"(?<=[.!?])\\s+\", body), key=lambda s: len(q & set(tokens(s)))) + \" [1]\"\n",
    "    n_out = record_usage(\"stand-in-small\", prompt, text)\n",
    "    Clock.advance(120 + 8 * n_out + rng.random() * 40)         # simulated model latency\n",
    "    return text\n",
    "\n",
    "@observe(\"answer_question\")\n",
    "def answer_question(question):\n",
    "    hits = retrieve(question)\n",
    "    return generate(question, hits)\n",
    "\n",
    "t = answer_question(\"What wave height stops the crew boats from sailing?\")\n",
    "trace = TRACES[-1]\n",
    "print(\"traza\", trace[\"id\"], trace[\"name\"], f\"{trace['end'] - trace['start']:.0f} ms\")\n",
    "for o in children(trace):\n",
    "    extra = f\"  model={o['usage']['model']} tokens in/out={o['usage']['input']}/{o['usage']['output']} cost=${o['usage']['cost']:.7f}\" if o[\"usage\"] else \"\"\n",
    "    print(f\"  tramo {o['name']:9s} [{o['type']}] {o['end'] - o['start']:6.0f} ms{extra}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e024c40b",
   "metadata": {},
   "source": [
    "Ese es todo el mecanismo: un decorador abre una observación, el anidamiento sigue la pila de llamadas a través\n",
    "de una variable de contexto y la generación lleva un registro de uso. Langfuse almacena exactamente esta forma\n",
    "(más una interfaz web) y su decorador `@observe` se usa igual, como verás en la sección 6.\n",
    "\n",
    "## 3. Tráfico real y puntuaciones asociadas a las trazas\n",
    "\n",
    "Una traza no te dice nada del sistema; cientos, sí. Simulamos ochenta peticiones en veinte sesiones de cuatro:\n",
    "las 16 preguntas del manual más 6 paráfrasis, muestreadas al azar. Una vez que existe cada traza, un **juez de\n",
    "estilo supervisor** (la idea del módulo 08, aquí en una versión barata basada en reglas) le asocia puntuaciones:\n",
    "\n",
    "* `correct`: 1 si la respuesta contiene el dato requerido, 0 si no (y 1 para un rechazo correcto)\n",
    "* `faithfulness`: la proporción de palabras de contenido de la respuesta que aparecen en los pasajes recuperados\n",
    "* `refused`: 1 si el sistema dijo que no encontraba la respuesta\n",
    "\n",
    "Las puntuaciones se asocian **después** de la petición, por otro proceso, en otro momento. Por eso las trazas y\n",
    "las puntuaciones son objetos distintos: el usuario ya tiene su respuesta y el veredicto de calidad llega más tarde."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7292008",
   "metadata": {},
   "source": [
    "### Imagen: cómo fluyen los datos en este módulo\n",
    "\n",
    "Una petición recorre el pipeline con trazado. El trazador registra la traza, un juez añade puntuaciones después,\n",
    "y el panel y la lista de peores trazas se calculan solo a partir de las trazas almacenadas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "93669b16",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.712876Z",
     "iopub.status.busy": "2026-11-17T21:26:01.712466Z",
     "iopub.status.idle": "2026-11-17T21:26:01.729748Z",
     "shell.execute_reply": "2026-11-17T21:26:01.727902Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 840 455\" width=\"100%\" font-family=\"Helvetica, Arial, sans-serif\"><defs><marker id=\"ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path d=\"M0,0 L10,5 L0,10 z\" fill=\"#555\"/></marker></defs><rect width=\"840\" height=\"455\" rx=\"12\" fill=\"#fbfbf8\" stroke=\"#d8d8cf\"/><text x=\"420.0\" y=\"26\" text-anchor=\"middle\" font-size=\"15\" font-weight=\"700\" fill=\"#222\">Flujo de datos: petición, traza, puntuaciones, panel</text><rect x=\"20\" y=\"80\" width=\"150\" height=\"60\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"95.0\" y=\"106.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Petición</text><text x=\"95.0\" y=\"123.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">una pregunta</text><line x1=\"170\" y1=\"110\" x2=\"200\" y2=\"110\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"200\" y=\"45\" width=\"375\" height=\"130\" rx=\"9\" fill=\"#dbeafe\" stroke=\"#3b82f6\" stroke-width=\"1.6\"/><text x=\"387.5\" y=\"114.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\"/><text x=\"387\" y=\"67\" text-anchor=\"middle\" font-size=\"13\" fill=\"#333\">Pipeline bajo @observe</text><rect x=\"215\" y=\"90\" width=\"110\" height=\"70\" rx=\"9\" fill=\"#f1f1f4\" stroke=\"#8a8aa0\" stroke-width=\"1.6\"/><text x=\"270.0\" y=\"121.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">retrieve</text><text x=\"270.0\" y=\"138.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">3 ms</text><rect x=\"345\" y=\"90\" width=\"215\" height=\"70\" rx=\"9\" fill=\"#dcfce7\" stroke=\"#16a34a\" stroke-width=\"1.6\"/><text x=\"452.5\" y=\"112.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">generate (llamada al modelo)</text><text x=\"452.5\" y=\"129.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">382 ms, tokens 122/32</text><text x=\"452.5\" y=\"146.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">coste $0.0000150</text><line x1=\"575\" y1=\"110\" x2=\"620\" y2=\"110\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"620\" y=\"65\" width=\"200\" height=\"90\" rx=\"9\" fill=\"#f1f1f4\" stroke=\"#8a8aa0\" stroke-width=\"1.6\"/><text x=\"720.0\" y=\"97.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Almacén de trazas</text><text x=\"720.0\" y=\"114.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">80 trazas</text><text x=\"720.0\" y=\"131.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">240 observaciones</text><line x1=\"720\" y1=\"155\" x2=\"720\" y2=\"205\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"620\" y=\"205\" width=\"200\" height=\"90\" rx=\"9\" fill=\"#ede9fe\" stroke=\"#7c3aed\" stroke-width=\"1.6\"/><text x=\"720.0\" y=\"229.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Puntuaciones</text><text x=\"720.0\" y=\"246.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">añadidas después:</text><text x=\"720.0\" y=\"263.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">correct, faithfulness,</text><text x=\"720.0\" y=\"280.0\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">refused</text><line x1=\"620\" y1=\"250\" x2=\"580\" y2=\"250\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"270\" y=\"205\" width=\"310\" height=\"130\" rx=\"9\" fill=\"#fff4c2\" stroke=\"#b89d28\" stroke-width=\"1.6\"/><text x=\"425.0\" y=\"240.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Panel (solo con las trazas)</text><text x=\"425.0\" y=\"257.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">latencia p50 / p95: 311 / 395 ms</text><text x=\"425.0\" y=\"274.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">coste por respuesta: $0.0000099</text><text x=\"425.0\" y=\"291.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">tasa de rechazo: 11.2%</text><text x=\"425.0\" y=\"308.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">correctas: 65.0%</text><rect x=\"20\" y=\"205\" width=\"215\" height=\"80\" rx=\"6\" fill=\"none\" stroke=\"#b89d28\" stroke-dasharray=\"5 4\"/><text x=\"30\" y=\"225\" font-size=\"12\" fill=\"#444\">Langfuse guarda esta misma</text><text x=\"30\" y=\"241\" font-size=\"12\" fill=\"#444\">forma y añade una interfaz</text><text x=\"30\" y=\"257\" font-size=\"12\" fill=\"#444\">web (sección 6)</text><line x1=\"425\" y1=\"335\" x2=\"425\" y2=\"375\" stroke=\"#555\" stroke-width=\"1.8\" marker-end=\"url(#ah)\"/><rect x=\"235\" y=\"375\" width=\"380\" height=\"55\" rx=\"9\" fill=\"#fee2e2\" stroke=\"#dc2626\" stroke-width=\"1.6\"/><text x=\"425.0\" y=\"398.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\" font-weight=\"700\">Peores trazas</text><text x=\"425.0\" y=\"415.5\" text-anchor=\"middle\" font-size=\"13\" fill=\"#222\">28 incorrectas, 9 preguntas distintas</text></svg>"
      ],
      "text/plain": [
       "<IPython.core.display.SVG object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "diagram(840, 455, [\n",
    "    (\"box\", 20, 80, 150, 60, \"Petición\\nuna pregunta\", \"gold\"),\n",
    "    (\"arrow\", 170, 110, 200, 110, \"\"),\n",
    "    (\"box\", 200, 45, 375, 130, \"\", \"blue\"),\n",
    "    (\"text\", 387, 67, \"Pipeline bajo @observe\", 13, \"middle\"),\n",
    "    (\"box\", 215, 90, 110, 70, \"retrieve\\n3 ms\", \"grey\"),\n",
    "    (\"box\", 345, 90, 215, 70, \"generate (llamada al modelo)\\n382 ms, tokens 122/32\\ncoste $0.0000150\", \"green\"),\n",
    "    (\"arrow\", 575, 110, 620, 110, \"\"),\n",
    "    (\"box\", 620, 65, 200, 90, \"Almacén de trazas\\n80 trazas\\n240 observaciones\", \"grey\"),\n",
    "    (\"arrow\", 720, 155, 720, 205, \"\"),\n",
    "    (\"box\", 620, 205, 200, 90, \"Puntuaciones\\nañadidas después:\\ncorrect, faithfulness,\\nrefused\", \"violet\"),\n",
    "    (\"arrow\", 620, 250, 580, 250, \"\"),\n",
    "    (\"box\", 270, 205, 310, 130, \"Panel (solo con las trazas)\\nlatencia p50 / p95: 311 / 395 ms\\ncoste por respuesta: $0.0000099\\ntasa de rechazo: 11.2%\\ncorrectas: 65.0%\", \"gold\"),\n",
    "    (\"note\", 20, 205, 215, 80, \"Langfuse guarda esta misma\\nforma y añade una interfaz\\nweb (sección 6)\"),\n",
    "    (\"arrow\", 425, 335, 425, 375, \"\"),\n",
    "    (\"box\", 235, 375, 380, 55, \"Peores trazas\\n28 incorrectas, 9 preguntas distintas\", \"red\"),\n",
    "], \"Flujo de datos: petición, traza, puntuaciones, panel\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8b2215ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.734110Z",
     "iopub.status.busy": "2026-11-17T21:26:01.733718Z",
     "iopub.status.idle": "2026-11-17T21:26:01.767365Z",
     "shell.execute_reply": "2026-11-17T21:26:01.764050Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "80 trazas, 240 observaciones\n"
     ]
    }
   ],
   "source": [
    "PARAPHRASES = [\n",
    "    (\"Are boats cancelled when the sea is too rough?\", \"1.5 m\"), (\"Can I go up the tower alone to fix something?\", \"forbidden\"),\n",
    "    (\"How big are the blades?\", \"164 m\"), (\"Who do I call in the middle of the night about an accident?\", \"control room\"),\n",
    "    (\"What if the oil in the gearbox gets dirty?\", \"filter change\"), (\"Do you stop work when whales or seals swim close?\", \"pause\"),\n",
    "]\n",
    "POOL = [(q[1], q[3]) for q in QUESTIONS] + PARAPHRASES            # (question, required fact or None)\n",
    "\n",
    "TRACES.clear(); OBS.clear(); Clock.ms = 0.0\n",
    "rng = random.Random(7)\n",
    "sample_rng = random.Random(11)\n",
    "for s in range(20):\n",
    "    for _ in range(4):\n",
    "        question, fact = sample_rng.choice(POOL)\n",
    "        answer_question(question)\n",
    "        tr = TRACES[-1]\n",
    "        tr[\"session\"], tr[\"fact\"] = f\"session-{s:02d}\", fact\n",
    "        Clock.advance(sample_rng.randint(2000, 9000))           # the user reads and types the next question\n",
    "print(len(TRACES), \"trazas,\", len(OBS), \"observaciones\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e535c2df",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.771231Z",
     "iopub.status.busy": "2026-11-17T21:26:01.770907Z",
     "iopub.status.idle": "2026-11-17T21:26:01.788599Z",
     "shell.execute_reply": "2026-11-17T21:26:01.786904Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'refused': 1, 'faithfulness': 1.0, 'correct': 1} <- puntuaciones de la primera traza\n"
     ]
    }
   ],
   "source": [
    "def judge(tr):\n",
    "    gen = next(o for o in children(tr) if o[\"type\"] == \"generation\")\n",
    "    retr = next(o for o in children(tr) if o[\"name\"] == \"retrieve\")\n",
    "    answer = tr[\"output\"]\n",
    "    refused = answer.startswith(\"I cannot find\")\n",
    "    context = \" \".join(chunks[ids.index(pid)] for pid, _ in retr[\"output\"]).lower()\n",
    "    words = [w for w in tokens(answer.replace(\"[1]\", \"\"))]\n",
    "    faith = 1.0 if refused else sum(w in context for w in words) / max(len(words), 1)\n",
    "    correct = int(refused) if tr[\"fact\"] is None else int((tr[\"fact\"] in answer) and not refused)\n",
    "    score(tr, \"refused\", int(refused)); score(tr, \"faithfulness\", round(faith, 3)); score(tr, \"correct\", correct)\n",
    "\n",
    "for tr in TRACES:\n",
    "    judge(tr)\n",
    "print({k: v for k, v in TRACES[0][\"scores\"].items()}, \"<- puntuaciones de la primera traza\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57bf96a2",
   "metadata": {},
   "source": [
    "## 4. El panel\n",
    "\n",
    "Todo lo que sigue se calcula solo a partir de la lista de trazas: esto es lo que muestra un panel de Langfuse\n",
    "y es lo primero que hay que mirar cuando alguien dice \"el asistente se nota peor esta semana\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "f3878d4c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:01.793082Z",
     "iopub.status.busy": "2026-11-17T21:26:01.792719Z",
     "iopub.status.idle": "2026-11-17T21:26:02.798174Z",
     "shell.execute_reply": "2026-11-17T21:26:02.796404Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trazas               80\n",
      "latencia p50 / p95  311 / 395 ms\n",
      "tokens ent. / sal.   88 / 19 por respuesta (media)\n",
      "coste por respuesta  $0.0000099 de media, $0.00079 en total\n",
      "tasa de rechazo       11.2%\n",
      "correctas             65.0%   fidelidad media 1.00\n",
      "los tokens de entrada son 54% del coste; correlación del coste con los tokens de entrada 0.87, con los de salida 0.89\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABWwAAAEiCAYAAABtFJsnAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAAkdVJREFUeJzs3XlcVFX/B/DPDMqgrC6sLoC74q5JuKQmSWYulWumiKal+KiRpmguaIq2GJaGS49hqbmV2pNluS/lrpj7FmKpLJosooAy5/eHv5kYZgZmhjsbfN6vFy+dc5fzPefee+6dM/eeKxNCCBARERERERERERGR1cmtHQARERERERERERERPcUOWyIiIiIiIiIiIiIbwQ5bIiIiIiIiIiIiIhvBDlsiIiIiIiIiIiIiG8EOWyIiIiIiIiIiIiIbwQ5bIiIiIiIiIiIiIhvBDlsiIiIiIiIiIiIiG8EOWyIiIiIiIiIiIiIbwQ5bIhu2Z88efPDBB8jJybF2KEREREREREREAIB//vkHMTExOHr0qLVDKZPYYWuihIQEyGQy3Lhxw9qhlNq+ffsgk8mwb98+a4dil0ytv9mzZ0Mmk+mdfv36dbzyyivw8vKCs7NzKaPU9tdff8HJyQm//fab5OsuzqBBgzBgwACL5klE5dfGjRtRtWpVPHjwQLJ1Dh8+HAEBAZKtr6yZOnUqgoODrR0G2Zjjx4+jffv2cHZ2hkwmQ2JiorVDKhNU30lOnDhh7VBKpaTr4vKK39OIyFpu3LgBmUyGhIQEndOFEBg2bBj27duHVq1aWTa4coIdtlawbt06xMXFWTsMMtIXX3yht7GSWl5eHvr3749x48Zh9OjRZsljzpw5CA4ORocOHcyyfn2mTJmC7777DmfOnLFovmS/fv/9d8yePRsZGRnWDoXM4Pbt25g9e7ZZOm8KCgowa9Ys/Oc//4GLi4vk67cVqi/0mzdv1jl93LhxWh0h+fn5WLx4MVq1agU3Nzd4eHggKCgIo0ePxqVLl9TzqTqDVH9OTk7w8/NDWFgYPvvsM2RnZ2vlN3HiRJw5cwY//PCDtAUlszNXe/v48WP0798f//zzDz799FN888038Pf3lzSPkly4cAGzZ8+2uZst5s+fj61bt1o7DDIAv8MRWRaPOdv24Ycf4saNG9iyZQscHR2tHU6ZxA5bK7C1hue5557Do0eP8Nxzz1k7FJumr8PW1Pp7//338ejRI53Tzp49i4iICMybN8+UUEuUnp6O1atX4+233zbL+ovTqlUrtG3bFp988onF8yb79PvvvyMmJoYdtmXU7du3ERMTY5YO2//973+4fPmy5D98rVy5EpcvX5Z0nZb22muv4d1330XTpk2xYMECxMTE4LnnnsPPP/+MI0eOaM0/Z84cfPPNN4iPj8d//vMfAE87Zps1a4Y//vhDY14fHx/06dMHH3/8sUXKQtIxV3t7/fp1JCcnY9KkSRg9ejTeeOMNVKlSRdI8SnLhwgXExMSww5ZMZmvf4YjKOh5z1uXv749Hjx5h6NChWtNyc3Px5MkT/PTTT/Dw8LB8cOVEBWsHQNYnl8vh5ORk7TCM9vDhQ1SuXNnaYZhcfxUqVECFCroPwbZt26Jt27alDU2vNWvWoEKFCujVq5fZ8ijOgAEDMGvWLHzxxRdl+q43IiEEcnNzUalSJYOXsZW2rSz46quv0KFDB9SoUUPS9VasWFGS9SiVSuTn51v8HHz8+HH8+OOPmDdvHqZNm6YxbcmSJTo763r06KFxXoqOjsaePXvw8ssvo3fv3rh48aLGfj5gwAD0798ff/75J+rUqWO2spB9SEtLAwCDvtTl5OSYZSgoIkvKzc2Fo6Mj5HLeH0VlR3H7NdtuaTx58gRKpdIm7lhVPVmli5OTE6ZPn27hiMofnkEktG3bNvTs2RN+fn5QKBSoW7cu5s6di4KCAvU8Xbp0wfbt25GcnKx+vLDwOHh5eXmYNWsW6tWrB4VCgVq1auG9995DXl6eRl4ymQzjxo3D1q1b0bRpUygUCgQFBWHHjh1acd26dQsjR45UxxUYGIgxY8YgPz8fgO6xkQ4ePIj+/fujdu3a6jjeeecdvXeEFqZ6fPLAgQN46623UK1aNbi5uWHYsGG4f/++1vxffPEFgoKCoFAo4Ofnh8jISK0vi126dEHTpk1x8uRJPPfcc6hcubLWl8zChg8fDhcXF/z5558ICwuDs7Mz/Pz8MGfOHAghNOZVKpWIi4tDUFAQnJyc4O3tjbfeeksj1oCAAJw/fx779+9Xb7cuXbrorT8AOHr0KF566SVUqVIFzs7OaN68ORYvXqyermusridPnmDu3LmoW7cuFAoFAgICMG3aNK3tHxAQgJdffhmHDh1Cu3bt4OTkhDp16uDrr7/WWyeFbd26FcHBwVqdpap6/uOPP9C5c2dUrlwZ9erVUz9mu3//fgQHB6NSpUpo2LAhdu3apbF8dnY2Jk6ciICAACgUCnh5eeGFF17AqVOnNOZ74YUXkJOTg507dxoUL9mektoVAPjzzz/Rv39/VK1aFZUrV8azzz6L7du3a63r888/R1BQECpXrowqVaqgbdu2WLduHYCnx8nkyZMBAIGBgerjr/AdUmvWrEGbNm1QqVIlVK1aFYMGDcJff/1VYhlUx+ClS5cwYMAAuLm5oVq1apgwYQJyc3M15jX22Pzll1/Qtm1bVKpUCcuXL9cbQ3Ftm6Hng507d6Jjx47w8PCAi4sLGjZsqNE+qtqoDRs2YNq0afDx8YGzszN69+6tVU8BAQEYPny4zjhVbZ6KFPHt27cPzzzzDAAgIiJCvX1VTzOU5lyUm5uLHTt2IDQ0VGua6hy6adMmNGnSBJUqVUJISAjOnj0LAFi+fDnq1asHJycndOnSReuOPF1j2CqVSixevBjNmjWDk5MTPD098eKLL2qMK6nKd+3aterznuq8ffr0afTo0QNubm5wcXFBt27ddN7pKoXr168DgM4hcRwcHFCtWjWD1vP8889jxowZSE5Oxpo1azSmqep927ZtpYyW7L29HT58ODp37gwA6N+/v8Y1lOp67fr163jppZfg6uqKIUOGADDs+gww7JooISEB/fv3BwB07dpVXbbC124///wzOnXqBGdnZ7i6uqJnz544f/58sWUDnr5wZdKkSWjWrBlcXFzg5uaGHj16GDT0k0wmQ05ODlavXq2OqXAbbGq7cP/+fbRr1w41a9ZUPw0g9XcMQ6/5dDl06BCeeeYZODk5oW7dusWeJ009xwNPj50RI0bA29tbXY5Vq1ZpzKM6R27cuBHz5s1DzZo14eTkhG7duuHatWvq+Yr7Dqdax/r16/H++++jRo0aqFy5MrKysozaP/7++2/07dsXzs7O8PLywjvvvKO1fYDSnRup7JPqnFHcfi1F2w08bXc7d+4MV1dXuLm54ZlnnlGfk4o75vLz8zFz5ky0adMG7u7ucHZ2RqdOnbB3716tPNavX482bdqo82jWrJnGd3JjFL5ub9++PSpVqoTAwEAsW7ZMa960tDSMHDkS3t7ecHJyQosWLbB69WqNeVRjxH788ceIi4tTf8+4cOGC3hhKuu4HpPsOoW8M2z179qjPlx4eHujTpw8uXryoMY/qe9a1a9cwfPhweHh4wN3dHREREXj48GGx9Uz/4h22EkpISICLiwuioqLg4uKCPXv2YObMmcjKysJHH30EAJg+fToyMzPx999/49NPPwUAdaeZUqlE7969cejQIYwePRqNGzfG2bNn8emnn+LKlStaj0sdOnQI33//PcaOHQtXV1d89tlneO2113Dz5k31l63bt2+jXbt2yMjIwOjRo9GoUSPcunULmzdvxsOHD/X+crNp0yY8fPgQY8aMQbVq1XDs2DF8/vnn+Pvvv7Fp0yaD6mPcuHHw8PDA7NmzcfnyZcTHxyM5OVnd+ANPD+SYmBiEhoZizJgx6vmOHz+O3377TeMupnv37qFHjx4YNGgQ3njjDXh7exebf0FBAV588UU8++yz+PDDD7Fjxw7MmjULT548wZw5c9TzvfXWW0hISEBERATGjx+PpKQkLFmyBKdPn1bHEBcXpx4DUfVLUnH579y5Ey+//DJ8fX0xYcIE+Pj44OLFi/jxxx8xYcIEvcu9+eabWL16Nfr164d3330XR48eRWxsLC5evIgtW7ZozHvt2jX069cPI0eORHh4OFatWoXhw4ejTZs2CAoK0pvH48ePcfz4cYwZM0bn9Pv37+Pll1/GoEGD0L9/f8THx2PQoEFYu3YtJk6ciLfffhuvv/46PvroI/Tr1w9//fUXXF1dAQBvv/02Nm/ejHHjxqFJkya4d+8eDh06hIsXL6J169bqPFQdJL/99hteeeUVvbGSbTKkXUlNTUX79u3x8OFDjB8/HtWqVcPq1avRu3dvbN68Wb3dV65cifHjx6Nfv37qjtI//vgDR48exeuvv45XX30VV65cwbfffotPP/0U1atXBwB4enoCAObNm4cZM2ZgwIABePPNN5Geno7PP/8czz33HE6fPm3Q3VwDBgxAQEAAYmNjceTIEXz22We4f/++xpd9Y47Ny5cvY/DgwXjrrbcwatQoNGzYsNj8dbVthp4Pzp8/j5dffhnNmzfHnDlzoFAocO3aNZ0vE5w3bx5kMhmmTJmCtLQ0xMXFITQ0FImJiUbdAQwYfr4qKb7GjRtjzpw5mDlzJkaPHo1OnToBANq3bw+gdOeikydPIj8/X6PtKezgwYP44YcfEBkZCQCIjY3Fyy+/jPfeew9ffPEFxo4di/v37+PDDz/EiBEjsGfPnmLzGzlyJBISEtCjRw+8+eabePLkCQ4ePIgjR45o3Jm6Z88ebNy4EePGjUP16tXVPwh26tQJbm5ueO+991CxYkUsX74cXbp0Uf9QJiXV2KFr165Fhw4d9D7tYYihQ4di2rRp+PXXXzFq1Ch1uru7O+rWrYvffvsN77zzTqljLq/KQnv71ltvoUaNGpg/fz7Gjx+PZ555RuMa6smTJwgLC0PHjh3x8ccfq58wMOT6TKWka6LnnnsO48ePx2effYZp06ahcePGAKD+95tvvkF4eDjCwsKwcOFCPHz4EPHx8ejYsSNOnz5d7EsG//zzT2zduhX9+/dHYGAgUlNTsXz5cnTu3BkXLlyAn5+f3mW/+eYbvPnmm2jXrp166Ja6desCgMntwt27d/HCCy/gn3/+wf79+1G3bl2zfMcw9JqvqLNnz6J79+7w9PTE7Nmz8eTJE8yaNUvndXVpzvGpqal49tln1R3Qnp6e+PnnnzFy5EhkZWVh4sSJGvMvWLAAcrkckyZNQmZmJj788EMMGTJE/ebz4r7DqcydOxeOjo6YNGkS8vLy4OjoiAsXLhi0fzx69AjdunXDzZs3MX78ePj5+eGbb77Ree6R4nsalU1SnjNUdO3XQOnb7oSEBIwYMQJBQUGIjo6Gh4cHTp8+jR07duD1118v9pjLysrCl19+icGDB2PUqFHIzs7Gf//7X4SFheHYsWNo2bIlgKffyQcPHoxu3bph4cKFAICLFy/it99+K/Y7eXHu37+Pl156CQMGDMDgwYOxceNGjBkzBo6OjhgxYgSAp8dzly5dcO3aNYwbNw6BgYHYtGkThg8fjoyMDK28v/rqK+Tm5mL06NFQKBSoWrWqzrwNue43x3eIwnbt2oUePXqgTp06mD17Nh49eoTPP/8cHTp0wKlTp7TOlwMGDEBgYCBiY2Nx6tQpfPnll/Dy8lJvDyqBIJN89dVXAoBISkpSpz18+FBrvrfeektUrlxZ5ObmqtN69uwp/P39teb95ptvhFwuFwcPHtRIX7ZsmQAgfvvtN3UaAOHo6CiuXbumTjtz5owAID7//HN12rBhw4RcLhfHjx/Xyk+pVAohhNi7d68AIPbu3VtsWWJjY4VMJhPJycla0wpT1U2bNm1Efn6+Ov3DDz8UAMS2bduEEEKkpaUJR0dH0b17d1FQUKCeb8mSJQKAWLVqlTqtc+fOAoBYtmxZsXmrhIeHCwDiP//5j0Z5e/bsKRwdHUV6eroQQoiDBw8KAGLt2rUay+/YsUMrPSgoSHTu3Fkrr6L19+TJExEYGCj8/f3F/fv3NeZV1bkQQsyaNUsUPgQTExMFAPHmm29qLDNp0iQBQOzZs0ed5u/vLwCIAwcOqNPS0tKEQqEQ7777brF1c+3aNa39REVVz+vWrVOnXbp0SQAQcrlcHDlyRJ3+yy+/CADiq6++Uqe5u7uLyMjIYvNXadCggejRo4dB85JtMaRdmThxogCg0Z5lZ2eLwMBAERAQoD7m+/TpI4KCgorN76OPPtJqb4UQ4saNG8LBwUHMmzdPI/3s2bOiQoUKWulFqY7B3r17a6SPHTtWABBnzpwRQph2bO7YsaPYvFX0tW2Gng8+/fRTAUDdpumiaqNq1KghsrKy1OkbN24UAMTixYs14g8PD9cZZ+H2T8r4jh8/rtWWqJTmXPTll18KAOLs2bNa0wAIhUKhsU8tX75cABA+Pj4a9RQdHa21/4WHh2ucx/fs2SMAiPHjx2vlVbjdV7Wl58+f15inb9++wtHRUVy/fl2ddvv2beHq6iqee+65YsspxL/beNOmTTqnR0ZGapxvlEqlet/z9vYWgwcPFkuXLtVZp6pzuq7jXcXd3V20atVKK7179+6icePGJcZP+pWV9lbfPqq6Xps6dapGujHXZ4ZeE23atEnreleIp3Xl4eEhRo0apZGekpIi3N3dtdKLys3N1biOFUKIpKQkoVAoxJw5c4pdVgghnJ2ddba7hrYLhY/RO3fuiKCgIFGnTh1x48YN9Tzm+I5hzDVf0XI5OTlptDcXLlwQDg4OGu1Uafe5kSNHCl9fX3H37l2N9EGDBgl3d3f1+UW1bzZu3Fjk5eWp51u8eLHWOUTfdzjVOurUqaN13jJ0/4iLixMAxMaNG9VpOTk5ol69epJ+T6OyTcpzRnH7dWnb7oyMDOHq6iqCg4PFo0ePdMYphP5j7smTJxrHqxBC3L9/X3h7e4sRI0ao0yZMmCDc3NzEkydPtNZhCtW10yeffKJOy8vLEy1bthReXl7qvg/V8bxmzRr1fPn5+SIkJES4uLiorzOTkpIEAOHm5ibS0tJKzN+Q62opr9FV8RW+RleV9d69e+q0M2fOCLlcLoYNG6ZOU33PKrw9hBDilVdeEdWqVSuxrPQUh0SQUOE7lLKzs3H37l106tQJDx8+1Hjjsj6bNm1C48aN0ahRI9y9e1f99/zzzwOA1i3+oaGh6l/hAaB58+Zwc3PDn3/+CeDprytbt25Fr169dI6HWvRxfH1lycnJwd27d9G+fXsIIXD69OkSywIAo0eP1rj7YcyYMahQoQJ++uknAE9/ncnPz8fEiRM1xsEZNWoU3NzctB7LUCgUiIiIMChvlXHjxqn/r/qFPT8/X/0o/6ZNm+Du7o4XXnhBo87btGkDFxcXnY9VlOT06dNISkrCxIkTtX75L67OVfUSFRWlkf7uu+8CgFZ9NGnSRH03GvD0DpiGDRuqt78+9+7dAwC9L/twcXHBoEGD1J8bNmwIDw8PNG7cWONuDtX/C+fn4eGBo0eP4vbt28XGoMr/7t27Jc5HtsXQduWnn35Cu3bt0LFjR/U0FxcXjB49Gjdu3FA/6uPh4YG///4bx48fNzqW77//HkqlEgMGDNA4fn18fFC/fn2Dj1/VHZYqqhcqqY5JY4/NwMBAhIWFGVwOXW2boecDVRuzbds2KJXKYvMZNmyY+m54AOjXrx98fX3V5TOGOeLTpTTnopLaum7dumncBaBq01577TWNetLV1hX13XffQSaTYdasWVrTirb7nTt3RpMmTdSfCwoK8Ouvv6Jv374aY736+vri9ddfx6FDh5CVlaU3b1PIZDL88ssv+OCDD1ClShV8++23iIyMhL+/PwYOHGj0C6dcXFyQnZ2tlc52vnTKYnurT9Gnfoy9PjP1mgh4egdWRkYGBg8erJGXg4MDgoODSyybQqFQX8cWFBTg3r176kdLDRkeQBdT2oW///4bnTt3xuPHj3HgwAH1nfSA9N8xAOOu+QqX65dffkHfvn1Ru3ZtdXrjxo21zpul2eeEEPjuu+/Qq1cvCCE0lg8LC0NmZqbWtomIiNB48lC1PxmyD6mEh4drPbFi6P7x008/wdfXF/369VOnVa5cWedLM6X4nkZlj9TnDBVd+7WKqW33zp07kZ2djalTp2qNkVrc92UVBwcH9fGqVCrxzz//4MmTJ2jbtq3GceXh4SH5MHwVKlTAW2+9pf7s6OiIt956C2lpaTh58iSAp3Xs4+ODwYMHq+erWLEixo8fjwcPHmD//v0a63zttdfUT7MUx5DranNeo9+5cweJiYkYPny4xl3AzZs3xwsvvKDzO0XRl5x36tQJ9+7dk/zatqxih62Ezp8/j1deeQXu7u5wc3ODp6cn3njjDQBAZmZmictfvXoV58+fh6enp8ZfgwYNAPz7wgaVwhc6KlWqVFGPD5Oeno6srCw0bdrU6LLcvHlTfSC6uLjA09NTPf6YIWUBgPr162t8dnFxga+vr3ostOTkZADQelzY0dERderUUU9XqVGjhlGDb8vlcq0XnajqUhXD1atXkZmZCS8vL616f/DggVadG0I1NqCx9Z6cnAy5XI569epppPv4+MDDw0OrPkra/iURRcbyValZs6bWidLd3R21atXSSgOgkd+HH36Ic+fOoVatWmjXrh1mz56t90JXCGHQCZlsi6HtSnJyss6hAFSPn6r25ylTpsDFxQXt2rVD/fr1ERkZWeKjOCpXr16FEAL169fXOn4vXrxo8PFbtK2qW7cu5HK5RltlzLEZGBhoUL4quto2Q88HAwcORIcOHfDmm2/C29sbgwYNwsaNG3VeeBUtp0wmQ7169Ux6Y7o54tNFinORvrauaBuqatMMaeuKun79Ovz8/PQ+wlZY0f0jPT0dDx8+1Hu8KJVK9XiN6enpSElJUf89ePCgxPz0USgUmD59Oi5evIjbt2/j22+/xbPPPqsersEYDx480OjkVmE7Xzplsb3VpUKFCqhZs6ZWfsZcn5Xmmujq1asAno7JXDSvX3/9tcSyKZVKfPrpp6hfvz4UCgWqV68OT09P/PHHHwa3U0UZ0y6oDB06FGlpadi/f7/Wixal/o4BGHfNV7hcjx490jofAdrfB0qzz6WnpyMjIwMrVqzQWlb1A2lJZVb92GfodTWg+/xv6P6RnJyMevXqabWZuvYBKc6NVPZIfc5Q0XddW5q229Tvy4WtXr0azZs3h5OTE6pVqwZPT09s375d4xgYO3YsGjRogB49eqBmzZoYMWKEznf+GMPPz0/r5WpF+xiSk5NRv359rZezGVvHRRlyXW3Oa3R9/Teqst29exc5OTka6VK0reUZx7CVSEZGBjp37gw3NzfMmTMHdevWhZOTE06dOoUpU6YY9OVUqVSiWbNmWLRokc7pRb9EOjg46JxP35dTQxUUFKjHvpoyZQoaNWoEZ2dn3Lp1C8OHDzfpLikpGDvGoiGUSiW8vLywdu1andMN+aVLaoZ+uTV1+6vGHtPXSOpbryH5DRgwAJ06dcKWLVvw66+/4qOPPsLChQvx/fffo0ePHhrL3b9/X+cFO5UvjRs3xuXLl/Hjjz9ix44d+O677/DFF19g5syZiImJKXZZpVIJmUyGn3/+Wef+WXRsOUPpOwYNPTaNbat0zW/o+aBSpUo4cOAA9u7di+3bt2PHjh3YsGEDnn/+efz66696j1t99JWxoKBAY12WiK+056LCbV3RLxRA6dq60ijNueyZZ57RuMifNWsWZs+erb47Rd8LZx4+fKj3Lb/A07v2Bg0ahNdeew1BQUHYuHEjEhISDBrb9u+//0ZmZqbWDxrA07pXjYNK1meL7S2geQdi4fyMuT4rzXGraku++eYb+Pj4aE0v6TiYP38+ZsyYgREjRmDu3LmoWrUq5HI5Jk6caNFr5ldffRVff/01Fi9ejNjYWI1p5viOYcw1nylKs8+p6v2NN95AeHi4znmaN2+u8VmKtl9X+y71/mGr39Oo7NJ33SJF222qNWvWYPjw4ejbty8mT54MLy8vODg4IDY2Vt0ZDABeXl5ITEzEL7/8gp9//hk///wzvvrqKwwbNkzrBWDWZOi1oSHX1db6DqGPua+ryzp22Epk3759uHfvHr7//ns899xz6vSkpCStefV9Ia5bty7OnDmDbt26SXJHiqenJ9zc3HDu3Dmjljt79iyuXLmC1atXY9iwYep0Yx8luHr1Krp27ar+/ODBA9y5cwcvvfQSgH9fenL58mWNO2Hz8/ORlJSk883exlAqlfjzzz/VvyYBwJUrVwBA/Rhs3bp1sWvXLnTo0KHEhtLQbaJ6hOzcuXNGlcHf3x9KpRJXr15V//oGPH1pQkZGhsajbaVRu3ZtVKpUSee+KQVfX1+MHTsWY8eORVpaGlq3bo158+ZpXLw/efIEf/31F3r37m2WGMh8DG1X/P391W+mLkw1PEzh/dnZ2RkDBw7EwIEDkZ+fj1dffRXz5s1DdHQ0nJycim0zhRAIDAzUOM6NdfXqVY1ftq9duwalUqluJyx1bBZmzPlALpejW7du6NatGxYtWoT58+dj+vTp2Lt3r0YbpLqLTEUIgWvXrml8aa1SpYrOx+GTk5M12mkp49O3fGnPRY0aNQLw9DzcrFkzg5YxVd26dfHLL7/gn3/+Megu28I8PT1RuXJlvceLXC5XX1yvXbtWo1NWtU0Kn091uXz5skH7acWKFdG8eXNcvXpV/ehxSb755hsA0DkMSFJSElq0aFHiOki3stjeGsqY6zNDFVc24OkXe1OuPTdv3oyuXbviv//9r0Z6RkaGQT9Y6IrLmHZB5T//+Q/q1auHmTNnwt3dHVOnTlVPk/o7hooh13xFy1WpUiWt8xGg3X6VZp/z9PSEq6srCgoKSv19ojBT6s7Q/cPf3x/nzp3TejKhaL1I9T2Nyh5znDOMZWjbXfj7sq4ffFX0HXObN29GnTp18P3332vMo2toKkdHR/Tq1Qu9evWCUqnE2LFjsXz5csyYMaPYvPW5ffs2cnJyNO6yLdrH4O/vjz/++ANKpVKjU1uKOi7putoc3yFUirvevHTpEqpXr6519zGVDodEkIjql4PCvxTk5+fjiy++0JrX2dlZ5+MqAwYMwK1bt7By5UqtaY8ePdK6vbwkcrkcffv2xf/+9z+cOHFCa7q+XzV0lUUIgcWLFxuV/4oVK/D48WP15/j4eDx58kR9ERcaGgpHR0d89tlnGnn997//RWZmJnr27GlUfrosWbJE/X8hBJYsWYKKFSuiW7duAJ7WeUFBAebOnau17JMnTzQ6LpydnQ0a169169YIDAxEXFyc1vzF/ZKk6siOi4vTSFf9OiZFfQBPv5C3bdtW5z5RGgUFBVr7tZeXF/z8/JCXl6eRfuHCBeTm5qrfBE/2w9B25aWXXsKxY8dw+PBh9bScnBysWLECAQEB6jE8VeOMqjg6OqJJkyYQQqjbD9WJv+jx9Oqrr8LBwQExMTFax5YQQmvd+ixdulTj8+effw4A6rbKUsdmYYaeD/755x+t6ao34xY97r7++muNcUY3b96MO3fuaHyxrlu3Lo4cOYL8/Hx12o8//qj16K2U8enbvqU9F7Vp0waOjo6St3W6vPbaaxBC6LxLsaQ7CBwcHNC9e3ds27ZNY3iK1NRUrFu3Dh07doSbmxsAoEOHDggNDVX/qTpsfX190bJlS6xZs0arHk+ePIkjR45obOerV6/i5s2bWrFkZGTg8OHDqFKlikF3wezZswdz585FYGAghgwZojEtMzMT169fZztfCmWxvTWUMddnhtJXtrCwMLi5uWH+/Pka160q6enpxa7XwcFBq042bdqEW7duGRyXrvbP0HahsBkzZmDSpEmIjo5GfHy8Ol3q7xjGXPMVLVdYWBi2bt2q0QZdvHgRv/zyi8a8pdnnHBwc8Nprr+G7777T2XlV0jbVR993uOIYun+89NJLuH37NjZv3qxOe/jwIVasWKG1PqD039Oo7JH6nGEKQ9vu7t27w9XVFbGxscjNzdUZJ6D/mNN1HBw9elSjTID2eU8ul6tvVCiurSrOkydPsHz5cvXn/Px8LF++HJ6enmjTpg2Ap3WckpKCDRs2aCz3+eefw8XFRT2EibEMua42x3cIFdX15urVqzXOW+fOncOvv/6q/s5E0uEdthJp3749qlSpgvDwcIwfPx4ymQzffPONzi9qbdq0wYYNGxAVFYVnnnkGLi4u6NWrF4YOHYqNGzfi7bffxt69e9GhQwcUFBTg0qVL2LhxI3755RedA4gXZ/78+fj111/RuXNnjB49Go0bN8adO3ewadMmHDp0SOulWMDTu5Lq1q2LSZMm4datW3Bzc8N3331n9Dgj+fn56NatGwYMGIDLly/jiy++QMeOHdV3VXp6eiI6OhoxMTF48cUX0bt3b/V8zzzzjHr8X1M5OTlhx44dCA8PR3BwMH7++Wds374d06ZNU38R7dy5M9566y3ExsYiMTER3bt3R8WKFXH16lVs2rQJixcvVg/+36ZNG8THx+ODDz5AvXr14OXlpR68uzC5XI74+Hj06tULLVu2REREBHx9fXHp0iWcP39e64JUpUWLFggPD8eKFSvUQ2wcO3YMq1evRt++fTXuVi6tPn36YPr06cjKytJ5wW+K7Oxs1KxZE/369UOLFi3g4uKCXbt24fjx4/jkk0805t25cycqV66MF154QZK8ybIMaVemTp2Kb7/9Fj169MD48eNRtWpVrF69GklJSfjuu+/UvzZ3794dPj4+6NChA7y9vXHx4kUsWbIEPXv2VI+Jqbr4mT59OgYNGoSKFSuiV69eqFu3Lj744ANER0fjxo0b6Nu3L1xdXZGUlIQtW7Zg9OjRmDRpUonlSUpKQu/evfHiiy/i8OHDWLNmDV5//XX1nYGWPDZVDD0fzJkzBwcOHEDPnj3h7++PtLQ0fPHFF6hZs6bGyyQAoGrVqujYsSMiIiKQmpqKuLg41KtXD6NGjVLP8+abb2Lz5s148cUXMWDAAFy/fh1r1qzRePmM1PHVrVsXHh4eWLZsGVxdXeHs7Izg4OBSn4ucnJzQvXt37Nq1C3PmzCnlFile165dMXToUHz22We4evUqXnzxRSiVShw8eBBdu3YtcUzYDz74ADt37kTHjh0xduxYVKhQAcuXL0deXh4+/PBDg2JYtGgRwsLC0LJlSwwfPhx+fn64ePEiVqxYAV9fX0RHR6vnPXPmDF5//XX06NEDnTp1QtWqVXHr1i2sXr0at2/fRlxcnNYjbD///DMuXbqEJ0+eIDU1FXv27MHOnTvh7++PH374QWvIhV27dkEIgT59+hhYi6RLWWtvDWXM9ZmhWrZsCQcHByxcuBCZmZlQKBR4/vnn4eXlhfj4eAwdOhStW7fGoEGD4OnpiZs3b2L79u3o0KGDxg0ARb388suYM2cOIiIi0L59e5w9exZr167Veo+CPm3atMGuXbuwaNEi+Pn5ITAwEMHBwSa3Cx999BEyMzMRGRkJV1dXvPHGG5J/xzDmmq+omJgY7NixA506dcLYsWPVnRhBQUH4448/1POVdp9bsGAB9u7di+DgYIwaNQpNmjTBP//8g1OnTmHXrl06OytKou87XHEM3T9GjRqFJUuWYNiwYTh58iR8fX3xzTffoHLlyhrzSfU9jcomKc8ZpjC07XZzc8Onn36KN998E8888wxef/11VKlSBWfOnMHDhw/VwxXoO+ZefvllfP/993jllVfQs2dPJCUlYdmyZWjSpInG+P5vvvkm/vnnHzz//POoWbMmkpOT8fnnn6Nly5YaT82p7ow15L0Ofn5+WLhwIW7cuIEGDRpgw4YNSExMxIoVK9QvXB89ejSWL1+O4cOH4+TJkwgICMDmzZvx22+/IS4uTue4/4Yw5LraHN8hCvvoo4/Qo0cPhISEYOTIkXj06BE+//xzuLu7Y/bs2SaVi4ohyCRfffWVACCSkpLUab/99pt49tlnRaVKlYSfn5947733xC+//CIAiL1796rne/DggXj99deFh4eHACD8/f3V0/Lz88XChQtFUFCQUCgUokqVKqJNmzYiJiZGZGZmqucDICIjI7Xi8vf3F+Hh4RppycnJYtiwYcLT01MoFApRp04dERkZKfLy8oQQQuzdu1crxgsXLojQ0FDh4uIiqlevLkaNGiXOnDkjAIivvvrKoLrZv3+/GD16tKhSpYpwcXERQ4YMEffu3dOaf8mSJaJRo0aiYsWKwtvbW4wZM0bcv39fY57OnTuLoKCgYvMtLDw8XDg7O4vr16+L7t27i8qVKwtvb28xa9YsUVBQoDX/ihUrRJs2bUSlSpWEq6uraNasmXjvvffE7du31fOkpKSInj17CldXVwFAdO7cWW/9CSHEoUOHxAsvvCBcXV2Fs7OzaN68ufj888/V02fNmiWKHoKPHz8WMTExIjAwUFSsWFHUqlVLREdHi9zcXI35/P39Rc+ePbXK0blzZ3VcxUlNTRUVKlQQ33zzjdbyuupZX36F98O8vDwxefJk0aJFC3WZW7RoIb744gut5YKDg8Ubb7xRYpxku0pqV4QQ4vr166Jfv37Cw8NDODk5iXbt2okff/xRYz3Lly8Xzz33nKhWrZpQKBSibt26YvLkyRrtnRBCzJ07V9SoUUPI5XKttve7774THTt2FM7OzsLZ2Vk0atRIREZGisuXLxdbBtUxeOHCBdGvXz/h6uoqqlSpIsaNGycePXqkMW9pj019imvbDDkf7N69W/Tp00f4+fkJR0dH4efnJwYPHiyuXLmiXo+qjfr2229FdHS08PLyEpUqVRI9e/YUycnJWvl+8sknokaNGkKhUIgOHTqIEydO6GxbpIpPCCG2bdsmmjRpIipUqKBxninNuUgIIb7//nshk8nEzZs3NdJ1nUOTkpIEAPHRRx9ppKvqb9OmTeq08PBwjXO3EEI8efJEfPTRR6JRo0bC0dFReHp6ih49eoiTJ08Wm6/KqVOnRFhYmHBxcRGVK1cWXbt2Fb///nuJZSzsyJEj4uWXXxZVqlQRFSpUEDVq1BBvvvmm+PvvvzXmS01NFQsWLBCdO3cWvr6+okKFCqJKlSri+eefF5s3b9aYV3VOV/05OjoKHx8f8cILL4jFixeLrKwsnbEMHDhQdOzY0aj4Sbey0N7qOo6E+Pd6TR9Drs+MuSZauXKlqFOnjnBwcNC6dtu7d68ICwsT7u7uwsnJSdStW1cMHz5cnDhxotiy5ebminfffVf4+vqKSpUqiQ4dOojDhw8bfE126dIl8dxzz4lKlSoJABrX8Ya0C6pj9Pjx4+q0goICMXjwYFGhQgWxdetWIYS03zGMuebTZf/+/aJNmzbC0dFR1KlTRyxbtkzndbEQpu9zQjxt6yIjI0WtWrVExYoVhY+Pj+jWrZtYsWKFeh59+6bqnFD4XKPvO5y+dQhh3P6RnJwsevfuLSpXriyqV68uJkyYIHbs2CHp9zQq+6Q6ZxS3X0vRdgshxA8//CDat28vKlWqJNzc3ES7du3Et99+q56u75hTKpVi/vz5wt/fXygUCtGqVSvx448/al2fbd68WXTv3l14eXkJR0dHUbt2bfHWW2+JO3fuaMRRvXp18eyzz5ZYt6rr9hMnToiQkBDh5OQk/P39xZIlS7TmTU1NFREREaJ69erC0dFRNGvWTOv41HftqY+h19VSXaPrageFEGLXrl2iQ4cO6u3Wq1cvceHCBY15VG16enq6RrqufjTSTyYER/slaSUkJCAiIgLHjx83+o5gqQwfPhybN28u1Ru0y7qRI0fiypUrOHjwoEXzTUxMROvWrXHq1Cn1YxdE1jB79mzExMQgPT29TL8Yad++fejatSs2bdpk9B1p9q6goABNmjTBgAEDdD6eR+aRkpKCwMBArF+/nnfYEhERkc26cOECgoKC8OOPP5Y4zFmXLl1w9+5do98RRGQqjmFLVE7NmjULx48fx2+//WbRfBcsWIB+/fqxs5aIzM7BwQFz5szB0qVL+QOeBcXFxaFZs2bsrCUiIiKbtnfvXoSEhJjlnRREpcUxbInKqdq1a2sN8m4J69evt3ieRFR+DRw4EAMHDrR2GOXKggULrB0CERERUYkiIyMRGRlp7TCIdOIdtkREREREREREREQ2gmPYEhEREREREREREdkI3mFLREREREREREREZCPYYUtERERERERERERkI9hhS0RERERERERERGQjKlg7gKKUSiVu374NV1dXyGQya4dDRAQhBLKzs+Hn5we5vGz9zsU2l4hsCdtbIiLLYHtLRGQZpra3Ntdhe/v2bdSqVcvaYRARafnrr79Qs2ZNa4chKba5RGSL2N4SEVkG21siIsswtr21uQ5bV1dXAE8L4ubmZpY8lEol0tPT4enpWeZ+TdSlPJW3PJUVKF/ltWZZs7KyUKtWLXX7VJZYos0tDXvbx+0pXnuKFbCveO0pVsC24mV7a1m2tO1NYc/x23PsgH3Hb8+xA9LFz/bWMPa+v0iJdaGJ9fEv1sW/dNWFqe2tzXXYqh5ZcHNzM2uHbW5uLtzc3MrFzlSeylueygqUr/LaQlnL4iNVlmhzS8MWtrsx7Clee4oVsK947SlWwDbjZXtrGba47Y1hz/Hbc+yAfcdvz7ED0sfP9rZ49r6/SIl1oYn18S/Wxb+Kqwtj29vyXZNERERERERERERENoQdtkREREREREREREQ2gh22RERERERERCS5AwcOoFevXvDz84NMJsPWrVtLXGbfvn1o3bo1FAoF6tWrh4SEBLPHSURka9hhS0RERERERGSH5v9wBkHTt6n/5v9wxtohacjJyUGLFi2wdOlSg+ZPSkpCz5490bVrVyQmJmLixIl488038csvv5g5UiIi22JzLx0jIiIiIiIiouLN/+EM1h69oZGm+jytdwvLB6RDjx490KNHD4PnX7ZsGQIDA/HJJ58AABo3boxDhw7h008/RVhYmLnCJCKyObzDloiIiIiIiMjOFO2sLSndHhw+fBihoaEaaWFhYTh8+LCVIiIisg7eYUtEREREREREVpeSkgJvb2+NNG9vb2RlZeHRo0eoVKmS1jJ5eXnIy8tTf87KygIAKJVKKJXKUsWjVCohhCj1esoC1oUm1se/WBf/0lUXptYLO2yLCJq+TZL1nJ/XR5L1EBERkW0reu0gB9DAA7iSARh6ecbrBiIiItPExsYiJiZGKz09PR25ubmlWrdSqURmZiaEEJDLy/cDyrZWF5HfHDV43qVDg41erqRlZABqOAO3cgChZzlLMqVcUrG1fcOadNVFdna2Setihy0RERERERGRnRkSHKBz+IMhwQEWj0UqPj4+SE1N1UhLTU2Fm5ubzrtrASA6OhpRUVHqz1lZWahVqxY8PT3h5uZWqniUSiVkMhk8PT3ZEWVjdXElw/B5vby8jF6upGVUNXA1Q/MH+sLLWZIp5ZKKre0b1qSrLpycnExaFztsiYiIiIiIiOyM6sVihTtthwQH2MwLx0wREhKCn376SSNt586dCAkJ0buMQqGAQqHQSpfL5ZJ0HslkMsnWZe9sqS6Meci8cLyGLmfIMuL/pxWebq26MaVcUrKlfcPaitaFqXXCDlsiIiIiIiIiOzStdwub7qB98OABrl27pv6clJSExMREVK1aFbVr10Z0dDRu3bqFr7/+GgDw9ttvY8mSJXjvvfcwYsQI7NmzBxs3bsT27dutVQQiIqtg1zcRERERERERSe7EiRNo1aoVWrVqBQCIiopCq1atMHPmTADAnTt3cPPmTfX8gYGB2L59O3bu3IkWLVrgk08+wZdffomwsDCrxE9EZC28w5aIiIiIiIiIJNelSxcIIfROT0hI0LnM6dOnzRgVEZHt4x22RERERERERERERDaCHbZERERERERERERENoIdtkREREREREREREQ2gh22RERERERERERERDaCHbZEREREVK4dOHAAvXr1gp+fH2QyGbZu3ap33rfffhsymQxxcXEWi4+IiIiIyhd22BIRERFRuZaTk4MWLVpg6dKlxc63ZcsWHDlyBH5+fhaKjIiIiIjKowrWDoCIiIiIyJp69OiBHj16FDvPrVu38J///Ae//PILevbsaaHIiIiIiKg8MrrD9sCBA/joo49w8uRJ3LlzB1u2bEHfvn3V04UQmDVrFlauXImMjAx06NAB8fHxqF+/vpRxExEREZVK0PRt1g6B7IRSqcTQoUMxefJkBAUFGbRMXl4e8vLy1J+zsrLU61IqlWaJ01hKpRJCCJuJx1j2HL89xw7Yd/z2HDsgXfz2Wn4iovLC6A5b1SNjI0aMwKuvvqo1/cMPP8Rnn32G1atXIzAwEDNmzEBYWBguXLgAJycnSYImIiIiIrKUhQsXokKFChg/frzBy8TGxiImJkYrPT09Hbm5uVKGZzKlUonMzEwIISCX299IafYcvz3HDth3/PYcOyBd/NnZ2RJGRUREUjO6w7a4R8aEEIiLi8P777+PPn36AAC+/vpreHt7Y+vWrRg0aFDpoiUiIiIisqCTJ09i8eLFOHXqFGQymcHLRUdHIyoqSv05KysLtWrVgqenJ9zc3MwRqtGUSiVkMhk8PT3ttuPKXuO359gB+47fnmMHpIufN1MREdk2ScewTUpKQkpKCkJDQ9Vp7u7uCA4OxuHDh3V22FrjcbHiHiOR6pRtS4+Y2PtjP8YoT2UFyld5rVlWKfIsbjiZx48f4/3338dPP/2EP//8E+7u7ggNDcWCBQuKfbHN7Nmzte7eatiwIS5dulTqeImI6KmDBw8iLS0NtWvXVqcVFBTg3XffRVxcHG7cuKFzOYVCAYVCoZUul8ttqpNIJpPZXEzGsOf47Tl2wL7jt+fYAWnit9eyExGVF5J22KakpAAAvL29NdK9vb3V04qyxuNixT1G0sBDmjzS0tKkWZEE7P2xH2NYs6yR3xwt9TqWDg02an5uW8uQ4pGx4oaTefjwIU6dOoUZM2agRYsWuH//PiZMmIDevXvjxIkTxa43KCgIu3btUn+uUIHvkiQiktLQoUM1bkYAgLCwMAwdOhQRERFWioqIiIiIyjKrf7O3xuNixT1GciVDmjy8vLykWZEE7P2xH2NYs6xS7DvG7jfctpYhxSNjxQ0n4+7ujp07d2qkLVmyBO3atcPNmzc17uoqqkKFCvDx8Sl1fERE5dmDBw9w7do19eekpCQkJiaiatWqqF27NqpVq6Yxf8WKFeHj44OGDRtaOlQiIiIiKgck7bBVdRqkpqbC19dXnZ6amoqWLVvqXMZaj4vpe4xEqoetba3zzN4f+zGGtcoqxb5jSszctuZnjbrNzMyETCaDh4dHsfNdvXoVfn5+cHJyQkhICGJjY4vt4LWHt5YXZm/DfthTvPYUK2CeeM11ZMsByIxcvzW3gy3tC9aK4cSJE+jatav6s+pmgvDwcCQkJFglJiIiIiIqvyTtsA0MDISPjw92796t7qDNysrC0aNHMWbMGCmzIiIqs3JzczFlyhQMHjy42CcNgoODkZCQgIYNG+LOnTuIiYlBp06dcO7cObi6uupcxh7eWl6YvQ37YU/x2lOsgHnilWoYpKJkAGo4P/2/MHAZaw6lZEv7grXeWt6lSxcIYejWgt5xa4mIiIiIpGB0h21Jj4xNnDgRH3zwAerXr4/AwEDMmDEDfn5+6pfrEBGRfo8fP8aAAQMghEB8fHyx8xYeYqF58+YIDg6Gv78/Nm7ciJEjR+pcxh7eWl6YvQ37YU/x2lOsgHnilWoYpKJU0V3NMPzpC2sOpWRL+wLfWk5EREREZEKHbUmPjL333nvIycnB6NGjkZGRgY4dO2LHjh28ACciKoGqszY5ORl79uwxugPVw8MDDRo00PhRrSh7eWt5YfY27Ic9xWtPsQLSx2vOh+/F/6/f0DysvQ1sZV+wdv5ERERERLbA6A7bkh4Zk8lkmDNnDubMmVOqwIiIyhNVZ+3Vq1exd+9erRfcGOLBgwe4fv06hg4daoYIiYiIiIiIiMgSeBsDEZEFPHjwAImJiUhMTATw73AyN2/exOPHj9GvXz+cOHECa9euRUFBAVJSUpCSkoL8/Hz1Orp164YlS5aoP0+aNAn79+/HjRs38Pvvv+OVV16Bg4MDBg8ebOniEREREREREZFEJH3pGBER6VbccDKzZ8/GDz/8AADqFzaq7N27F126dAEAXL9+HXfv3lVP+/vvvzF48GDcu3cPnp6e6NixI44cOQJPT0/zFoaIiIjIDkzdcAL/++OW+nOv5jUwv39rK0ZERERkGHbYEhFZQEnDyRjydvKibyVfv359acMiIiIiKpOKdtYC+P/PAhM717JOUERERAZihy0REVEZFjR9m850OYAGHsCVjJJfjHV+Xh+JoyIiIjKvop21Ktv/uM0OWyIisnnssCUiIiIiIiK7dPByCmK3n0Nq1iN4u1VCdM+m6NTQx9phERERlQpfOkZERERERER25+DlFLz99VEk38tB7mMlku/l4O2vj+Lg5RRrh0ZERFQq7LAlIiIiIiIiuxO7/Zze9F7Na+ic1rO5nzlDIiIikgQ7bImIiIiIiMjupGY90pu+YGBbrU7bXs1rYH7/NpYIjYiIqFQ4hi0RERERERHZHW+3Ski+l6MzHQAWDGyLBQPbakxTKkt61SYREZH18Q5bIiIiIiIisjvRPZsalU5ERGQv2GFLREREREREdqdTQx8sGxYM/2rOcKooh381ZywbFoxODX0skv/8H84gaPo29d/8H85YJF97s3TpUgQEBMDJyQnBwcE4duxYsfPHxcWhYcOGqFSpEmrVqoV33nkHubm5FoqWiMg2cEgEIiIiIiIiskudGvpYrIO2sPk/nMHaozc00lSfp/VuYfF4bNWGDRsQFRWFZcuWITg4GHFxcQgLC8Ply5fh5eWlNf+6deswdepUrFq1Cu3bt8eVK1cwfPhwyGQyLFq0yAolICKyDt5hS0RERERERGSEop21JaWXV4sWLcKoUaMQERGBJk2aYNmyZahcuTJWrVqlc/7ff/8dHTp0wOuvv46AgAB0794dgwcPLvGuXCKisoZ32BIRERERERGRpPLz83Hy5ElER0er0+RyOUJDQ3H48GGdy7Rv3x5r1qzBsWPH0K5dO/z555/46aefMHToUL355OXlIS8vT/05KysLwNMXzJX2JXNKpRJCCL6sDrZXF8bcfVg4ZkOXK2kZOQCZjmnWqh9TyiUVW9s3rElXXZhaL+ywJSIiIiIiIiJJ3b17FwUFBfD29tZI9/b2xqVLl3Qu8/rrr+Pu3bvo2LEjhBB48uQJ3n77bUybNk1vPrGxsYiJidFKT09PL/XYt0qlEpmZmRBCQC4v3w8o21pdNPAwfN60tDSjlytpGRmAGs5P/y/0LGdJppRLKra2b1iTrrrIzs42aV3ssCUiIiIiIiK7lJSejVUHr+HSnUw08nXHiE71EOjpavZ8hwQH6Bz+YEhwgNnzLsv27duH+fPn44svvkBwcDCuXbuGCRMmYO7cuZgxY4bOZaKjoxEVFaX+nJWVhVq1asHT0xNubm6likepVEImk8HT05MdUTZWF1cyDJ+38HjJhi5X0jKqGriaARS+f1LX2MyWYEq5pGJr+4Y16aoLJycnk9bFDlsiIiIiKtcOHDiAjz76CCdPnsSdO3ewZcsW9O3bFwDw+PFjvP/++/jpp5/w559/wt3dHaGhoViwYAH8/PysGzhROZeUno0BX+xH3hMlCpQCl1OysOPsLWwc29nsnbaqF4sV7rQdEhzAF44VUr16dTg4OCA1NVUjPTU1FT4+ul8UN2PGDAwdOhRvvvkmAKBZs2bIycnB6NGjMX36dJ2dQQqFAgqFQitdLpdL0nkkk8kkW5e9s6W6MOYh88LxGrqcIcuI/59WeLq16saUcknJlvYNaytaF6bWCWuSiIiIiMq1nJwctGjRAkuXLtWa9vDhQ5w6dQozZszAqVOn8P333+Py5cvo3bu3FSIlosJWHbym7qwFgAKlQN4TJVYdvGaR/Kf1boHz8/pgdp/mcFY4YN3RG2g350dsOpZkkfxtnaOjI9q0aYPdu3er05RKJXbv3o2QkBCdyzx8+FCrc8PBwQEAIITQtQgRUZnEO2yJiIiIqFzr0aMHevTooXOau7s7du7cqZG2ZMkStGvXDjdv3kTt2rUtESIR6XDpTqa6s1alQClw6U6mRfKf/8MZrWERcvIKMHvbHwCA/u0CLRKHLYuKikJ4eDjatm2Ldu3aIS4uDjk5OYiIiAAADBs2DDVq1EBsbCwAoFevXli0aBFatWqlHhJhxowZ6NWrl7rjloioPGCHLRERERGRETIzMyGTyeDh4WHtUIjKtUa+7rickqXRaesgl6GRr7vZ89bVWVvYRzvOs8MWwMCBA5Geno6ZM2ciJSUFLVu2xI4dO9QvIrt586bGHbXvv/8+ZDIZ3n//fdy6dQuenp7o1asX5s2bZ60iEBFZBTtsiYiIiIgMlJubiylTpmDw4MHFvswmLy8PeXl56s9ZWVkAnj4OrFQaM/Ke+SiVSgghbCYeY9lz/PYcO2A78Ud0rINfzv6N/CcCBUrAQQ44VpAhomMdvbFJFfu3R28UO75gbl6BWepHqvgtue3GjRuHcePG6Zy2b98+jc8VKlTArFmzMGvWLAtERkRku9hhS0RERERkgMePH2PAgAEQQiA+Pr7YeWNjYxETE6OVnp6ejtzcXHOFaBSlUonMzEwIIezyJSH2HL89xw7YTvyVAawc1BQ7z9/G3/cfomaVynghyA+VlY+QlvZI5zJSxd7Ao/jpThXlSEtLM3n9+kgVf3Z2toRRERGR1NhhS0RERERUAlVnbXJyMvbs2VPs3bUAEB0djaioKPXnrKws1KpVC56eniUuaylKpRIymQyenp5222lor/Hbc+yAbcXv5QU0q+9v8PxSxX4lo/jpM3sHwcvLy+T16yNV/E5OThJGRUREUmOHLRERERFRMVSdtVevXsXevXtRrVq1EpdRKBRQKBRa6XK53OodXIXJZDKbi8kY9hy/PccO2Hf8UsQ+ODhA5xi2FeTA+72am3X8Winit8ftRkRUnkjeYVtQUIDZs2djzZo1SElJgZ+fH4YPH64ePJyIiIiIyJY8ePAA165dU39OSkpCYmIiqlatCl9fX/Tr1w+nTp3Cjz/+iIKCAqSkpAAAqlatCkdHR2uFTURWNK13CwDQ6LQdEhygTif7EzR9m1aaHE+Hv7iSARQe9ff8vD4WikqTrhj1KW2MRfPSVxeF8zI1PkuWy9bZel1Y8jix9bowN8k7bBcuXIj4+HisXr0aQUFBOHHiBCIiIuDu7o7x48dLnR0RERERUamcOHECXbt2VX9WDWUQHh6O2bNn44cffgAAtGzZUmO5vXv3okuXLpYKk4hszLTeLdhBS0REZiF5h+3vv/+OPn36oGfPngCAgIAAfPvttzh27JjUWRERERERlVqXLl0ghNA7vbhpRERERERSk3zgmvbt22P37t24cuUKAODMmTM4dOgQevToIXVWRER248CBA+jVqxf8/Pwgk8mwdetWjelCCMycORO+vr6oVKkSQkNDcfXq1RLXu3TpUgQEBMDJyQnBwcH8cYyIiIiIiIjIzkl+h+3UqVORlZWFRo0awcHBAQUFBZg3bx6GDBmic/68vDzk5eWpP2dlZQF4+vZLpbLoyCjSUCqVEELoXL9UPdjmit0UxZW3rLFmWaXYd4yNm9vWcnmXVk5ODlq0aIERI0bg1Vdf1Zr+4Ycf4rPPPsPq1asRGBiIGTNmICwsDBcuXND7Ft8NGzYgKioKy5YtQ3BwMOLi4hAWFobLly+b5a3ERERERERERGR+knfYbty4EWvXrsW6desQFBSExMRETJw4EX5+fggPD9eaPzY2FjExMVrp6enpyM3NNSrvyG+OGjSfDEANZ+BWDlD0AbcGHkZlqVdaWpo0K5KAUqlEZmYmhBBl/m2g1iyrFPtO/0/+Z9T8+vblpUODSx+MjbHmts3Ozi71Onr06KH3SQMhBOLi4vD++++jT5+ng6V//fXX8Pb2xtatWzFo0CCdyy1atAijRo1CREQEAGDZsmXYvn07Vq1ahalTp5Y6ZiIiIqLyZNOxJHy04zwe5hWgssIBk18MQv92gdYOi4iIyiHJO2wnT56MqVOnqjsYmjVrhuTkZMTGxurssI2Ojla/2AF4eodtrVq14OnpCTc3N6PyvpJh2Hyqrp6rGdpvN5SKLd3dplQqIZPJ4OnpWS46bK1VVkP3Pynp25dtaf+TijW3rb47XKWSlJSElJQUhIaGqtPc3d0RHByMw4cP6+ywzc/Px8mTJxEdHa1Ok8vlCA0NxeHDh/XmZY2nGkrD3u4it8V49R0tcjz90ceQo8kWymOOujVXS2JM3apYs45tab+1hRiIqHzadCwJs7f9of6ck1eg/sxOWyIisjTJO2wfPnyo1Zni4OCg9wJcoVBAoVBopcvlcqM7ZYy5xBf/P7+5vhbYWseoTCYzqU7tkbXKaq2vmLr25bK6na21bc2dX0pKCgDA29tbI93b21s9rai7d++ioKBA5zKXLl3Sm5eUTzVYgr09IWCL8eq7+191hz6g/bRJUbbw1Ig56laqp2qKMqZuVaxZx7a030rxRAMRWcbqg1exJvGo+hq0V/MaWDCwrVVjKo2PdpzXm84OWyIisjTJO2x79eqFefPmoXbt2ggKCsLp06exaNEijBgxQuqsiIjISFI+1WAJ9vaEgC3Gq+/uf2OeNrGFu/bNUbfmejLClCd5rFnHtrTfmvuJBiKSxrRNJ3H15j8aaf/74xYAYMHAtjh4OQWx288hNesRvN0qIbpnU3Rq6GONUA32MK/AqHQiIiJzkrzD9vPPP8eMGTMwduxYpKWlwc/PD2+99RZmzpwpdVZERGWCj8/TLzCpqanw9fVVp6empqJly5Y6l6levTocHByQmpqqkZ6amqpeny5SPtVgKfb2hICtxVtch6GhT5vYSlmkrltzPhlh7JM81q5jW9lvrZ0/ERlm+x+3dT6l8L8/bqFny5p4++t/3y2SfC8Hb399FMuGBdt0p21lhQNydHTOVlY4WCEaIiIq7yS/KnZ1dUVcXBySk5Px6NEjXL9+HR988AEcHR2lzoqIqEwIDAyEj48Pdu/erU7LysrC0aNHERISonMZR0dHtGnTRmMZpVKJ3bt3612GiIiIyNxit58zKt2cDl5OwUuLdqHN7P/hpUW7cPCy7qGmAGDyi0FGpRMREZkTb2MgIrKABw8eIDExEYmJiQCevmgsMTERN2/ehEwmw8SJE/HBBx/ghx9+wNmzZzFs2DD4+fmhb9++6nV069YNS5YsUX+OiorCypUrsXr1aly8eBFjxoxBTk4OIiIiLFw6IiIioqdSsx4ZlW4uBy+n4O2vjyL5Xg5yHyvVd/oeupKqc/7+7QIxu09zOCscIAPgrHDA7D7NOX4tERFZheRDIhARkbYTJ06ga9eu6s+qcWTDw8ORkJCA9957Dzk5ORg9ejQyMjLQsWNH7NixQ2M8x+vXr+Pu3bvqzwMHDkR6ejpmzpyJlJQUtGzZEjt27NB6ERkRERGRlHo298PVm7e10ns1r4E/bmUg+V6O1jRvt0qWCE1N3x29C386h/++3kzntP7tAtlBS0RENoEdtkREFtClSxcIof998TKZDHPmzMGcOXP0znPjxg2ttHHjxmHcuHFShEhERERkkPn92+Cj73JxJfHfF4/1al5D/cKxwmPYqkT3bGqWWOJ3X0T8visoUAIOcmBMlwYY062xzdzpS0REZAp22BIREREREZFRwjvVx+TXvLReFtipoQ+WDQtG7PZzSM16BG+3Soju2dQsLxyL330RS/ZcUX8uUEL92dutksXv9D14OcUi5SYiorKPHbZEREREREQkmU4NfSzSURm/74re9KVvBOu803fKS+a507foncWqMXOXDQtmpy0RERmNLx0jIiIiIiIiu1Og1J+uutPXv5oznCrK4V/NGcuGBaNjA/OM9a9vzFx96URERMXhHbZERERERERkdxzkujttHf7/tiRdd/oqlXp6eUuJY+YSEZGUeIctERERERER2Z0xXRoYlW5O+sbGNeeYuUREVHaxw5aIiIiIiIjszphujTHu+QbqO2od5MC45xtgTLfGFo8luqfusXH1pRMRERWHQyIQERERERGRXRrTrbFVOmiLUo2ZG7v9HFKzHsHbrRKiezblC8eIiMgk7LAlIiKjBU3fJsl6zs/rI8l6iIiIiKxN15i5REREpuCQCERERERUrh04cAC9evWCn58fZDIZtm7dqjFdCIGZM2fC19cXlSpVQmhoKK5evWqdYImIiIiozGOHLRERERGVazk5OWjRogWWLl2qc/qHH36Izz77DMuWLcPRo0fh7OyMsLAw5ObmWjhSItInfvdFNJ+xDUHTt6H5jG2I333R2iERERGZjEMiEBEREVG51qNHD/To0UPnNCEE4uLi8P7776NPn6fDuHz99dfw9vbG1q1bMWjQIEuGSkQ6xO++iCV7rqg/FyiBJXuu4H5OPh49UeLSnUw08nXHiE714F/N2YqREhERGYZ32BIRERER6ZGUlISUlBSEhoaq09zd3REcHIzDhw9bMTIiUonfd0Vn+tqjN7Dt9F+4cDsT207/hQFf7MeNu9kWjo6WLl2KgIAAODk5ITg4GMeOHSt2/oyMDERGRsLX1xcKhQINGjTATz/9ZKFoiYhsA++wJSIiIiLSIyUlBQDg7e2tke7t7a2epkteXh7y8vLUn7OysgAASqUSSqXSDJEaT6lUQghhM/EYy57jt+fYAduLXyj134kklALy///38ZMCfHXoGsaG+NlM7MaSqu4tVf4NGzYgKioKy5YtQ3BwMOLi4hAWFobLly/Dy8tLa/78/Hy88MIL8PLywubNm1GjRg0kJyfDw8PDIvESEdkKdtgSEREREUksNjYWMTExWunp6ek2M/atUqlEZmYmhBCQy+3vwTt7jt+eYwdsL/5GVQClMGzeB/fvISOjss3Ebiyp6j472zJ3Gi9atAijRo1CREQEAGDZsmXYvn07Vq1ahalTp2rNv2rVKvzzzz/4/fffUbFiRQBAQECARWIlIrIl7LAlIiIiItLDx8cHAJCamgpfX191empqKlq2bKl3uejoaERFRak/Z2VloVatWvD09ISbm5vZ4jWGUqmETCaDp6en3XZc2Wv89hw7YHvxP9+yPr7Ye1UrXQagcD+ugxxoUrcaPDw8bCZ2Y0lV905OThJGpVt+fj5OnjyJ6OhodZpcLkdoaKjeIWV++OEHhISEIDIyEtu2bYOnpydef/11TJkyBQ4ODjqXkeqJBl21KcfT/ajoNGvdoW3MFi9tjEXz0lcXhfMyNT5TljN3XiUtY659w5LlMoU91IU16XoKwtTY2GFLRERERKRHYGAgfHx8sHv3bnUHbVZWFo4ePYoxY8boXU6hUEChUGily+Vym+okkslkNheTMew5fnuOHbCt+MeENgFkMsTvu4IC5dOO2UHPBGDL6b+Q90SJAqWAg1yGihXkiOhYDzLlI5uJ3RRS1L0lyn737l0UFBToHFLm0qVLOpf5888/sWfPHgwZMgQ//fQTrl27hrFjx+Lx48eYNWuWzmWkeqKhgYd2mgxAjf9/T13hzv+0tDSD1yslXTHqU9oYi+alry4K52VqfKYsZ+68SlrGXPuGJctlCnuoC2vS9RSEqU80sMOWiIiIiMq1Bw8e4Nq1a+rPSUlJSExMRNWqVVG7dm1MnDgRH3zwAerXr4/AwEDMmDEDfn5+6Nu3r/WCJiINY7o1xphujTXSBofUwaqD13DpTiYa+bpjRKd68K/mjLS0R1aKkkqiVCrh5eWFFStWwMHBAW3atMGtW7fw0Ucf6e2wleqJhisZ2mmqbu2rGUDhe+R0jb9rCbpi1Ke0MRbNS19dFM7L1PhMWc7ceZW0jLn2DUuWyxT2UBfWpOspCFOfaGCHLRERERGVaydOnEDXrl3Vn1Vf/MPDw5GQkID33nsPOTk5GD16NDIyMtCxY0fs2LHDIo8UE5HpAj1dMffVVhpptvLYbHlQvXp1ODg4IDU1VSM9NTVVPdxMUb6+vqhYsaLG8AeNGzdGSkoK8vPz4ejoqLWMVE806NszxP9PKzzdWndnG7P3ljZGXXnpqovCeZkanynLmTsvQ5Yxx75hyXKZwh7qwtqKPgVhamy2UyIiIiIiIivo0qULhBBafwkJCQCeXnjPmTMHKSkpyM3Nxa5du9CgQQPrBk1EZOMcHR3Rpk0b7N69W52mVCqxe/duhISE6FymQ4cOuHbtmkbH+pUrV+Dr66uzs5aIqKxihy0RERERERERSS4qKgorV67E6tWrcfHiRYwZMwY5OTmIiIgAAAwbNkzjpWRjxozBP//8gwkTJuDKlSvYvn075s+fj8jISGsVgYjIKjgkAhERERERERFJbuDAgUhPT8fMmTORkpKCli1bYseOHeoXkd28eVPjceFatWrhl19+wTvvvIPmzZujRo0amDBhAqZMmWKtIhARWYVZOmxv3bqFKVOm4Oeff8bDhw9Rr149fPXVV2jbtq05siMiIipzgqZvs3YIZEFSbe/z8/pIsh4iIiKpjBs3DuPGjdM5bd++fVppISEhOHLkiJmjIiKybZJ32N6/fx8dOnRA165d8fPPP8PT0xNXr15FlSpVpM6KiIiIiIiIiIiIqEyRfAzbhQsXolatWvjqq6/Qrl07BAYGonv37qhbt67UWRERlRkBAQGQyWRaf/rG60pISNCal28rJyIiIiIiIrJ/knfY/vDDD2jbti369+8PLy8vtGrVCitXrpQ6GyKiMuX48eO4c+eO+m/nzp0AgP79++tdxs3NTWOZ5ORkS4VLRERERERERGYi+ZAIf/75J+Lj4xEVFYVp06bh+PHjGD9+PBwdHREeHq41f15eHvLy8tSfs7KyAABKpRJKpdKovA3tfZYDkBkxvymMjd2clEolhBA2FZO5WLOs5tyfistT175cFre1NbetJfL09PTU+LxgwQLUrVsXnTt31ruMTCaDj4+PuUMjIiIiIiIiIguSvMNWqVSibdu2mD9/PgCgVatWOHfuHJYtW6azwzY2NhYxMTFa6enp6cjNzTUq7wYehs0nA1DD+en/hVE5GK7/J/+TZD1LhwaXeh1KpRKZmZkQQmi8gdPeRX5zVCtNtW1v5Ri+baWoY8Dw/U9K+vbltLQ0ywdjZtbcj7Ozsy2aX35+PtasWYOoqCjIZDK98z148AD+/v5QKpVo3bo15s+fj6CgIAtGSkRERERERERSk7zD1tfXF02aNNFIa9y4Mb777jud80dHRyMqKkr9OSsrC7Vq1YKnpyfc3NyMyvtKhmHzqbp6rmYAtn4fopeXV6nXoVQqIZPJ4OnpWaY6bHVtb1O2rRR1rC8ec9NXXqnKZEusuR9bemzYrVu3IiMjA8OHD9c7T8OGDbFq1So0b94cmZmZ+Pjjj9G+fXucP38eNWvW1LucVE81SLUFSsrT3p4QkDJec+/lxjxtYgv1b459wVx1bIknefQxpX5s6TizhRiIiIiIiKxN8g7bDh064PLlyxppV65cgb+/v875FQoFFAqFVrpcLje6U8aYS3zx//Pb+tcCqTqmZDKZSXVqy/RtO2O3rVR1Yq19SVd5y9J2Lsxa+7Gl8/vvf/+LHj16wM/PT+88ISEhCAkJUX9u3749GjdujOXLl2Pu3Ll6l5PqqQap7igv6W5we3tCQMp4zX3XvjFPm9jCXfvm2BfMVceWeJJHH1O2lS0dZ5Z+ooGIiIiIyBZJ3mH7zjvvoH379pg/fz4GDBiAY8eOYcWKFVixYoXUWRERlTnJycnYtWsXvv/+e6OWq1ixIlq1aoVr164VO59UTzVIdUd5SXeD29sTAlLGa+679o15IsEW7to3x75grjq25pM8pmwrWzrOLP1EAxERERGRLZK8w/aZZ57Bli1bEB0djTlz5iAwMBBxcXEYMmSI1FkREZU5X331Fby8vNCzZ0+jlisoKMDZs2fx0ksvFTufVE81SNUJZUie9vaEgFTxWqKjz9AnEmyl7qXeF8xZx9Z6ksfUurGV48za+RMRERER2QLJO2wB4OWXX8bLL79sjlUTEZVZSqUSX331FcLDw1GhgmbzPGzYMNSoUQOxsbEAgDlz5uDZZ59FvXr1kJGRgY8++gjJycl48803rRE6EREREREREUnELB22RERkvF27duHmzZsYMWKE1rSbN29q3Hl2//59jBo1CikpKahSpQratGmD33//Xeulj0RERERERERkX9hhS0RkI7p37w4hdL+iaN++fRqfP/30U3z66acWiIqIiIiIiIiILIkDhRERERERERERERHZCN5hS0RERERERERkgqDp2wye9/y8PmaMRDdbj68sM7Tuy0O9W3I/LCv7PO+wJSIiIiIiIiIiIrIR7LAlIiIiIiIiIiIishHssCUiIiIiIiIiIiKyERzDloiIiIioGAUFBZg9ezbWrFmDlJQU+Pn5Yfjw4Xj//fchk8msHR5RuTNu9WHsvZKmc9qQ4ABM693CwhERERFJix22RERERETFWLhwIeLj47F69WoEBQXhxIkTiIiIgLu7O8aPH2/t8IjKleI6awFg7dEbAMBOWyIismscEoGIiIiIqBi///47+vTpg549eyIgIAD9+vVD9+7dcezYMWuHRlTuFNdZq6LqtCUiIrJXvMOWiIiIiKgY7du3x4oVK3DlyhU0aNAAZ86cwaFDh7Bo0SK9y+Tl5SEvL0/9OSsrCwCgVCqhVCrNHrMhlEolhBA2E4+x7Dl+e44dsG78ht5xpC821v2/6yEiItvFDlsq94Kmb7N2CJKTqkzn5/WRZD1SxCMHsPM/waUPhoiIyEhTp05FVlYWGjVqBAcHBxQUFGDevHkYMmSI3mViY2MRExOjlZ6eno7c3FxzhmswpVKJzMxMCCEgl9vfg3f2HL89xw5YN/4GHobNl5am+05c1v1T2dnZEkZFRERSY4ctEREREVExNm7ciLVr12LdunUICgpCYmIiJk6cCD8/P4SHh+tcJjo6GlFRUerPWVlZqFWrFjw9PeHm5map0IulVCohk8ng6elptx1X9hq/PccOWDd+Py9P7LuSXuw8g9v5w8vLS+c01v1TTk5OEkZFRERSY4ctEREREVExJk+ejKlTp2LQoEEAgGbNmiE5ORmxsbF6O2wVCgUUCoVWulwut6lOIplMZnMxGcOe47fn2AHrxf95ePtiXzw2JDigxBeOse5ht2UnIiov2GFLRERERFSMhw8fanVuODg4cAxIIhPF776I+H1XUKAEHOTAmC4NMKZbY4OXXxIeYsboiIiIrI8dtkRERERExejVqxfmzZuH2rVrIygoCKdPn8aiRYswYsQIa4dGZHfid1/Ekj1X1J8LlFB/NqbTloiIqCzjcxBERERERMX4/PPP0a9fP4wdOxaNGzfGpEmT8NZbb2Hu3LnWDo3I7sTvu2JUOhERUXnEO2yJiIjIrgRN32btEKiccXV1RVxcHOLi4qwdCpHdK9Azkoi+dCIiovKId9gSERERERGRRTjo+QaqL53s39KlSxEQEAAnJycEBwfj2LFjBi23fv16yGQy9O3b17wBEhHZIJ4WiYiIiIiIyCLGdGlgVDrZtw0bNiAqKgqzZs3CqVOn0KJFC4SFhSEtLa3Y5W7cuIFJkyahU6dOFoqUiMi2sMOWiIiIiIiILGJMt8YY93wD9R21DnJg3PMN+MKxMmrRokUYNWoUIiIi0KRJEyxbtgyVK1fGqlWr9C5TUFCAIUOGICYmBnXq1LFgtEREtoNj2BIREREREZHFjOnWmB205UB+fj5OnjyJ6OhodZpcLkdoaCgOHz6sd7k5c+bAy8sLI0eOxMGDB0vMJy8vD3l5eerPWVlZAAClUgml0vDBkXXdzSYHINMxrfB6jbkLzph4dDElL1PjK7qcvrowR16GLGfuvEpaRsp9Q4r9yZS8TGHJ48SS+5NUlEolhBAa6zU1D3bYEhEREREREZGk7t69i4KCAnh7e2uke3t749KlSzqXOXToEP773/8iMTHR4HxiY2MRExOjlZ6eno7c3FyD19PAQztNBqCG89P/i0LphYd00LWcPiUNBVESU/IyNb6iy+mrC3PkZchy5s6rpGWk3Dek2J9MycsUljxOLLk/SUWpVCIzMxNCCMjlT7uOs7OzTVoXO2yJiIiIiIiIyKqys7MxdOhQrFy5EtWrVzd4uejoaERFRak/Z2VloVatWvD09ISbm5vB67mSoZ2mulPvagZQ+B45Ly+vYpfTp/BypjAlL1PjK7qcvrowR16GLGfuvEpaRsp9Q4r9yZS8TGHJ48SS+5NUlEolZDIZPD091R22Tk5OJq2LHbZEREREREREJKnq1avDwcEBqampGumpqanw8fHRmv/69eu4ceMGevXqpU5TPUpcoUIFXL58GXXr1tVaTqFQQKFQaKXL5XJ1h4kh9D20LP5/WuHphddrzMPOxsSjiyl5mRqfruV01YW58ippOXPnZcgyUu0bUuxPpuRlCkseJ5bcn6Qkk8k02h9T8zD7S8cWLFgAmUyGiRMnmjsrIiIiIiIispL43RfRfMY2BE3fhuYztiF+90Vrh0RW5OjoiDZt2mD37t3qNKVSid27dyMkJERr/kaNGuHs2bNITExU//Xu3Rtdu3ZFYmIiatWqZcnwiYisyqwdtsePH8fy5cvRvHlzc2ZDRGT3Zs+eDZlMpvHXqFGjYpfZtGkTGjVqBCcnJzRr1gw//fSThaIlIiIi0hS/+yKW7LmCgv+/talACSzZc4WdtuVcVFQUVq5cidWrV+PixYsYM2YMcnJyEBERAQAYNmyY+qVkTk5OaNq0qcafh4cHXF1d0bRpUzg6OlqzKEREFmW2DtsHDx5gyJAhWLlyJapUqWKubIiIyoygoCDcuXNH/Xfo0CG98/7+++8YPHgwRo4cidOnT6Nv377o27cvzp07Z8GIiYiIiJ6K33fFqHQqHwYOHIiPP/4YM2fORMuWLZGYmIgdO3aoX0R28+ZN3Llzx8pREhHZHrONYRsZGYmePXsiNDQUH3zwgd758vLykJeXp/6clZUF4OmjEqrxagxlaO+zHE/fYmf28SAkYGwd6FuHEEKSddkSXdvPnratFMxdXqn2GSnikwNW248tlWeFChV0juely+LFi/Hiiy9i8uTJAIC5c+di586dWLJkCZYtW2bOMImIiIi0FOi5XNKXTuXHuHHjMG7cOJ3T9u3bV+yyCQkJ0gdERGQHzNJhu379epw6dQrHjx8vcd7Y2FjExMRopaenpyM3N9eofBt4GDafDEAN56f/F0blYHn9P/lfqdchAzD35QYQQphlQGVr0bW97WnbSsHc5ZVi/wMMPzaLIwOQkZFhlf04OzvbIvlcvXoVfn5+cHJyQkhICGJjY1G7dm2d8x4+fFjjbbgAEBYWhq1btxabh1Q/kkm1BUrK095+cJIyXnPv5cb84GML9V+4bm39TGbNHw9N2Va2dJzZQgxEZBoHue7OWQdbb7SJiIhskOQdtn/99RcmTJiAnTt3wsnJqcT5o6OjNTodsrKyUKtWLXh6esLNzc2ovK9kGDaf6prhaoZxb4+zV3IAHh4e8PT0LFMdtrq2d3nctkD5KK8192ND2rLSCg4ORkJCAho2bIg7d+4gJiYGnTp1wrlz5+Dq6qo1f0pKivpRMhVvb2+kpKQUm49UP5JJ0QkPAGlpacVOVyqVyMzMtJsfnKSMV6o61seYH3xK2k6GivzmqMnLquK9lWP+uikta/54aMq20rXflmZbFbZ0aLBR81vqBzIikt6YLg2wZI/28AdjujSwQjRERET2TfIO25MnTyItLQ2tW7dWpxUUFODAgQNYsmQJ8vLy4ODgoJ6mUCigUCi01iOXy43+smtMh5X4//nLeieXikwmM6lObZm+bVfetm15Kq+19mNL5NejRw/1/5s3b47g4GD4+/tj48aNGDlypGT5SPUjmaE/kJXEy8ur2OlKpRIymcxufnCSMl6p6lgfY37wKWk7Gao0ZbKnH6isGasp20rXfmupY7woS/xARkTmMaZbYwBPx6wtUD69s3ZMlwbqdCIiIjKc5B223bp1w9mzZzXSIiIi0KhRI0yZMkWjs5aIiHTz8PBAgwYNcO3aNZ3TfXx8kJqaqpGWmppa4hi4Uv1IJlUnlCF52tsPTlLFa4mOPkN/8JGq7ktbJnv6gcpasZq6rYrut5Y8xkszPxHZljHdGrODloiISAKSXxW7urqiadOmGn/Ozs6oVq0amjZtKnV2RERl0oMHD3D9+nX4+vrqnB4SEoLdu3drpO3cuRMhISGWCI+IiIiIiIiIzIS3MRAR2YBJkyZh//79uHHjBn7//Xe88sorcHBwwODBgwEAw4YNQ3R0tHr+CRMmYMeOHfjkk09w6dIlzJ49GydOnND7Bl4iIiIiIiIisg+SD4mgy759+yyRDRGR3fr7778xePBg3Lt3D56enujYsSOOHDkCT09PAMDNmzc1HhVu37491q1bh/fffx/Tpk1D/fr1sXXrVj7JQERERERERGTnLNJhS0RExVu/fn2x03X98NW/f3/079/fTBERERERERERkTVwSAQiIiIiohLcunULb7zxBqpVq4ZKlSqhWbNmOHHihLXDIiIiIqIyiHfYEhEREREV4/79++jQoQO6du2Kn3/+GZ6enrh69SqqVKli7dCIiIiIqAxihy0RERERUTEWLlyIWrVq4auvvlKnBQYGWjEiIiIiIirL2GFLRERERFSMH374AWFhYejfvz/279+PGjVqYOzYsRg1apTeZfLy8pCXl6f+nJWVBQBQKpVQKpVmj9kQSqUSQgibicdY9hy/PccO2Hf89hw7IF389lp+IqLygh22RERERETF+PPPPxEfH4+oqChMmzYNx48fx/jx4+Ho6Ijw8HCdy8TGxiImJkYrPT09Hbm5ueYO2SBKpRKZmZkQQkAut79XW9hz/PYcO2Df8dtz7IB08WdnZ0sYFRERSY0dtkRERERExVAqlWjbti3mz58PAGjVqhXOnTuHZcuW6e2wjY6ORlRUlPpzVlYWatWqBU9PT7i5uVkk7pIolUrIZDJ4enrabceVvcZvz7ED9h2/PccOSBe/k5OThFEREZHU2GFLRERERFQMX19fNGnSRCOtcePG+O677/Quo1AooFAotNLlcrlNdRLJZDKbi8kY9hy/PccO2Hf89hw7IE389lp2IqLygq00EREREVExOnTogMuXL2ukXblyBf7+/laKiIiIiIjKMnbYEhEREREV45133sGRI0cwf/58XLt2DevWrcOKFSsQGRlp7dCIiIiIqAxihy0RERERUTGeeeYZbNmyBd9++y2aNm2KuXPnIi4uDkOGDLF2aERERERUBnEMWzJK0PRtpV7H+Xl9JIiEyqPIb47iSgagLMU6uP8REZEpXn75Zbz88svWDoOIiIiIygHeYUtERERERERERERkI9hhS0RERERERERERGQj2GFLREREREREREREZCM4hi0REREREREREVE5Ysw7isr6u2BssS54hy0RERERERERERGRjWCHLREREREREREREZGNYIctERERERERERERkY1ghy0RERERERERERGRjWCHLRERERERERGZxdKlSxEQEAAnJycEBwfj2LFjeudduXIlOnXqhCpVqqBKlSoIDQ0tdn4iorKKHbZEREREREREJLkNGzYgKioKs2bNwqlTp9CiRQuEhYUhLS1N5/z79u3D4MGDsXfvXhw+fBi1atVC9+7dcevWLQtHTkRkXeywJSIiIiIiIiLJLVq0CKNGjUJERASaNGmCZcuWoXLlyli1apXO+deuXYuxY8eiZcuWaNSoEb788ksolUrs3r3bwpETEVkXO2yJiIiIiIiISFL5+fk4efIkQkND1WlyuRyhoaE4fPiwQet4+PAhHj9+jKpVq5orTCIim1RB6hXGxsbi+++/x6VLl1CpUiW0b98eCxcuRMOGDaXOioiIiIiIiIhs0N27d1FQUABvb2+NdG9vb1y6dMmgdUyZMgV+fn4anb5F5eXlIS8vT/05KysLAKBUKqFUKg2OV9fdbHIAMh3TCq/XmLvgjIlHF1PyMjW+osvpqwtz5GXIcubOq6RlpNw3pNifLJWXJY8Te9ifdE0TQmjMY+pxL3mH7f79+xEZGYlnnnkGT548wbRp09C9e3dcuHABzs7OUmdHRFQmmPJjV0JCAiIiIjTSFAoFcnNzzR0uEREREZFZLViwAOvXr8e+ffvg5OSkd77Y2FjExMRopaenpxt1XdzAQztNBqDG/3djiELphcfg1bWcPvrG7jWUKXmZGl/R5fTVhTnyMmQ5c+dV0jJS7htS7E+WysuSx4k97E9FKZVKZGZmQggBufxpN3B2drbhKy9E8g7bHTt2aHxOSEiAl5cXTp48ieeee07q7IiIygRTf+xyc3PD5cuX1Z9lMpklwiUiIiIiKlb16tXh4OCA1NRUjfTU1FT4+PgUu+zHH3+MBQsWYNeuXWjevHmx80ZHRyMqKkr9OSsrC7Vq1YKnpyfc3NwMjvdKhnaa6q67qxlA4XvkvLy8il1On8LLmcKUvEyNr+hy+urCHHkZspy58yppGSn3DSn2J0vlZcnjxB72p6KUSiVkMhk8PT3VHbbF/eBUHMk7bIvKzMwEAL1jzkj1+AJg+C3Mxd3KXxbJAa1bskuzrtKSIg7AuFvxy6ryVF6pymrK/ifVPlscU3/skslkJV7wEhERERFZmqOjI9q0aYPdu3ejb9++AKB+gdi4ceP0Lvfhhx9i3rx5+OWXX9C2bdsS81EoFFAoFFrpcrlc3WFiCH1X/OL/pxWeXni9xnxTMCYeXUzJy9T4dC2nqy7MlVdJy5k7L0OWkWrfkGJ/slReljxO7GF/0kUmk2m0P6Ye92btsFUqlZg4cSI6dOiApk2b6pxHqscXAMNvYS7uVv6ySAYgIyND45ZsUxlzm7g+pX0MRMWYW/HLqvJUXqnKasr+Z+ojDKVR0o9dKg8ePIC/vz+USiVat26N+fPnIygoSO/85hzjyxQl5alrDCBbJmW85v4hxpgfQcz5Q5sxy9rLD1TWjNXUH6WK7reWOsZLOz8REdm2qKgohIeHo23btmjXrh3i4uKQk5OjHtZr2LBhqFGjBmJjYwEACxcuxMyZM7Fu3ToEBAQgJSUFAODi4gIXFxerlYOIyNLM2mEbGRmJc+fO4dChQ3rnkerxBcDwW5iLu5W/LJID8PDw0Lgl21TG3CauT2kfA1Ex5lb8sqo8lVeqspqy/5n6CIOpDPmxCwAaNmyIVatWoXnz5sjMzMTHH3+M9u3b4/z586hZs6bOZcw5xpcpSupA1zUGkC2TMl6p6lgfY34EMecPbYaypx+orBmrKdtK135rqWO8KGv8QEZEROYzcOBApKenY+bMmUhJSUHLli2xY8cO9YvIbt68qXHNFB8fj/z8fPTr109jPbNmzcLs2bMtGToRkVWZrcN23Lhx+PHHH3HgwAG9HQeAdI8vAMZ14ui7lb+sKnpLtqmkqC+pOl2MuRW/LCtP5ZWirKbsf5buKDTkxy4ACAkJQUhIiPpz+/bt0bhxYyxfvhxz587VuYw5x/gyRUkd6LrGALJlUsYrVR3rY8yPIOb8oc1Q9vQDlTVjNWVb6dpvLXWMF2XpH8iIiMj8xo0bp3cIhH379ml8vnHjhvkDIiKyA5J32Aoh8J///AdbtmzBvn37EBgYKHUWRERllqE/dulSsWJFtGrVCteuXdM7j7nH+DKWIXlK9YOTpdjSD2QlMfRHEHP/0GYoe/qBylqxmrqtiu63ljzGSzO/tSxYsADR0dGYMGEC4uLirB0OEREREZUxkl8VR0ZGYs2aNVi3bh1cXV2RkpKClJQUPHr0SOqsiIjKDCEExo0bhy1btmDPnj0m/dhVUFCAs2fPwtfX1wwREhERABw/fhzLly8v8a3lRERERESmkrzDNj4+HpmZmejSpQt8fX3Vfxs2bJA6KyKiMsOQH7uGDRuG6Oho9ec5c+bg119/xZ9//olTp07hjTfeQHJyMt58801rFIGIqMx78OABhgwZgpUrV6JKlSrWDoeIiIiIyiizDIlARETGiY+PBwB06dJFI/2rr77C8OHDAWi/lOH+/fsYNWoUUlJSUKVKFbRp0wa///47mjRpYqmwiYjKlcjISPTs2ROhoaH44IMPrB0OEREREZVRZnvpGBERGc6QH7uKvpTh008/xaeffmqmiIiIqLD169fj1KlTOH78uEHz5+XlIS8vT/05KysLwNOXvCmVtjESs1KphBDCZuIxlj3Hb8+xA/Ydvz3HDkgXv72Wn4iovGCHLRERERFRMf766y9MmDABO3fuhJOTk0HLxMbGIiYmRis9PT0dubm5UodoEqVSiczMTAgh7OaFb4XZc/z2HDtg3/Hbc+yAdPFnZ2dLGBUREUmNHbZERERERMU4efIk0tLS0Lp1a3VaQUEBDhw4gCVLliAvLw8ODg4ay0RHRyMqKkr9OSsrC7Vq1YKnpyfc3NwsFntxlEolZDIZPD097bbjyl7jt+fYAfuO355jB6SL39Afn4iIyDrYYUtEREREVIxu3brh7NmzGmkRERFo1KgRpkyZotVZCwAKhQIKhUIrXS6X21QnkUwms7mYjGHP8dtz7IB9x2/PsQPSxG+vZSciKi/YYUtEREREVAxXV1c0bdpUI83Z2RnVqlXTSiciIiIiKi3+rEZERERERERERERkI3iHLRERERGRkfbt22ftEIiIiIiojOIdtkREREREREREREQ2gh22RERERERERERERDaCHbZERERERERERERENoIdtkREREREREREREQ2gi8dKycivzmKKxmA0tqBAAiavs3aIRAREREREREREdkk3mFLREREREREREREZCPYYUtERERERERERERkIzgkAhERERGRHUlKz8aqg9dw6U4mGvm6Y0Snegj0dLV2WEREREQkEXbYEhERERHZgaT0bCzeeRG7LtyBEE/TLqdkYcfZW9g4tjM7bYmIiIjKCHbYEhERERHZmHGrD2PvlTT15+CAqjh7OxOP8gsgCs1XoBTIe6LEqoPXMPfVVpYPlIiIiIgkxw5bIiIiIiIbUrSzFgCO3vhH7/wFSoFLdzLNHRYRERERWQhfOkZEREREZEOKdtaWxEEuQyNfdzNFQ0RERESWxg5bIiIiIiI7pqggx4hO9awdBhERERFJhEMiEBERERHZCQe5DAVKARkAmQzo1sQXE15ozBeOEREREZUh7LAlIiIiIrIhXRt46RwWITigKmpUc8GlO5lo5OuOEZ3qsaOWiIiIqAxihy0RERERkQ1ZEh6i9eKxrg28sCQ8xIpREREREZGlsMOWiIiIiMjGsHOWiIiIqPwy20vHli5dioCAADg5OSE4OBjHjh0zV1ZERGWGsW3npk2b0KhRIzg5OaFZs2b46aefLBQpEREREVHJeH1LRGQ8s3TYbtiwAVFRUZg1axZOnTqFFi1aICwsDGlp2mNxERHRU8a2nb///jsGDx6MkSNH4vTp0+jbty/69u2Lc+fOWThyIiIiIiJtvL4lIjKNWTpsFy1ahFGjRiEiIgJNmjTBsmXLULlyZaxatcoc2RERlQnGtp2LFy/Giy++iMmTJ6Nx48aYO3cuWrdujSVLllg4ciIiIiIibby+JSIyjeQdtvn5+Th58iRCQ0P/zUQuR2hoKA4fPix1dkREZYIpbefhw4c15geAsLAwtrVEREREZHW8viUiMp3kLx27e/cuCgoK4O3trZHu7e2NS5cuac2fl5eHvLw89efMzEwAQEZGBpRKpVF5i7yHhs0H4EkuIPKe/r+sK0/lLU9lBcpXeaUqa0ZGhtHLZGVlPY1BmK+WjW07ASAlJUXn/CkpKXrzkarNNbS9LUlJ20OpVCIrKwuOjo6Qy8027LpkpIxXqjrWu34YfkyZctzozLMUZbKn9s6asZqyrXTtt5Y6xouyRHtrLaoyqcpoC5RKJbKzs+Hk5GQXbWxR9hy/PccO2Hf89hw7IF38vL7Vpuvcp++cXvj8Zsw5s7TXVKbkZWp8RZcr7vpG6rwMWc7ceZW0jJT7hhT7k6XysuRxYg/7U1G6rqtNbW8l77A1VmxsLGJiYrTS/f39zZrvebOu3faUp/KWp7IC5au8UpS1yiemL5udnQ13d3cJorAea7W5+pRme1DpGXpM2cp2sqf2zlqx2sq2UjE1nrLQ3haVnZ0NAKhVq5aVIyEi+ldZaG/NfX2r65xu6vnNkudpU/IqaRl91zfmyEvK5cy1jFT7hq3XhSHLWfs4sYe8jG1vJe+wrV69OhwcHJCamqqRnpqaCh8fH635o6OjERUVpf6sVCrxzz//oFq1apDJZFKHB+Bp73atWrXw119/wc3NzSx52JLyVN7yVFagfJXXmmUVQiA7Oxt+fn5my8PYthMAfHx8jJofsE6bWxr2to/bU7z2FCtgX/HaU6yAbcVrifbWWvz8/PDXX3/B1dXVZtpbW9r2prDn+O05dsC+47fn2AHp4uf1rWHsfX+REutCE+vjX6yLf+mqC1PbW8k7bB0dHdGmTRvs3r0bffv2BfC0wdy9ezfGjRunNb9CoYBCodBI8/DwkDosndzc3MrVzlSeylueygqUr/Jaq6zmvvPA2LYTAEJCQrB7925MnDhRnbZz506EhITozceabW5p2Ns+bk/x2lOsgH3Fa0+xArYTr73f6aWPXC5HzZo1rR2GTray7U1lz/Hbc+yAfcdvz7ED0sTP61vD2fv+IiXWhSbWx79YF/8qWhemtLdmGRIhKioK4eHhaNu2Ldq1a4e4uDjk5OQgIiLCHNkREZUJJbWdw4YNQ40aNRAbGwsAmDBhAjp37oxPPvkEPXv2xPr163HixAmsWLHCmsUgIiIiIgLA61siIlOZpcN24MCBSE9Px8yZM5GSkoKWLVtix44dWoOHExHRv0pqO2/evKnxcon27dtj3bp1eP/99zFt2jTUr18fW7duRdOmTa1VBCIiIiIiNV7fEhGZxmwvHRs3bpzexxysTaFQYNasWVqPTZRV5am85amsQPkqb3kpa3Ft5759+7TS+vfvj/79+5s5Kuuxt+1uT/HaU6yAfcVrT7EC9hcvScfet709x2/PsQP2Hb89xw7YZ/z2fH1rj/VtLqwLTayPf7Eu/iVlXciEEEKCmIiIiIiIiIiIiIiolOQlz0JERERERERERERElsAOWyIiIiIiIiIiIiIbwQ5bIiIiIiIiIiIiIhtRZjpsY2Nj8cwzz8DV1RVeXl7o27cvLl++rDFPbm4uIiMjUa1aNbi4uOC1115Damqqxjw3b95Ez549UblyZXh5eWHy5Ml48uSJJYtiEEPK26VLF8hkMo2/t99+W2MeeyhvfHw8mjdvDjc3N7i5uSEkJAQ///yzenpZ2q5AyeUtK9tVlwULFkAmk2HixInqtLK2fcl4uvYLW3Pr1i288cYbqFatGipVqoRmzZrhxIkT1g5Lp4KCAsyYMQOBgYGoVKkS6tati7lz58IWhrQ/cOAAevXqBT8/P8hkMmzdulVjuhACM2fOhK+vLypVqoTQ0FBcvXrVOsGi+HgfP36MKVOmoFmzZnB2doafnx+GDRuG27dv22S8Rb399tuQyWSIi4uzWHxkHlJdI9sKU68VrKWk84OttWuFGXK+sKX4pTiH/PPPPxgyZAjc3Nzg4eGBkSNH4sGDB1aN3dDzibViL8uWLl2KgIAAODk5ITg4GMeOHbN2SFYxe/Zsre+fjRo1snZYFmFv16bmVlJ9DB8+XGtfefHFF60TrBlZ6tqqzHTY7t+/H5GRkThy5Ah27tyJx48fo3v37sjJyVHP88477+B///sfNm3ahP379+P27dt49dVX1dMLCgrQs2dP5Ofn4/fff8fq1auRkJCAmTNnWqNIxTKkvAAwatQo3LlzR/334YcfqqfZS3lr1qyJBQsW4OTJkzhx4gSef/559OnTB+fPnwdQtrYrUHJ5gbKxXYs6fvw4li9fjubNm2ukl7XtS8bRt1/Ykvv376NDhw6oWLEifv75Z1y4cAGffPIJqlSpYu3QdFq4cCHi4+OxZMkSXLx4EQsXLsSHH36Izz//3NqhIScnBy1atMDSpUt1Tv/www/x2WefYdmyZTh69CicnZ0RFhaG3NxcC0f6VHHxPnz4EKdOncKMGTNw6tQpfP/997h8+TJ69+5thUifKql+VbZs2YIjR47Az8/PQpGROUlxjWwrTL1WsBZDzg+21q4VZsj5wpbil+IcMmTIEJw/fx47d+7Ejz/+iAMHDmD06NFWjd3Q84m1Yi+rNmzYgKioKMyaNQunTp1CixYtEBYWhrS0NGuHZhVBQUEa3z8PHTpk7ZAswt6uTc3NkGvJF198UWNf+fbbby0YoWVY7NpKlFFpaWkCgNi/f78QQoiMjAxRsWJFsWnTJvU8Fy9eFADE4cOHhRBC/PTTT0Iul4uUlBT1PPHx8cLNzU3k5eVZtgBGKlpeIYTo3LmzmDBhgt5l7Lm8VapUEV9++WWZ364qqvIKUTa3a3Z2tqhfv77YuXOnRvnKy/Yl3fTtF7ZmypQpomPHjtYOw2A9e/YUI0aM0Eh79dVXxZAhQ6wUkW4AxJYtW9SflUql8PHxER999JE6LSMjQygUCvHtt99aIUJNRePV5dixYwKASE5OtkxQxdAX799//y1q1Kghzp07J/z9/cWnn35q8djIvEy5RrYFpblWsJaSzg+23q6VdL6w5fhNOYdcuHBBABDHjx9Xz/Pzzz8LmUwmbt26ZbXYdSl6PrGV2MuSdu3aicjISPXngoIC4efnJ2JjY60YlXXMmjVLtGjRwtphWJ29XZuam662Kjw8XPTp08cq8ViTua6tyswdtkVlZmYCAKpWrQoAOHnyJB4/fozQ0FD1PI0aNULt2rVx+PBhAMDhw4fRrFkzeHt7q+cJCwtDVlaWxt2NtqhoeVXWrl2L6tWro2nTpoiOjsbDhw/V0+yxvAUFBVi/fj1ycnIQEhJS5rdr0fKqlLXtGhkZiZ49e2psR6DsH7dUPH37ha354Ycf0LZtW/Tv3x9eXl5o1aoVVq5cae2w9Grfvj12796NK1euAADOnDmDQ4cOoUePHlaOrHhJSUlISUnR2B/c3d0RHBysbg9sXWZmJmQyGTw8PKwdik5KpRJDhw7F5MmTERQUZO1wyExMuUa2BaW5VrCWks4Ptt6ulXS+sPX4CzMk1sOHD8PDwwNt27ZVzxMaGgq5XI6jR49aPObiFD2f2FPs9iA/Px8nT57U2F/kcjlCQ0Ntbt+2lKtXr8LPzw916tTBkCFDcPPmTWuHZHX21AZa0r59++Dl5YWGDRtizJgxuHfvnrVDMjtzXVtVkDZM26BUKjFx4kR06NABTZs2BQCkpKTA0dFR60uSt7c3UlJS1PMU7vRRTVdNs1W6ygsAr7/+Ovz9/eHn54c//vgDU6ZMweXLl/H9998DsK/ynj17FiEhIcjNzYWLiwu2bNmCJk2aIDExsUxuV33lBcrWdgWA9evX49SpUzh+/LjWtLJ83FLxitsvbM2ff/6J+Ph4REVFYdq0aTh+/DjGjx8PR0dHhIeHWzs8LVOnTkVWVhYaNWoEBwcHFBQUYN68eRgyZIi1QyuW6njWdbzbw7Gem5uLKVOmYPDgwXBzc7N2ODotXLgQFSpUwPjx460dCpmJqdfI1lbaawVrKen8YOvtWknnC1uPvzBDYk1JSYGXl5fG9AoVKqBq1ao2VR5d5xN7id1e3L17FwUFBTr3l0uXLlkpKusJDg5GQkICGjZsiDt37iAmJgadOnXCuXPn4Orqau3wrMae2kBLefHFF/Hqq68iMDAQ169fx7Rp09CjRw8cPnwYDg4O1g7PLMx5bVUmO2wjIyNx7ty5cjOuir7yFh6zqFmzZvD19UW3bt1w/fp11K1b19JhlkrDhg2RmJiIzMxMbN68GeHh4di/f7+1wzIbfeVt0qRJmdquf/31FyZMmICdO3fCycnJ2uGQjbC3/UKpVKJt27aYP38+AKBVq1Y4d+4cli1bZpMdths3bsTatWuxbt06BAUFITExERMnToSfn59NxlsWPH78GAMGDIAQAvHx8dYOR6eTJ09i8eLFOHXqFGQymbXDITOxx2tkezsnFGZv54eieL6wPfZwPqGyp/BTWM2bN0dwcDD8/f2xceNGjBw50oqRka0ZNGiQ+v/NmjVD8+bNUbduXezbtw/dunWzYmTmY85rqzI3JMK4cePw448/Yu/evahZs6Y63cfHB/n5+cjIyNCYPzU1FT4+Pup5ir61TfVZNY+t0VdeXYKDgwEA165dA2Bf5XV0dES9evXQpk0bxMbGokWLFli8eHGZ3a76yquLPW/XkydPIi0tDa1bt0aFChVQoUIF7N+/H5999hkqVKgAb2/vMrl9qXgl7RcFBQXWDlGDr6+v+g54lcaNG9vso2KTJ0/G1KlTMWjQIDRr1gxDhw7FO++8g9jYWGuHVizV8azreLflY1315To5ORk7d+602btrDx48iLS0NNSuXVt93CUnJ+Pdd99FQECAtcMjCZTmGtmapLhWsJaSzg+23q6VdL6w9fgLMyRWHx8frRdKPXnyBP/8849NlKe484mtx25vqlevDgcHB7vYt63Bw8MDDRo0UH//LK/sqQ20ljp16qB69epldl8x97VVmemwFUJg3Lhx2LJlC/bs2YPAwECN6W3atEHFihWxe/duddrly5dx8+ZN9digISEhOHv2rMbJTnUyLHqxZW0llVeXxMREAE8vHgH7Km9RSqUSeXl5ZW676qMqry72vF27deuGs2fPIjExUf3Xtm1bDBkyRP3/8rB9SVNJ+4WtPU7ToUMHXL58WSPtypUr8Pf3t1JExXv48CHkcs3Tv4ODA5RKpZUiMkxgYCB8fHw02oOsrCwcPXpUY4xvW6L6cn316lXs2rUL1apVs3ZIeg0dOhR//PGHxnHn5+eHyZMn45dffrF2eFQKUlwjW5MU1wrWUtL5wdbbtZLOF7Yef2GGxBoSEoKMjAycPHlSPc+ePXugVCrVN0hYS0nnE1uO3R45OjqiTZs2GvuLUqnE7t27bW7ftoYHDx7g+vXr6u+f5ZU9tYHW8vfff+PevXtlbl+x2LWVhC9Gs6oxY8YId3d3sW/fPnHnzh3138OHD9XzvP3226J27dpiz5494sSJEyIkJESEhISopz958kQ0bdpUdO/eXSQmJoodO3YIT09PER0dbY0iFauk8l67dk3MmTNHnDhxQiQlJYlt27aJOnXqiOeee069Dnsp79SpU8X+/ftFUlKS+OOPP8TUqVOFTCYTv/76qxCibG1XIYovb1narvoUfvOzEGVv+5Jpiu4XtuTYsWOiQoUKYt68eeLq1ati7dq1onLlymLNmjXWDk2n8PBwUaNGDfHjjz+KpKQk8f3334vq1auL9957z9qhiezsbHH69Glx+vRpAUAsWrRInD59Wv0W7AULFggPDw+xbds28ccff4g+ffqIwMBA8ejRI5uLNz8/X/Tu3VvUrFlTJCYmapyr8/LybC5eXfz9/cWnn35q2SBJclJcI9saY68VrMWQ84OttWuFGXK+sKX4pTiHvPjii6JVq1bi6NGj4tChQ6J+/fpi8ODBVo3d0POJtWIvq9avXy8UCoVISEgQFy5cEKNHjxYeHh4iJSXF2qFZ3Lvvviv27dsnkpKSxG+//SZCQ0NF9erVRVpamrVDMzt7uzY1t+LqIzs7W0yaNEkcPnxYJCUliV27donWrVuL+vXri9zcXGuHLilLXVuVmQ5bADr/vvrqK/U8jx49EmPHjhVVqlQRlStXFq+88oq4c+eOxnpu3LghevToISpVqiSqV68u3n33XfH48WMLl6ZkJZX35s2b4rnnnhNVq1YVCoVC1KtXT0yePFlkZmZqrMceyjtixAjh7+8vHB0dhaenp+jWrZu6s1aIsrVdhSi+vGVpu+pT9EtYWdu+ZBpb7rAVQoj//e9/omnTpkKhUIhGjRqJFStWWDskvbKyssSECRNE7dq1hZOTk6hTp46YPn261ToRC9u7d6/Oc1t4eLgQQgilUilmzJghvL29hUKhEN26dROXL1+2yXiTkpL0nqv37t1rc/Hqwg7bskGqa2RbYsq1grWUdH6wtXatMEPOF7YUvxTnkHv37onBgwcLFxcX4ebmJiIiIkR2drZVYzf0fGKt2Muyzz//XNSuXVs4OjqKdu3aiSNHjlg7JKsYOHCg8PX1FY6OjqJGjRpi4MCB4tq1a9YOyyLs7drU3Iqrj4cPH4ru3bsLT09PUbFiReHv7y9GjRpVJn/ksNS1lez/MyMiIiIiIiIiIiIiKyszY9gSERERERERERER2Tt22BIRERERERERERHZCHbYEhEREREREREREdkIdtgSERERERERERER2Qh22BIRERERERERERHZCHbYEhEREREREREREdkIdtgSERERERERERER2Qh22BIRERERERERERHZCHbYEhERERFRudGlSxdMnDjR2mEUKyEhAR4eHtYOwyJmz56Nli1bWjsMIiIim8IOWyIiIiIisjv20PFq78pTxzEREZEtYYctERERERERmSw/P9/aIRAREZUp7LAlIiIiIiK7Mnz4cOzfvx+LFy+GTCaDTCbDjRs3AAD79+9Hu3btoFAo4Ovri6lTp+LJkyd617V9+3a4u7tj7dq1AIC//voLAwYMgIeHB6pWrYo+ffqo163Ku2/fvvj444/h6+uLatWqITIyEo8fP1bP88UXX6B+/fpwcnKCt7c3+vXrV2x5EhISULt2bVSuXBmvvPIK7t27pzXPtm3b0Lp1azg5OaFOnTqIiYkptlwA8OWXX6Jx48ZwcnJCo0aN8MUXX6in3bhxAzKZDN9//z26du2KypUro0WLFjh8+DAAYN++fYiIiEBmZqa6jmfPng0ACAgIwNy5czFs2DC4ublh9OjRAIApU6agQYMGqFy5MurUqYMZM2Zo1AsALFiwAN7e3nB1dcXIkSORm5urMf348eN44YUXUL16dbi7u6Nz5844depUseUkIiIqa9hhS0REREREdmXx4sUICQnBqFGjcOfOHdy5cwe1atXCrVu38NJLL+GZZ57BmTNnEB8fj//+97/44IMPdK5n3bp1GDx4MNauXYshQ4bg8ePHCAsLg6urKw4ePIjffvsNLi4uePHFFzXuIt27dy+uX7+OvXv3YvXq1UhISEBCQgIA4MSJExg/fjzmzJmDy5cvY8eOHXjuuef0luXo0aMYOXIkxo0bh8TERHTt2lUr3oMHD2LYsGGYMGECLly4gOXLlyMhIQHz5s3Tu961a9di5syZmDdvHi5evIj58+djxowZWL16tcZ806dPx6RJk5CYmIgGDRpg8ODBePLkCdq3b4+4uDi4ubmp63jSpEnq5T7++GO0aNECp0+fxowZMwAArq6uSEhIwIULF7B48WKsXLkSn376qXqZjRs3Yvbs2Zg/fz5OnDgBX19fjU5kAMjOzkZ4eDgOHTqEI0eOoH79+njppZeQnZ2tt6xERERljUwIIawdBBERERERkTG6dOmCli1bIi4uTp02ffp0fPfdd7h48SJkMhmAp3e7TpkyBZmZmZDL5erl6tevj+nTp2Pbtm3o3LkzAGDNmjX44IMPNJbPz8+Hh4cHtm7diu7du2P48OHYt28frl+/DgcHBwDAgAEDIJfLsX79enz//feIiIjA33//DVdX1xLL8frrryMzMxPbt29Xpw0aNAg7duxARkYGACA0NBTdunVDdHS0ep41a9bgvffew+3bt3Wut169epg7dy4GDx6sTvvggw/w008/4ffff8eNGzcQGBiIL7/8EiNHjgQAXLhwAUFBQbh48SIaNWqEhIQETJw4UR2HSkBAAFq1aoUtW7YUW7aPP/4Y69evx4kTJwAA7du3R6tWrbB06VL1PM8++yxyc3ORmJiocx1KpRIeHh5Yt24dXn755WLzIyIiKit4hy0REREREZUJFy9eREhIiLqzFQA6dOiABw8e4O+//1anbd68Ge+88w527typ7qwFgDNnzuDatWtwdXWFi4sLXFxcULVqVeTm5uL69evq+YKCgtSdtQDg6+uLtLQ0AMALL7wAf39/1KlTB0OHDsXatWvx8OHDYmMODg7WSAsJCdH4fObMGcyZM0cdk4uLi/ruYl3rzsnJwfXr1zFy5EiNZT744AONcgBA8+bNNcoBQF2W4rRt21YrbcOGDejQoQN8fHzg4uKC999/Hzdv3jSqrKmpqRg1ahTq168Pd3d3uLm54cGDBxrrISIiKusqWDsAIiIiIiIiS2rVqhVOnTqFVatWoW3btuoO3gcPHqBNmzbq8WwL8/T0VP+/YsWKGtNkMhmUSiWAp8MCnDp1Cvv27cOvv/6KmTNnYvbs2Th+/Dg8PDxMivfBgweIiYnBq6++qjXNyclJ5/wAsHLlSq0O0sIdzUXLoqoHVVmK4+zsrPH58OHDGDJkCGJiYhAWFgZ3d3esX78en3zySYnrKiw8PBz37t3D4sWL4e/vD4VCgZCQEL7YjIiIyhV22BIRERERkd1xdHREQUGBRlrjxo3x3XffQQih7nz87bff4Orqipo1a6rnq1u3Lj755BN06dIFDg4OWLJkCQCgdevW2LBhA7y8vODm5mZybBUqVEBoaChCQ0Mxa9YseHh4YM+ePTo7XBs3boyjR49qpB05ckTjc+vWrXH58mXUq1fPoPy9vb3h5+eH/2vvDkKZ8eM4jn/+uNgFBy60oVgjTWy1WiEXkqykdhCpOYxmMuzCiCPKXKQQJc3JaRYKpazIRXFwWbazWsmBC//D01/teSg9l/+ep/fr+Ovbr+/3d/z06/dLJBLq7e397Tk+O+OvxONxmUwmTU1Nfawlk8mMmv9m7e/v/1j7edaLiwutrq6qo6ND0o9P4B4fH393BAAA/kgEtgAAAAD+OOXl5bq8vNTDw8PH0wXDw8MKh8MaGRmRz+fT/f29ZmdnFQgElJOT+RpcdXW1zs7O1NLSory8PIXDYfX29mpxcVEul0vz8/MqKytTMpnU/v6+gsFgRuj7lWg0qkQioaamJhUVFSkWi+nt7U1ms/nTer/fL6fTqaWlJblcLh0dHenw8DCjZmZmRp2dnTIajerp6VFOTo5ubm50e3v75Ydqc3Nz8vv9KigoUHt7u15fX3V9fa10Oq1AIPDtM35+ftbJyYmsVqsMBoMMBsOntVVVVUqlUtrb25PdbtfBwcEvb9yOjo5qYGBANptNTqdTu7u7uru7U2VlZcY+Ozs7stlsenp60uTkpPLz87/VLwAAfwvesAUAAADwx5mYmFBubq5qampUXFysVCql0tJSxWIxXV1dyWq1yuv1yuPxaHp6+tM9zGazTk9PFYlEND4+LoPBoPPzcxmNRnV3d8tiscjj8ejl5eXbN24LCwu1v7+v1tZWWSwWra2tKRKJqLa29tN6h8Oh9fV1raysyGq16vj4+Jd+29raFI1GdXx8LLvdLofDoeXlZZlMpi/7GBwc1MbGhra2tlRXV6fm5mZtb2+roqLiW3NIPz4J83q9crvdKi4u1sLCwpe1XV1dGhsbk8/nU319veLxuEKhUEaN2+1WKBRSMBhUY2OjksmkhoaGMmo2NzeVTqfV0NCgvr4++f1+lZSUfLtnAAD+Bv+8v7+//99NAAAAAAAAAAC4YQsAAAAAAAAAWYPAFgAAAAAAAACyBIEtAAAAAAAAAGQJAlsAAAAAAAAAyBIEtgAAAAAAAACQJQhsAQAAAAAAACBLENgCAAAAAAAAQJYgsAUAAAAAAACALEFgCwAAAAAAAABZgsAWAAAAAAAAALIEgS0AAAAAAAAAZAkCWwAAAAAAAADIEv8Cwm2p0Lhg/c0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1400x300 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def duration(tr): return tr[\"end\"] - tr[\"start\"]\n",
    "def gen_of(tr): return next(o for o in children(tr) if o[\"type\"] == \"generation\")\n",
    "lat = np.array([duration(t) for t in TRACES])\n",
    "cost = np.array([gen_of(t)[\"usage\"][\"cost\"] for t in TRACES])\n",
    "tok_in = np.array([gen_of(t)[\"usage\"][\"input\"] for t in TRACES]); tok_out = np.array([gen_of(t)[\"usage\"][\"output\"] for t in TRACES])\n",
    "refused = np.array([t[\"scores\"][\"refused\"] for t in TRACES]); correct = np.array([t[\"scores\"][\"correct\"] for t in TRACES])\n",
    "faith = np.array([t[\"scores\"][\"faithfulness\"] for t in TRACES])\n",
    "\n",
    "print(f\"trazas               {len(TRACES)}\")\n",
    "print(f\"latencia p50 / p95  {np.percentile(lat, 50):.0f} / {np.percentile(lat, 95):.0f} ms\")\n",
    "print(f\"tokens ent. / sal.   {tok_in.mean():.0f} / {tok_out.mean():.0f} por respuesta (media)\")\n",
    "print(f\"coste por respuesta  ${cost.mean():.7f} de media, ${cost.sum():.5f} en total\")\n",
    "print(f\"tasa de rechazo       {refused.mean():.1%}\")\n",
    "print(f\"correctas             {correct.mean():.1%}   fidelidad media {faith.mean():.2f}\")\n",
    "in_cost = tok_in * PRICES[\"stand-in-small\"][0] / 1e6\n",
    "print(f\"los tokens de entrada son {in_cost.sum() / cost.sum():.0%} del coste; correlación del coste con los tokens de entrada {np.corrcoef(cost, tok_in)[0, 1]:.2f}, con los de salida {np.corrcoef(cost, tok_out)[0, 1]:.2f}\")\n",
    "\n",
    "fig, ax = plt.subplots(1, 4, figsize=(14, 3))\n",
    "ax[0].hist(lat, bins=15, color=\"#2a7ab0\"); ax[0].set_title(\"latencia por petición (ms)\")\n",
    "ax[1].hist(cost * 1e6, bins=15, color=\"#2a7ab0\"); ax[1].set_title(\"coste por respuesta (micro-USD)\")\n",
    "ax[2].scatter(tok_in, cost * 1e6, s=14, color=\"#2a7ab0\"); ax[2].set_xlabel(\"tokens de entrada\"); ax[2].set_title(\"coste frente a tokens de entrada\")\n",
    "by_session = [np.mean([t[\"scores\"][\"correct\"] for t in TRACES if t[\"session\"] == f\"session-{s:02d}\"]) for s in range(20)]\n",
    "ax[3].bar(range(20), by_session, color=\"#2a7ab0\"); ax[3].set_title(\"correctas, por sesión\"); ax[3].set_ylim(0, 1.05)\n",
    "for a in ax: a.grid(alpha=.3)\n",
    "plt.tight_layout(); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b206084",
   "metadata": {},
   "source": [
    "Tres hábitos que conviene sacar del panel:\n",
    "\n",
    "1. **Informa de percentiles, no de medias.** El p95 es la experiencia que tiene uno de cada veinte usuarios.\n",
    "2. **Sabe qué impulsa el coste.** Cada respuesta paga los pasajes que recibió más la respuesta que escribe. En\n",
    "   esta ejecución, el reparto impreso muestra cuánto viene de la entrada y cuánto de la salida, y cuál de las dos\n",
    "   sigue más de cerca al coste. Los tokens de salida cuestan más cada uno, así que con un modelo real y más\n",
    "   locuaz el panorama cambia: registra el reparto con tus propias trazas antes de decidir si recuperar menos\n",
    "   pasajes o pedir respuestas más cortas.\n",
    "3. **Una tasa se lee junto a su calidad.** Una tasa de rechazo que baja es buena noticia solo si `correct` y\n",
    "   `faithfulness` se mantienen; si no, el sistema simplemente ha empezado a adivinar. (Aquí la fidelidad es 1,00\n",
    "   en todas las respuestas **por construcción**: un generador extractivo solo puede copiar frases que recibió, así\n",
    "   que esta puntuación no discrimina. Con un modelo real es la puntuación que detecta afirmaciones inventadas.\n",
    "   Una métrica que nunca se mueve es una métrica de la que hay que dudar.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5442e789",
   "metadata": {},
   "source": [
    "## 5. Encontrar las peores trazas\n",
    "\n",
    "Los agregados te dicen que algo va mal; **las trazas individuales te dicen qué**. Ordena por una puntuación y\n",
    "abre las malas: obtienes la pregunta, los pasajes que se recuperaron y la respuesta, todo en una pantalla."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "53bfbdd3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:02.804825Z",
     "iopub.status.busy": "2026-11-17T21:26:02.803661Z",
     "iopub.status.idle": "2026-11-17T21:26:02.827135Z",
     "shell.execute_reply": "2026-11-17T21:26:02.825497Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28 trazas incorrectas, 9 preguntas distintas:\n",
      "\n",
      "Q: Are boats cancelled when the sea is too rough?\n",
      "   recuperó [('p15', 2.6), ('p22', 2.4)] (necesitaba '1.5 m')\n",
      "   A: Nothing is discharged at sea. [1]\n",
      "\n",
      "Q: What if the oil in the gearbox gets dirty?\n",
      "   recuperó [('p07', 6.9), ('p06', 2.7)] (necesitaba 'filter change')\n",
      "   A: Gearbox oil is sampled every six months. [1]\n",
      "\n",
      "Q: What do I do if the pitch fault comes back after a reset?\n",
      "   recuperó [('p10', 8.0), ('p16', 2.2)] (necesitaba 'replace the hydraulic accumulator')\n",
      "   A: A pitch system fault (code PF-203) usually means a low accumulator pressure. [1]\n",
      "\n",
      "Q: Can I go up the tower alone to fix something?\n",
      "   recuperó [('p01', 0.0), ('p02', 0.0)] (necesitaba 'forbidden')\n",
      "   A: I cannot find that in the handbook.\n",
      "\n",
      "Q: Who do I call in the middle of the night about an accident?\n",
      "   recuperó [('p14', 3.0), ('p06', 2.7)] (necesitaba 'control room')\n",
      "   A: Between October and March, turbines HV01 to HV08 are curtailed at night during the peak migration periods announced by the ornithology team. [1]\n",
      "\n",
      "Q: What is the largest amount of power the farm is allowed to send ashore?\n",
      "   recuperó [('p15', 3.2), ('p21', 3.0)] (necesitaba '300 MW')\n",
      "   A: Used oil, filters and composite offcuts are brought ashore and logged in the waste register. [1]\n",
      "\n",
      "Q: What qualification do I need before getting on the boat?\n",
      "   recuperó [('p22', 2.4), ('p01', 2.1)] (necesitaba 'GWO Basic Safety Training')\n",
      "   A: New staff complete a two-day induction covering the handbook, emergency drills and the sea survival course before their first transfer. [1]\n",
      "\n",
      "Q: How big are the blades?\n",
      "   recuperó [('p08', 2.7), ('p01', 0.0)] (necesitaba '164 m')\n",
      "   A: Blades are inspected by drone every 12 months and after any lightning alarm. [1]\n",
      "\n",
      "Q: When can the turbines restart after a blackout?\n",
      "   recuperó [('p12', 4.4), ('p05', 2.6)] (necesitaba '10 minutes')\n",
      "   A: On loss of grid connection the turbines stop and the converters isolate. [1]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "bad = [t for t in TRACES if t[\"scores\"][\"correct\"] == 0]\n",
    "seen = {}\n",
    "for t in bad:\n",
    "    seen.setdefault(t[\"input\"], t)\n",
    "print(f\"{len(bad)} trazas incorrectas, {len(seen)} preguntas distintas:\\n\")\n",
    "for question, t in seen.items():\n",
    "    retr = next(o for o in children(t) if o[\"name\"] == \"retrieve\")\n",
    "    print(f\"Q: {question}\\n   recuperó {[(p, round(s, 1)) for p, s in retr['output']]} (necesitaba {t['fact']!r})\\n   A: {t['output']}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96b8029",
   "metadata": {},
   "source": [
    "Cada fallo es de uno de los dos tipos del módulo 01: o los ids recuperados no incluyen el pasaje que contiene\n",
    "el dato (un fallo de recuperación, que se arregla con las técnicas del módulo 02 o editando los documentos), o\n",
    "sí lo incluyen y el generador eligió mal (se arregla en el prompt o en el modelo). Sin la traza solo sabrías que\n",
    "\"respondió mal\".\n",
    "\n",
    "## 6. Lo real: Langfuse\n",
    "\n",
    "Langfuse es de código abierto, se puede autoalojar o usar como servicio en la nube, y su SDK de Python (versión\n",
    "5, basado en OpenTelemetry) te da los mismos objetos que nuestro trazador. La celda siguiente es la versión de\n",
    "producción de las secciones 2 y 3. **No se ejecuta en este notebook**: necesita un proyecto de Langfuse y claves\n",
    "(`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_HOST`), y los resultados ejecutados de este curso nunca\n",
    "dependen de una clave. Con las variables sin definir, la celda solo imprime un mensaje."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6b8f35d6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-11-17T21:26:02.831143Z",
     "iopub.status.busy": "2026-11-17T21:26:02.830766Z",
     "iopub.status.idle": "2026-11-17T21:26:02.840324Z",
     "shell.execute_reply": "2026-11-17T21:26:02.838792Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LANGFUSE_* sin definir: se omitió la celda del SDK real (es el estado esperado en el curso).\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "if os.getenv(\"LANGFUSE_PUBLIC_KEY\") and os.getenv(\"LANGFUSE_SECRET_KEY\"):\n",
    "    from langfuse import observe, get_client\n",
    "\n",
    "    langfuse = get_client()                                   # reads LANGFUSE_* from the environment\n",
    "\n",
    "    @observe(as_type=\"generation\")                            # a model call: usage and cost are attached to it\n",
    "    def generate_real(question, passages):\n",
    "        langfuse.update_current_generation(model=\"your-model-id\", usage_details={\"input\": 812, \"output\": 61})\n",
    "        return \"...\"\n",
    "\n",
    "    @observe()                                                # the trace root: nested @observe calls become spans\n",
    "    def answer_real(question):\n",
    "        langfuse.update_current_trace(session_id=\"session-01\", user_id=\"user-17\")\n",
    "        hits = [\"...\"]\n",
    "        return generate_real(question, hits)\n",
    "\n",
    "    answer_real(\"What wave height stops the crew boats from sailing?\")\n",
    "    # the supervisor of module 08 attaches its verdict later, from another process, by trace id:\n",
    "    langfuse.create_score(trace_id=\"<trace id>\", name=\"faithfulness\", value=0.92, data_type=\"NUMERIC\", comment=\"judge: all claims cited\")\n",
    "    langfuse.flush()                                          # send pending events before the process exits\n",
    "else:\n",
    "    print(\"LANGFUSE_* sin definir: se omitió la celda del SDK real (es el estado esperado en el curso).\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "142d2cf3",
   "metadata": {},
   "source": [
    "Dos integraciones importan más que el decorador, porque trazan código que tú no escribiste:\n",
    "\n",
    "* **LiteLLM** (módulo 04): `litellm.success_callback = [\"langfuse\"]` y `litellm.failure_callback = [\"langfuse\"]`\n",
    "  registran cada llamada a un modelo que pase por él, con modelo, tokens y coste, sea cual sea el proveedor detrás.\n",
    "* **LangChain y LangGraph** (módulos 03 y 05): pasa `from langfuse.langchain import CallbackHandler` como callback\n",
    "  en `config={\"callbacks\": [handler]}` de la llamada y cada paso de la cadena, cada llamada al recuperador y cada\n",
    "  llamada al modelo se convierte en una observación anidada.\n",
    "\n",
    "En el agente de analítica de parques eólicos de este portafolio, Langfuse se conectó exactamente así: a través del\n",
    "enrutador LiteLLM y de los callbacks de LangChain, con el trazado desactivado cuando no hay claves configuradas,\n",
    "de modo que la aplicación funciona igual con él o sin él.\n",
    "\n",
    "## 7. LangSmith, y por qué no lo usamos\n",
    "\n",
    "**LangSmith** es el producto de trazado, evaluación y conjuntos de datos de los creadores de LangChain. Hace el\n",
    "mismo trabajo básico que Langfuse (trazas, ejecuciones, puntuaciones de feedback) y añade catálogos de prompts y\n",
    "ejecuciones de evaluación guiadas por conjuntos de datos, estrechamente integrados con LangChain y LangGraph.\n",
    "\n",
    "| | Langfuse | LangSmith |\n",
    "|---|---|---|\n",
    "| Licencia y alojamiento | código abierto, autoalojable o en la nube | producto alojado (el autoalojamiento es una opción empresarial) |\n",
    "| Encaje con frameworks | cualquier código, más LiteLLM, LangChain, SDK estilo OpenAI, OpenTelemetry | diseñado en torno a LangChain y LangGraph, funciona con otros |\n",
    "| Trazas, puntuaciones, sesiones | sí | sí (ejecuciones, feedback, hilos) |\n",
    "| Conjuntos de datos y experimentos | sí | sí, un punto fuerte |\n",
    "| Gestión de prompts | sí | sí (catálogo de prompts) |\n",
    "| Usado en este portafolio | **sí** | **no** |\n",
    "\n",
    "**La razón honesta por la que no usamos LangSmith:** Langfuse ya cubría el trazado, y una segunda pila habría\n",
    "supuesto otra cuenta externa, otro juego de claves y otro lugar donde guardar prompts y respuestas, sin ninguna\n",
    "ganancia funcional. No hay nada malo en LangSmith; para un equipo totalmente centrado en LangChain y que quiera su\n",
    "flujo de evaluación alojado es una primera opción razonable. Elige según dónde pueden vivir tus datos, cómo llama\n",
    "tu código a los modelos y qué ejecutas ya, y recuerda que, como ambas hablan en trazas, tramos y puntuaciones,\n",
    "pasar de una a otra es un cambio pequeño en la capa de instrumentación, no una reescritura.\n",
    "\n",
    "En este curso no se ejecuta código de LangSmith, por la misma razón.\n",
    "\n",
    "## Ejercicios\n",
    "\n",
    "1. Añade a la generación un campo `retrieved_ids` y una puntuación `hit` (¿está el pasaje de referencia entre\n",
    "   ellos?). Después dibuja `hit` frente a `correct` sobre las trazas. ¿Qué dice la tabla cruzada sobre de dónde\n",
    "   vienen los fallos?\n",
    "2. Cambia `PRICES` a diez veces el precio de salida. ¿Cómo se mueve el coste por respuesta y qué lado domina\n",
    "   ahora? Después cambia `k` en `retrieve` de 2 a 4: ¿cuánto se mueve el coste y cómo cambia `correct`?\n",
    "3. Haz que el generador de sustitución lance de vez en cuando una excepción. ¿Dónde necesitaría tu trazador un\n",
    "   campo `status` para que las llamadas fallidas aparezcan en el panel en lugar de desaparecer?\n",
    "4. Registra una puntuación de sesión: la proporción de trazas de una sesión que fueron rechazos. ¿Qué sesiones\n",
    "   abrirías primero y qué buscarías?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
