repoview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (I). Pick top-growing Github repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/homebrew/anaconda3/lib/python3.11/site-packages/pandas/core/arrays/masked.py:60: UserWarning: Pandas requires version '1.3.6' or newer of 'bottleneck' (version '1.3.5' currently installed).\n",
      "  from pandas.core import (\n"
     ]
    }
   ],
   "source": [
    "from tools.agent import RAgent, clone_repo, decide_temp_repo\n",
    "from tools.llm import get_openai_response\n",
    "\n",
    "# (I). Pick your favorite github repository\n",
    "# fastest_repos = get_fastest_growing_repos(days_ago=3, top_n = 10)\n",
    "repo_url = \"https://github.com/xjdr-alt/entropix.git\"\n",
    "temp_repo = decide_temp_repo(repo_url)\n",
    "\n",
    "# Initialize RepoAgent\n",
    "clone_repo(repo_url, temp_repo)\n",
    "agent = RAgent(temp_repo, get_llm_response=get_openai_response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (II). Visualize & Animate Repository Depedency Structure\n",
    "\n",
    "<div align=\"center\">\n",
    "  <img src=\"anime_entropix.gif\" width=\"1200\" alt=\"Entropix Codebase\">\n",
    "</div>\n",
    "<div align=\"center\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "success: successfully compiled sandbox/entropix_dag.d2 to sandbox/entropix_dag.png in 999.863999ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'sandbox/entropix_dag.png'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Static File-Dependency Visualization\n",
    "agent.visualize_file()\n",
    "\n",
    "# GIF animation of File-Dependency\n",
    "# agent.animate_file(frame_count=50, fps=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (III). Pick a file and visualize dependency graph within that file\n",
    "\n",
    "<div align=\"center\">\n",
    "  <img src=\"anime_entropix_stats.gif\" width=\"800\" alt=\"Entropix Codebase\">\n",
    "</div>\n",
    "<div align=\"center\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "success: successfully compiled d2_output/evolve_graph_0.d2 to d2_output/evolve_graph_0.png in 492.760709ms\n",
      "success: successfully compiled d2_output/evolve_graph_1.d2 to d2_output/evolve_graph_1.png in 456.588875ms\n",
      "success: successfully compiled d2_output/evolve_graph_2.d2 to d2_output/evolve_graph_2.png in 464.505709ms\n",
      "success: successfully compiled d2_output/evolve_graph_3.d2 to d2_output/evolve_graph_3.png in 497.796625ms\n",
      "success: successfully compiled d2_output/evolve_graph_4.d2 to d2_output/evolve_graph_4.png in 493.050167ms\n",
      "success: successfully compiled d2_output/evolve_graph_5.d2 to d2_output/evolve_graph_5.png in 485.671791ms\n",
      "success: successfully compiled d2_output/evolve_graph_6.d2 to d2_output/evolve_graph_6.png in 480.871083ms\n",
      "success: successfully compiled d2_output/evolve_graph_7.d2 to d2_output/evolve_graph_7.png in 488.986874ms\n",
      "success: successfully compiled d2_output/evolve_graph_8.d2 to d2_output/evolve_graph_8.png in 489.573042ms\n",
      "success: successfully compiled d2_output/evolve_graph_9.d2 to d2_output/evolve_graph_9.png in 493.892042ms\n",
      "success: successfully compiled d2_output/evolve_graph_10.d2 to d2_output/evolve_graph_10.png in 501.5245ms\n",
      "success: successfully compiled d2_output/evolve_graph_11.d2 to d2_output/evolve_graph_11.png in 504.314083ms\n",
      "success: successfully compiled d2_output/evolve_graph_12.d2 to d2_output/evolve_graph_12.png in 515.471292ms\n",
      "success: successfully compiled d2_output/evolve_graph_13.d2 to d2_output/evolve_graph_13.png in 517.661209ms\n",
      "success: successfully compiled d2_output/evolve_graph_14.d2 to d2_output/evolve_graph_14.png in 562.175958ms\n",
      "success: successfully compiled d2_output/evolve_graph_15.d2 to d2_output/evolve_graph_15.png in 517.130458ms\n",
      "success: successfully compiled d2_output/evolve_graph_16.d2 to d2_output/evolve_graph_16.png in 580.556792ms\n",
      "success: successfully compiled d2_output/evolve_graph_17.d2 to d2_output/evolve_graph_17.png in 519.050041ms\n",
      "success: successfully compiled d2_output/evolve_graph_18.d2 to d2_output/evolve_graph_18.png in 514.444375ms\n",
      "success: successfully compiled d2_output/evolve_graph_19.d2 to d2_output/evolve_graph_19.png in 507.10075ms\n",
      "success: successfully compiled d2_output/evolve_graph_20.d2 to d2_output/evolve_graph_20.png in 506.201542ms\n",
      "success: successfully compiled d2_output/evolve_graph_21.d2 to d2_output/evolve_graph_21.png in 539.32175ms\n",
      "success: successfully compiled d2_output/evolve_graph_22.d2 to d2_output/evolve_graph_22.png in 541.537959ms\n",
      "success: successfully compiled d2_output/evolve_graph_23.d2 to d2_output/evolve_graph_23.png in 530.287208ms\n",
      "success: successfully compiled d2_output/evolve_graph_24.d2 to d2_output/evolve_graph_24.png in 539.344833ms\n",
      "success: successfully compiled d2_output/evolve_graph_25.d2 to d2_output/evolve_graph_25.png in 553.049708ms\n",
      "success: successfully compiled d2_output/evolve_graph_26.d2 to d2_output/evolve_graph_26.png in 559.38725ms\n",
      "success: successfully compiled d2_output/evolve_graph_27.d2 to d2_output/evolve_graph_27.png in 588.235ms\n",
      "success: successfully compiled d2_output/evolve_graph_28.d2 to d2_output/evolve_graph_28.png in 570.825958ms\n",
      "success: successfully compiled d2_output/evolve_graph_29.d2 to d2_output/evolve_graph_29.png in 576.019958ms\n",
      "success: successfully compiled d2_output/evolve_graph_30.d2 to d2_output/evolve_graph_30.png in 575.124542ms\n",
      "success: successfully compiled d2_output/evolve_graph_31.d2 to d2_output/evolve_graph_31.png in 578.1555ms\n",
      "success: successfully compiled d2_output/evolve_graph_32.d2 to d2_output/evolve_graph_32.png in 616.529ms\n",
      "success: successfully compiled d2_output/evolve_graph_33.d2 to d2_output/evolve_graph_33.png in 603.218584ms\n",
      "success: successfully compiled d2_output/evolve_graph_34.d2 to d2_output/evolve_graph_34.png in 595.865416ms\n",
      "success: successfully compiled d2_output/evolve_graph_35.d2 to d2_output/evolve_graph_35.png in 616.821292ms\n",
      "success: successfully compiled d2_output/evolve_graph_36.d2 to d2_output/evolve_graph_36.png in 629.240459ms\n",
      "success: successfully compiled d2_output/evolve_graph_37.d2 to d2_output/evolve_graph_37.png in 652.425084ms\n",
      "success: successfully compiled d2_output/evolve_graph_38.d2 to d2_output/evolve_graph_38.png in 637.684624ms\n",
      "success: successfully compiled d2_output/evolve_graph_39.d2 to d2_output/evolve_graph_39.png in 657.270125ms\n",
      "success: successfully compiled d2_output/evolve_graph_40.d2 to d2_output/evolve_graph_40.png in 648.485249ms\n",
      "success: successfully compiled d2_output/evolve_graph_41.d2 to d2_output/evolve_graph_41.png in 706.311667ms\n",
      "success: successfully compiled d2_output/evolve_graph_42.d2 to d2_output/evolve_graph_42.png in 710.936917ms\n",
      "success: successfully compiled d2_output/evolve_graph_43.d2 to d2_output/evolve_graph_43.png in 709.67825ms\n",
      "success: successfully compiled d2_output/evolve_graph_44.d2 to d2_output/evolve_graph_44.png in 707.698125ms\n",
      "success: successfully compiled d2_output/evolve_graph_45.d2 to d2_output/evolve_graph_45.png in 751.004708ms\n",
      "success: successfully compiled d2_output/evolve_graph_46.d2 to d2_output/evolve_graph_46.png in 780.171249ms\n",
      "success: successfully compiled d2_output/evolve_graph_47.d2 to d2_output/evolve_graph_47.png in 720.901834ms\n",
      "success: successfully compiled d2_output/evolve_graph_48.d2 to d2_output/evolve_graph_48.png in 718.820084ms\n",
      "success: successfully compiled d2_output/evolve_graph_49.d2 to d2_output/evolve_graph_49.png in 700.017417ms\n",
      "success: successfully compiled d2_output/evolve_graph_50.d2 to d2_output/evolve_graph_50.png in 718.989666ms\n",
      "success: successfully compiled d2_output/evolve_graph_51.d2 to d2_output/evolve_graph_51.png in 696.348167ms\n",
      "success: successfully compiled d2_output/evolve_graph_52.d2 to d2_output/evolve_graph_52.png in 705.914249ms\n",
      "success: successfully compiled d2_output/evolve_graph_53.d2 to d2_output/evolve_graph_53.png in 704.008708ms\n",
      "success: successfully compiled d2_output/evolve_graph_54.d2 to d2_output/evolve_graph_54.png in 689.452958ms\n",
      "success: successfully compiled d2_output/evolve_graph_55.d2 to d2_output/evolve_graph_55.png in 692.891168ms\n",
      "success: successfully compiled d2_output/evolve_graph_56.d2 to d2_output/evolve_graph_56.png in 696.234458ms\n",
      "success: successfully compiled d2_output/evolve_graph_57.d2 to d2_output/evolve_graph_57.png in 692.887291ms\n",
      "success: successfully compiled d2_output/evolve_graph_58.d2 to d2_output/evolve_graph_58.png in 721.118374ms\n",
      "success: successfully compiled d2_output/evolve_graph_59.d2 to d2_output/evolve_graph_59.png in 717.894166ms\n",
      "success: successfully compiled d2_output/evolve_graph_60.d2 to d2_output/evolve_graph_60.png in 695.8665ms\n",
      "success: successfully compiled d2_output/evolve_graph_61.d2 to d2_output/evolve_graph_61.png in 709.516083ms\n",
      "success: successfully compiled d2_output/evolve_graph_62.d2 to d2_output/evolve_graph_62.png in 735.120626ms\n",
      "success: successfully compiled d2_output/evolve_graph_63.d2 to d2_output/evolve_graph_63.png in 737.639959ms\n",
      "success: successfully compiled d2_output/evolve_graph_64.d2 to d2_output/evolve_graph_64.png in 709.025542ms\n",
      "success: successfully compiled d2_output/evolve_graph_65.d2 to d2_output/evolve_graph_65.png in 695.273041ms\n",
      "success: successfully compiled d2_output/evolve_graph_66.d2 to d2_output/evolve_graph_66.png in 700.520209ms\n",
      "success: successfully compiled d2_output/evolve_graph_67.d2 to d2_output/evolve_graph_67.png in 715.577376ms\n",
      "success: successfully compiled d2_output/evolve_graph_68.d2 to d2_output/evolve_graph_68.png in 724.262291ms\n",
      "success: successfully compiled d2_output/evolve_graph_69.d2 to d2_output/evolve_graph_69.png in 736.074584ms\n",
      "success: successfully compiled d2_output/evolve_graph_70.d2 to d2_output/evolve_graph_70.png in 704.808083ms\n",
      "success: successfully compiled d2_output/evolve_graph_71.d2 to d2_output/evolve_graph_71.png in 730.7775ms\n",
      "success: successfully compiled d2_output/evolve_graph_72.d2 to d2_output/evolve_graph_72.png in 737.938708ms\n",
      "success: successfully compiled d2_output/evolve_graph_73.d2 to d2_output/evolve_graph_73.png in 717.573457ms\n",
      "success: successfully compiled d2_output/evolve_graph_74.d2 to d2_output/evolve_graph_74.png in 733.125916ms\n",
      "success: successfully compiled d2_output/evolve_graph_75.d2 to d2_output/evolve_graph_75.png in 715.271625ms\n",
      "success: successfully compiled d2_output/evolve_graph_76.d2 to d2_output/evolve_graph_76.png in 711.962667ms\n",
      "success: successfully compiled d2_output/evolve_graph_77.d2 to d2_output/evolve_graph_77.png in 741.896375ms\n",
      "success: successfully compiled d2_output/evolve_graph_78.d2 to d2_output/evolve_graph_78.png in 716.038375ms\n",
      "success: successfully compiled d2_output/evolve_graph_79.d2 to d2_output/evolve_graph_79.png in 735.188583ms\n",
      "success: successfully compiled d2_output/evolve_graph_80.d2 to d2_output/evolve_graph_80.png in 759.497916ms\n",
      "success: successfully compiled d2_output/evolve_graph_81.d2 to d2_output/evolve_graph_81.png in 717.404041ms\n",
      "success: successfully compiled d2_output/evolve_graph_82.d2 to d2_output/evolve_graph_82.png in 711.103625ms\n",
      "success: successfully compiled d2_output/evolve_graph_83.d2 to d2_output/evolve_graph_83.png in 724.407251ms\n",
      "success: successfully compiled d2_output/evolve_graph_84.d2 to d2_output/evolve_graph_84.png in 724.344042ms\n",
      "success: successfully compiled d2_output/evolve_graph_85.d2 to d2_output/evolve_graph_85.png in 722.739ms\n",
      "success: successfully compiled d2_output/evolve_graph_86.d2 to d2_output/evolve_graph_86.png in 709.430166ms\n",
      "success: successfully compiled d2_output/evolve_graph_87.d2 to d2_output/evolve_graph_87.png in 723.478041ms\n",
      "success: successfully compiled d2_output/evolve_graph_88.d2 to d2_output/evolve_graph_88.png in 716.763292ms\n",
      "success: successfully compiled d2_output/evolve_graph_89.d2 to d2_output/evolve_graph_89.png in 714.262292ms\n",
      "Creating GIF: 100%|██████████| 90/90 [00:22<00:00,  3.98it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GIF file saved in path:  anime_entropix_config.py.gif\n"
     ]
    }
   ],
   "source": [
    "file_name_or_number = \"entropix/model.py\"\n",
    "\n",
    "# Visualize Module-Dependency Graph starting from a specific file\n",
    "# agent.visualize_module(file_name_or_number, cap_node_number=15, depth=3)\n",
    "\n",
    "# Animating GIF of Module-Dependency Graph\n",
    "agent.animate_module(file_name_or_number, frame_count=50, fps=10, cap_node_number=15, depth=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (IV). Relevant code to prompt (Ready for Agentic QA)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Repository to prompt \n",
    "repo_prompt = agent._to_prompt()\n",
    "\n",
    "# File prompt (including all its dependency function's code snippet)\n",
    "# file_prompt = agent._to_prompt(file_name_or_number)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (V). Podcast Generation (~ 1 min)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Podcast created and saved as sandbox/podcast_entropix.mp3\n"
     ]
    }
   ],
   "source": [
    "# Repository-level Podcast \n",
    "agent.generate_podcast()\n",
    "\n",
    "# Module-level Podcast\n",
    "# agent.generate_podcast(file_name_or_number)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (VI). Agentic Question Answering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "success: successfully compiled sandbox/entropix_stats.py_dag.d2 to sandbox/entropix_stats.py_dag.png in 1.484072958s\n",
      "success: successfully compiled sandbox/entropix_stats.py_dag.d2 to sandbox/entropix_stats.py_dag.png in 1.48597525s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'This diagram represents the structure and relationships within the `model.py` file in the `entropix` module. Here’s a breakdown of the components and their interactions:\\n\\n### Modules\\n- **stats**: Contains `AttnStats` class with functions like `update`, `std_error`, `new`, and `avg_entropy`.\\n- **config**: Contains `ModelParams` for configuration settings.\\n- **weights**: Includes `LayerWeights` and functions to `load_weights` and handle `XfnmWeights`.\\n- **kvcache**: Manages key-value caching with the `update` and `new` functions.\\n- **model**: Core model structures with functions like `xfnmr` (transformer), `feed_forward`, `rms_norm`, `attention`, and `apply_rotary_emb`.\\n\\n### Interactions\\n- The red dashed lines represent dependencies or calls between components.\\n- The black solid lines indicate more direct function calls or class utilization within the same module.\\n\\n### Key Functionality\\n- **Attention Mechanism**: The `attention` function interacts with `apply_rotary_emb`, likely for position embeddings within the model.\\n- **Parameter Management**: `weights` module handles the loading and organization of model parameters.\\n- **Configuration and Statistics**: The `config` and `stats` modules provide necessary configurations and statistical tracking for model performance.\\n\\nThis visualization helps in understanding the modular design and interdependencies within the `model.py` file, aiding in debugging and development.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# agent.respond(\"how many files are there in the repository?\")\n",
    "\n",
    "# agent.respond(\"visualize the repository for me\")\n",
    "\n",
    "# agent.respond(\"can you tell me more about the model.py file?\")\n",
    "\n",
    "agent.respond(\"help me visualize model.py file\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}