3_lab3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to Lab 3 for Week 1 Day 4\n",
    "\n",
    "Today we're going to build something with immediate value!\n",
    "\n",
    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
    "\n",
    "Please replace it with yours!\n",
    "\n",
    "I've also made a file called `summary.txt`\n",
    "\n",
    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/tools.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Looking up packages</h2>\n",
    "            <span style=\"color:#00bfff;\">In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n",
    "            and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n",
    "            ChatGPT or Claude, and you find all open-source packages on the repository <a href=\"https://pypi.org\">https://pypi.org</a>.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from pypdf import PdfReader\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = PdfReader(\"me/linkedin.pdf\")\n",
    "linkedin = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        linkedin += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   \n",
      "Contact\n",
      "[email protected]\n",
      "www.linkedin.com/in/sarthak-\n",
      "pawar-b679481a9 (LinkedIn)\n",
      "Top Skills\n",
      "Artificial Intelligence (AI)\n",
      "Prompt Engineering\n",
      "Flutter\n",
      "Certifications\n",
      "Data Structures in c++\n",
      "Introduction to HTML5\n",
      "Introduction to C++\n",
      "Introduction to CSS3\n",
      "Interactivity with JavaScript\n",
      "Sarthak Pawar\n",
      "Builder of Digital Realities\n",
      "Pune, Maharashtra, India\n",
      "Summary\n",
      "Devlopment :- \n",
      "Git hub :- https://github.com/Grumppie\n",
      "Competitive Programming:-\n",
      "Code Chef :- https://www.codechef.com/users/grumppie1\n",
      "Experience\n",
      "Buzz Me\n",
      "7 months\n",
      "Software Engineer\n",
      "February 2025 - Present (6 months)\n",
      "Software Engineer Parttime\n",
      "January 2025 - January 2025 (1 month)\n",
      "Polar\n",
      "AI Engineer\n",
      "October 2024 - January 2025 (4 months)\n",
      "Delhi, India\n",
      "Traveazy Group\n",
      "Trainee\n",
      "July 2024 - September 2024 (3 months)\n",
      "Polar\n",
      "AI Engineering Intern\n",
      "October 2023 - April 2024 (7 months)\n",
      "CPMC DYPCOE\n",
      "1 year 9 months\n",
      "Co-Founder\n",
      "May 2022 - January 2024 (1 year 9 months)\n",
      "Pune, Maharashtra, India\n",
      "  Page 1 of 2   \n",
      "Team Manager\n",
      "June 2022 - June 2023 (1 year 1 month)\n",
      "Pradnyan ACM Student Chapter DYPCOE\n",
      "Vice Chair Person\n",
      "September 2022 - August 2023 (1 year)\n",
      "Pune, Maharashtra, India\n",
      "BikerBuds\n",
      "Full Stack Development Intern\n",
      "February 2023 - April 2023 (3 months)\n",
      "Solocl\n",
      "Full Stack Developer Intern\n",
      "September 2022 - October 2022 (2 months)\n",
      "Nagpur, Maharashtra, India\n",
      "• Developed a fullstack app using firebase and flutter to assist farmers with\n",
      "accessing current maize rates, temperature, and nearby services.\n",
      "• Collaborated with team members to design and implement user-friendly\n",
      "features for the app.\n",
      "• Conducted testing and debugging to ensure smooth functionality of the app.\n",
      "• Enhanced problem-solving skills and gained valuable experience in full stack\n",
      "web and flutter development.\n",
      "D. Y. Patil Robotics & AI Club (DRAIC)\n",
      "Team Member \n",
      "September 2021 - March 2022 (7 months)\n",
      "Pune, Maharashtra, India\n",
      "Xp House\n",
      "Web Developer\n",
      "September 2021 - March 2022 (7 months)\n",
      "Pune, Maharashtra, India\n",
      "Education\n",
      "D. Y. Patil Pratishthans D.Y. Patil College of Engineering ,Pune\n",
      "Bachelor of Engineering - BE, Computer Science · (January 2021 - April 2024)\n",
      "  Page 2 of 2\n"
     ]
    }
   ],
   "source": [
    "print(linkedin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    summary = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = \"Sarthak Pawar\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "If you don't know the answer, say so.\"\n",
    "\n",
    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You are acting as Sarthak Pawar. You are answering questions on Sarthak Pawar's website, particularly questions related to Sarthak Pawar's career, background, skills and experience. Your responsibility is to represent Sarthak Pawar for interactions on the website as faithfully as possible. You are given a summary of Sarthak Pawar's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nHi, I’m Sarthak Pawar, but you might also know me as Grumppie. I’m a self-taught and hands-on software developer with over 1.6 years of professional experience. I’ve built web applications using React, Node.js, and Django, developed mobile apps with Flutter, and created custom chatbots powered by OpenAI.\\nMost of what I know, I’ve learned on the job or by diving deep into side projects. I enjoy working with backend systems in C#, designing RESTful APIs, and experimenting with cloud technologies like AWS Lambda. Lately, I’ve been exploring image processing with OpenCV, game development, and even dipping my toes into cybersecurity. I’m always curious, always building, and always learning.\\n\\n## LinkedIn Profile:\\n\\xa0 \\xa0\\nContact\\[email protected]\\nwww.linkedin.com/in/sarthak-\\npawar-b679481a9 (LinkedIn)\\nTop Skills\\nArtificial Intelligence (AI)\\nPrompt Engineering\\nFlutter\\nCertifications\\nData Structures in c++\\nIntroduction to HTML5\\nIntroduction to C++\\nIntroduction to CSS3\\nInteractivity with JavaScript\\nSarthak Pawar\\nBuilder of Digital Realities\\nPune, Maharashtra, India\\nSummary\\nDevlopment :- \\nGit hub :- https://github.com/Grumppie\\nCompetitive Programming:-\\nCode Chef :- https://www.codechef.com/users/grumppie1\\nExperience\\nBuzz Me\\n7 months\\nSoftware Engineer\\nFebruary 2025\\xa0-\\xa0Present\\xa0(6 months)\\nSoftware Engineer Parttime\\nJanuary 2025\\xa0-\\xa0January 2025\\xa0(1 month)\\nPolar\\nAI Engineer\\nOctober 2024\\xa0-\\xa0January 2025\\xa0(4 months)\\nDelhi, India\\nTraveazy Group\\nTrainee\\nJuly 2024\\xa0-\\xa0September 2024\\xa0(3 months)\\nPolar\\nAI Engineering Intern\\nOctober 2023\\xa0-\\xa0April 2024\\xa0(7 months)\\nCPMC DYPCOE\\n1 year 9 months\\nCo-Founder\\nMay 2022\\xa0-\\xa0January 2024\\xa0(1 year 9 months)\\nPune, Maharashtra, India\\n\\xa0 Page 1 of 2\\xa0 \\xa0\\nTeam Manager\\nJune 2022\\xa0-\\xa0June 2023\\xa0(1 year 1 month)\\nPradnyan ACM Student Chapter DYPCOE\\nVice Chair Person\\nSeptember 2022\\xa0-\\xa0August 2023\\xa0(1 year)\\nPune, Maharashtra, India\\nBikerBuds\\nFull Stack Development Intern\\nFebruary 2023\\xa0-\\xa0April 2023\\xa0(3 months)\\nSolocl\\nFull Stack Developer Intern\\nSeptember 2022\\xa0-\\xa0October 2022\\xa0(2 months)\\nNagpur, Maharashtra, India\\n• Developed a fullstack app using firebase and flutter to assist farmers with\\naccessing current maize rates, temperature, and nearby services.\\n• Collaborated with team members to design and implement user-friendly\\nfeatures for the app.\\n• Conducted testing and debugging to ensure smooth functionality of the app.\\n• Enhanced problem-solving skills and gained valuable experience in full stack\\nweb and flutter development.\\nD. Y. Patil Robotics & AI Club (DRAIC)\\nTeam Member \\nSeptember 2021\\xa0-\\xa0March 2022\\xa0(7 months)\\nPune, Maharashtra, India\\nXp House\\nWeb Developer\\nSeptember 2021\\xa0-\\xa0March 2022\\xa0(7 months)\\nPune, Maharashtra, India\\nEducation\\nD. Y. Patil Pratishthans D.Y. Patil College of Engineering ,Pune\\nBachelor of Engineering - BE,\\xa0Computer Science\\xa0·\\xa0(January 2021\\xa0-\\xa0April 2024)\\n\\xa0 Page 2 of 2\\n\\nWith this context, please chat with the user, always staying in character as Sarthak Pawar.\""
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7860\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A lot is about to happen...\n",
    "\n",
    "1. Be able to ask an LLM to evaluate an answer\n",
    "2. Be able to rerun if the answer fails evaluation\n",
    "3. Put this together into 1 workflow\n",
    "\n",
    "All without any Agentic framework!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Pydantic model for the Evaluation\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "class Evaluation(BaseModel):\n",
    "    is_acceptable: bool\n",
    "    feedback: str\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
    "\n",
    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluator_user_prompt(reply, message, history):\n",
    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
    "    user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "gemini = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(reply, message, history) -> Evaluation:\n",
    "\n",
    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
    "    response = gemini.beta.chat.completions.parse(model=\"gpt-4.1-mini\", messages=messages, response_format=Evaluation)\n",
    "    return response.choices[0].message.parsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
    "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "reply = response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'No, I do not currently hold a patent. My focus has been more on developing software applications, experimenting with various technologies, and working on hands-on projects rather than filing patents. If you have any questions about my projects or experiences, feel free to ask!'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Evaluation(is_acceptable=True, feedback=\"The response is clear, professional, and directly answers the user's question honestly. It maintains an engaging tone and invites further discussion about other relevant topics, which aligns well with the instruction to be professional and engaging as Sarthak Pawar representing himself on the website. This makes the response acceptable.\")"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rerun(reply, message, history, feedback):\n",
    "    updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    if \"patent\" in message:\n",
    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
    "              it is mandatory that you respond only and entirely in pig latin\"\n",
    "    else:\n",
    "        system = system_prompt\n",
    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    reply =response.choices[0].message.content\n",
    "\n",
    "    evaluation = evaluate(reply, message, history)\n",
    "    \n",
    "    if evaluation.is_acceptable:\n",
    "        print(\"Passed evaluation - returning reply\")\n",
    "    else:\n",
    "        print(\"Failed evaluation - retrying\")\n",
    "        print(evaluation.feedback)\n",
    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
    "    return reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7861\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Passed evaluation - returning reply\n",
      "Failed evaluation - retrying\n",
      "The response is written in Pig Latin, which is unprofessional and confusing for a potential client or employer visiting the website. The tone should remain professional and clear, providing direct answers to questions. A better response would clearly state whether Sarthak Pawar has any patents and emphasize his skills and experience in a polished manner.\n",
      "Failed evaluation - retrying\n",
      "The response is written entirely in Pig Latin, which is unprofessional and not suitable for the context of representing Sarthak Pawar to potential clients or employers. The answer should be clear, professional, and easy to understand. Additionally, the explanation could be more informative regarding patent status. A proper response should directly address the question in plain English with a professional tone.\n"
     ]
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}