{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Welcome to Lab 3 for Week 1 Day 4\n", "\n", "Today we're going to build something with immediate value!\n", "\n", "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n", "\n", "Please replace it with yours!\n", "\n", "I've also made a file called `summary.txt`\n", "\n", "We're not going to use Tools just yet - we're going to add the tool tomorrow." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Looking up packages

\n", " In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n", " and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n", " ChatGPT or Claude, and you find all open-source packages on the repository https://pypi.org.\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n", "\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "from pypdf import PdfReader\n", "import gradio as gr" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "load_dotenv(override=True)\n", "openai = OpenAI()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "reader = PdfReader(\"me/linkedin.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "   \n", "Contact\n", "ysaadoun@gmail.com\n", "www.linkedin.com/in/alexandre-\n", "ygal-saadoun-92b18630 (LinkedIn)\n", "Top Skills\n", "Web Development\n", "Data Science\n", "Machine Learning\n", "Languages\n", "French (Native or Bilingual)\n", "Hebrew (Full Professional)\n", "Spanish (Limited Working)\n", "English (Native or Bilingual)\n", "Certifications\n", "100 days Python Training\n", "Google Business Intelligence\n", "Specialization\n", "CORNELL / PRODUCT\n", "MANAGEMENT 360\n", "CORNELL / DATA ANALYTICS 360\n", "IBM AI Developer Professional\n", "Certificate\n", "Publications\n", "The right to digital privacy: a\n", "european survey\n", "Alexandre Ygal Saadoun\n", "Business Executive | Communication Expert | Data Scientist | LLM\n", "Engineer |\n", "Brooklyn, New York, United States\n", "Summary\n", "With over 20 years of experience spanning journalism, business\n", "management, and cutting-edge AI technologies, I bring a unique\n", "perspective to driving innovation and growth. From reporting on\n", "world events to leading sales teams, my career has been defined by\n", "an ability to communicate effectively, manage complex projects, and\n", "deliver results.\n", "In recent years, I have focused on leveraging data and AI to optimize\n", "business strategies. As a certified expert in Business Intelligence\n", "and AI development, I am passionate about transforming insights\n", "into actionable strategies that drive profitability and innovation.\n", "Fluent in five languages and adept at navigating global markets, I\n", "thrive in dynamic, high-stakes environments.\n", "Additionally, my expertise includes Python programming and large\n", "language model (LLM) engineering. I specialize in designing,\n", "training, and fine-tuning machine learning models for diverse\n", "applications, from natural language processing to advanced\n", "analytics.\n", "Experience\n", "MLDSAYS\n", "Head of Machine Learning & Chief Revenue Officer\n", "January 2024 - Present (1 year 7 months)\n", "New York, New York, United States\n", "• AI Solutions for Legal Professionals: Designing and implementing\n", "comprehensive LLM-based solutions for law firms, focusing on contract\n", "analysis, legal document processing, and regulatory compliance automation to\n", "transform traditional legal workflows.\n", "• LLM & NLP Engineering: Implemented fine-tuning workflows for transformer-\n", "based models (GPT-3.5/4, BERT, T5) using parameter-efficient techniques\n", "like LoRA and QLoRA for domain-specific legal document comprehension\n", "tasks. Designed and deployed retrieval-augmented generation (RAG) systems\n", "  Page 1 of 5   \n", "combining vector search with knowledge graphs for enhanced factual accuracy\n", "and contextual understanding in legal research. \n", "• Multi-Agent AI Architecture: Architected sophisticated multi-agent LLM\n", "systems for complex legal document workflows, orchestrating specialized\n", "agents using LangGraph and Dagster for tasks including contract review, due\n", "diligence, and legal research. \n", "• Advanced Workflow Orchestration: Integrated temporal reasoning\n", "frameworks using Allen's Interval Algebra and PyTemporal with Neo4j for\n", "improved sequencing of events in legal case narratives and contract timelines.\n", "• ML Operations & Production Deployment: Designed comprehensive\n", "evaluation frameworks with custom metrics for hallucination detection,\n", "factual accuracy, and relevance in legal document processing tasks.\n", "Deployed and monitored ML models in production environments using Docker\n", "containerization and Google Cloud Vertex AI. Implemented automated\n", "testing protocols to ensure model performance stability across different legal\n", "document types and jurisdictions.\n", "• Rapid Prototyping & Client Engagement: Leveraged interactive frameworks\n", "such as Gradio, Streamlit, and QT to quickly develop and refine proof-\n", "of-concepts for legal technology applications, ensuring client needs are\n", "met effectively and swiftly through consultative and proactive engagement\n", "approaches\n", "GOTHAM STONE LLC & LMG \n", "VP, Wholesale & Construction\n", "January 2014 - January 2023 (9 years 1 month)\n", "NEW YORK\n", "LMG TILE & GOTHAM STONE – New York, NY\n", "2014–2024\n", "VP, Wholesale & Construction (2014–2021); CEO (2021–2024)\n", "• Accelerated growth by independently launching and rapidly scaling a new\n", "business division from\n", "inception to $4M annual revenue in a year, demonstrating a proactive mindset,\n", "strategic agility, and relentless drive to exceed ambitious sales targets.\n", "• Closed multimillion-dollar contracts by proactively identifying and swiftly\n", "capitalizing on business\n", "opportunities, cultivating influential client relationships, and consistently\n", "surpassing market share and revenue goals.\n", "• Executed robust risk management strategies under demanding conditions,\n", "effectively employing\n", "  Page 2 of 5   \n", "sophisticated analytics such as Monte Carlo simulations to secure profitability\n", "and enhance competitive market\n", "positioning.\n", "• Managed complex, multi-priority projects leading diverse, cross-functional\n", "teams with agility (Agile\n", "methodologies), effectively balancing strategic objectives and operational\n", "demands in high-paced, dynamic environments.\n", "United Nations\n", "Public Information Officer\n", "September 2012 - March 2014 (1 year 7 months)\n", "New York\n", "Delivered comprehensive coverage and analysis of the General Assembly's\n", "2nd & 3rd Committees, focusing on economic, social, climate change, and\n", "sustainable development issues.\n", "• Engaged with high-level officials, diplomats, and experts, effectively\n", "conveying complex information to diverse audiences while upholding\n", "confidentiality and integrity.\n", "• Managed multiple assignments under tight deadlines, conducting extensive\n", "research to support accurate and credible reporting in line with UN standards.\n", "FRANCE 24\n", "5 years 9 months\n", "NYC NEWS CORRESPONDENT / BROADCAST JOURNALIST\n", "2011 - May 2012 (1 year)\n", "New York, NY\n", "• Live TV Reporting: Delivered real-time coverage of New York area business\n", "and political events for two major French news TV outlets. Demonstrated\n", "exceptional on-air presence and the ability to provide immediate, accurate\n", "updates during live broadcasts.\n", "• Time Management and Efficiency: Exhibited outstanding ability to work under\n", "time pressure, consistently producing high-quality reports on tight deadlines,\n", "ensuring timely and relevant information for viewers.\n", "• Expert Communication: Engaged with a diverse range of stakeholders,\n", "including political figures and business leaders, enhancing the depth and\n", "accuracy of reports through expert communication.\n", "Cairo Bureau Chief\n", "November 2007 - March 2011 (3 years 5 months)\n", "Established and opened the bureau in November 2007. \n", "  Page 3 of 5   \n", "• Reported under strict deadlines and challenging conditions for both TV and\n", "print outlets. \n", "• Comprehensive Coverage: Led coverage and monitoring of political,\n", "economic, and environmental affairs, providing detailed and insightful reporting\n", "on significant developments in the region.\n", "• Navigating Dictatorship Constraints: Successfully managed to work in Egypt,\n", "operating under a dictatorship and navigating harsh legal constraints as well\n", "as episodic violence, ensuring comprehensive and accurate reporting despite\n", "significant risks and challenges.\n", "• Key Correspondent Role: Served as the main correspondent during the\n", "Egyptian Revolution in 2011, delivering frontline reports and in-depth analysis\n", "during a critical period of change.\n", "• Crisis Leadership: Supervised a large team during violent phases of unrest,\n", "navigating severe challenges, ensuring the safety of team members, and\n", "maintaining the integrity and accuracy of the coverage under extreme\n", "conditions.\n", "• Interviewed major political, religious and business leaders including US\n", "Secretary of State Hillary Clinton, French President Nicolas Sarkozy, former\n", "UN Secretary-General Boutros Boutros-Ghali, Egyptian National Democratic\n", "party’s Gamal Mubarak, Muslim Brotherhood Leader Mehdi Akef, Al Azhar\n", "Sheikhs Mohammed Tantawi and Ahmad El Tayeb, Orascom Telecom CEO\n", "Naguib Sawiris.\n", "• Covered US President Barack Obama’s visit to Cairo in June 2009.\n", "• Daily coverage from the Egypt-Gaza Border during the January 2009 Israeli\n", "“Cast Lead” offensive on Gaza.\n", "• Covered the African Soccer Cup won by Egypt in 2008 and 2010.\n", "Business Editor\n", "September 2006 - November 2007 (1 year 3 months)\n", "• Full-time member of the initial launching team of the first French international\n", "news channel with broadcasts in French, English and Arabic. \n", "• Responsible for editorial content, line up and output of business morning\n", "programs. \n", "• Daily coverage of world stock markets including currencies, commodities and\n", "indices. \n", "• Coordinated and dispatched desk journalists and edited their output for daily\n", "news programs.\n", "Public Senat\n", "Program editor\n", "January 2006 - September 2006 (9 months)\n", "  Page 4 of 5   \n", "•Chose the content and focus of various prime-time talk shows in daily editorial\n", "meetings. \n", "•Selected and pre-interviewed guests for French political, business and social\n", "talk shows.\n", "•Interviewed and profiled high-ranking French political figures such as Minister\n", "Nicolas Sarkozy, President Jacques Chirac, Francois Hollande, Segolene\n", "Royal, in addition to various ministers and public figures.\n", "Education\n", "Université Panthéon Sorbonne (Paris I)\n", "Masters Trade law, law · (2011 - 2012)\n", "Cornell University\n", "MACHINE LEARNING HIGHER CERTIFICATE, MACHINE\n", "LEARNING · (March 2024 - June 2024)\n", "Cornell University\n", "Data Analytics 360, Data Analytics · (December 2023 - June 2024)\n", "Cornell University\n", "INNOVATION STRATEGY, INNOVATION STRATEGY · (November\n", "2023 - June 2024)\n", "Cornell University\n", "Product Management for Engineers, Business · (December 2023 - February\n", "2024)\n", "  Page 5 of 5\n" ] } ], "source": [ "print(linkedin)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "name = \"Alexandre Saadoun\"" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n", "particularly questions related to {name}'s career, background, skills and experience. \\\n", "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n", "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n", "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "If you don't know the answer, say so.\"\n", "\n", "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"You are acting as Alexandre Saadoun. You are answering questions on Alexandre Saadoun's website, particularly questions related to Alexandre Saadoun's career, background, skills and experience. Your responsibility is to represent Alexandre Saadoun for interactions on the website as faithfully as possible. You are given a summary of Alexandre Saadoun's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nMy name is Alexandre. I'm a Business Executive, Communication Expert, Data Scientist, and LLM Engineer in Brooklyn, NY. I'm originally from Paris, France, but I moved to NYC in 2011.\\nI love all foods, particularly French food. \\n\\n## LinkedIn Profile:\\n\\xa0 \\xa0\\nContact\\nysaadoun@gmail.com\\nwww.linkedin.com/in/alexandre-\\nygal-saadoun-92b18630 (LinkedIn)\\nTop Skills\\nWeb Development\\nData Science\\nMachine Learning\\nLanguages\\nFrench (Native or Bilingual)\\nHebrew (Full Professional)\\nSpanish (Limited Working)\\nEnglish (Native or Bilingual)\\nCertifications\\n100 days Python Training\\nGoogle Business Intelligence\\nSpecialization\\nCORNELL / PRODUCT\\nMANAGEMENT 360\\nCORNELL / DATA ANALYTICS 360\\nIBM AI Developer Professional\\nCertificate\\nPublications\\nThe right to digital privacy: a\\neuropean survey\\nAlexandre Ygal Saadoun\\nBusiness Executive | Communication Expert | Data Scientist | LLM\\nEngineer |\\nBrooklyn, New York, United States\\nSummary\\nWith over 20 years of experience spanning journalism, business\\nmanagement, and cutting-edge AI technologies, I bring a unique\\nperspective to driving innovation and growth. From reporting on\\nworld events to leading sales teams, my career has been defined by\\nan ability to communicate effectively, manage complex projects, and\\ndeliver results.\\nIn recent years, I have focused on leveraging data and AI to optimize\\nbusiness strategies. As a certified expert in Business Intelligence\\nand AI development, I am passionate about transforming insights\\ninto actionable strategies that drive profitability and innovation.\\nFluent in five languages and adept at navigating global markets, I\\nthrive in dynamic, high-stakes environments.\\nAdditionally, my expertise includes Python programming and large\\nlanguage model (LLM) engineering. I specialize in designing,\\ntraining, and fine-tuning machine learning models for diverse\\napplications, from natural language processing to advanced\\nanalytics.\\nExperience\\nMLDSAYS\\nHead of Machine Learning & Chief Revenue Officer\\nJanuary 2024\\xa0-\\xa0Present\\xa0(1 year 7 months)\\nNew York, New York, United States\\n• AI Solutions for Legal Professionals: Designing and implementing\\ncomprehensive LLM-based solutions for law firms, focusing on contract\\nanalysis, legal document processing, and regulatory compliance automation to\\ntransform traditional legal workflows.\\n• LLM & NLP Engineering: Implemented fine-tuning workflows for transformer-\\nbased models (GPT-3.5/4, BERT, T5) using parameter-efficient techniques\\nlike LoRA and QLoRA for domain-specific legal document comprehension\\ntasks. Designed and deployed retrieval-augmented generation (RAG) systems\\n\\xa0 Page 1 of 5\\xa0 \\xa0\\ncombining vector search with knowledge graphs for enhanced factual accuracy\\nand contextual understanding in legal research. \\n• Multi-Agent AI Architecture: Architected sophisticated multi-agent LLM\\nsystems for complex legal document workflows, orchestrating specialized\\nagents using LangGraph and Dagster for tasks including contract review, due\\ndiligence, and legal research. \\n• Advanced Workflow Orchestration: Integrated temporal reasoning\\nframeworks using Allen's Interval Algebra and PyTemporal with Neo4j for\\nimproved sequencing of events in legal case narratives and contract timelines.\\n• ML Operations & Production Deployment: Designed comprehensive\\nevaluation frameworks with custom metrics for hallucination detection,\\nfactual accuracy, and relevance in legal document processing tasks.\\nDeployed and monitored ML models in production environments using Docker\\ncontainerization and Google Cloud Vertex AI. Implemented automated\\ntesting protocols to ensure model performance stability across different legal\\ndocument types and jurisdictions.\\n• Rapid Prototyping & Client Engagement: Leveraged interactive frameworks\\nsuch as Gradio, Streamlit, and QT to quickly develop and refine proof-\\nof-concepts for legal technology applications, ensuring client needs are\\nmet effectively and swiftly through consultative and proactive engagement\\napproaches\\nGOTHAM STONE LLC & LMG \\nVP, Wholesale & Construction\\nJanuary 2014\\xa0-\\xa0January 2023\\xa0(9 years 1 month)\\nNEW YORK\\nLMG TILE & GOTHAM STONE – New York, NY\\n2014–2024\\nVP, Wholesale & Construction (2014–2021); CEO (2021–2024)\\n• Accelerated growth by independently launching and rapidly scaling a new\\nbusiness division from\\ninception to $4M annual revenue in a year, demonstrating a proactive mindset,\\nstrategic agility, and relentless drive to exceed ambitious sales targets.\\n• Closed multimillion-dollar contracts by proactively identifying and swiftly\\ncapitalizing on business\\nopportunities, cultivating influential client relationships, and consistently\\nsurpassing market share and revenue goals.\\n• Executed robust risk management strategies under demanding conditions,\\neffectively employing\\n\\xa0 Page 2 of 5\\xa0 \\xa0\\nsophisticated analytics such as Monte Carlo simulations to secure profitability\\nand enhance competitive market\\npositioning.\\n• Managed complex, multi-priority projects leading diverse, cross-functional\\nteams with agility (Agile\\nmethodologies), effectively balancing strategic objectives and operational\\ndemands in high-paced, dynamic environments.\\nUnited Nations\\nPublic Information Officer\\nSeptember 2012\\xa0-\\xa0March 2014\\xa0(1 year 7 months)\\nNew York\\nDelivered comprehensive coverage and analysis of the General Assembly's\\n2nd & 3rd Committees, focusing on economic, social, climate change, and\\nsustainable development issues.\\n• Engaged with high-level officials, diplomats, and experts, effectively\\nconveying complex information to diverse audiences while upholding\\nconfidentiality and integrity.\\n• Managed multiple assignments under tight deadlines, conducting extensive\\nresearch to support accurate and credible reporting in line with UN standards.\\nFRANCE 24\\n5 years 9 months\\nNYC NEWS CORRESPONDENT / BROADCAST JOURNALIST\\n2011\\xa0-\\xa0May 2012\\xa0(1 year)\\nNew York, NY\\n• Live TV Reporting: Delivered real-time coverage of New York area business\\nand political events for two major French news TV outlets. Demonstrated\\nexceptional on-air presence and the ability to provide immediate, accurate\\nupdates during live broadcasts.\\n• Time Management and Efficiency: Exhibited outstanding ability to work under\\ntime pressure, consistently producing high-quality reports on tight deadlines,\\nensuring timely and relevant information for viewers.\\n• Expert Communication: Engaged with a diverse range of stakeholders,\\nincluding political figures and business leaders, enhancing the depth and\\naccuracy of reports through expert communication.\\nCairo Bureau Chief\\nNovember 2007\\xa0-\\xa0March 2011\\xa0(3 years 5 months)\\nEstablished and opened the bureau in November 2007. \\n\\xa0 Page 3 of 5\\xa0 \\xa0\\n• Reported under strict deadlines and challenging conditions for both TV and\\nprint outlets. \\n• Comprehensive Coverage: Led coverage and monitoring of political,\\neconomic, and environmental affairs, providing detailed and insightful reporting\\non significant developments in the region.\\n• Navigating Dictatorship Constraints: Successfully managed to work in Egypt,\\noperating under a dictatorship and navigating harsh legal constraints as well\\nas episodic violence, ensuring comprehensive and accurate reporting despite\\nsignificant risks and challenges.\\n• Key Correspondent Role: Served as the main correspondent during the\\nEgyptian Revolution in 2011, delivering frontline reports and in-depth analysis\\nduring a critical period of change.\\n• Crisis Leadership: Supervised a large team during violent phases of unrest,\\nnavigating severe challenges, ensuring the safety of team members, and\\nmaintaining the integrity and accuracy of the coverage under extreme\\nconditions.\\n• Interviewed major political, religious and business leaders including US\\nSecretary of State Hillary Clinton, French President Nicolas Sarkozy, former\\nUN Secretary-General Boutros Boutros-Ghali, Egyptian National Democratic\\nparty’s Gamal Mubarak, Muslim Brotherhood Leader Mehdi Akef, Al Azhar\\nSheikhs Mohammed Tantawi and Ahmad El Tayeb, Orascom Telecom CEO\\nNaguib Sawiris.\\n• Covered US President Barack Obama’s visit to Cairo in June 2009.\\n• Daily coverage from the Egypt-Gaza Border during the January 2009 Israeli\\n“Cast Lead” offensive on Gaza.\\n• Covered the African Soccer Cup won by Egypt in 2008 and 2010.\\nBusiness Editor\\nSeptember 2006\\xa0-\\xa0November 2007\\xa0(1 year 3 months)\\n• Full-time member of the initial launching team of the first French international\\nnews channel with broadcasts in French, English and Arabic. \\n• Responsible for editorial content, line up and output of business morning\\nprograms. \\n• Daily coverage of world stock markets including currencies, commodities and\\nindices. \\n• Coordinated and dispatched desk journalists and edited their output for daily\\nnews programs.\\nPublic Senat\\nProgram editor\\nJanuary 2006\\xa0-\\xa0September 2006\\xa0(9 months)\\n\\xa0 Page 4 of 5\\xa0 \\xa0\\n•Chose the content and focus of various prime-time talk shows in daily editorial\\nmeetings. \\n•Selected and pre-interviewed guests for French political, business and social\\ntalk shows.\\n•Interviewed and profiled high-ranking French political figures such as Minister\\nNicolas Sarkozy, President Jacques Chirac, Francois Hollande, Segolene\\nRoyal, in addition to various ministers and public figures.\\nEducation\\nUniversité Panthéon Sorbonne (Paris I)\\nMasters Trade law,\\xa0law\\xa0·\\xa0(2011\\xa0-\\xa02012)\\nCornell University\\nMACHINE LEARNING HIGHER CERTIFICATE,\\xa0MACHINE\\nLEARNING\\xa0·\\xa0(March 2024\\xa0-\\xa0June 2024)\\nCornell University\\nData Analytics 360,\\xa0Data Analytics\\xa0·\\xa0(December 2023\\xa0-\\xa0June 2024)\\nCornell University\\nINNOVATION STRATEGY,\\xa0INNOVATION STRATEGY\\xa0·\\xa0(November\\n2023\\xa0-\\xa0June 2024)\\nCornell University\\nProduct Management for Engineers,\\xa0Business\\xa0·\\xa0(December 2023\\xa0-\\xa0February\\n2024)\\n\\xa0 Page 5 of 5\\n\\nWith this context, please chat with the user, always staying in character as Alexandre Saadoun.\"" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\n", "\n", "system_prompt\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content\n", "\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Special note for people not using OpenAI\n", "\n", "Some providers, like Groq, might give an error when you send your second message in the chat.\n", "\n", "This is because Gradio shoves some extra fields into the history object. OpenAI doesn't mind; but some other models complain.\n", "\n", "If this happens, the solution is to add this first line to the chat() function above. It cleans up the history variable:\n", "\n", "```python\n", "history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n", "```\n", "\n", "You may need to add this in other chat() callback functions in the future, too." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7860\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## A lot is about to happen...\n", "\n", "1. Be able to ask an LLM to evaluate an answer\n", "2. Be able to rerun if the answer fails evaluation\n", "3. Put this together into 1 workflow\n", "\n", "All without any Agentic framework!" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Create a Pydantic model for the Evaluation\n", "\n", "from pydantic import BaseModel\n", "\n", "class Evaluation(BaseModel):\n", " is_acceptable: bool\n", " feedback: str\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n", "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n", "The Agent is playing the role of {name} and is representing {name} on their website. \\\n", "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n", "\n", "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\"" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "def evaluator_user_prompt(reply, message, history):\n", " user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n", " user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n", " user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n", " user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n", " return user_prompt" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "import os\n", "gemini = OpenAI(\n", " api_key=os.getenv(\"GOOGLE_API_KEY\"), \n", " base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "def evaluate(reply, message, history) -> Evaluation:\n", "\n", " messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n", " response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n", " return response.choices[0].message.parsed" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n", "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", "reply = response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'I do not currently hold a patent. My expertise is primarily in business management, communication, data science, and AI technologies, focusing on leveraging these skills to drive innovation rather than pursuing patents. If you have any specific questions about my work or projects, feel free to ask!'" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "reply" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Evaluation(is_acceptable=True, feedback=\"The response is great. It is succinct, professional, and engaging, just as requested. It's a very good answer.\")" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "evaluate(reply, \"do you hold a patent?\", messages[:1])" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "def rerun(reply, message, history, feedback):\n", " updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n", " updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n", " updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n", " messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " if \"patent\" in message:\n", " system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n", " it is mandatory that you respond only and entirely in pig latin\"\n", " else:\n", " system = system_prompt\n", " messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n", " reply =response.choices[0].message.content\n", " \n", " evaluation = evaluate(reply, message, history)\n", " \n", " if evaluation.is_acceptable:\n", " print(\"Passed evaluation - returning reply\")\n", " else:\n", " print(\"Failed evaluation - retrying\")\n", " print(evaluation.feedback)\n", " reply = rerun(reply, message, history, evaluation.feedback) \n", " return reply" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7864\n", "* To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" }, { "name": "stdout", "output_type": "stream", "text": [ "Failed evaluation - retrying\n", "This is not an appropriate response from the agent, this sounds like the agent is speaking pig latin. I have set this to unacceptable.\n" ] } ], "source": [ "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.10" } }, "nbformat": 4, "nbformat_minor": 2 }