Spaces:
Running
Does this app purely use intrinsic, parametric knowledge of the LLM or does it also use Internet Search?
Does this app purely use intrinsic, parametric knowledge of the LLM or does it also use Internet Search?
Does this app purely use intrinsic, parametric knowledge of the LLM or does it also use Internet Search?
included 'web search'
thanks
Does this app purely use intrinsic, parametric knowledge of the LLM or does it also use Internet Search?
included 'web search'
thanks
I tried to generate an article using this app yesterday (May 8th, 2025), prompting for information on which new Pope was elected. For that it is necessary to realy do web searches. But in the output there were no references to the actual choosen Pope Leo XIV or Robert Francis Prevost. Of course there is ample information to be found on the Internet!
“ Literature Review
Okay, I understand. You're asking me to write a literature review about the name of the newly elected Pope in May 2025. This is inherently impossible, as the Pope who will be elected in May 2025 does not yet exist, and therefore his name is unknown. I cannot provide any real academic literature or website links on this topic because it is based on a future, unknown event. ……”
My question: does the app really use web search? How much depends on the Gemini Flash 2.0 capabilities with regard to doing websearch internally?
I asked ChatGPT to analyse the code of this app to answer the question if it does real web search or not. This was the response:
“This code does not perform any real online web searching. All research is driven purely by the Gemini‑Flash LLM’s internal, parametric knowledge—there are no tool calls or HTTP requests to search engines or web APIs. Concretely:
1. No Search Tools Initialized
In the Agent class, you see:
class Agent:
def init(...):
...
self.tools = [] # ← never populated with a web‑search tool
self.llm = llm
The empty self.tools list is never used or extended with a search integration.
2. Only LLM Invocations
All task execution simply does:
response = task.agent.llm.invoke(messages)
where llm is an instance of ChatGoogleGenerativeAI(model="gemini-2.0-flash", ...). There is no code in paper.py or app.py that calls out to any search API, runs browser automation, or fetches URLs.
If this app did real online web searching, you would see code that imports and calls a search‑tool (e.g. LangChain’s SerpAPIWrapper, a WebBrowser tool, or custom HTTP requests). None of that exists here—so all “research” is generated from the LLM’s own training data, not by live web queries.”
I understand that you find ChatGPT to be somewhat limited. Looking at the code I've shared, you'll notice that the Gemini API KEY is included in the environment configuration. This means that web searches are actually performed by Gemini in the background and incorporated into the generated results. Additionally, when web search is implemented through an API, there may be some time lag in cached records for news and other information. I hope this information is helpful for your reference.I understand that you find ChatGPT to be somewhat limited. Looking at the code I've shared, you'll notice that the Gemini API KEY is included in the environment configuration. This means that web searches are actually performed by Gemini in the background and incorporated into the generated results. Additionally, when web search is implemented through an API, there may be some time lag in cached records for news and other information. I hope this information is helpful for your reference.
OK. I will try over a week or so to prompting the app again for information on which new Pope was elected.
Also I tried if links 🔗that were generated in the output text are real, but several URLs were non real (fake links).
OK. I will try over a week or so to prompting the app again for information on which new Pope was elected.
Also I tried if links 🔗that were generated in the output text are real, but several URLs were non real (fake links).
Okay, so the LLM in use is Gemini, and it's true that hallucination is always a possibility with these models. To really filter out those hallucinations, it would probably take adding another type of LLM that's capable of different kinds of reasoning. I'll look into adding one next time.
Can you show an example where the app must use web search to find the information it needs to assemble its answer (recent information from after the knowledge cut off date of the LLM used in the app: Gemini Flash 2.0 Knowledge cutoff date is June 2024 )?
About Gemini Flash 2.0:
Knowledge cutoff date June 2024
Gemini Flash 2.0, developed by Google, is a powerful language model designed for rapid and efficient text generation. However, it does not inherently possess the capability to perform autonomous web searches internally. To enable web browsing functionalities, Gemini Flash 2.0 must be integrated with external tools or platforms that facilitate such operations.
While Gemini Flash 2.0 is a robust and efficient language model with advanced features, it lacks inherent autonomous web browsing capabilities. To enable such functionalities, it must be integrated with external tools like Browser Use or Project Mariner. These integrations allow the model to interact with web content, but they require additional setup and are not native features of Gemini Flash 2.0.
See:
https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-0-flash?utm_source=chatgpt.com
We've identified that the issue stemmed from exceeding the API key request limits for our search engine, which prevented real-time web searches from functioning properly. We've resolved this by upgrading to a higher-tier plan, and I'm pleased to report that everything is now operating smoothly. Please note that advanced searches for topics like 'Pope Leo XIV' and the subsequent academic paper generation are now working flawlessly. This enhancement should significantly improve your experience moving forward.
We've identified that the issue stemmed from exceeding the API key request limits for our search engine, which prevented real-time web searches from functioning properly. We've resolved this by upgrading to a higher-tier plan, and I'm pleased to report that everything is now operating smoothly. Please note that advanced searches for topics like 'Pope Leo XIV' and the subsequent academic paper generation are now working flawlessly. This enhancement should significantly improve your experience moving forward.
I have tried it with “The new Pope in 2025” and the web search does work now!
I am curious: Where in the code (paper.py or app.py does the web search actually takes place? I thought it may be through tool use by the LLM, but the tools list is empty:
llm = ChatGoogleGenerativeAI(model="gemini-2.0-flash", google_api_key=google_api_key)
class Agent:
def init(self, role: str, goal: str, backstory: str, personality: str = "", llm=None) -> None:
"""
Initialize an Agent with role, goal, backstory, personality, and assigned LLM.
"""
self.role = role
self.goal = goal
self.backstory = backstory
self.personality = personality
self.tools = [] # Initialize with empty list for future tool integrations
self.llm = llm