domain_investigation_tools_rod

README.md

@@ -8,84 +8,9 @@ sdk_version: 5.32.1
 app_file: app.py
 pinned: false
 license: apache-2.0
-tags:
- - mcp-server-track
-short_description: Cybersecurity MCP tools to enhance threat insights
+short_description: tdb
 ---
 
-
-# TDAgentTools & TDAgent: Empowering Cybersecurity with Agentic AI
-
-Welcome to TDAgentTools & TDAgent, our innovative proof of concept (PoC) crafted for the Agents-MCP Hackathon. Our initiatives focus on leveraging Agentic AI to enhance cybersecurity threat analysis, providing robust tools for data enrichment and strategic advice for incident handling.
-
-## Team Introduction
-
-We are an AI-focused team within a company, dedicated to empowering other teams by implementing AI solutions. Our expertise lies in automating processes to enhance productivity and tackle complex tasks that AI excels in. Our hackathon team members include:
-
-- Pedro Completo Bento
-- Josep Pon Farreny
-- Sofia Jeronimo dos Santos
-- Rodrigo Dominguez Sanz
-- Miguel Rodin
-
-## Project Overview
-
-### Track 1: MCP Tool - TDAgentTools
-
-TDAgentTools serves as an MCP server built using Gradio, offering a wide array of cybersecurity intelligence tools. These tools enable users to augment their LLMs' capabilities by integrating with various publicly available cybersecurity intel resources. Our TDAgentTools are accessible via the following link: [TDAgentTools Space](https://huggingface.co/spaces/Agents-MCP-Hackathon/TDAgentTools).
-
-#### Available Tools:
-1. **TDAgentTools_get_url_http_content**: Retrieve URL content through an HTTP GET request.
-2. **TDAgentTools_query_abuseipdb**: Query AbuseIPDB to check if an IP is reported for abusive behavior.
-3. **TDAgentTools_query_rdap**: Gather information about internet resources such as domain names and IP addresses.
-4. **TDAgentTools_get_virus_total_url_info**: Fetch URL information using VirusTotal URL Scanner.
-5. **TDAgentTools_get_geolocation**: Obtain location details from an IP address.
-6. **TDAgentTools_enumerate_dns**: Access DNS configuration details for a given domain.
-7. **TDAgentTools_scrap_subdomains_for_domain**: Retrieve subdomains related to a domain.
-8. **TDAgentTools_retrieve_ioc_from_threatfox**: Get potential IoC information from ThreatFox.
-9. **TDAgentTools_get_stix_object_of_attack_id**: Access a STIX object using an ATT&CK ID.
-10. **TDAgentTools_lookup_user**: Seek user details from the Company User Lookup System.
-11. **TDAgentTools_lookup_cloud_account**: Investigate cloud account information.
-12. **TDAgentTools_send_email**: Simulate emailing from [email protected].
-
-> **Note:** TDAgentTools rely on publicly provided APIs and some of which require API keys. If any of these API keys are revoked, certain tools may not function as intended.
-
-[Track1 Demo link](https://youtu.be/c7Yg_jOD6J0)
-
-
-### Track 3: Agentic Demo Showcase - TDAgent
-
-TDAgent is an adaptive and interactive AI agent. This agent facilitates a dynamic AI experience, allowing users to switch the LLM used and adjust the system prompt to refine the agent’s behavior and objectives. It uses TDAgentTools to enrich threat data. Explore it here: [TDAgent Space](https://huggingface.co/spaces/Agents-MCP-Hackathon/TDAgent).
-
-#### Key Features:
-- **Intelligent API Interactions**: The agent autonomously interacts with APIs for data enrichment and analysis without explicit user guidance.
-- **Enhanced Data Enrichment**: Automatically enriches initial incident data, providing deeper insights.
-- **Actionable Intelligence**: Suggests actions based on enriched data and analysis, displaying concise outputs for clearer communication.
-- **Versatile Adaptability**: Capable of switching LLMs for varied results and enhanced debugging.
-
-## Motivation and Goals
-
-Our primary motivation is to explore Agentic AI applications in the cybersecurity realm, focusing on AI agent support for:
-1. Enriching reported threat data.
-2. Assisting analysts in threat analysis.
-
-We aimed to:
-- Explore Agentic AI technologies like Gradio and MCP.
-- Enhance AI agent data enrichment with custom tools.
-- Enable agent autonomy in API interaction and threat assessment.
-- Equip the agent to propose specific incident response actions.
-
-[Track3 Demo link](https://youtu.be/C6Z9EOW-3lE)
-
-## Insights & Conclusions
-
-- **Agent's Autonomy**: Demonstrated autonomous API interactions and data enrichment capabilities.
-- **Enhanced Decision-Making**: The agent suggests data-driven insights beyond API outputs.
-- **Future Improvements**: Plan to fine-tune threat escalation logic and introduce additional decision layers for enhanced threat management.
-
-Our projects successfully demonstrated rapid prototyping with Gradio and Hugging Face Spaces, achieving all intended objectives while providing an engaging and rewarding experience for our team. This PoC shows the potential for future expansions and refinements in the realm of cybersecurity AI support!
-
-
 # TDA Agent Tools
 
 # Development setup

app.py

@@ -1,133 +1,24 @@
-from pathlib import Path
-from typing import NamedTuple
-
 import gradio as gr
-import gradio.themes as gr_themes
-import markdown
 
-from tdagent.tools.get_domain_information import (
-    dns_enumeration_tool,
-    extractor_of_ioc_from_threatfox_tool,
-    geo_location_tool,
-    scrap_subdomains_tool,
-)
-from tdagent.tools.get_url_content import gr_make_http_request
+from tdagent.tools.get_url_content import gr_get_url_http_content
 from tdagent.tools.internal_company_user_search import gr_internal_company
-from tdagent.tools.lookup_company_cloud_account_information import (
-    gr_lookup_company_cloud_account_information,
-)
+from tdagent.tools.letter_counter import gr_letter_counter
+from tdagent.tools.lookup_company_cloud_account_information import gr_lookup_company_cloud_account_information
 from tdagent.tools.query_abuse_ip_db import gr_query_abuseipdb
-from tdagent.tools.rdap import gr_query_rdap
-from tdagent.tools.retrieve_from_mitre_attack import gr_get_stix_of_attack_id
 from tdagent.tools.send_email import gr_send_email
-from tdagent.tools.virus_total import gr_virus_total_url_info
-
-
-# from tdagent.tools.whois import gr_query_whois
-
-
-## Tools to load into the application interface ##
-
-
-def _read_markdown_body_as_html(path: str = "README.md") -> str:
-    with Path(path).open(encoding="utf-8") as f:  # Default mode is "r"
-        lines = f.readlines()
-
-    # Skip YAML front matter if present
-    if lines and lines[0].strip() == "---":
-        for i in range(1, len(lines)):
-            if lines[i].strip() == "---":
-                lines = lines[i + 1:]  # skip metadata block
-                break
-
-    markdown_body = "".join(lines).strip()
-    return markdown.markdown(markdown_body)
-
-
-class ToolInfo(NamedTuple):
-    """Gradio MCP tool info."""
-
-    name: str
-    interface: gr.Interface
-
-
-TOOLS = (
-    ToolInfo("Make an HTTP request to a URL with specified method and parameters", gr_make_http_request),
-    ToolInfo("Query AbuseIPDB", gr_query_abuseipdb),
-    # Whois does not work from Spaces (port 43 blocked)
-    # ToolInfo("Query WHOIS", gr_query_whois),
-    ToolInfo("Query RDAP", gr_query_rdap),
-    ToolInfo("Virus Total URL info", gr_virus_total_url_info),
-    ToolInfo("Get IP's Location", geo_location_tool),
-    ToolInfo("DNS Enumerator", dns_enumeration_tool),
-    ToolInfo("Subdomain Retriever", scrap_subdomains_tool),
-    ToolInfo("Extractor of IoCs", extractor_of_ioc_from_threatfox_tool),
-    ToolInfo("ATT&CK STIX information", gr_get_stix_of_attack_id),
-    ## Fake tools
-    ToolInfo("Fake company directory", gr_internal_company),
-    ToolInfo(
-        "Fake company cloud accounts",
+from tdagent.tools.virus_total import gr_virus_total
+
+gr_app = gr.TabbedInterface(
+    [
+        gr_get_url_http_content,
+        gr_letter_counter,
+        gr_query_abuseipdb,
+        gr_virus_total,
+        gr_internal_company,
         gr_lookup_company_cloud_account_information,
-    ),
-    ToolInfo("Send email", gr_send_email),
+        gr_send_email,
+    ],
 )
 
-## Application Interface ##
-
-custom_css = """
-.main-header {
-    background: linear-gradient(135deg, #00a388 0%, #ffae00 100%);
-    padding: 30px;
-    border-radius: 5px;
-    margin-bottom: 20px;
-    text-align: center;
-}
-"""
-with (
-    gr.Blocks(
-        theme=gr_themes.Origin(
-            primary_hue="teal",
-            spacing_size="sm",
-            font="sans-serif",
-        ),
-        title="TDAgent",
-        fill_height=True,
-        fill_width=True,
-        css=custom_css,
-    ) as gr_app,
-):
-    gr.HTML(
-        """
-    <div class="main-header">
-        <h1>👩‍💻 TDAgentTools & TDAgent 👨‍💻</h1>
-        <p style="font-size: 1.2em; margin: 10px 0 0 0;">
-            Empowering Cybersecurity with Agentic AI
-        </p>
-    </div>
-    """,
-    )
-    with gr.Tabs():
-        with gr.TabItem("About"):
-            html_content = _read_markdown_body_as_html("README.md")
-            gr.Markdown(html_content)
-        with gr.TabItem("TDAgentTools"):
-            gr.TabbedInterface(
-                interface_list=[t_info.interface for t_info in TOOLS],
-                tab_names=[t_info.name for t_info in TOOLS],
-                title="TDAgentTools",
-            )
-        with gr.TabItem("Demo"):
-            gr.Markdown(
-                """
-            This is a demo of TDAgentTools, a simple MCP server.
-            Be carefull with using well-known urls for malware distribution
-            when using the url content extractor tool.
-            """,
-            )
-            gr.HTML(
-                """<iframe width="560" height="315" src="https://youtube.com/embed/c7Yg_jOD6J0" frameborder="0" allowfullscreen></iframe>""",
-                # noqa: E501
-            )
-
 if __name__ == "__main__":
     gr_app.launch(mcp_server=True)

packages.txt

@@ -1 +0,0 @@
-whois

pyproject.toml

@@ -12,18 +12,7 @@ authors = [
 requires-python = ">=3.10,<4"
 readme = "README.md"
 license = ""
-dependencies = [
-    "attackcti>=0.5.4",
-    "audioop-lts>=0.2.1 ; python_full_version >= '3.13'",
-    "black>=25.1.0",
-    "cachetools>=6.0.0",
-    "dnspython>=2.7.0",
-    "gradio[mcp]>=5.32.1",
-    "markdown>=3.8",
-    "python-whois>=0.9.5",
-    "requests>=2.32.3",
-    "vt-py~=0.21.0",
-]
+dependencies = ["gradio[mcp]>=5.32.1", "requests>=2.32.3", "vt-py~=0.21.0"]
 
 [project.scripts]
 
@@ -109,7 +98,7 @@ line-length = 88
 
 [tool.ruff.lint]
 select = ["ALL"]
-ignore = ["D100", "D104", "D107", "D401", "EM102", "ERA001", "TRY003", "UP038"]
+ignore = ["D100", "D104", "D107", "D401", "EM102", "ERA001", "TRY003"]
 
 [tool.ruff.lint.flake8-quotes]
 inline-quotes = "double"
@@ -133,5 +122,4 @@ convention = "google"
 [tool.ruff.lint.per-file-ignores]
 "*/__init__.py" = ["F401"]
 "tdagent/cli/**/*.py" = ["D103", "T201"]
-"tdagent/tools/rdap.py" = ["PLR2004"]
 "tests/*.py" = ["D103", "PLR2004", "S101"]

requirements-dev.txt

@@ -6,14 +6,12 @@ aiofiles==24.1.0
     #   vt-py
 aiohappyeyeballs==2.6.1
     # via aiohttp
-aiohttp==3.12.9
+aiohttp==3.12.8
     # via vt-py
 aiosignal==1.3.2
     # via aiohttp
 annotated-types==0.7.0
     # via pydantic
-antlr4-python3-runtime==4.9.3
-    # via stix2-patterns
 anyio==4.9.0
     # via
     #   gradio
@@ -23,22 +21,14 @@ anyio==4.9.0
     #   starlette
 async-timeout==5.0.1 ; python_full_version < '3.11'
     # via aiohttp
-attackcti==0.5.4
-    # via tdagent
 attrs==25.3.0
     # via aiohttp
 audioop-lts==0.2.1 ; python_full_version >= '3.13'
-    # via
-    #   gradio
-    #   tdagent
-black==25.1.0
-    # via tdagent
+    # via gradio
 boolean-py==5.0
     # via license-expression
 cachecontrol==0.14.3
     # via pip-audit
-cachetools==6.0.0
-    # via tdagent
 certifi==2025.4.26
     # via
     #   httpcore
@@ -48,9 +38,8 @@ cfgv==3.4.0
     # via pre-commit
 charset-normalizer==3.4.2
     # via requests
-click==8.2.1
+click==8.2.1 ; sys_platform != 'emscripten'
     # via
-    #   black
     #   typer
     #   uvicorn
 colorama==0.4.6 ; sys_platform == 'win32'
@@ -66,8 +55,6 @@ defusedxml==0.7.1
     # via py-serializable
 distlib==0.3.9
     # via virtualenv
-dnspython==2.7.0
-    # via tdagent
 exceptiongroup==1.3.0 ; python_full_version < '3.11'
     # via
     #   anyio
@@ -89,7 +76,7 @@ fsspec==2025.5.1
     # via
     #   gradio-client
     #   huggingface-hub
-gradio==5.33.0
+gradio==5.32.1
     # via tdagent
 gradio-client==1.10.2
     # via gradio
@@ -99,7 +86,7 @@ h11==0.16.0
     # via
     #   httpcore
     #   uvicorn
-hf-xet==1.1.3 ; platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'
+hf-xet==1.1.2 ; platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'
     # via huggingface-hub
 httpcore==1.0.9
     # via httpx
@@ -129,8 +116,6 @@ jinja2==3.1.6
     # via gradio
 license-expression==30.4.1
     # via cyclonedx-python-lib
-markdown==3.8
-    # via tdagent
 markdown-it-py==3.0.0
     # via rich
 markupsafe==3.0.2
@@ -149,9 +134,7 @@ multidict==6.4.4
     #   yarl
 mypy==1.16.0
 mypy-extensions==1.1.0
-    # via
-    #   black
-    #   mypy
+    # via mypy
 nodeenv==1.9.1
     # via pre-commit
 numpy==2.2.6
@@ -160,23 +143,20 @@ numpy==2.2.6
     #   pandas
 orjson==3.10.18
     # via gradio
-packageurl-python==0.17.0
+packageurl-python==0.16.0
     # via cyclonedx-python-lib
 packaging==25.0
     # via
-    #   black
     #   gradio
     #   gradio-client
     #   huggingface-hub
     #   pip-audit
     #   pip-requirements-parser
     #   pytest
-pandas==2.3.0
+pandas==2.2.3
     # via gradio
 pathspec==0.12.1
-    # via
-    #   black
-    #   mypy
+    # via mypy
 pillow==11.2.1
     # via gradio
 pip==25.1.1
@@ -188,7 +168,6 @@ pip-requirements-parser==32.0.1
     # via pip-audit
 platformdirs==4.3.8
     # via
-    #   black
     #   pip-audit
     #   virtualenv
 pluggy==1.6.0
@@ -202,7 +181,6 @@ py-serializable==2.0.0
     # via cyclonedx-python-lib
 pydantic==2.11.5
     # via
-    #   attackcti
     #   fastapi
     #   gradio
     #   mcp
@@ -224,22 +202,15 @@ pytest==7.4.4
 pytest-cov==4.1.0
 pytest-randomly==3.16.0
 python-dateutil==2.9.0.post0
-    # via
-    #   pandas
-    #   python-whois
+    # via pandas
 python-dotenv==1.1.0
     # via pydantic-settings
 python-multipart==0.0.20
     # via
     #   gradio
     #   mcp
-python-whois==0.9.5
-    # via tdagent
 pytz==2025.2
-    # via
-    #   pandas
-    #   stix2
-    #   taxii2-client
+    # via pandas
 pyyaml==6.0.2
     # via
     #   gradio
@@ -250,8 +221,6 @@ requests==2.32.3
     #   cachecontrol
     #   huggingface-hub
     #   pip-audit
-    #   stix2
-    #   taxii2-client
     #   tdagent
 rich==14.0.0
     # via
@@ -265,13 +234,8 @@ semantic-version==2.10.0
     # via gradio
 shellingham==1.5.4 ; sys_platform != 'emscripten'
     # via typer
-simplejson==3.20.1
-    # via stix2
 six==1.17.0
-    # via
-    #   python-dateutil
-    #   stix2-patterns
-    #   taxii2-client
+    # via python-dateutil
 sniffio==1.3.1
     # via anyio
 sortedcontainers==2.4.0
@@ -283,21 +247,14 @@ starlette==0.46.2
     #   fastapi
     #   gradio
     #   mcp
-stix2==3.0.1
-    # via attackcti
-stix2-patterns==2.0.0
-    # via stix2
-taxii2-client==2.3.0
-    # via attackcti
 toml==0.10.2
     # via pip-audit
 tomli==2.2.1 ; python_full_version <= '3.11'
     # via
-    #   black
     #   coverage
     #   mypy
     #   pytest
-tomlkit==0.13.3
+tomlkit==0.13.2
     # via gradio
 tqdm==4.67.1
     # via huggingface-hub
@@ -306,7 +263,6 @@ typer==0.16.0 ; sys_platform != 'emscripten'
 typing-extensions==4.14.0
     # via
     #   anyio
-    #   black
     #   exceptiongroup
     #   fastapi
     #   gradio

requirements.txt

@@ -1,19 +1,17 @@
 # This file was autogenerated by uv via the following command:
-#    uv export --format requirements-txt --no-hashes --no-dev -o requirements.txt
+#    uv pip compile pyproject.toml -o requirements.txt
 aiofiles==24.1.0
     # via
     #   gradio
     #   vt-py
 aiohappyeyeballs==2.6.1
     # via aiohttp
-aiohttp==3.12.9
+aiohttp==3.12.8
     # via vt-py
 aiosignal==1.3.2
     # via aiohttp
 annotated-types==0.7.0
     # via pydantic
-antlr4-python3-runtime==4.9.3
-    # via stix2-patterns
 anyio==4.9.0
     # via
     #   gradio
@@ -21,20 +19,10 @@ anyio==4.9.0
     #   mcp
     #   sse-starlette
     #   starlette
-async-timeout==5.0.1 ; python_full_version < '3.11'
+async-timeout==5.0.1
     # via aiohttp
-attackcti==0.5.4
-    # via tdagent
 attrs==25.3.0
     # via aiohttp
-audioop-lts==0.2.1 ; python_full_version >= '3.13'
-    # via
-    #   gradio
-    #   tdagent
-black==25.1.0
-    # via tdagent
-cachetools==6.0.0
-    # via tdagent
 certifi==2025.4.26
     # via
     #   httpcore
@@ -44,22 +32,10 @@ charset-normalizer==3.4.2
     # via requests
 click==8.2.1
     # via
-    #   black
     #   typer
     #   uvicorn
-colorama==0.4.6 ; sys_platform == 'win32'
-    # via
-    #   click
-    #   pytest
-    #   tqdm
-coverage==7.8.2
-    # via pytest-cov
-dnspython==2.7.0
-    # via tdagent
-exceptiongroup==1.3.0 ; python_full_version < '3.11'
-    # via
-    #   anyio
-    #   pytest
+exceptiongroup==1.3.0
+    # via anyio
 fastapi==0.115.12
     # via gradio
 ffmpy==0.6.0
@@ -74,8 +50,8 @@ fsspec==2025.5.1
     # via
     #   gradio-client
     #   huggingface-hub
-gradio==5.33.0
-    # via tdagent
+gradio==5.32.1
+    # via tdagent (pyproject.toml)
 gradio-client==1.10.2
     # via gradio
 groovy==0.1.2
@@ -84,7 +60,7 @@ h11==0.16.0
     # via
     #   httpcore
     #   uvicorn
-hf-xet==1.1.3 ; platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'
+hf-xet==1.1.2
     # via huggingface-hub
 httpcore==1.0.9
     # via httpx
@@ -106,13 +82,9 @@ idna==3.10
     #   httpx
     #   requests
     #   yarl
-iniconfig==2.1.0
-    # via pytest
 jinja2==3.1.6
     # via gradio
-markdown==3.8
-    # via tdagent
-markdown-it-py==3.0.0 ; sys_platform != 'emscripten'
+markdown-it-py==3.0.0
     # via rich
 markupsafe==3.0.2
     # via
@@ -120,14 +92,12 @@ markupsafe==3.0.2
     #   jinja2
 mcp==1.9.0
     # via gradio
-mdurl==0.1.2 ; sys_platform != 'emscripten'
+mdurl==0.1.2
     # via markdown-it-py
 multidict==6.4.4
     # via
     #   aiohttp
     #   yarl
-mypy-extensions==1.1.0
-    # via black
 numpy==2.2.6
     # via
     #   gradio
@@ -136,28 +106,19 @@ orjson==3.10.18
     # via gradio
 packaging==25.0
     # via
-    #   black
     #   gradio
     #   gradio-client
     #   huggingface-hub
-    #   pytest
-pandas==2.3.0
+pandas==2.2.3
     # via gradio
-pathspec==0.12.1
-    # via black
 pillow==11.2.1
     # via gradio
-platformdirs==4.3.8
-    # via black
-pluggy==1.6.0
-    # via pytest
 propcache==0.3.1
     # via
     #   aiohttp
     #   yarl
 pydantic==2.11.5
     # via
-    #   attackcti
     #   fastapi
     #   gradio
     #   mcp
@@ -168,58 +129,38 @@ pydantic-settings==2.9.1
     # via mcp
 pydub==0.25.1
     # via gradio
-pygments==2.19.1 ; sys_platform != 'emscripten'
+pygments==2.19.1
     # via rich
-pytest==7.4.4
-    # via
-    #   pytest-cov
-    #   pytest-randomly
-pytest-cov==4.1.0
-pytest-randomly==3.16.0
 python-dateutil==2.9.0.post0
-    # via
-    #   pandas
-    #   python-whois
+    # via pandas
 python-dotenv==1.1.0
     # via pydantic-settings
 python-multipart==0.0.20
     # via
     #   gradio
     #   mcp
-python-whois==0.9.5
-    # via tdagent
 pytz==2025.2
-    # via
-    #   pandas
-    #   stix2
-    #   taxii2-client
+    # via pandas
 pyyaml==6.0.2
     # via
     #   gradio
     #   huggingface-hub
 requests==2.32.3
     # via
+    #   tdagent (pyproject.toml)
     #   huggingface-hub
-    #   stix2
-    #   taxii2-client
-    #   tdagent
-rich==14.0.0 ; sys_platform != 'emscripten'
+rich==14.0.0
     # via typer
-ruff==0.11.12 ; sys_platform != 'emscripten'
+ruff==0.11.12
     # via gradio
 safehttpx==0.1.6
     # via gradio
 semantic-version==2.10.0
     # via gradio
-shellingham==1.5.4 ; sys_platform != 'emscripten'
+shellingham==1.5.4
     # via typer
-simplejson==3.20.1
-    # via stix2
 six==1.17.0
-    # via
-    #   python-dateutil
-    #   stix2-patterns
-    #   taxii2-client
+    # via python-dateutil
 sniffio==1.3.1
     # via anyio
 sse-starlette==2.3.6
@@ -229,27 +170,15 @@ starlette==0.46.2
     #   fastapi
     #   gradio
     #   mcp
-stix2==3.0.1
-    # via attackcti
-stix2-patterns==2.0.0
-    # via stix2
-taxii2-client==2.3.0
-    # via attackcti
-tomli==2.2.1 ; python_full_version <= '3.11'
-    # via
-    #   black
-    #   coverage
-    #   pytest
-tomlkit==0.13.3
+tomlkit==0.13.2
     # via gradio
 tqdm==4.67.1
     # via huggingface-hub
-typer==0.16.0 ; sys_platform != 'emscripten'
+typer==0.16.0
     # via gradio
 typing-extensions==4.14.0
     # via
     #   anyio
-    #   black
     #   exceptiongroup
     #   fastapi
     #   gradio
@@ -269,17 +198,14 @@ typing-inspection==0.4.1
 tzdata==2025.2
     # via pandas
 urllib3==2.4.0
-    # via
-    #   gradio
-    #   requests
-uvicorn==0.34.3 ; sys_platform != 'emscripten'
+    # via requests
+uvicorn==0.34.3
     # via
     #   gradio
     #   mcp
 vt-py==0.21.0
-    # via tdagent
+    # via tdagent (pyproject.toml)
 websockets==15.0.1
     # via gradio-client
-xdoctest==1.2.0
 yarl==1.20.0
     # via aiohttp

subdomains/subdomains.txt

@@ -1,999 +0,0 @@
-www
-mail
-ftp
-localhost
-webmail
-smtp
-pop
-ns1
-webdisk
-ns2
-cpanel
-whm
-autodiscover
-autoconfig
-m
-imap
-test
-ns
-blog
-pop3
-dev
-www2
-admin
-forum
-news
-vpn
-ns3
-mail2
-new
-mysql
-old
-lists
-support
-mobile
-mx
-static
-docs
-beta
-shop
-sql
-secure
-demo
-cp
-calendar
-wiki
-web
-media
-email
-images
-img
-www1
-intranet
-portal
-video
-sip
-dns2
-api
-cdn
-stats
-dns1
-ns4
-www3
-dns
-search
-staging
-server
-mx1
-chat
-wap
-my
-svn
-mail1
-sites
-proxy
-ads
-host
-crm
-cms
-backup
-mx2
-lyncdiscover
-info
-apps
-download
-remote
-db
-forums
-store
-relay
-files
-newsletter
-app
-live
-owa
-en
-start
-sms
-office
-exchange
-ipv4
-mail3
-help
-blogs
-helpdesk
-web1
-home
-library
-ftp2
-ntp
-monitor
-login
-service
-correo
-www4
-moodle
-it
-gateway
-gw
-i
-stat
-stage
-ldap
-tv
-ssl
-web2
-ns5
-upload
-nagios
-smtp2
-online
-ad
-survey
-data
-radio
-extranet
-test2
-mssql
-dns3
-jobs
-services
-panel
-irc
-hosting
-cloud
-de
-gmail
-s
-bbs
-cs
-ww
-mrtg
-git
-image
-members
-poczta
-s1
-meet
-preview
-fr
-cloudflare-resolve-to
-dev2
-photo
-jabber
-legacy
-go
-es
-ssh
-redmine
-partner
-vps
-server1
-sv
-ns6
-webmail2
-av
-community
-cacti
-time
-sftp
-lib
-facebook
-www5
-smtp1
-feeds
-w
-games
-ts
-alumni
-dl
-s2
-phpmyadmin
-archive
-cn
-tools
-stream
-projects
-elearning
-im
-iphone
-control
-voip
-test1
-ws
-rss
-sp
-wwww
-vpn2
-jira
-list
-connect
-gallery
-billing
-mailer
-update
-pda
-game
-ns0
-testing
-sandbox
-job
-events
-dialin
-ml
-fb
-videos
-music
-a
-partners
-mailhost
-downloads
-reports
-ca
-router
-speedtest
-local
-training
-edu
-bugs
-manage
-s3
-status
-host2
-ww2
-marketing
-conference
-content
-network-ip
-broadcast-ip
-english
-catalog
-msoid
-mailadmin
-pay
-access
-streaming
-project
-t
-sso
-alpha
-photos
-staff
-e
-auth
-v2
-web5
-web3
-mail4
-devel
-post
-us
-images2
-master
-rt
-ftp1
-qa
-wp
-dns4
-www6
-ru
-student
-w3
-citrix
-trac
-doc
-img2
-css
-mx3
-adm
-web4
-hr
-mailserver
-travel
-sharepoint
-sport
-member
-bb
-agenda
-link
-server2
-vod
-uk
-fw
-promo
-vip
-noc
-design
-temp
-gate
-ns7
-file
-ms
-map
-cache
-painel
-js
-event
-mailing
-db1
-c
-auto
-img1
-vpn1
-business
-mirror
-share
-cdn2
-site
-maps
-tickets
-tracker
-domains
-club
-images1
-zimbra
-cvs
-b2b
-oa
-intra
-zabbix
-ns8
-assets
-main
-spam
-lms
-social
-faq
-feedback
-loopback
-groups
-m2
-cas
-loghost
-xml
-nl
-research
-art
-munin
-dev1
-gis
-sales
-images3
-report
-google
-idp
-cisco
-careers
-seo
-dc
-lab
-d
-firewall
-fs
-eng
-ann
-mail01
-mantis
-v
-affiliates
-webconf
-track
-ticket
-pm
-db2
-b
-clients
-tech
-erp
-monitoring
-cdn1
-images4
-payment
-origin
-client
-foto
-domain
-pt
-pma
-directory
-cc
-public
-finance
-ns11
-test3
-wordpress
-corp
-sslvpn
-cal
-mailman
-book
-ip
-zeus
-ns10
-hermes
-storage
-free
-static1
-pbx
-banner
-mobil
-kb
-mail5
-direct
-ipfixe
-wifi
-development
-board
-ns01
-st
-reviews
-radius
-pro
-atlas
-links
-in
-oldmail
-register
-s4
-images6
-static2
-id
-shopping
-drupal
-analytics
-m1
-images5
-images7
-img3
-mx01
-www7
-redirect
-sitebuilder
-smtp3
-adserver
-net
-user
-forms
-outlook
-press
-vc
-health
-work
-mb
-mm
-f
-pgsql
-jp
-sports
-preprod
-g
-p
-mdm
-ar
-lync
-market
-dbadmin
-barracuda
-affiliate
-mars
-users
-images8
-biblioteca
-mc
-ns12
-math
-ntp1
-web01
-software
-pr
-jupiter
-labs
-linux
-sc
-love
-fax
-php
-lp
-tracking
-thumbs
-up
-tw
-campus
-reg
-digital
-demo2
-da
-tr
-otrs
-web6
-ns02
-mailgw
-education
-order
-piwik
-banners
-rs
-se
-venus
-internal
-webservices
-cm
-whois
-sync
-lb
-is
-code
-click
-w2
-bugzilla
-virtual
-origin-www
-top
-customer
-pub
-hotel
-openx
-log
-uat
-cdn3
-images0
-cgi
-posta
-reseller
-soft
-movie
-mba
-n
-r
-developer
-nms
-ns9
-webcam
-construtor
-ebook
-ftp3
-join
-dashboard
-bi
-wpad
-admin2
-agent
-wm
-books
-joomla
-hotels
-ezproxy
-ds
-sa
-katalog
-team
-emkt
-antispam
-adv
-mercury
-flash
-myadmin
-sklep
-newsite
-law
-pl
-ntp2
-x
-srv1
-mp3
-archives
-proxy2
-ps
-pic
-ir
-orion
-srv
-mt
-ocs
-server3
-meeting
-v1
-delta
-titan
-manager
-subscribe
-develop
-wsus
-oascentral
-mobi
-people
-galleries
-wwwtest
-backoffice
-sg
-repo
-soporte
-www8
-eu
-ead
-students
-hq
-awstats
-ec
-security
-school
-corporate
-podcast
-vote
-conf
-magento
-mx4
-webservice
-tour
-s5
-power
-correio
-mon
-mobilemail
-weather
-international
-prod
-account
-xx
-pages
-pgadmin
-bfn2
-webserver
-www-test
-maintenance
-me
-magazine
-syslog
-int
-view
-enews
-ci
-au
-mis
-dev3
-pdf
-mailgate
-v3
-ss
-internet
-host1
-smtp01
-journal
-wireless
-opac
-w1
-signup
-database
-demo1
-br
-android
-career
-listserv
-bt
-spb
-cam
-contacts
-webtest
-resources
-1
-life
-mail6
-transfer
-app1
-confluence
-controlpanel
-secure2
-puppet
-classifieds
-tunet
-edge
-biz
-host3
-red
-newmail
-mx02
-sb
-physics
-ap
-epaper
-sts
-proxy1
-ww1
-stg
-sd
-science
-star
-www9
-phoenix
-pluto
-webdav
-booking
-eshop
-edit
-panelstats
-xmpp
-food
-cert
-adfs
-mail02
-cat
-edm
-vcenter
-mysql2
-sun
-phone
-surveys
-smart
-system
-twitter
-updates
-webmail1
-logs
-sitedefender
-as
-cbf1
-sugar
-contact
-vm
-ipad
-traffic
-dm
-saturn
-bo
-network
-ac
-ns13
-webdev
-libguides
-asp
-tm
-core
-mms
-abc
-scripts
-fm
-sm
-test4
-nas
-newsletters
-rsc
-cluster
-learn
-panelstatsmail
-lb1
-usa
-apollo
-pre
-terminal
-l
-tc
-movies
-sh
-fms
-dms
-z
-base
-jwc
-gs
-kvm
-bfn1
-card
-web02
-lg
-editor
-metrics
-feed
-repository
-asterisk
-sns
-global
-counter
-ch
-sistemas
-pc
-china
-u
-payments
-ma
-pics
-www10
-e-learning
-auction
-hub
-sf
-cbf8
-forum2
-ns14
-app2
-passport
-hd
-talk
-ex
-debian
-ct
-rc
-2012
-imap4
-blog2
-ce
-sk
-relay2
-green
-print
-geo
-multimedia
-iptv
-backup2
-webapps
-audio
-ro
-smtp4
-pg
-ldap2
-backend
-profile
-oldwww
-drive
-bill
-listas
-orders
-win
-mag
-apply
-bounce
-mta
-hp
-suporte
-dir
-pa
-sys
-mx0
-ems
-antivirus
-web8
-inside
-play
-nic
-welcome
-premium
-exam
-sub
-cz
-omega
-boutique
-pp
-management
-planet
-ww3
-orange
-c1
-zzb
-form
-ecommerce
-tmp
-plus
-openvpn
-fw1
-hk
-owncloud
-history
-clientes
-srv2
-img4
-open
-registration
-mp
-blackboard
-fc
-static3
-server4
-s6
-ecard
-dspace
-dns01
-md
-mcp
-ares
-spf
-kms
-intranet2
-accounts
-webapp
-ask
-rd
-www-dev
-gw2
-mall
-bg
-teste
-ldap1
-real
-m3
-wave
-movil
-portal2
-kids
-gw1
-ra
-tienda
-private
-po
-2013
-cdn4
-gps
-km
-ent
-tt
-ns21
-at
-athena
-cbf2
-webmail3
-mob
-matrix
-ns15
-send
-lb2
-pos
-2
-cl
-renew
-admissions
-am
-beta2
-gamma
-mx5
-portfolio
-contest
-box
-mg
-wwwold
-neptune
-mac
-pms
-traveler
-media2
-studio
-sw
-imp
-bs
-alfa
-cbf4
-servicedesk
-wmail
-video2
-switch
-sam
-sky
-ee
-widget
-reklama
-msn
-paris
-tms
-th
-vega
-trade
-intern
-ext
-oldsite
-learning
-group
-f1
-ns22
-ns20
-demo3
-bm
-dom
-pe
-annuaire
-portail
-graphics
-iris
-one
-robot
-ams
-s7
-foro
-gaia
-vpn3

tdagent/constants.py

@@ -1,8 +0,0 @@
-import enum
-
-
-class HttpContentType(str, enum.Enum):
-    """Http content type values."""
-
-    HTML = "text/html"
-    JSON = "application/json"

tdagent/tools/get_domain_information.py

@@ -1,368 +0,0 @@
-import json
-import os
-from concurrent.futures import ThreadPoolExecutor
-from pathlib import Path
-from typing import Any
-
-import cachetools
-import gradio as gr
-import requests
-import urllib3
-from dns import message
-
-
-_DNS_SERVER = "https://dns.google/dns-query"  # can use others
-_DNS_RECORD_TYPES = [
-    "A",
-    "AAAA",
-    "CNAME",
-    "MX",
-    "NS",
-    "SOA",
-    "TXT",
-    "RP",
-    "LOC",
-    "CAA",
-    "SPF",
-    "SRV",
-    "NSEC",
-    "RRSIG",
-]
-
-_COMMON_SUBDOMAINS_TXT_PATH = Path("./subdomains/subdomains.txt")
-
-_CACHE_MAX_SIZE = 4096
-_CACHE_TTL_SECONDS = 3600
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def get_geolocation(ip: str) -> dict[str, Any] | str:
-    """Get location information from an ip address.
-
-    Returns the following information on an ip address:
-        1. IPv4
-        2. city
-        4. country_code
-        5. country_name
-        6. latitude
-        7. longitude
-        8. postal
-        9. state
-
-    Example:
-    >>> from pprint import pprint
-    >>> pprint(get_location("103.100.104.0"))
-    ... {'IPv4': '103.100.104.0',
-        'city': None,
-        'country_code': 'NZ',
-        'country_name': 'New Zealand',
-        'latitude': -41,
-        'longitude': 174,
-        'postal': None,
-        'state': None}
-
-    Args:
-        ip: ip address
-
-    Returns:
-        Location information on the ip address.
-    """
-    try:
-        return requests.get(
-            f"https://geolocation-db.com/json/{ip.strip()}",
-            timeout=1,
-        ).json()
-    except Exception as e:  # noqa: BLE001
-        return str(e)
-
-
-def _request_dns_record(  # noqa: D417
-    domain: str,
-    record_type: str,
-    timeout: float = 0.5,
-) -> list[str]:
-    """Utility to build dns resolve requests that do not use port 53.
-
-    Args:
-        domain: domain to investigate
-        record_type: record type
-
-    Returns:
-        Information about the dns record type for the domain.
-    """
-    q = message.make_query(domain, record_type)
-    response = requests.post(
-        _DNS_SERVER,
-        headers={
-            "Content-Type": "application/dns-message",
-            "Accept": "application/dns-message",
-        },
-        data=q.to_wire(),
-        verify=True,
-        timeout=timeout,
-    )
-    dns_message = message.from_wire(response.content)
-    return [str(rdata) for rdata in dns_message.answer[0]] if dns_message.answer else []
-
-
-# see: https://thepythoncode.com/article/dns-enumeration-with-python
-# https://dnspython.readthedocs.io
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def enumerate_dns(domain_name: str) -> dict[str, Any] | None:
-    r"""Enumerates information about a specific domain's DNS configuration.
-
-    Information collected about the domain name:
-        1. A records: the IPv4 associated with the domain
-        2. AAAA records: the IPv6 associated with the domain
-        3. CAA records: used by owners to specify which Certificate Authorities
-            are authorized to issue SSL/TLS certificates for their domains.
-        4. CNAME records: alias of one name to another - the DNS lookup will
-            continue by retrying the lookup with the new name.
-        5. LOC records: geographic location associated with a domain name.
-        6. MX records: associated email servers to the domain.
-        7. NS records: DNS servers that are authoritative for a particular domain.
-            These may be use to inquire information about the domain.
-        8. SOA records: defines authoritative information about a DNS zone,
-            including zone transfers and cache expiration.
-        9. TXT records: used for domain verification and email security.
-        10. RP records: the responsible person for a domain.
-        11. SPF records: defines authorized email servers.
-        12. SRV records: specifies location of specific services
-            (port and host) for the domain.
-        14. NSEC records: proves non-existence of DNS records
-            and prevents zone enumeration.
-        15. RRSIG records: contains cryptographic signatures for DNSSEC-signed
-            records, providing authentication and integrity.
-
-    Example:
-    >>> from pprint import pprint
-    >>> pprint(enumerate_dns("youtube.com"))
-    ... {'A': 'youtube.com. 300 IN A 142.250.200.142',
-        'AAAA': 'youtube.com. 286 IN AAAA 2a00:1450:4003:80f::200e',
-        'CAA': 'youtube.com. 14352 IN CAA 0 issue "pki.goog"',
-        'CNAME': None,
-        'LOC': None,
-        'MX': 'youtube.com. 300 IN MX 0 smtp.google.com.',
-        'NS': 'youtube.com. 21600 IN NS ns4.google.com.\n'
-            'youtube.com. 21600 IN NS ns1.google.com.\n'
-            'youtube.com. 21600 IN NS ns2.google.com.\n'
-            'youtube.com. 21600 IN NS ns3.google.com.',
-        'NSEC': None,
-        'RP': None,
-        'RRSIG': None,
-        'SOA': 'youtube.com. 60 IN SOA ns1.google.com. dns-admin.google.com. '
-                '766113658 900 900 1800 60',
-        'SPF': None,
-        'SRV': None,
-        'TXT': 'youtube.com. 3586 IN TXT "v=spf1 include:google.com mx -all"\n'
-                'youtube.com. 3586 IN TXT '
-                '"facebook-domain-verification=64jdes7le4h7e7lfpi22rijygx58j1"\n'
-                'youtube.com. 3586 IN TXT '
-                '"google-site-verification=QtQWEwHWM8tHiJ4s-jJWzEQrD_fF3luPnpzNDH-Nw-w"'}
-
-    Args:
-        domain_name: domain name for which to
-            enumerate the DNS configuration.
-
-    Returns:
-        The domain's DNS configuration.
-    """
-    enumeration = {}
-    for record_type in _DNS_RECORD_TYPES:
-        try:
-            record = _request_dns_record(domain_name.strip(), record_type, timeout=1)
-            if record:
-                enumeration[record_type] = record
-        except Exception as e:  # noqa: BLE001, PERF203
-            enumeration[record_type] = [str(e)]
-    return enumeration if enumeration else None
-
-
-def resolve_subdomain_ipv4(domain: str) -> str | None:
-    """Resolve the IPv4 address of a domain.
-
-    Args:
-        domain: domain name
-
-    Returns:
-        The domain is returned provided
-            it was resolved. Otherwise nothing
-            is returned.
-    """
-    try:
-        ipv4 = _request_dns_record(domain, "A", timeout=0.6)
-        if ipv4:
-            return domain
-        msg = "Cannot resolve it: it is likely non-existing"
-        raise Exception(msg)  # noqa: TRY002, TRY301
-    except Exception:  # noqa: BLE001
-        return None
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def scrap_subdomains_for_domain(domain_name: str) -> list[str]:
-    """Retrieves subdomains associated to a domain if any.
-
-    The information retrieved from a domain is its subdomains
-    provided they are the top 1000 subdomain prefixes as
-    indicated by https://github.com/rbsec/dnscan/tree/master
-
-    Importantly, it finds subdomains only if their prefixes
-    are along the top 1000 most common. Hence, it may not
-    yield all the subdomains associated to the domain.
-
-    Example:
-    >>> scrap_subdomains_for_domain("github.com")
-    ... ['www.github.com', 'smtp.github.com', 'ns1.github.com',
-        'ns2.github.com','autodiscover.github.com', 'test.github.com',
-        'blog.github.com', 'admin.github.com', 'support.github.com',
-        'docs.github.com', 'shop.github.com', 'wiki.github.com',
-        'api.github.com', 'live.github.com', 'help.github.com',
-        'jobs.github.com', 'services.github.com', 'de.github.com',
-        'cs.github.com', 'fr.github.com', 'ssh.github.com',
-        'partner.github.com', 'community.github.com',
-        'mailer.github.com', 'training.github.com', ...]
-
-    Args:
-        domain_name: domain name for which to retrieve a
-            list of subdomains
-
-    Returns:
-        List of subdomains if any.
-    """
-    try:
-        with open(_COMMON_SUBDOMAINS_TXT_PATH) as file:  # noqa: PTH123
-            subdomains = [line.strip() for line in file if line.strip()]
-    except FileNotFoundError:
-        return []
-
-    potential_subdomains = [
-        f"{subdomain}.{domain_name.strip()}" for subdomain in subdomains
-    ]
-    with ThreadPoolExecutor(max_workers=None) as executor:
-        results = executor.map(resolve_subdomain_ipv4, potential_subdomains)
-        return [domain for domain in results if domain]
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def retrieve_ioc_from_threatfox(potentially_ioc: str) -> str:
-    r"""Retrieves information about a potential IoC from ThreatFox.
-
-    It may be used to retrieve information of indicators of compromise
-    (IOCs) associated with malware, with the infosec community, AV
-    vendors and cyber threat intelligence providers.
-
-    Examples:
-    >>> retrieve_ioc_from_threatfox("139.180.203.104")
-    ... {
-    "query_status": "ok",
-    "data": [
-        {
-            "id": "12",
-            "ioc": "139.180.203.104:443",
-            "threat_type": "botnet_cc",
-            "threat_type_desc": "Indicator that identifies a botnet command&control...",
-            "ioc_type": "ip:port",
-            "ioc_type_desc": "ip:port combination that is used for botnet Command&...,
-            "malware": "win.cobalt_strike",
-            "malware_printable": "Cobalt Strike",
-            "malware_alias": "Agentemis,BEACON,CobaltStrike",
-            "malware_malpedia": "https:\/\/malpedia.caad.fkie.fraunhofer.de\/...",
-            "confidence_level": 75,
-            "first_seen": "2020-12-06 09:10:23 UTC",
-            "last_seen": null,
-            "reference": null,
-            "reporter": "abuse_ch",
-            "tags": null,
-            "malware_samples": [
-                {
-                    "time_stamp": "2021-03-23 08:18:06 UTC",
-                    "md5_hash": "5b7e82e051ade4b14d163eea2a17bf8b",
-                    "sha256_hash": "b325c92fa540edeb89b95dbfd4400c1cb33599c66859....",
-                    "malware_bazaar": "https:\/\/bazaar.abuse.ch\/sample\/b325c...\/"
-                },
-            ]
-
-        }
-    ]
-    }
-
-    Args:
-        potentially_ioc: this can be a url, a domain, a hash,
-            or any other type of IoC.
-
-    Returns:
-        Information of the input as an IoC: threat type, malware type andsamples,
-            confidence level, first/last seen dates, and more IoC information.
-    """
-    headers = {"Auth-Key": os.environ["THREATFOX_APIKEY"]}
-    pool = urllib3.HTTPSConnectionPool(
-        "threatfox-api.abuse.ch",
-        port=443,
-        maxsize=50,
-        headers=headers,
-        timeout=5,
-    )
-    data = {
-        "query": "search_ioc",
-        "search_term": potentially_ioc.strip(),
-    }
-    json_data = json.dumps(data)
-    try:
-        response = pool.request("POST", "/api/v1/", body=json_data)
-        return response.data.decode("utf-8", "ignore")
-    except Exception as e:  # noqa: BLE001
-        return str(e)
-
-
-geo_location_tool = gr.Interface(
-    fn=get_geolocation,
-    inputs=gr.Textbox(label="ip"),
-    outputs=gr.JSON(label="Geolocation of IP"),
-    title="Domain Associated Geolocation Finder",
-    description="Retrieves the geolocation associated to an input ip address",
-    theme="default",
-    examples=["1.0.3.255", "59.34.7.3"],
-)
-
-dns_enumeration_tool = gr.Interface(
-    fn=enumerate_dns,
-    inputs=gr.Textbox(label="domain"),
-    outputs=gr.JSON(label="DNS records"),
-    title="DNS record enumerator of domains",
-    description="Retrieves several dns record types for the input domain names",
-    theme="default",
-    examples=["owasp.org", "nist.gov"],
-)
-
-scrap_subdomains_tool = gr.Interface(
-    fn=scrap_subdomains_for_domain,
-    inputs=gr.Textbox(label="domain"),
-    outputs=gr.JSON(label="Subdomains managed by domain"),
-    title="Subdomains Extractor of domains",
-    description="Retrieves the subdomains for the input domain if they are common",
-    theme="default",
-    examples=["github.com", "netacea.com"],
-)
-
-extractor_of_ioc_from_threatfox_tool = gr.Interface(
-    fn=retrieve_ioc_from_threatfox,
-    inputs=gr.Textbox(label="IoC - url, domains or hash"),
-    outputs=gr.Text(label="Entity information as an IoC"),
-    title="IoC information extractor associated to particular entities",
-    description=(
-        "If information as an Indicator of Compromise (IoC) exists "
-        "for the input url, domain or hash, it retrieves it"
-    ),
-    theme="default",
-    examples=["advertipros.com", "dev.couplesparks.com"],
-    example_labels=["👾 IoC 1", "👾 IoC 2"],
-)

tdagent/tools/get_url_content.py

@@ -1,124 +1,65 @@
+import enum
 from collections.abc import Sequence
-from typing import Literal, Optional, Dict, Any
-import json
 
 import gradio as gr
 import requests
 
-from tdagent.constants import HttpContentType
 
-# Define valid HTTP methods
-HttpMethod = Literal["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD"]
+class HttpContentType(str, enum.Enum):
+    """Http content type values."""
 
-def make_http_request(
+    HTML = "text/html"
+    JSON = "application/json"
+
+
+def get_url_http_content(
     url: str,
-    method: HttpMethod = "GET",
-    content_type: str = "",
-    body: str = "",
-    timeout: float = 30,
-    custom_headers: str = ""
+    content_type: Sequence[HttpContentType] | None = None,
+    timeout: int = 30,
 ) -> tuple[str, str]:
-    """Make an HTTP request to a URL with specified method and parameters.
+    """Get the content of a URL using an HTTP GET request.
 
     Args:
-        url: The URL to make the request to.
-        method: HTTP method to use (GET, POST, PUT, DELETE, PATCH, HEAD).
-        content_type: Comma-separated string of content types.
-        body: Request body for methods that support it (POST, PUT, PATCH).
+        url: The URL to fetch the content from.
+        content_type: If given it should contain the expected
+            content types in the response body. The server may
+            not honor the requested content types.
         timeout: Request timeout in seconds. Defaults to 30.
-        custom_headers: JSON string of additional headers.
 
     Returns:
-        A pair of strings (content, error_message).
+        A pair of strings (content, error_message). If there is an
+        error getting content from the URL the `content` will be
+        empty and `error_message` will, usually, contain the error
+        cause. Otherwise, `error_message` will be empty and the
+        content will be filled with data fetched from the URL.
     """
-    # Initialize headers dictionary
     headers = {}
 
-    # Parse content type
     if content_type:
-        headers["Accept"] = content_type
-
-    # Parse custom headers
-    if custom_headers:
-        try:
-            custom_headers_dict = json.loads(custom_headers)
-            headers.update(custom_headers_dict)
-        except json.JSONDecodeError:
-            return "", "Invalid JSON format in custom headers"
-
-    # Prepare request parameters
-    request_params: Dict[str, Any] = {
-        "url": url,
-        "headers": headers,
-        "timeout": timeout,
-    }
-
-    # Add body for methods that support it
-    if method in ["POST", "PUT", "PATCH"] and body:
-        request_params["data"] = body
-
-    try:
-        response = requests.request(method, **request_params)
-    except requests.exceptions.MissingSchema as err:
-        return "", str(err)
-    except requests.exceptions.RequestException as err:
-        return "", str(err)
+        headers["Accept"] = ",".join(content_type)
+    response = requests.get(
+        url,
+        headers=headers,
+        timeout=timeout,
+    )
 
     try:
         response.raise_for_status()
     except requests.HTTPError as err:
         return "", str(err)
 
-    # For HEAD requests, return headers as content
-    if method == "HEAD":
-        return str(dict(response.headers)), ""
-
     return response.text, ""
 
-# Create the Gradio interface
-gr_make_http_request = gr.Interface(
-    fn=make_http_request,
-    inputs=[
-        gr.Textbox(label="URL"),
-        gr.Dropdown(
-            choices=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD"],
-            label="HTTP Method",
-            value="GET"
-        ),
-        gr.Textbox(
-            label="Content Type",
-            placeholder="text/html,application/json"
-        ),
-        gr.Textbox(
-            label="Request Body (for POST/PUT/PATCH)",
-            lines=3,
-            placeholder='{"key": "value"}'
-        ),
-        gr.Number(
-            label="Timeout (seconds)",
-            value=30,
-            minimum=1,
-            maximum=300
-        ),
-        gr.Textbox(
-            label="Custom Headers (JSON format)",
-            placeholder='{"Authorization": "Bearer token"}'
-        )
-    ],
-    outputs=gr.Text(label="Response"),
-    title="Make HTTP Requests",
+
+gr_get_url_http_content = gr.Interface(
+    fn=get_url_http_content,
+    inputs=["text", "text"],
+    outputs="text",
+    title="Get the content of a URL using an HTTP GET request.",
     description=(
-        "Make HTTP requests with different methods and parameters. "
-        "Supports GET, POST, PUT, DELETE, PATCH, and HEAD methods. "
-        "For POST, PUT, and PATCH requests, you can include a request body. "
-        "Custom headers can be added in JSON format. "
-        "Be cautious when accessing unknown URLs."
+        "Get the content of a URL in one of the specified content types."
+        " The server may not honor the content type and if it fails the"
+        " reason should also be returned with the corresponding HTTP"
+        " error code."
     ),
-    examples=[
-        ["https://google.com", "GET", "text/html", "", 30, ""],
-        ["https://api.example.com/data", "POST", "application/json", '{"key": "value"}', 30, '{"Authorization": "Bearer token"}'],
-    ],
 )
-
-if __name__ == "__main__":
-    gr_make_http_request.launch()

tdagent/tools/internal_company_user_search.py

@@ -1,6 +1,5 @@
 import gradio as gr
 
-
 # Fake user database
 users_db = {
     "jsmith": {
@@ -10,7 +9,7 @@ users_db = {
         "user_id": "US789456",
         "jobtitle": "Software Engineer",
         "department": "Engineering",
-        "country": "United States",
+        "country": "United States"
     },
     "mhacker": {
         "username": "mhacker",
@@ -19,19 +18,16 @@ users_db = {
         "user_id": "US123789",
         "jobtitle": "Security Specialist",
         "department": "Pentests",
-        "country": "Germany",
-    },
+        "country": "Germany"
+    }
 }
 
 
-def lookup_user(username: str) -> str:
-    """Function to lookup user information.
-
+def lookup_user(username):
+    """
+    Function to lookup user information.
     Company User Lookup System. Enter a username to get user details.
-
-    Returns:
-        A formatted string with user details if found, otherwise an
-        error message.
+    Returns a formatted string with user details if found, otherwise returns error message
     """
     if username in users_db:
         user = users_db[username]
@@ -43,8 +39,8 @@ User ID: {user['user_id']}
 Job Title: {user['jobtitle']}
 Department: {user['department']}
 Country: {user['country']}"""
-
-    return """User Not Found
+    else:
+        return """User Not Found
 Username: Not found
 Email: N/A
 Name: N/A
@@ -60,6 +56,6 @@ gr_internal_company = gr.Interface(
     inputs=["text"],
     outputs=["text"],
     title="Company User Lookup System",
-    description="Company User Lookup System.",
-    theme="default",
+    description="Company User Lookup System. Enter a username to get user details",
+    theme="default"
 )

tdagent/tools/lookup_company_cloud_account_information.py

@@ -1,6 +1,5 @@
 import gradio as gr
 
-
 # Fake cloud accounts database
 cloud_accounts = {
     ("AWS", "123456789012"): {
@@ -9,7 +8,7 @@ cloud_accounts = {
         "cloud_account_name": "Production-Main",
         "owner_user_id": "AWS001",
         "owner_email": "[email protected]",
-        "deployed_applications": ["ERP System", "Customer Portal", "Data Lake"],
+        "deployed_applications": ["ERP System", "Customer Portal", "Data Lake"]
     },
     ("AWS", "098765432109"): {
         "public_cloud_provider": "AWS",
@@ -17,7 +16,7 @@ cloud_accounts = {
         "cloud_account_name": "Development-Team1",
         "owner_user_id": "AWS002",
         "owner_email": "[email protected]",
-        "deployed_applications": ["Test Environment", "CI/CD Pipeline"],
+        "deployed_applications": ["Test Environment", "CI/CD Pipeline"]
     },
     ("Azure", "sub-abc-123-def-456"): {
         "public_cloud_provider": "Azure",
@@ -25,7 +24,7 @@ cloud_accounts = {
         "cloud_account_name": "Enterprise-Solutions",
         "owner_user_id": "AZ001",
         "owner_email": "[email protected]",
-        "deployed_applications": ["Microsoft 365 Integration", "Azure AD Connect"],
+        "deployed_applications": ["Microsoft 365 Integration", "Azure AD Connect"]
     },
     ("Azure", "sub-xyz-789-uvw-321"): {
         "public_cloud_provider": "Azure",
@@ -33,7 +32,7 @@ cloud_accounts = {
         "cloud_account_name": "Research-Division",
         "owner_user_id": "AZ002",
         "owner_email": "[email protected]",
-        "deployed_applications": ["ML Platform", "Research Portal"],
+        "deployed_applications": ["ML Platform", "Research Portal"]
     },
     ("GCP", "project-id-123456"): {
         "public_cloud_provider": "GCP",
@@ -41,7 +40,7 @@ cloud_accounts = {
         "cloud_account_name": "Analytics-Platform",
         "owner_user_id": "GCP001",
         "owner_email": "[email protected]",
-        "deployed_applications": ["BigQuery Data Warehouse", "Kubernetes Cluster"],
+        "deployed_applications": ["BigQuery Data Warehouse", "Kubernetes Cluster"]
     },
     ("GCP", "project-id-789012"): {
         "public_cloud_provider": "GCP",
@@ -49,49 +48,45 @@ cloud_accounts = {
         "cloud_account_name": "ML-Operations",
         "owner_user_id": "GCP002",
         "owner_email": "[email protected]",
-        "deployed_applications": ["TensorFlow Training", "Model Serving Platform"],
-    },
+        "deployed_applications": ["TensorFlow Training", "Model Serving Platform"]
+    }
 }
 
 
-def lookup_cloud_account(public_cloud_provider: str, account_id: str) -> dict[str, str]:
-    """Function to lookup cloud account information.
-
-    Returns a formatted string with account details if
-    found, otherwise returns error message.
+def lookup_cloud_account(public_cloud_provider, account_id):
+    """
+    Function to lookup cloud account information
+    Returns a formatted string with account details if found, otherwise returns error message
     """
     account_key = (public_cloud_provider, account_id)
 
     if account_key in cloud_accounts:
         account = cloud_accounts[account_key]
-        return {
-            "public_cloud_provider": f"{account['public_cloud_provider']}",
-            "account_id": f"{account['account_id']}",
-            "cloud_account_name": f"{account['cloud_account_name']}",
-            "owner_user_id": f"{account['owner_user_id']}",
-            "owner_email": f"{account['owner_email']}",
-            "deployed_applications": f"{account['deployed_applications']!s}",
-        }
-
-    return {
-        "error": "Account not found",
-        "public_cloud_provider": f"{public_cloud_provider}",
-        "account_id": f"{account_id}",
-        "message": "No cloud account information found",
-    }
+        return f"""{{
+    "public_cloud_provider": "{account['public_cloud_provider']}",
+    "account_id": "{account['account_id']}",
+    "cloud_account_name": "{account['cloud_account_name']}",
+    "owner_user_id": "{account['owner_user_id']}",
+    "owner_email": "{account['owner_email']}",
+    "deployed_applications": {str(account['deployed_applications'])}
+}}"""
+    else:
+        return f"""{{
+    "error": "Account not found",
+    "public_cloud_provider": "{public_cloud_provider}",
+    "account_id": "{account_id}",
+    "message": "No cloud account information found"
+}}"""
 
 
 # Create Gradio Interface
 gr_lookup_company_cloud_account_information = gr.Interface(
     fn=lookup_cloud_account,
-    inputs=[
-        gr.Dropdown(
-            choices=["AWS", "Azure", "GCP"],
-            label="Cloud Provider",
-            info="Select the cloud provider",
-        ),
-        "text",
-    ],
+    inputs=[gr.Dropdown(
+        choices=["AWS", "Azure", "GCP"],
+        label="Cloud Provider",
+        info="Select the cloud provider"
+    ), "text"],
     outputs="text",
     title="Company Cloud Account Lookup System",
     description="""
@@ -102,5 +97,5 @@ gr_lookup_company_cloud_account_information = gr.Interface(
     Azure: sub-abc-123-def-456, sub-xyz-789-uvw-321
     GCP: project-id-123456, project-id-789012
     """,
-    theme="default",
+    theme="default"
 )

tdagent/tools/query_abuse_ip_db.py

@@ -1,19 +1,16 @@
-from __future__ import annotations
-
 import os
-from dataclasses import dataclass
-from datetime import datetime
-from typing import Any
 
-import gradio as gr
 import requests
+from datetime import datetime
+from dataclasses import dataclass
+from typing import List, Optional, Dict, Any, Tuple, Union
+import gradio as gr
 
 
 # API docs: https://docs.abuseipdb.com/#check-endpoint
 
-
 @dataclass
-class AbuseReport:  # noqa: D101
+class AbuseReport:
     date: str
     categories: str
     comment: str
@@ -21,7 +18,7 @@ class AbuseReport:  # noqa: D101
 
 
 @dataclass
-class IPCheckResult:  # noqa: D101
+class IPCheckResult:
     ip_address: str
     abuse_confidence_score: int
     total_reports: int
@@ -29,10 +26,10 @@ class IPCheckResult:  # noqa: D101
     domain: str
     isp: str
     last_reported: str
-    reports: list[AbuseReport]
+    reports: List[AbuseReport]
 
     def format_summary(self) -> str:
-        """Format a summary of the IP check result."""
+        """Format a summary of the IP check result"""
         return f"""
         IP Address: {self.ip_address}
         Abuse Confidence Score: {self.abuse_confidence_score}%
@@ -44,8 +41,9 @@ class IPCheckResult:  # noqa: D101
         """
 
 
-def check_ip(ip_address: str, api_key: str, days: str = "30") -> dict[str, Any]:
-    """Query the AbuseIPDB API to check if an IP address has been reported.
+def check_ip(ip_address: str, api_key: str, days: str = "30") -> Dict[str, Any]:
+    """
+    Query the AbuseIPDB API to check if an IP address has been reported.
 
     Args:
         ip_address: The IP address to check
@@ -55,33 +53,30 @@ def check_ip(ip_address: str, api_key: str, days: str = "30") -> dict[str, Any]:
     Returns:
         API response data as a dictionary
     """
-    url = "https://api.abuseipdb.com/api/v2/check"
+    url = 'https://api.abuseipdb.com/api/v2/check'
 
-    headers = {"Accept": "application/json", "Key": api_key}
+    headers = {
+        'Accept': 'application/json',
+        'Key': api_key
+    }
 
     params = {
-        "ipAddress": ip_address,
-        "maxAgeInDays": days,
-        "verbose": str(True),
+        'ipAddress': ip_address,
+        'maxAgeInDays': days,
+        'verbose': True
     }
 
     try:
-        response = requests.get(
-            url,
-            headers=headers,
-            params=params,
-            timeout=30,
-        )
+        response = requests.get(url, headers=headers, params=params)
         response.raise_for_status()  # Raise exception for HTTP errors
         return response.json()
     except requests.exceptions.RequestException as e:
         return {"error": str(e)}
 
 
-def parse_response(
-    response: dict[str, Any],
-) -> tuple[IPCheckResult | None, str]:
-    """Parse the API response into a dataclass.
+def parse_response(response: Dict[str, Any]) -> Tuple[Optional[IPCheckResult], Optional[str]]:
+    """
+    Parse the API response into a dataclass
 
     Args:
         response: The API response dictionary
@@ -98,21 +93,18 @@ def parse_response(
 
     # Create a list of AbuseReport objects
     reports = []
-    if data.get("reports"):
+    if "reports" in data and data["reports"]:
         for report in data["reports"]:
-            reported_at = datetime.fromisoformat(
-                report["reportedAt"].replace("Z", "+00:00"),
-            ).strftime("%Y-%m-%d %H:%M:%S")
+            reported_at = datetime.fromisoformat(report["reportedAt"].replace("Z", "+00:00")).strftime(
+                "%Y-%m-%d %H:%M:%S")
             categories = ", ".join([str(cat) for cat in report.get("categories", [])])
 
-            reports.append(
-                AbuseReport(
-                    date=reported_at,
-                    categories=categories,
-                    comment=report.get("comment", ""),
-                    reporter=str(report.get("reporterId", "Anonymous")),
-                ),
-            )
+            reports.append(AbuseReport(
+                date=reported_at,
+                categories=categories,
+                comment=report.get("comment", ""),
+                reporter=str(report.get("reporterId", "Anonymous"))
+            ))
 
     # Create the main result object
     result = IPCheckResult(
@@ -123,14 +115,15 @@ def parse_response(
         domain=data.get("domain", "N/A"),
         isp=data.get("isp", "N/A"),
         last_reported=data.get("lastReportedAt", "Never"),
-        reports=reports,
+        reports=reports
     )
 
-    return result, ""
+    return result, None
 
 
 def query_abuseipdb(ip_address: str, days: int = 30) -> str:
-    """Query AbuseIP to find if an IP has been reported for abusive behavior.
+    """
+    Main function to query AbuseIPDB and format the response for Gradio
 
     Args:
         ip_address: The IP address to check
@@ -148,21 +141,16 @@ def query_abuseipdb(ip_address: str, days: int = 30) -> str:
     response = check_ip(ip_address, api_key, str(days))
     result, error = parse_response(response)
 
-    if result:
-        return result.format_summary()
+    if error:
+        return error
 
-    return error
+    return result.format_summary()
 
 
 gr_query_abuseipdb = gr.Interface(
     fn=query_abuseipdb,
-    inputs=gr.Textbox(label="ip"),
-    outputs=gr.Text(label="Report on abusive behaviour"),
+    inputs=["text"],
+    outputs="text",
     title="AbuseIPDB IP Checker",
-    description=(
-        "Check if an IP address has been reported for abusive behavior"
-        " using AbuseIP DB API"
-    ),
-    examples=["5.252.155.14", "77.239.99.248"],
-    example_labels=["👾 Malicious IP 1", "👾 Malicious IP 2"],
+    description="Check if an IP address has been reported for abusive behavior using AbuseIP DB API",
 )

tdagent/tools/rdap.py

@@ -1,110 +0,0 @@
-import enum
-
-import cachetools
-import gradio as gr
-import requests
-import whois
-
-from tdagent.constants import HttpContentType
-
-
-# one of domain, ip, autnum, entity etc
-_RDAP_URL_TEMPLATE = r"https://rdap.org/{rdap_type}/{rdap_object}"
-_CACHE_MAX_SIZE = 4096
-_CACHE_TTL_SECONDS = 3600
-
-
-class RdapTypes(str, enum.Enum):
-    """RDAP object types."""
-
-    DOMAIN = "domain"
-    IP = "ip"
-    AUTNUM = "autnum"
-    ENTITY = "entity"
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def query_rdap(  # noqa: PLR0911
-    url_or_ip: str,
-    timeout: int = 30,
-) -> dict[str, str | int | float]:
-    """Query RDAP to get information about Internet resources.
-
-    The Registration Data Access Protocol (RDAP) is the successor to WHOIS.
-    Like WHOIS, RDAP provides access to information about Internet resources
-    (domain names, autonomous system numbers, and IP addresses).
-
-    Args:
-        url_or_ip: URL, domain or IP to query for RDAP information.
-        timeout: Request timeout in seconds. Defaults to 30.
-
-    Returns:
-        A JSON formatted string with RDAP information. In there is
-        an error, the JSON will contain the key "error" with an
-        error message.
-    """
-    rdap_type = RdapTypes.DOMAIN
-    rdap_object = url_or_ip
-    if whois.IPV4_OR_V6.match(url_or_ip):
-        rdap_type = RdapTypes.IP
-    else:
-        rdap_object = whois.extract_domain(url_or_ip)
-
-    query_url = _RDAP_URL_TEMPLATE.format(rdap_type=rdap_type, rdap_object=rdap_object)
-    response = requests.get(
-        query_url,
-        timeout=timeout,
-        headers={"Accept": HttpContentType.JSON},
-    )
-
-    try:
-        response.raise_for_status()
-    except requests.HTTPError as err:
-        if err.response.status_code == 302:
-            if "Location" in err.response.headers:
-                return {
-                    "message": "Follow the location to find RDAP information",
-                    "location": err.response.headers["Location"],
-                }
-            return {
-                "error": (
-                    "Information not found in RDAP.org but it knows of"
-                    " a service which is authoritative for the requested resource."
-                ),
-            }
-        if err.response.status_code == 400:
-            return {
-                "error": (
-                    "Invalid request (malformed path, unsupported object "
-                    " type, invalid IP address, etc)"
-                ),
-            }
-        if err.response.status_code == 403:
-            return {
-                "error": "You've been blocked due to abuse or other misbehavior",
-            }
-        if err.response.status_code == 404:
-            return {
-                "error": (
-                    "RDAP.org doesn't know of an RDAP service which is"
-                    " authoritative for the requested resource. RDAP.org"
-                    " only knows about servers that are registered with IANA"
-                ),
-            }
-        return {
-            "error": str(err),
-        }
-
-    return response.json()
-
-
-gr_query_rdap = gr.Interface(
-    fn=query_rdap,
-    inputs=gr.Textbox(label="url or ip"),
-    outputs=gr.JSON(label="Report from RDAP"),
-    title="Get RDAP information for a given URL.",
-    description="Query a RDAP database to gather information about a url or domain.",
-    examples=["8.8.8.8", "pastebin.com"],
-)

tdagent/tools/retrieve_from_mitre_attack.py

@@ -1,59 +0,0 @@
-from typing import Any
-
-import cachetools
-import gradio as gr
-from attackcti import attack_client
-
-
-_CACHE_MAX_SIZE = 4096
-_CACHE_TTL_SECONDS = 3600
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def get_stix_object_of_attack_id(
-    attack_id: str,
-    object_type: str = "attack-pattern",
-) -> dict[str, Any]:
-    """Retrieves a STIX object identified by an ATT&CK ID in all ATT&CK matrices.
-
-    Args:
-        attack_id (str): The ATT&CK ID (e.g., 'T1234') of the STIX object to retrieve.
-        object_type (str): The type of STIX object to retrieve, such as
-            'attack-pattern', 'course-of-action', 'intrusion-set',
-            'malware', 'tool', or 'x-mitre-data-component'. Default is 'attack-pattern'
-
-    Returns:
-        A list containing the matched STIX object, either in its raw STIX format
-        or as a custom dictionary following the structure defined by the relevant
-        Pydantic model, depending on the 'stix_format' flag.
-    """
-    try:
-        lift = attack_client()
-        return lift.get_object_by_attack_id(
-            object_type=object_type.strip(),
-            attack_id=attack_id.strip(),
-            stix_format=False,
-        )[0]
-    except Exception as e:  # noqa: BLE001
-        return {"Exception": str(e)}
-
-
-gr_get_stix_of_attack_id = gr.Interface(
-    fn=get_stix_object_of_attack_id,
-    inputs=[
-        gr.Textbox(label="Mitre technique ID"),
-        gr.Textbox(label="Mitre object type"),
-    ],
-    outputs=gr.JSON(label="Mitre report"),
-    title="MITRE ATT&CK STIX information",
-    description=(
-        "Retrieves a specific STIX object identified by an ATT&CK ID across all ATT&CK"
-        " matrices"
-    ),
-    examples=[
-        ["T1568.002", "attack-pattern"],
-        ["M1042", "course-of-action"],
-    ],
-)

tdagent/tools/send_email.py

@@ -1,48 +1,53 @@
+import gradio as gr
 import datetime
 import random
 
-import gradio as gr
-
 
-def send_email(recipient: str, subject: str, message: str) -> str:
-    """Simulates sending an email from [email protected].
+def send_email(recipient, subject, message):
+    """
+    Simulates sending an email from [email protected] and prints it to the console in a well-formatted way.
 
     This function takes email details, formats them into a standard email structure
     with headers and body, prints the formatted email to the console, and returns
     a success message. The sender is always set to [email protected].
 
-    Args:
-        recipient: The email address of the recipient (To field)
-        subject: The subject line of the email
-        message: The main body content of the email
+    Parameters:
+    -----------
+    recipient : str
+        The email address of the recipient (To field)
+    subject : str
+        The subject line of the email
+    message : str
+        The main body content of the email
 
     Returns:
+    --------
+    str
         A success message indicating that the email was sent, including
         the recipient address and the current time
 
     Example:
-        >>> send_email("[email protected]", "Security Alert", "Please update your password.")
-        ------ EMAIL SENT ------
-        From: [email protected]
-        To: jane@example.com
-        Subject: Security Alert
-        Date: Wed, 04 Jun 2025 16:40:58 +0000
-        Message-ID: <123456.7890123456@mail-server>
-
-        Please update your password.
-        ------------------------
-        'Email successfully sent to [email protected] at 16:40:58'
-    """  # noqa: E501
+    --------
+    >>> send_email("[email protected]", "Security Alert", "Please update your password.")
+    ------ EMAIL SENT ------
+    From: cert@company.com
+    To: [email protected]
+    Subject: Security Alert
+    Date: Wed, 04 Jun 2025 16:40:58 +0000
+    Message-ID: <123456.7890123456@mail-server>
+
+    Please update your password.
+    ------------------------
+    'Email successfully sent to [email protected] at 16:40:58'
+    """
     # Fixed sender email
     sender = "[email protected]"
 
     # Generate a random message ID
-    message_id = f"<{random.randint(100000, 999999)}.{random.randint(1000000000, 9999999999)}@mail-server>"  # noqa: E501, S311
+    message_id = f"<{random.randint(100000, 999999)}.{random.randint(1000000000, 9999999999)}@mail-server>"
 
     # Get current timestamp
-    timestamp = datetime.datetime.now(datetime.timezone.utc).strftime(
-        "%a, %d %b %Y %H:%M:%S +0000",
-    )
+    timestamp = datetime.datetime.now().strftime("%a, %d %b %Y %H:%M:%S +0000")
 
     # Format the email
     email_format = f"""
@@ -58,13 +63,10 @@ Message-ID: {message_id}
 """
 
     # Print the email to console
-    print(email_format)  # noqa: T201
+    print(email_format)
 
     # Return a success message
-    return (
-        f"Email successfully sent from {sender} to {recipient} at "
-        + datetime.datetime.now(datetime.timezone.utc).strftime("%H:%M:%S")
-    )
+    return f"Email successfully sent from {sender} to {recipient} at {datetime.datetime.now().strftime('%H:%M:%S')}"
 
 
 # Create Gradio Interface
@@ -73,5 +75,5 @@ gr_send_email = gr.Interface(
     inputs=["text", "text", "text"],
     outputs="text",
     title="Email Sender Simulator",
-    description="This tool simulates sending an email.",
+    description="This tool simulates sending an email by formatting and printing it to the console."
 )

tdagent/tools/virus_total.py

@@ -1,26 +1,21 @@
-import os
-from datetime import datetime, timezone
-
-import cachetools
 import gradio as gr
 import vt
-
-
-_CACHE_MAX_SIZE = 4096
-_CACHE_TTL_SECONDS = 3600
+import os
+from datetime import datetime
+from functools import lru_cache
+from typing import Optional
+import time
+import asyncio
 
 # Get API key from environment variable
-API_KEY = os.getenv("VT_API_KEY")
-
+API_KEY = os.getenv('VT_API_KEY')
 
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def get_virus_total_url_info(url: str) -> str:
-    """Get URL Info from VirusTotal URL Scanner. Scan URL is not available."""
-    if not API_KEY:
-        return "Error: Virus total API key not configured."
 
+@lru_cache(maxsize=100)
+def get_url_info_cached(url: str, timestamp: Optional[int] = None) -> str:
+    """
+    Get URL Info from VirusTotal URL Scanner. Scan URL is not available
+    """
     try:
         # Create a new client for each request (thread-safe)
         with vt.Client(API_KEY) as client:
@@ -38,15 +33,12 @@ def get_virus_total_url_info(url: str) -> str:
             if last_analysis_date:
                 # Convert to datetime if it's a timestamp
                 if isinstance(last_analysis_date, (int, float)):
-                    last_analysis_date = datetime.fromtimestamp(
-                        last_analysis_date,
-                        timezone.utc,
-                    )
-                date_str = last_analysis_date.strftime("%Y-%m-%d %H:%M:%S UTC")
+                    last_analysis_date = datetime.utcfromtimestamp(last_analysis_date)
+                date_str = last_analysis_date.strftime('%Y-%m-%d %H:%M:%S UTC')
             else:
                 date_str = "Not available"
 
-            return f"""
+            result = f"""
 URL: {url}
 Last Analysis Date: {date_str}
 
@@ -61,18 +53,31 @@ Reputation Score: {url_analysis.reputation}
 Times Submitted: {url_analysis.times_submitted}
 
 Cache Status: Hit
-            """.strip()
+            """
+
+            return result
+
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+
+def get_url_info(url: str) -> str:
+    """
+    Wrapper function to handle the cached URL info retrieval
+    """
+    # Clean the URL to ensure consistent caching
+    url = url.strip().lower()
+
+    # Use current timestamp rounded to nearest hour to maintain cache for an hour
+    timestamp = int(time.time()) // 3600
 
-    except Exception as err:  # noqa: BLE001
-        return f"Error: {err}"
+    return get_url_info_cached(url, timestamp)
 
 
-gr_virus_total_url_info = gr.Interface(
-    fn=get_virus_total_url_info,
-    inputs=gr.Textbox(label="url"),
-    outputs=gr.Text(label="VirusTotal report"),
+gr_virus_total = gr.Interface(
+    fn=get_url_info,
+    inputs=["text"],
+    outputs="text",
     title="VirusTotal URL Scanner",
     description="Get URL Info from VirusTotal URL Scanner. Scan URL is not available",
-    examples=["https://advertipros.com//?u=script", "https://google.com"],
-    example_labels=["👾 Malicious URL", "🧑‍💻 Benign URL"],
 )

tdagent/tools/whois.py

@@ -1,49 +0,0 @@
-import json
-import shutil
-
-import cachetools
-import gradio as gr
-import whois
-
-from tdagent.utils.json_utils import TDAgentJsonEncoder
-
-
-_WHOIS_BINARY = "whois"
-_CACHE_MAX_SIZE = 4096
-_CACHE_TTL_SECONDS = 3600
-
-
-@cachetools.cached(
-    cache=cachetools.TTLCache(maxsize=_CACHE_MAX_SIZE, ttl=_CACHE_TTL_SECONDS),
-)
-def query_whois(url: str) -> str:
-    """Query a WHOIS database to gather information about a url or domain.
-
-    WHOIS information includes: domain names, IP address blocks and autonomous
-    systems, but it is also used for a wider range of other information.
-
-    Args:
-        url: URL to query for WHOIS information.
-
-    Returns:
-        A JSON formatted string with the gathered information
-    """
-    try:
-        whois_result = whois.whois(
-            url,
-            command=shutil.which(_WHOIS_BINARY) is not None,
-            executable=_WHOIS_BINARY,
-        )
-    except whois.parser.PywhoisError as err:
-        return json.dumps({"error": str(err)})
-
-    return json.dumps(whois_result, cls=TDAgentJsonEncoder)
-
-
-gr_query_whois = gr.Interface(
-    fn=query_whois,
-    inputs=["text"],
-    outputs="text",
-    title="Get WHOIS information for a given URL.",
-    description="Query a WHOIS database to gather information about a url or domain.",
-)

tdagent/utils/json_utils.py

@@ -1,14 +0,0 @@
-import datetime
-import json
-
-
-class TDAgentJsonEncoder(json.JSONEncoder):
-    """Extend JSON encoder with known types."""
-
-    def default(self, o: object) -> object:  # noqa: D102
-        if isinstance(o, datetime.datetime):
-            return {"__type__": "datetime", "value": o.isoformat()}
-        if isinstance(o, datetime.date):
-            return {"__type__": "date", "value": o.isoformat()}
-
-        return super().default(o)