README_API.md

📊 Dense Captioning Platform API Documentation

Overview

The Dense Captioning Platform provides comprehensive chart analysis through a Gradio-based API. It can classify chart types, detect chart elements, and segment data points from uploaded images.

API Access

Base URL: https://hanszhu-dense-captioning-platform.hf.space

API Type: Gradio Client API (not RESTful)

Installation

Prerequisites

pip install gradio-client

Quick Start

Python Client (Recommended)

from gradio_client import Client, handle_file

# Initialize client with direct URL
client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Analyze a chart image using file path
result = client.predict(
    image=handle_file('path/to/your/chart.png'),
    fn_index=0
)

print(result)

Using a URL

from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Use a publicly accessible image URL
result = client.predict(
    image=handle_file("https://example.com/chart.png"),
    fn_index=0
)

print(result)

Input Parameters

Parameter	Type	Required	Description
image	File/URL	Yes	Chart image to analyze (PNG, JPG, JPEG supported)

Important Notes

✅ Working Approach

• Use fn_index=0 instead of api_name="/predict"
• Use direct URL "https://hanszhu-dense-captioning-platform.hf.space"
• Always use handle_file() for both local files and URLs
• This is a Gradio Client API, not a RESTful API

❌ What Doesn't Work

• Direct HTTP POST requests to /predict
• Using api_name="/predict" with this setup
• Using Client("hanszhu/Dense-Captioning-Platform") (use direct URL instead)

Output Format

The API returns a JSON object with the following structure:

{
    "chart_type_id": 4,
    "chart_type_label": "Bar plot",
    "element_result": {
        "bboxes": [...],
        "segments": [...]
    },
    "datapoint_result": {
        "bboxes": [...],
        "segments": [...]
    },
    "status": "Full analysis completed",
    "processing_time": 2.345
}

Output Fields

Field	Type	Description
chart_type_id	int	Numeric identifier for chart type (0-27)
chart_type_label	string	Human-readable chart type name
element_result	object/string	Detected chart elements (titles, axes, legends, etc.)
datapoint_result	object/string	Segmented data points and regions
status	string	Processing status message
processing_time	float	Time taken for analysis in seconds

Supported Chart Types

The platform can classify 28 different chart types:

ID	Chart Type	ID	Chart Type
0	Line graph	14	Histogram
1	Natural image	15	Box plot
2	Table	16	Vector plot
3	3D object	17	Pie chart
4	Bar plot	18	Surface plot
5	Scatter plot	19	Algorithm
6	Medical image	20	Contour plot
7	Sketch	21	Tree diagram
8	Geographic map	22	Bubble chart
9	Flow chart	23	Polar plot
10	Heat map	24	Area chart
11	Mask	25	Pareto chart
12	Block diagram	26	Radar chart
13	Venn diagram	27	Confusion matrix

Chart Elements Detected

The element detection model identifies:

• Titles & Labels: Chart title, subtitle, axis labels
• Axes: X-axis, Y-axis, tick labels
• Legend: Legend title, legend items, legend text
• Data Elements: Data points, data lines, data bars, data areas
• Structural Elements: Grid lines, plot areas

Error Handling

The API returns error messages in the response fields when issues occur:

{
    "chart_type_id": "Error: Model not available",
    "chart_type_label": "Error: Model not available",
    "element_result": "Error: Invalid image format",
    "datapoint_result": "Error: Processing failed",
    "status": "Error in chart classification",
    "processing_time": 0.0
}

Rate Limits

• Free Tier: Limited requests per hour
• Processing Time: Typically 2-5 seconds per image
• Image Size: Recommended max 10MB

Complete Working Example

Here's a complete example that demonstrates all the working patterns:

from gradio_client import Client, handle_file
import json

def analyze_chart(image_path_or_url):
    """
    Analyze a chart image using the Dense Captioning Platform API
    
    Args:
        image_path_or_url (str): Path to local image file or URL to image
    
    Returns:
        dict: Analysis results with chart type, elements, and data points
    """
    try:
        # Initialize client with direct URL
        client = Client("https://hanszhu-dense-captioning-platform.hf.space")
        
        # Make prediction using the working approach
        result = client.predict(
            image=handle_file(image_path_or_url),
            fn_index=0
        )
        
        return result
        
    except Exception as e:
        return {
            "error": f"API call failed: {str(e)}",
            "status": "Error",
            "processing_time": 0.0
        }

# Example usage
if __name__ == "__main__":
    # Test with a local file
    local_result = analyze_chart("path/to/your/chart.png")
    print("Local file result:", json.dumps(local_result, indent=2))
    
    # Test with a URL
    url_result = analyze_chart("https://example.com/chart.png")
    print("URL result:", json.dumps(url_result, indent=2))

Examples

Example 1: Bar Chart Analysis

from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Analyze a bar chart
result = client.predict(
    image=handle_file('bar_chart.png'),
    fn_index=0
)

print(f"Chart Type: {result['chart_type_label']}")
print(f"Processing Time: {result['processing_time']}s")

Example 2: Batch Processing

from gradio_client import Client, handle_file
import os

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Process multiple charts
chart_files = ['chart1.png', 'chart2.png', 'chart3.png']
results = []

for chart_file in chart_files:
    if os.path.exists(chart_file):
        result = client.predict(
            image=handle_file(chart_file),
            fn_index=0
        )
        results.append(result)
        print(f"Processed {chart_file}: {result['chart_type_label']}")

Example 3: Test with Public Image

from gradio_client import Client, handle_file

client = Client("https://hanszhu-dense-captioning-platform.hf.space")

# Test with a public image URL
result = client.predict(
    image=handle_file("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png"),
    fn_index=0
)

print("✅ API Test Successful!")
print(f"Chart Type: {result['chart_type_label']}")
print(f"Status: {result['status']}")

Troubleshooting

Common Issues

1. "Model not available": The models are still loading, wait a moment and retry
2. "Invalid image format": Ensure the image is in PNG, JPG, or JPEG format
3. "Processing failed": The image might be corrupted or too large
4. "Expecting value: line 1 column 1": Use fn_index=0 instead of api_name="/predict"
5. "Cannot find a function with api_name": Use direct URL and fn_index=0

Best Practices

1. Image Quality: Use clear, high-resolution images for best results
2. Format: PNG or JPG formats work best
3. Size: Keep images under 10MB for faster processing
4. Client Setup: Always use direct URL and fn_index=0
5. File Handling: Always use handle_file() for both local files and URLs
6. Retry Logic: Implement retry logic for failed requests

Quick Test

To verify the API is working, run this test:

from gradio_client import Client, handle_file

try:
    client = Client("https://hanszhu-dense-captioning-platform.hf.space")
    result = client.predict(
        image=handle_file("https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png"),
        fn_index=0
    )
    print("✅ API is working!")
    print(f"Chart Type: {result['chart_type_label']}")
except Exception as e:
    print(f"❌ API test failed: {e}")

Support

For issues or questions:

• Check the Hugging Face Space
• Review the error messages in the API response
• Ensure your image format and size are within limits