Add application file

README.md

@@ -1,10 +1,144 @@
----
-title: Llm Apiku
-emoji: 🏆
-colorFrom: pink
-colorTo: gray
-sdk: docker
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Panduan Penggunaan Docker untuk Hugging Face Model API
+
+## Struktur File
+```
+your-project/
+├── Dockerfile
+├── docker-compose.yml
+├── requirements.txt
+├── app.py
+├── cache/           # Folder untuk cache model
+└── README.md
+```
+
+## Cara Menjalankan
+
+### 1. Build dan Jalankan dengan Docker Compose
+```bash
+# Clone atau buat direktori project
+mkdir gema-model-api
+cd gema-model-api
+
+# Copy semua file yang telah dibuat
+# Kemudian jalankan:
+docker-compose up --build
+```
+
+### 2. Atau Build Manual
+```bash
+# Build image
+docker build -t gema-model-api .
+
+# Run container
+docker run -p 8000:8000 -v $(pwd)/cache:/root/.cache/huggingface gema-model-api
+```
+
+## Testing API
+
+### 1. Health Check
+```bash
+curl http://localhost:8000/health
+```
+
+### 2. Generate Text
+```bash
+curl -X POST "http://localhost:8000/generate" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "inputs": "Apa kabar dunia teknologi hari ini?"
+  }'
+```
+
+### 3. Generate dengan Parameter Custom
+```bash
+curl -X POST "http://localhost:8000/generate" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "inputs": "Jelaskan tentang kecerdasan buatan",
+    "max_length": 200,
+    "temperature": 0.8,
+    "top_p": 0.95
+  }'
+```
+
+## Mengakses dari Aplikasi Lain
+
+### Python
+```python
+import requests
+
+url = "http://localhost:8000/generate"
+data = {
+    "inputs": "CONTOH INPUT USER"
+}
+
+response = requests.post(url, json=data)
+result = response.json()
+print(result["generated_text"])
+```
+
+### JavaScript/Node.js
+```javascript
+const response = await fetch('http://localhost:8000/generate', {
+    method: 'POST',
+    headers: {
+        'Content-Type': 'application/json',
+    },
+    body: JSON.stringify({
+        inputs: 'CONTOH INPUT USER'
+    })
+});
+
+const result = await response.json();
+console.log(result.generated_text);
+```
+
+## Dokumentasi API
+Setelah menjalankan container, buka browser dan akses:
+- API Docs: `http://localhost:8000/docs`
+- ReDoc: `http://localhost:8000/redoc`
+
+## Tips Optimasi
+
+### 1. Untuk GPU Support
+Jika Anda memiliki GPU NVIDIA, update `app.py`:
+```python
+# Ubah gpu_layers dari 0 ke jumlah yang sesuai
+gpu_layers=50  # Atau sesuai kemampuan GPU Anda
+```
+
+Dan update `docker-compose.yml`:
+```yaml
+services:
+  gema-model-api:
+    # ... konfigurasi lainnya
+    runtime: nvidia  # Untuk GPU support
+    environment:
+      - NVIDIA_VISIBLE_DEVICES=all
+```
+
+### 2. Untuk Production
+- Gunakan reverse proxy (nginx)
+- Implement authentication
+- Add rate limiting
+- Set up monitoring dan logging
+- Use environment variables untuk konfigurasi
+
+### 3. Memory Management
+Model ini memerlukan RAM yang cukup. Sesuaikan memory limits di docker-compose.yml berdasarkan spesifikasi server Anda.
+
+## Troubleshooting
+
+### Model Loading Issues
+- Pastikan koneksi internet stabil saat pertama kali menjalankan
+- Model akan diunduh otomatis dan disimpan di cache
+- Jika gagal, coba hapus folder cache dan jalankan ulang
+
+### Memory Issues
+- Kurangi `context_length` di app.py
+- Sesuaikan memory limits di docker-compose.yml
+- Gunakan swap file jika diperlukan
+
+### Port Conflicts
+- Ubah port di docker-compose.yml jika port 8000 sudah digunakan
+- Contoh: `"8080:8000"` untuk menggunakan port 8080

app.py

@@ -0,0 +1,93 @@
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel
+from ctransformers import AutoModelForCausalLM
+import os
+import uvicorn
+from typing import Optional, List
+import logging
+
+# Set up logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+app = FastAPI(title="Gema 4B Model API", version="1.0.0")
+
+# Request model - fleksibel untuk menerima semua parameter
+class TextRequest(BaseModel):
+    inputs: str
+    system_prompt: Optional[str] = None
+    max_tokens: Optional[int] = 10
+    temperature: Optional[float] = 0.7
+    top_k: Optional[int] = 50
+    top_p: Optional[float] = 0.9
+    repeat_penalty: Optional[float] = 2
+    stop: Optional[List[str]] = None
+
+# Response model
+class TextResponse(BaseModel):
+    generated_text: str
+
+# Global model variable
+model = None
+
+@app.on_event("startup")
+async def load_model():
+    global model
+    try:
+        logger.info("Loading model...")
+        model = AutoModelForCausalLM.from_pretrained(
+            "Dnfs/gema-4b-indra10k-model1-Q4_K_M-GGUF",
+            model_file="gema-4b-indra10k-model1-q4_k_m.gguf",
+            model_type="llama",
+            gpu_layers=0,  # Set to appropriate number if using GPU
+            context_length=2048,
+            threads=os.cpu_count()
+        )
+        logger.info("Model loaded successfully!")
+    except Exception as e:
+        logger.error(f"Failed to load model: {e}")
+        raise e
+
+@app.post("/generate", response_model=TextResponse)
+async def generate_text(request: TextRequest):
+    if model is None:
+        raise HTTPException(status_code=500, detail="Model not loaded")
+    
+    try:
+        # Buat prompt - gunakan system_prompt jika ada, atau langsung input user
+        if request.system_prompt:
+            full_prompt = f"{request.system_prompt}\n\nUser: {request.inputs}\nAssistant:"
+        else:
+            full_prompt = request.inputs
+        
+        # Generate text dengan parameter dari request
+        generated_text = model(
+            full_prompt,
+            max_new_tokens=request.max_tokens,
+            temperature=request.temperature,
+            top_p=request.top_p,
+            top_k=request.top_k,
+            repetition_penalty=request.repeat_penalty,
+            stop=request.stop or []
+        )
+        
+        # Bersihkan response dari system prompt jika ada
+        if "Assistant:" in generated_text:
+            generated_text = generated_text.split("Assistant:")[-1].strip()
+        
+        return TextResponse(generated_text=generated_text)
+    
+    except Exception as e:
+        logger.error(f"Generation error: {e}")
+        raise HTTPException(status_code=500, detail=f"Generation failed: {str(e)}")
+
+@app.get("/health")
+async def health_check():
+    return {"status": "healthy", "model_loaded": model is not None}
+
+@app.get("/")
+async def root():
+    return {"message": "Gema 4B Model API", "docs": "/docs"}
+
+if __name__ == "__main__":
+    uvicorn.run(app, host="0.0.0.0", port=8000)

docker-compose.yml

@@ -0,0 +1,23 @@
+version: '3.8'
+
+services:
+  gema-model-api:
+    build: .
+    ports:
+      - "8000:8000"
+    environment:
+      - PYTHONUNBUFFERED=1
+    volumes:
+      - ./cache:/root/.cache/huggingface  # Cache model downloads
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+    deploy:
+      resources:
+        limits:
+          memory: 8G  # Adjust based on your system
+        reservations:
+          memory: 4G

dockerfile

@@ -0,0 +1,27 @@
+# Dockerfile
+FROM python:3.10-slim
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    curl \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy requirements first (for better caching)
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application files
+COPY . .
+
+# Expose port
+EXPOSE 8000
+
+# Command to run the application
+CMD ["python", "app.py"]

requirements.txt

@@ -0,0 +1,8 @@
+transformers==4.36.0
+torch==2.1.0
+fastapi==0.104.1
+uvicorn==0.24.0
+huggingface-hub==0.19.4
+pydantic==2.5.0
+accelerate==0.25.0
+ctransformers==0.2.27