main/logs/llm_api.log

2025-01-09 15:54:08,215 - hf_validation - WARNING - No .env file found. Fine if you're on Huggingface, but you need one to run locally on your PC.
2025-01-09 15:54:08,215 - hf_validation - ERROR - No HF_TOKEN found in environment variables
2025-01-09 15:54:08,215 - main - INFO - Starting LLM API server
2025-01-09 15:54:08,216 - llm_api - INFO - Initializing LLM API
2025-01-09 15:54:08,216 - llm_api - INFO - LLM API initialized successfully
2025-01-09 15:54:08,216 - api_routes - INFO - Router initialized with LLM API instance
2025-01-09 15:54:08,218 - main - INFO - FastAPI application created successfully
2025-01-09 16:46:10,118 - api_routes - INFO - Received request to download model: microsoft/phi-4
2025-01-09 16:46:10,118 - llm_api - INFO - Starting download of model: microsoft/phi-4
2025-01-09 16:46:10,118 - llm_api - INFO - Enabling stdout logging for download
2025-01-09 17:00:32,400 - llm_api - INFO - Disabling stdout logging
2025-01-09 17:00:32,400 - llm_api - INFO - Saving model to main/models/phi-4
2025-01-09 17:02:39,928 - llm_api - INFO - Successfully downloaded model: microsoft/phi-4
2025-01-09 17:02:41,075 - api_routes - INFO - Successfully downloaded model: microsoft/phi-4
2025-01-09 17:02:41,080 - api_routes - INFO - Received request to initialize model: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
2025-01-09 17:02:41,080 - llm_api - INFO - Initializing generation model: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
2025-01-09 17:02:41,081 - llm_api - INFO - Loading model from source: huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
2025-01-09 17:02:41,377 - llm_api - ERROR - Failed to initialize generation model huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.26.0'`
2025-01-09 17:02:41,377 - api_routes - ERROR - Error initializing model: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install 'accelerate>=0.26.0'`