from omegaconf import OmegaConf from query import VectaraQuery import os from PIL import Image import gradio as gr from huggingface_hub import InferenceClient import logging logging.basicConfig(level=logging.DEBUG) # """ # For more information on `huggingface_hub` Inference API support, please check the docs: https://huggingface.co/docs/huggingface_hub/v0.22.2/en/guides/inference # """ # client = InferenceClient("HuggingFaceH4/zephyr-7b-beta") # def respond(message, history: list[tuple[str, str]], system_message, max_tokens, temperature, top_p): # messages = [{"role": "system", "content": system_message}] # for val in history: # if val[0]: # messages.append({"role": "user", "content": val[0]}) # if val[1]: # messages.append({"role": "assistant", "content": val[1]}) # messages.append({"role": "user", "content": message}) # response = "" # for message in client.chat_completion(messages, max_tokens=max_tokens, stream=True, temperature=temperature, top_p=top_p): # token = message.choices[0].delta.content # response += token # yield response def isTrue(x) -> bool: if isinstance(x, bool): return x return x.strip().lower() == 'true' corpus_ids = str(os.environ['corpus_ids']).split(',') cfg = OmegaConf.create({ 'customer_id': str(os.environ['customer_id']), 'corpus_ids': corpus_ids, 'api_key': str(os.environ['api_key']), 'title': os.environ['title'], 'description': os.environ['description'], 'source_data_desc': os.environ['source_data_desc'], 'streaming': isTrue(os.environ.get('streaming', False)), 'prompt_name': os.environ.get('prompt_name', None) }) cfg.description = f'''

{cfg.description}

''' vq = VectaraQuery(cfg.api_key, cfg.customer_id, cfg.corpus_ids, cfg.prompt_name) def respond(message, history): if cfg.streaming: # Call stream response and stream output stream = vq.submit_query_streaming(message) outputs = "" for output in stream: outputs += output yield outputs else: logging.debug(f"Calling normal query with message: {message}") # Call non-stream response and return message output response = vq.submit_query(message) logging.debug(f"Chat response: {response}") return response def random_fun(message, history): return message + '!' demo = gr.ChatInterface(respond, title = cfg.title, description = cfg.description) """ For information on how to customize the ChatInterface, peruse the gradio docs: https://www.gradio.app/docs/chatinterface """ # demo = gr.ChatInterface(respond) if __name__ == "__main__": demo.launch()