amokrov commited on
Commit
b985b27
·
verified ·
1 Parent(s): 740f779

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -18
README.md CHANGED
@@ -79,34 +79,37 @@ The provided OpenVINO™ IR model is compatible with:
79
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
80
 
81
  ```
82
- pip install "langchain-community>=0.2.15" optimum[openvino] huggingface_hub
83
  ```
84
 
85
  2. Run model inference:
86
 
87
  ```
88
- from langchain_community.embeddings import OpenVINOBgeEmbeddings
 
89
 
 
90
 
91
- embedding_model_name = 'OpenVINO/bge-base-en-v1.5-int8-ov'
92
- embedding_model_kwargs = {"device": "CPU", "compile": False}
93
- encode_kwargs = {
94
- "mean_pooling": False,
95
- "normalize_embeddings": True,
96
- "batch_size": 4,
97
- }
98
 
99
- embedding = OpenVINOBgeEmbeddings(
100
- model_name_or_path=embedding_model_name,
101
- model_kwargs=embedding_model_kwargs,
102
- encode_kwargs=encode_kwargs,
103
- )
 
 
 
 
 
 
 
104
 
105
- embedding.ov_model.compile()
 
106
 
107
- text = "This is a test document."
108
- embedding_result = embedding.embed_query(text)
109
- embedding_result[:3]
110
  ```
111
 
112
  For more examples and possible optimizations, refer to the [Inference with Optimum Intel](https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-optimum-intel.html).
 
79
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
80
 
81
  ```
82
+ pip install optimum[openvino]
83
  ```
84
 
85
  2. Run model inference:
86
 
87
  ```
88
+ import torch
89
+ from transformers import AutoTokenizer
90
 
91
+ from optimum.intel.openvino import OVModelForFeatureExtraction
92
 
 
 
 
 
 
 
 
93
 
94
+ # Sentences we want sentence embeddings for
95
+ sentences = ["Sample Data-1", "Sample Data-2"]
96
+
97
+ # Load model from HuggingFace Hub
98
+ tokenizer = AutoTokenizer.from_pretrained('OpenVINO/bge-base-en-v1.5-int8-ov')
99
+ model = OVModelForFeatureExtraction.from_pretrained('OpenVINO/bge-base-en-v1.5-int8-ov')
100
+
101
+ # Tokenize sentences
102
+ encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
103
+
104
+ # Compute token embeddings
105
+ model_output = model(**encoded_input)
106
 
107
+ # Perform pooling. In this case, cls pooling.
108
+ sentence_embeddings = model_output[0][:, 0]
109
 
110
+ # normalize embeddings
111
+ sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
112
+ print("Sentence embeddings:", sentence_embeddings)
113
  ```
114
 
115
  For more examples and possible optimizations, refer to the [Inference with Optimum Intel](https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-optimum-intel.html).