Project: datalens
81 entity types
Matrix/User Interface/backend/app/services/embedding_service.py
PageUser Interface

backend/app/services/embedding_service.py

Embedding service uses Ollama running on elin GPU for batch embedding of document chunks. Embedding service uses the nomic-embed-text 768-dimensional model via Ollama for GPU accelerated vector embedding. EmbeddingService is used by the Docling extraction system to produce GPU-accelerated embeddings for semantic chunk vectors. batch_vectorize_job depends on EmbeddingService to perform batch vectorization of extracted document chunks. BatchProcessor uses EmbeddingService to vectorize extracted data in the processing pipeline. EmbeddingService integrates with OpenClawHttpClient to use Ollama on elin GPU for embedding computations. The embedding_service.py provides centralized embedding generation by communicating with Ollama running on elin GPU. GPU-first document extraction uses the embedding service in backend/app/services/embedding_service.py which communicates with Ollama on the GPU for embeddings. The embedding service performs batch embeddings using Ollama's nomic-embed-text model on GPU, supporting GPU utilization monitoring and automatic retries. The extraction worker chains extraction results to the embedding service for batch vectorization on GPU after successful DOCX/PPTX extraction.