Project: datalens
81 entity types
Matrix/Operations/RQ worker
ServerOperations

RQ worker

DataLens depends on the RQ worker to process queued extraction jobs for the SVGV dataset files. The RQ worker processes extraction jobs for the 132 SVGV dataset files. Redis functions as the queue backend supporting the RQ worker processing extraction jobs. The Backend API interacts with the RQ worker to manage extraction job queues and status for SVGV files. The extract_file_job function is executed by the RQ worker to process extraction of files. The worker container runs the RQ worker instance responsible for processing extraction jobs. The worker container is idle, waiting for extraction jobs in the RQ worker queue after the reset. The RQ worker is used by the FastAPI backend to handle asynchronous jobs such as embeddings and summary generation.