Project: datalens
81 entity types
Matrix/Data Model/132 files
PhysicalTableData Model

132 files

The reset-and-reextract.py script processes the 132 files for full reset and re-extraction of the SVGV dataset. The monitor-extraction.sh script monitors the extraction progress of the 132 files including queue size and extracted/pending counts. The RQ worker processes extraction jobs for the 132 SVGV dataset files. The project_14 schema in PostgreSQL stores extracted data from the 132 SVGV dataset files. The file_uploads data entity tracks the processing status of the 132 files in the project_14 schema. The 132 SVGV files map to DuckDB tables, which are used to store extracted budget data PostgreSQL owns metadata for the 132 SVGV files in the project catalog The SVGV bulk extraction process processes the 132 SVGV files to extract data into DuckDB tables The full SVGV dataset reset and re-extraction process queues 132 extraction jobs for processing. The 132 extraction jobs are processed by the RQ worker.