DuckDB

DuckDB is used by DataLens for local structured data storage and querying from extracted analysis files. It is central to analysis and is used by SQLAgent. The platform is migrating to PostgreSQL to eliminate DuckDB's write-lock, enhancing concurrency. It stores tables like sales, surveys, and test data, supporting the platform's in-memory querying and analysis processes. Qdrant indexes embeddings generated from text chunks stored in DuckDB, enabling semantic search in the platform. OpenClaw Skill API queries DuckDB which contains 473 extracted budget tables for analytical data. DuckDB hosts the data tables extracted for Project 14 from SVGV files for analytical queries. The implementation uses DuckDB as a unified database for storing extracted data. CSV Extractor loads validated and cleaned CSV files into DuckDB. Excel Extractor loads normalized Excel data into DuckDB. PDF Extractor extracts tables from PDFs and loads them into DuckDB. SQLAgent executes generated SQL queries on DuckDB. The system uses LangChain framework alongside DuckDB for data pipeline management. RAPIDS cuDF is optionally used for large dataframe acceleration alongside DuckDB.