DataLens Agent Mode implements the explore-schema skill to profile project data schemas.

ExploreSchemaSkill produces SkillResult when executing to discover and profile project data schema. AgentWarmingService uses ExploreSchemaSkill to assemble warm context by computing schema profiles.

ExportService uses DuckDBService to export query results in various formats. ExportService converts SkillResult data into CSV, Excel, or JSON formats.

Initially missing or broken, now fixed to show accurate extraction progress.

The Extraction coordinator is part of the workflow that triggers AI Summary Generation after file extraction completes. BatchProcessor uses ExtractionCoordinator to coordinate data extraction in the pipeline. ExtractionCoordinator uses PrepareDataSkill to process data after extraction across CPU and GPU services. The extraction coordinator service depends on the DuckDB service for managing extracted text chunks and related data. BatchProcessor orchestrates the full pipeline that involves ExtractionCoordinator for extraction tasks. ExtractionCoordinator coordinates extraction processes that are prioritized by FilePrioritizer.

DataLens Platform uses an extraction pipeline involving DS-STAR extractors and DuckDB for data processing. The extraction pipeline depends on the RQ queue for batch extraction job management. The extraction pipeline previously wrote extracted data into DuckDB, causing write locks during extraction. The extraction pipeline was modified to write extracted data into PostgreSQL enabling concurrent query operation. The extraction pipeline depends on the RQ queue for batch extraction job management. The extraction pipeline previously wrote extracted data into DuckDB, causing write locks during extraction. The extraction pipeline was modified to write extracted data into PostgreSQL enabling concurrent query operation. The Extraction Pipeline depends on the RQ Worker to process the extraction queue for files asynchronously. Extraction Pipeline stores extracted file data and catalog information in PostgreSQL database with language support. The Backend implements the extraction pipeline business process including DS-STAR integration. The DataLens Platform includes an extraction pipeline that converts CSV, Excel, and PDF files into DuckDB usable data. The Extraction pipeline in the DataLens Platform uses pandas for data manipulation and loading extracted data. The Extraction Pipeline depends on the RQ Worker to process files asynchronously in the extraction queue.

Must show over 90% accuracy in extracting tables, maintaining document structure, and semantic chunking, validated through tests.

Simple, in-memory keyword-based index for table search when Qdrant is unavailable, in backend/app/services/table_index.py.

The file extraction process triggers the async embedding queue to generate embeddings asynchronously after extraction completes.

File summaries are developed to accurately reflect content, relevance, and key questions for each uploaded data file, informing users and linking to project goals. Files are cataloged within DataLens, supporting comprehensive AI-generated summaries to enhance data understanding.

The project goal is used to inform the file summary prompt replacing the previous hardcoded budget analysis description.

Generates concise summaries describing file contents, relevance, and questions they can answer, during file ingestion and cataloging, to improve data cataloging and documentation. File Summary Generation uses the project's scope instead of a hardcoded string to contextualize the summaries.

DataLens Platform includes a file upload feature for CSV, Excel, and PDF files. DS-STAR FileAnalyzer integration depends on the file upload feature to automatically catalog files upon upload. FileUpload physical table entries are associated with Project entities, storing files related to projects. The Backend implements the file upload requirement. AI Summary Generation stores the generated summaries in the ai_summary column of the FileUpload records. Project physical table contains multiple FileUpload physical tables representing uploaded files associated with the project. FileUpload physical table depends on ProcessingJob physical table representing background processing jobs of uploaded files.

The Full Findings Visualization Layer capability requires the support of 7 filter categories for filtering findings in the UI.

Image file showing test results or validation outcome for visualization features.

FindingsGenerator uses Finding to represent individual analytical findings generated from query results. Finding is a data structure created and managed by FindingsGenerator during analytical finding generation.

findings_generator logging is a specific logging to be added at the start of findings generation for diagnostics. findings_generator logging is to be implemented in backend/app/services/findings_generator.py.

The frontend /ask-stream switch uses the backend /ask-stream endpoint to enable streaming query responses and prevent client timeout errors. The frontend /ask-stream switch is implemented in the SvelteKit Framework to provide streaming UI experience and progress states.

Fixed frontend API call to `/api/v1/projects/{project_id}/files` endpoint to correctly display AI summaries in the project dashboard.

Frontend Integration uses the FindingsPanelNew component to display findings in the agent page. The Multilingual Support (Danish) epic includes the Frontend UI Translation use case. The Frontend UI Translation use case uses SvelteKit i18n plugin to provide UI translations. The Frontend UI Translation use case involves modifying Frontend files to implement i18n. The Frontend UI Translation use case includes providing a language selector on the Login page. The Frontend UI Translation use case includes a language selector on the Projects page.

The frontend/tests/discovery-svgv.spec.ts file is part of the comprehensive E2E test suite for validating DataLens features on the SVGV dataset. The frontend tests discovery-svgv.spec.ts are part of the Playwright E2E test suite for the Discovery feature.

Full Danish Language Support was implemented in the Backend to handle user language preference and LLM summary language. Full Danish Language Support was implemented in the Frontend with 150+ UI strings translated to Danish and language selector support. Admin User was configured with language set to Danish to receive all summaries and UI in Danish.