APIs registered for Discovery feature: /discovery, /tables, /validate, enabling data discovery, review, and validation workflows.

Discovery API router was registered in the backend app main.py to expose discovery endpoints. DiscoveryFlow depends on the Discovery API router for backend data and functionality. The Discovery API router integrates with the DataLens backend system. Coolify auto-deploy integrates with the Discovery API router deployment process to automate updates.

The Discovery backend service implements the Discovery API endpoints. The UI components depend on the Discovery backend service to function correctly.

An integrated system component within DataLens that automates table ranking, join discovery, and consolidation, dramatically improving query success rate and transparency, with full functionality validated through the latest end-to-end tests. The Discovery feature is validated by Playwright E2E tests to ensure functionality. Frontend tests validate the Discovery feature's user interface and experience. Backend tests validate the Discovery feature's API endpoints and server-side logic.

Expected to be less than 10 seconds during normal operation.

The Data Discovery feature uses the Discovery service. DiscoveryService generates ConsolidationRecommendation for intelligent table consolidation based on questions. DSStarService integrates with DiscoveryService as client for DS-STAR Agent API on elin.

Established process involving data discovery, consolidation, and analysis driven by a comprehensive workflow.

Playwright test suite with 15 real-data validation tests for Discovery feature, scheduled to be integrated into deployment pipeline.

The Data Discovery feature contains the discovery.py service. The discovery.py service uses the TableIndex for semantic table matching. The Discovery Service processes Danish questions for entity extraction, table ranking, and join detection. The Discovery Service uses a 4-factor relevance score to rank tables by matching criteria. The Discovery Service applies the Known keys join strategy to discover joins with 95% confidence. The Discovery Service applies the ID matching join strategy for join discovery with 85% confidence. The Discovery Service applies the Value overlap join strategy to identify joins by data overlap with 75% confidence. The Discovery Service implements the Schema consolidation mechanism to improve query success rate. DiscoveryService uses Table to represent database tables with metadata in consolidation recommendations. DiscoveryService uses JoinPath to represent joins between database tables for consolidation. DiscoveryService produces ConsolidationRecommendation to suggest table consolidations for questions. FilePrioritizer uses DiscoveryService outputs to prioritize project files relevant to analytical questions. The discovery.py service implements semantic table matching through TableIndex. The Data Discovery Feature includes the Backend discovery service which performs entity extraction, table ranking, and join discovery. The Discovery Service uses Qwen3 LLM for table selection in the intelligent table discovery process. The Discovery Service passes unified schemas to Arctic LLM for SQL generation after table consolidation. The Data Discovery Feature includes the Backend discovery service which performs entity extraction, table ranking, and join discovery. The Discovery Service uses Qwen3 LLM for table selection in the intelligent table discovery process. The Discovery Service passes unified schemas to Arctic LLM for SQL generation after table consolidation.

The Discovery Build Summary.md document outlines the implementation and workflow for the data discovery features, including intelligent table consolidation and guided exploration components.

Guides detailing the development and integration of the discovery system.

Documentation of all test cases validating the discovery feature.

Documentation detailing the multi-stage discovery process, backend services, UI components, and integration steps for the data discovery feature.

The DiscoveryFlow page uses Frontend UI components as part of the Data Discovery user experience. The DiscoveryFlow component is intended to be added and used in the /projects/[id]/analyze page file. The DiscoveryFlow component is intended to be added and used in the /projects/[id]/analyze page file. DiscoveryFlow component was integrated into the frontend analysis page to enable discovery UI. DiscoveryFlow depends on the Discovery API router for backend data and functionality. DiscoveryFlow is a UI component part of the Data Discovery system. DataLens system includes the DiscoveryFlow UI component for data discovery and consolidation. The DiscoveryFlow component is implemented using the Svelte framework. The DiscoveryFlow component uses Tailwind CSS for styling and responsive design. The DiscoveryFlow component is built with the shadcn stack, enabling modern UI features like gradients and animations. The Data Discovery Feature includes the DiscoveryFlow UI component used in analysis page for user interaction. Integration requires adding the DiscoveryFlow component to the analysis page for front-end functionality. Integration requires adding the DiscoveryFlow component to the analysis page for front-end functionality.

DiscoveryFlow Svelte component is part of the Data Discovery feature for guided data exploration. The Data Discovery feature includes the DiscoveryFlow Svelte component for guided data exploration.

A responsive UI component displaying loading state during data discovery processes.

Infrastructure includes Disk space as a monitored specification. The plan monitors disk space usage for DuckDB and Qdrant data storage.

The text_chunks table includes a file_id referencing the file from which the text chunk originated. Each text chunk in the text_chunks table is associated with a project via project_id. File uploads contain text_chunks tables which store chunks of text extracted from uploaded files.

DataLens Agent Mode deployment involves running IronClaw Service as a Docker container. The DataLens platform uses Docker for deployment including backend, frontend, PostgreSQL, and Redis containers. Docker deployment is defined using the docker-compose.yml configuration file.

The DataLens Platform backend and dependencies are deployed using a Docker Compose Stack specification. The Docker Compose Stack includes the Platform Backend service as one of its containers alongside PostgreSQL and Redis.

The Docker deployment uses docker-compose.yml to orchestrate the stack The Docker deployment includes the Frontend The Docker deployment includes the Backend The Docker deployment includes PostgreSQL The Docker deployment includes Redis The Docker deployment uses health checks for monitoring service health The Docker deployment configures volumes for data persistence The Docker deployment configures networking for container communication The Docker deployment includes a Coolify-ready configuration

The DataLens Platform includes Docker Deployment for its full stack. Docker Deployment integrates with the Coolify Application deployment system. Docker Deployment contains the docker-compose.yml file for full stack orchestration. Docker Deployment includes the DEPLOYMENT.md acceptance document guiding deployment via Coolify. Docker Deployment includes the COOLIFY_DEPLOY.md acceptance document with deployment instructions. The Docker deployment environment contains the Frontend, Backend, PostgreSQL, and Redis services. The Docker deployment includes Health checks for components. The Docker deployment includes volumes configurations for persistent storage. The Docker deployment includes network configurations for service communication. The Docker deployment configuration is designed to be ready for deployment with Coolify.

The Frontend's build process is constrained to use Docker multi-stage build technology. The Frontend uses Docker multi-stage build for containerization

The backend container may require a manual restart via docker restart command to fix potential deployment issues blocking query execution after thinking messages.

The Backend is designed to support Docker deployment as a technical constraint. The Backend includes Docker support

The Docker-compose configuration depends on the RQ extraction queue on redis for job processing. docker-compose.coolify.yml is part of the Docker-compose configuration used in deployment. The Docker-compose configuration was constrained by a misconfiguration of the RQ extraction queue on redis causing job processing issues. The Backend container depends on the Docker-compose configuration for deployment orchestration. The Coolify deployment uses the Docker-compose configuration for service orchestration. The backend container's deployment problems can be remedied by manual deployment through docker-compose on theo if Coolify is stuck. DataLens Platform uses Docker Compose for deployment. Backend API service is deployed via Docker Compose orchestration. Frontend service is deployed with Docker Compose orchestration. DataLens Platform uses Docker Compose for deployment. Backend API service is deployed via Docker Compose orchestration. Frontend service is deployed with Docker Compose orchestration.