1. Main development threads
|  PLANNED |  STARTED |  FINISHED |
This page presents a status of the ongoing developments
| Status | Priority | Summary | JIRA Ticket |
|---|---|---|---|
| HIGH | Refactoring the Configuration service | DM-28860 - Getting issue details... STATUS |
| HIGH | Redesign of Qserv's "Empty Chunk Lists" into "Chunk Inventory" | |
| MEDIUM | Add chunk versioning | Create a ticket  |
| CRITICAL | Huge memory leak in the Master Replication Controller when Qserv workers are down | |
| HIGH | Integration test for the Replication System | Create a ticket |
| MEDIUM | Security reinforced protocol for Controllers/Workers communications and worker database operations | |
| Reinforce the controllers / workers and file client / file server protocols to carry over and verify a unique identifier of a Qserv instance, so that the Replication system's configuration errors were properly detected and handled, and cross-talks between two or many instances of the Replication system were prevented. | DM-23191 - Getting issue details... STATUS | |
| A Design of the Qserv Monitoring System | DM-13751 - Getting issue details... STATUS | |
| Implement Qserv Monitoring Services (the Dashboard) | DM-13752 - Getting issue details... STATUS | |
|
| ||
| Master replication controller with built-in REST API support | DM-10425 - Getting issue details... STATUS | |
| Deploying, configuring and operating the Qserv Replication system | ||
| Improved container build procedure for the Replication System's tools | DM-16790 - Getting issue details... STATUS | |
| Migrate binary tools of the Replication system to the common Application framework | DM-17113 - Getting issue details... STATUS | |
| Add REST services for monitoring and managing the Replication System | DM-16788 - Getting issue details... STATUS | |
| DM-17655 - Getting issue details... STATUS | ||
| Minor code refactoring | DM-18401 - Getting issue details... STATUS | |
| Evaluate MySQL table partitioning for the reliable ingest with the rollback option | No ticket required for this effort. All details are found within a document linked on this row. | |
| Worker database management | DM-17812 - Getting issue details... STATUS worker service needs to connect directly to the MariaDB service of the Qserv worker | |
| Implement a framework for sampling monitoring parameters of Qserv workers | DM-19226 - Getting issue details... STATUS | |
| Add support for super-transactions | DM-19156 - Getting issue details... STATUS | |
| Integrate Qserv replication system with CSS | DM-18004 - Getting issue details... STATUS Master Replication Controller needs to connect directly to the MariaDB service of the Qserv master | |
| Log REST requests to the Master Replication Controller into the Event Log | This was implemented as a refactoring stage of the Replication system's code before a work on the new Ingest system began. | |
| Improve tracking of the asynchronous operations (jobs/requests) | DM-18743 - Getting issue details... STATUS | |
| Garbage collection at Replication workers | DM-22428 - Getting issue details... STATUS |
2. General code improvements, refactoring, cleanup
| PLANNED | STARTED | FINISHED |
These projects have generally low priority, and they're meant to improve the overall quality and robustness of the code, as well as make it more maintainable in the long run.
| Status | Priority | Summary | JIRA Ticket |
|---|---|---|---|
| MEDIUM | Improved version number handling in the REST API | Create a ticket |
| MEDIUM | Testing framework for the REST API | Create a ticket |
| LOW | Fix incomplete implementation of the method: json HttpConfigurationModule::_addTable() {
debug(__func__);
// FIXME: extend the service to accept all attributes of the table.
// Create a method in the base class to extract standard attributes from
// from the requests's body. Use this method here and in the method HttpIngestModule::_addTable
| |
| LOW | Fix incorrect naming convention for the object identifiers in the director and dependent tables | |
| LOW | Eliminate Doxygen style comments from the code | No ticket is required |
| LOW | Allow protobuf dependencies in the worker-side replication requests | Partially implemented for a family of SQL requests when implementing a new Ingest system. More work is needed for other types of worker-side requests. |
| LOW | Migrate to using std::chrono in the public API of the Replication Framework | |
 | LOW | More robust implementation of the asynchronous handlers | |
| LOW | Refactor operations with chunked table names to class ChunkedTable | |
| LOW | Code refactoring in the asynchronous event handlers of Requests and Jobs | |
| LOW | Style refactoring in the range-based for loops | |
| Optimized implementation of class SqlRemoveTablePartitionsJob | DM-22135 - Getting issue details... STATUS | |
| Refactor the HTTP Processor of the Master Replication Controller | DM-21803 - Getting issue details... STATUS | |
| Refactor results reporting in the implementation of class ControllerApp | DM-21524 - Getting issue details... STATUS |
3. Known bugs, outstanding issues, etc.
- Investigating problems with XRootD/SSI at workers
- Huge memory leak in the Master Replication Controller when Qserv workers are down