- Please register topics below
|Progress from the previous meeting Database meeting 2022-03-09|
Grand Unified Repo:
Subchunk sizes and overlap:
|Optimizations in processing results of the N-N queries|
John Gates, team
|Update on worker load imbalance problem||Fritz Mueller|
The context was set in the previous meeting (see the link Database meeting 2022-03-09):
|IDF worker crash this morning||Fritz Mueller|
How do we investigate this?
Possible short-term solutions:
There is a (potentially?) related issue exhibiting itself in the worker logs as follows:
Further investigation shows that these harmless messages are posted by:
The work on modifying the workflow to begin using ASYNC ingest service is still in progress.
The SYNC mode worked successfully for ingesting 50 TB catalog
Fabrice Jammes is still working on the Operator to integrate the change.