In close collaboration with engineers from Microsoft, we discovered an error in the Azure framework hosting Continia Cloud OCR, which have lead to some of the difficulties we have been struggling with for the past week. Multiple solutions to mitigate these issues have been applied, and we have closely monitored the Cloud OCR system to determine the level of stability and performance. We now believe the system is in a state of reliable behaviour, and the operation of the system is back to normal.
Should you still experience slow OCR processing, documents stuck in the processing state or other issues regarding our OCR service, please let us know by creating a support ticket in Continia Partnerzone.
We apologize for the inconvenience and appreciate the patience and understanding.
Posted Feb 17, 2023 - 23:29 CET
Update
We have closely monitored the stability and performance of Cloud OCR all day and we have not noticed any issues.
A smaller subset of documents may still be stuck in the Cloud OCR system for various reasons. We continue the manual recovery process of those documents, as we come across them.
Posted Feb 14, 2023 - 14:36 CET
Update
We are continuing to monitor for any further issues.
Posted Feb 14, 2023 - 07:12 CET
Update
The OCR processing queue is empty, and we see a normalized behavior across the system. We are continuing to monitor for any further issues.
Posted Feb 13, 2023 - 23:13 CET
Update
We keep monitoring the system for any issues or performance degradation that might occur. We see good progress on OCR processing the queue of documents. The current expectation is that the queue will be empty within an hour or two.
A smaller subset of documents might still be stuck in the OCR process. Those documents will be manually recovered once the queue is completed.
Posted Feb 13, 2023 - 20:37 CET
Monitoring
We have implemented changes to the system to make it more stable and increase the processing output. Right now, we keep it at a lower run rate and will scale further up as we see fit. We will closely monitor the service as it continuously processes the queue of documents.
Please expect a delay in OCR processing for at least three to four hours still, as we need to go through what has piled up throughout the day.
Once again, we deeply apologize for the inconvenience and thank you for your understanding.
Posted Feb 13, 2023 - 16:53 CET
Identified
The issue appears to be related to a scaling issue in Azure, and we have implemented a change that we are currently testing.
PDF files received are still being queued and processed at a reduced pace. When the system is operational at full speed, all received files will be processed as soon as possible.
We cannot provide a time when the Cloud OCR will be back to normal operations, but it is our highest priority.
We apologize for the inconvenience caused.
Posted Feb 13, 2023 - 13:21 CET
Investigating
We have received reports of delayed processing times in Cloud OCR. We expect this to be related to the maintenance task performed yesterday and are currently investigating.
We apologize for the inconvenience and thank you for your understanding.