The work-around has resulted in better than expected performance with all Footprints workspaces being able to keep up with incoming emails. We are marking this incident as "Resolved" for now but will continue to work with the vendor to find a longer term solution.
Posted Dec 05, 2019 - 16:18 EST
Monitoring
A work-around has been implemented to allow Footprints to better keep up with the processing of incoming emails. Although we expect to see delays once it gets busy we are estimating they will be less than 30 minutes as opposed to the 2-3 hours we've been seeing the past 2 days. We continue to work with the vendor support/developer in order to find a proper fix and will update later today. Once again, thank you for your patience and understanding.
Posted Dec 05, 2019 - 09:21 EST
Update
All workspaces have caught up on email intake for now but will most likely fall behind again once volume picks up tomorrow. The issue has now been escalated to a developer who will be available Thursday morning. We will continue to investigate and trouble-shoot at that time.
Posted Dec 04, 2019 - 17:41 EST
Update
All attempted fixes have provided only marginal improvement meaning that we are still about 2 hours behind in the processing of emails in Footprints. We continue to work with the vendor to determine the root issue. Once again, thank you for your patience.
Posted Dec 04, 2019 - 14:51 EST
Identified
In consultation with vendor support we will be making a configuration change then restarting the Footprints server. It will be unavailable from approximately 12:15-12:30pm. We will provide an update after we have observed the results of the restart. Thank you for your patience.
Posted Dec 04, 2019 - 11:40 EST
Investigating
Unfortunately, the issue does seem to be load related as we again are having slowness issues beginning at 9:00am when email requests started arriving in earnest. We are re-engaging with vendor support to continue trouble-shooting. In the meantime, if you have urgent requests and have not received a ticket notification email please call the Help Centre at ext 58888.
Posted Dec 04, 2019 - 09:33 EST
Monitoring
In consultation with vendor support we have implemented a configuration change to reduce the email processing time. Although it appears to have helped we cannot be sure to what degree as it is now a "quiet time" with no new tickets coming into any workspace and therefore very little system contention. We will continue to monitor performance as the business day picks up again Wednesday.
Posted Dec 03, 2019 - 18:59 EST
Investigating
Footprints is currently processing incoming email very slowly (45-90 minutes instead of the usual 2-4 minutes). This results in longer than usual response times to Help Centre tickets. If your request is urgent and you have not received a ticket notification message please call the Help Centre at ext 58888.
All other Footprints work spaces are less impacted with delays between 5-10 minutes. We have involved vendor support and continue to investigate.