Issues with foreUP
Incident Report for foreUP
Postmortem

Dear Valued Customers, 

We deeply apologize for the recent software outages that affected our services. We understand the frustration and inconvenience this may have caused, and we want to assure you that resolving the issue and minimizing any future disruptions is our top priority. 

foreUP is committed to providing you with reliable and seamless software solutions, and we sincerely regret falling short of your expectations this time. Our technical team has been working tirelessly to identify the root cause of the outage and implement measures to prevent similar incidents from occurring in the future. 

As part of our commitment to transparency and our partnership, we would like to provide a few details regarding the outage.  During the day on Friday, May 26th and Sunday, May 28th, our database experienced significant unexpected spikes, which ultimately resulted in system outages.  We recently scaled up our database in preparation for more traffic during our busier season; however, this unexpected surge created a circumstance that exceeded busy season levels and our increased capacity. Our team is actively implementing safeguards and reinforcing our infrastructure to prevent similar issues from impacting our services. 

In the short term, we are taking precautions to prevent an outage by making sure the proper use of our third-party APIs is in place and any unusual situations do not impact our normal course of business. Additionally, the team is investigating any additional query optimization to ensure the most efficient and effective processes are running on our database.  Longer term, we will make database structural changes that will make it easier to scale up resources when unanticipated surges occur in the future, however, due to the inherent high risk of these types of database structural changes, we will need to schedule them outside of the peak busy season.    

We understand the impact these outages may have had on your operations and workflows. Please rest assured that we are doing everything in our power to restore full functionality and to support you in any way we can during this time. Our customer support team is available to assist you with any questions or concerns you may have. 

Once again, we sincerely apologize for any inconvenience caused, and we thank you for your patience and understanding. We greatly value your business and the trust you have placed in us. We are committed to learning from this experience and improving our systems to provide you with a more reliable and robust service moving forward. 

Thank you for your continued support.

Posted May 31, 2023 - 14:20 MDT

Resolved
This incident has been resolved.
Posted May 26, 2023 - 13:47 MDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted May 26, 2023 - 13:29 MDT
Update
We are continuing to investigate an issue with checking in tee times and performing sales in foreUP. Other functions of the software are affected as well.
Posted May 26, 2023 - 13:22 MDT
Investigating
We are currently investigating this issue.
Posted May 26, 2023 - 13:09 MDT
This incident affected: foreUP System (Database).