Website Instability

Incident Report for Ninja Bot

Monitoring

We have found the Root Cause of the Websites instability issues to be Generated by:

• Outdated, Deprecated or Unsupported Modules
• Multiple errors generating from Ninja Bots API
• Multiple errors generating from the Dashboard

And a few more issues that don’t matter in terms of relevance to this case.

The Good Stuff:

1) We have rolled out some Patches which should solve All the errors being generated at this time, due to the Size and Nature of these Patches this could take some time and could possibly result in Degraded Performance on the Bot, Website, Dashboard and API

2) Website is back Online and Operational but still Under Maintenance, Due to this users will be Redirected to a Maintenance Mode message throughout the entire website

The Bad Stuff:

1) Currently for all of the Patches to be uploaded and Pushed to development this requires our Devs to be tied up mainly on this issue.

(Support & Dev Response times for the bot could vary during this time)

2) Unfortunately due to the Size of these Changes, the Upload itself is expected to take about 1 - 2 Hours

3) Though the Bot is still online we are Estimating that the Websites Maintenance Mode will be Enabled for at least upwards of 12 - 24 Hours

Please Note:
1) The given Deadlines set above are subject to change.

2) Estimated Times are created with the Developers in mind. Changes to the Code have to be made
+ Uploaded to the Server
+ Pushed To Development
+ Monitored for a Decent amount of time to watch for and work out any errors or inconsistencies

This is all Part of what we do in Terms of making sure that the Services we offer are made available at all times.
Posted Jul 11, 2020 - 00:57 MDT

Investigating

The Ninja Bot Website is showing signs of intermittent Database issues including:

* Connection String
* Connection Methods
* Database Access
* Corrupted Content

We will investigate this issue as soon as we have worked out the issues on the Bots Back-End.
Posted Jul 10, 2020 - 15:30 MDT
This incident affects: Website.