When: 17:06 ~ 17:34 PM March 7, 2021 UTC (4:22 ~ 4:34 AM March 8, 2021 AEDT)
Impact: Customers in the region may have experienced chat failures during that time.
Possible cause: This issue has come from AWS, their AP-SOUTHEAST-2 region encountered an internet connectivity issue.
Next step: - 4:06 AM AEDT, AWS confirmed that there was an internet connectivity issue in the region AP-SOUTHEAST-2 and they started the investigation. - 4:22 AM, Traffic to the servers dropped to the bottom. - 4:24 AM, We got an HC failure alarm. - 4:27 AM, DB connection failure started happening and our API service got partially impact. Some API requests failed. - 4:34 AM, The DB connection failure stopped occurring and all of the systems were back functioning. All the HC failures stopped occurring.
We're working with AWS to find out the root cause of the issue.
Posted Mar 08, 2021 - 04:34 AEDT
Investigating
We're experiencing an elevated level of API errors and are currently looking into the issue.