BluePay Gateway Issue
Incident Report for BluePay
Resolved
All BluePay Gateway issues related to the 09/14/2020 fiber cut have been resolved. We apologize for any inconvenience this may have caused. Your business is very important to us, and we strive to provide you with exceptional and secure payment processing solutions.
Posted Sep 18, 2020 - 10:29 CDT
Update
The BluePay transactions that were unavailable in CardPointe are now available.
Posted Sep 18, 2020 - 10:15 CDT
Update
Transactions processed 09/14/2020 01:41pm CDT until 09/15/2020 07:00am CDT are currently unavailable in CardPointe. Work is underway to make them available.
Posted Sep 17, 2020 - 16:51 CDT
Update
Transactions processed 09/14/2020 01:41pm CDT until 09/15/2020 07:00am CDT are currently unavailable in CardPointe. Work is underway to make them available.
Posted Sep 17, 2020 - 16:50 CDT
Update
First Data North 09/15/2020 settlement has been delayed to 09/16/2020 for merchants with settlement scheduled between 7:05am CDT and 11:59pm CDT. This is due to the 09/14/2020 settlement being delayed to 09/15/2020 7:05am CDT. Merchants using the default 2:00am CDT settlement schedule are unaffected by this delay.
Posted Sep 16, 2020 - 08:48 CDT
Update
Processing using the TSYS platform was unavailable 09/14/2020 1:41pm through 09/15/2020 6:58am. Any missed settlements will settle at the next scheduled time.
Posted Sep 15, 2020 - 11:42 CDT
Update
First Data Nashville 8583 settlements missed between 09/14/2020 1:41pm CDT until 09/15/2020 6:58am CDT will settle at their next scheduled settlement time.
Posted Sep 15, 2020 - 10:57 CDT
Update
Any ACH Direct settlements missed between 09/14/2020 1:41pm CDT until 09/15/2020 6:58am CDT will settle at their next scheduled settlement time.
Posted Sep 15, 2020 - 10:29 CDT
Update
All First Data North settlements between 09/14/2020 1:05pm CDT and 09/15/2020 7:05am were all settled at 09/15/2020 7:05am.
Posted Sep 15, 2020 - 10:18 CDT
Update
BlueChex settlement reporting is updated.
Posted Sep 15, 2020 - 08:54 CDT
Update
Notification queue is caught up. Email receipts, settlement emails, webhook posts and CardPointe updates are now working normally.
Posted Sep 15, 2020 - 08:41 CDT
Update
Full capacity of file upload processing has been restored.
Posted Sep 15, 2020 - 08:21 CDT
Update
Quickbooks Online is now available.
Posted Sep 15, 2020 - 07:03 CDT
Update
BluePay transaction processing continues from our secondary data center while a service provider works to restore connectivity to our primary data center.

The timeline so far.

09/14/2020 1:41pm Fiber cut at primary data center causes traffic to automatically route to secondary data center. Hosted payment forms become unavailable. Email receipt and webhook processing stopped.
09/14/2020 2:50pm Hosted payment forms available.
09/14/2020 4:00pm Server resources added to address load
09/14/2020 5:08pm Email receipt and webhook processing started.

The following services remain affected.
Email receipts operating at reduced capacity.
Webhook posts operating at reduced capacity.
Quickbooks Online unavailable.
BlueChex settlement reporting delayed.
ACH direct settlement delayed.
Transaction reporting to CardPointe delayed.

Thank you for your patience while we work through these remaining issues.
Posted Sep 15, 2020 - 06:26 CDT
Update
Additional server resources have been added to address response errors.
Posted Sep 14, 2020 - 16:02 CDT
Monitoring
We have implemented a solution and are monitoring the incident until it is resolved. Processing times may be slightly elevated.
Posted Sep 14, 2020 - 14:51 CDT
Identified
We have identified the cause of the incident affecting the BluePay Gateway and are working to find a resolution. We appreciate your patience during this time.
Posted Sep 14, 2020 - 14:43 CDT
This incident affected: Web Interface and APIs.