tag:status.luckyorange.com,2005:/historyLucky Orange Status - Incident History2024-03-28T07:25:26-05:00Lucky Orangetag:status.luckyorange.com,2005:Incident/194979842023-12-21T07:51:16-06:002023-12-21T07:51:16-06:00Issues with capturing data<p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>07:51</var> CST</small><br><strong>Resolved</strong> - The issue has been resolved. We identified that this did only effect a few accounts and those accounts should now be tracking visitors as normal again. Please reach out to support with any questions.</p><p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>05:30</var> CST</small><br><strong>Identified</strong> - There are some accounts having issues collecting data. We have identified the issue and are working to resolve this right away.</p>tag:status.luckyorange.com,2005:Incident/190035052023-11-02T17:11:01-05:002023-11-02T17:11:01-05:00Data Not Available<p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>17:11</var> CDT</small><br><strong>Resolved</strong> - All data is now caught up and there should be no further delays in display of the data.</p><p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>15:10</var> CDT</small><br><strong>Monitoring</strong> - There are some accounts where your data may not be available for a short amount of time. All data is still being collected and will appear as soon as an update to our data provider is completed. This should take an hour or two at most.</p>tag:status.luckyorange.com,2005:Incident/178989812023-07-19T13:25:11-05:002023-07-19T13:25:11-05:00Visitor Session Playback Delay<p><small>Jul <var data-var='date'>19</var>, <var data-var='time'>13:25</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>19</var>, <var data-var='time'>10:25</var> CDT</small><br><strong>Monitoring</strong> - The update has been completed and any missing data will be backfilled over the coming hours. Please reach out with any questions or concerns at this time.</p><p><small>Jul <var data-var='date'>19</var>, <var data-var='time'>06:02</var> CDT</small><br><strong>Identified</strong> - We are making some updates to our database which may cause data from the past few days to not display right away. All data will be backfilled as soon as this update is completed.</p>tag:status.luckyorange.com,2005:Incident/178625512023-07-15T18:39:25-05:002023-07-15T18:39:25-05:00Recording Data Delayed<p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>18:39</var> CDT</small><br><strong>Resolved</strong> - The data is finishing backfilling and is available to review within all accounts.</p><p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>09:20</var> CDT</small><br><strong>Monitoring</strong> - An update has been implemented to resolve the issue and data will slowly start to populate back in over the coming hours. We will update this as soon as data collection is caught back up.</p><p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>02:30</var> CDT</small><br><strong>Investigating</strong> - We are currently experiencing a delay in new recording data showing within the account. We are working on resolving this issue and will provide an update shortly.</p>tag:status.luckyorange.com,2005:Incident/176912712023-06-26T16:15:13-05:002023-06-27T07:46:29-05:00App Status<p><small>Jun <var data-var='date'>26</var>, <var data-var='time'>16:15</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>26</var>, <var data-var='time'>14:22</var> CDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jun <var data-var='date'>26</var>, <var data-var='time'>14:08</var> CDT</small><br><strong>Identified</strong> - We are experiencing delays with data loading throughout the app. We have identified the cause, and should have all systems back up and running shortly. We'll update this page with additional information as it becomes available. Thank you for your understanding and patience.</p>tag:status.luckyorange.com,2005:Incident/172548312023-05-15T11:09:17-05:002023-05-15T11:09:17-05:00Account Access<p><small>May <var data-var='date'>15</var>, <var data-var='time'>11:09</var> CDT</small><br><strong>Resolved</strong> - We have resolved the latency issues and access to the app should be fully operational.</p><p><small>May <var data-var='date'>15</var>, <var data-var='time'>08:17</var> CDT</small><br><strong>Identified</strong> - We have identified latency issues with accessing accounts on Lucky Orange. We are working to resolve the issue and will provide updates as we have them.</p>tag:status.luckyorange.com,2005:Incident/169166832023-04-19T22:00:49-05:002023-04-20T07:03:07-05:00Shopify Payment Issues<p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>22:00</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>19</var>, <var data-var='time'>19:00</var> CDT</small><br><strong>Identified</strong> - Shopify is currently experiencing an outage with their payment gateway that is preventing merchants from upgrading to paid plans on apps. This is preventing any Lucky Orange customers that use the Shopify app from upgrading at this time. Shopify is diligently looking into this and we will provide an update as soon as this is resolved.</p>tag:status.luckyorange.com,2005:Incident/168652052023-04-14T09:35:46-05:002023-04-14T09:35:46-05:00Slow response with data delays within app<p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>09:35</var> CDT</small><br><strong>Resolved</strong> - We have resolved any delays that were occurring within the app and all is back up and running. Thanks for your patience</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>07:00</var> CDT</small><br><strong>Investigating</strong> - We are experiencing delays with data loading throughout the app. We are investigating what is causing the issue and will provide updates as soon as we have identified and resolved what is causing this display issue to occur.</p>tag:status.luckyorange.com,2005:Incident/166941982023-03-30T07:59:39-05:002023-03-30T07:59:48-05:00Access to Lucky Orange<p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>07:59</var> CDT</small><br><strong>Resolved</strong> - All is back to normal and the update has been completed. Have a wonderful day and thanks again for your patience.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>07:03</var> CDT</small><br><strong>Monitoring</strong> - The maintenance has been completed and are slowly opening access back up to the app. We will continue to monitor performance as access is restored. We appreciate your patience.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>06:04</var> CDT</small><br><strong>Update</strong> - We are almost done with the system updates and should have access restored shortly. Thanks again for your patience and please reach out to support@luckyorange.com with any additional questions.</p><p><small>Mar <var data-var='date'>30</var>, <var data-var='time'>00:41</var> CDT</small><br><strong>Identified</strong> - Maintenance is ongoing and we will continue to provide updates here as access becomes restored.</p><p><small>Mar <var data-var='date'>29</var>, <var data-var='time'>23:00</var> CDT</small><br><strong>Investigating</strong> - We are currently performing an update that will prevent our customers from having the ability to login to the Lucky Orange app. These updates will take a few hours and we will provide updates as soon as access has been restored. Thank you for your patience.</p>tag:status.luckyorange.com,2005:Incident/158112432023-01-10T10:30:14-06:002023-01-10T11:59:27-06:00Real-time, live view, chat presence<p><small>Jan <var data-var='date'>10</var>, <var data-var='time'>10:30</var> CST</small><br><strong>Resolved</strong> - The updates have been completed and everything is operating as expected. We will continue to monitor and provide updates, as needed, moving forward.</p><p><small>Jan <var data-var='date'>10</var>, <var data-var='time'>09:04</var> CST</small><br><strong>Investigating</strong> - We are making some updates to enhance performance of these features. There may be instances over the next few days where connection to the app is inconsistent. We appreciate your patience as we perform these updates.</p>tag:status.luckyorange.com,2005:Incident/158036372023-01-09T11:23:23-06:002023-01-09T11:23:23-06:00Access to Lucky Orange App<p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>11:23</var> CST</small><br><strong>Resolved</strong> - All access has been restored. We appreciate your patience and please reach out to support at support@luckyorange.com if you continue to have trouble accessing your account.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>10:51</var> CST</small><br><strong>Update</strong> - The updates are taking a bit longer than expected, we will make sure to update everyone as soon as access is restored. Thanks for your patience.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>10:35</var> CST</small><br><strong>Investigating</strong> - We are currently restarting a few of our services and access to the app may be down momentarily. We will let you know as soon as access has been fully restored.</p>tag:status.luckyorange.com,2005:Incident/145268702022-12-06T17:20:44-06:002022-12-06T17:20:44-06:00Log in temporarily unavailable<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>17:20</var> CST</small><br><strong>Resolved</strong> - All systems are fully operational and we have put tests in place to catch this issue in case it occurs again in the future.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>16:47</var> CST</small><br><strong>Monitoring</strong> - All log in access has been restored. We are monitoring to make sure no furhter updates are needed.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>15:29</var> CST</small><br><strong>Investigating</strong> - We are currently investigating the issue causing customers to not have the ability to login to the Lucky Orange app. We will provide updates as soon as they are available.</p>tag:status.luckyorange.com,2005:Incident/145313922022-12-06T17:19:39-06:002022-12-06T17:19:39-06:00Launching heatmaps<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>17:19</var> CST</small><br><strong>Resolved</strong> - Heatmaps are again operational and will launch again without issue.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>16:49</var> CST</small><br><strong>Investigating</strong> - The ability to render heatmaps is currently stalling for some users. We are looking in to this issue and will update once we have more information.</p>tag:status.luckyorange.com,2005:Incident/108484222022-08-19T21:06:20-05:002022-08-19T21:06:20-05:00Issues with some app features<p><small>Aug <var data-var='date'>19</var>, <var data-var='time'>21:06</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>19</var>, <var data-var='time'>12:07</var> CDT</small><br><strong>Update</strong> - All features should be back to normal and working reliably. Some customers might see missing data in their dashboard temporarily as we ensure that everything else is working. Access to that data will be restored soon. Thank you for your patience.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>16:11</var> CDT</small><br><strong>Monitoring</strong> - We are investigating an ongoing issue with our database provider causing some features in the app to work unreliably. This includes funnels, filtering throughout the app, and certain dashboard cards. We apologize for the disruption and hope to have everything working smoothly soon.</p>tag:status.luckyorange.com,2005:Incident/108413242022-08-16T20:21:59-05:002022-08-16T20:21:59-05:00Data not being collected<p><small>Aug <var data-var='date'>16</var>, <var data-var='time'>20:21</var> CDT</small><br><strong>Resolved</strong> - The primary incident has been resolved and data is being collected properly. We will continue to investigate the cause and share more information soon.</p><p><small>Aug <var data-var='date'>16</var>, <var data-var='time'>19:34</var> CDT</small><br><strong>Identified</strong> - We have identified that both our primary and backup data collection queues were taken offline. During this time, data was unable to be collected by Lucky Orange and is unrecoverable. The backup queue has been moved to secondary infrastructure to mitigate the failure and is now collecting new data. We are still working to mitigate the failure of our primary queue.<br /><br />We are also still investigating what could cause a double outage like this. Additionally, our alerts that are in place to notify us when a queue is unavailable never triggered. Updates will be provided as we know more.</p><p><small>Aug <var data-var='date'>16</var>, <var data-var='time'>18:38</var> CDT</small><br><strong>Investigating</strong> - We are investigating an issue where data is not currently being collected from browser visitors.</p>tag:status.luckyorange.com,2005:Incident/108315742022-08-15T18:05:45-05:002022-08-15T18:05:45-05:00Slow API response times<p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>18:05</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>12:12</var> CDT</small><br><strong>Identified</strong> - We are currently investigating slow API response times when loading event and visitor data. This can cause the app to feel slow or not show data in some cases.</p>tag:status.luckyorange.com,2005:Incident/108199602022-08-13T18:30:34-05:002022-08-13T18:30:34-05:00Infrastructure changes<p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>18:30</var> CDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>12:31</var> CDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>13</var>, <var data-var='time'>12:27</var> CDT</small><br><strong>Scheduled</strong> - We will be making changes to the infrastructure that powers Lucky Orange. During this time, some data might not be visible in the app. This page will be updated once the changes have been completed.</p>tag:status.luckyorange.com,2005:Incident/102023822022-06-08T15:30:00-05:002022-06-09T09:41:27-05:00Missing events for customers with 30 day retention<p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>15:30</var> CDT</small><br><strong>Resolved</strong> - An ongoing issue with one of our vendors lead to misconfiguration in the database that stores events (sessions, pages, etc.) for customers with 30 day data retention. This caused our data collection system to incorrectly mark events destined for that database as ignorable and the events were silently ignored without being processed and stored. Because these events were silently ignored, no monitoring alarms were triggered as the system believed everything was working properly.<br /><br />Therefore, for a period of time yesterday, no events are available for customers with 30 day retention. We sincerely apologize for the missing data and have since added alerts to catch this scenario if it were to happen again. Additionally, we are taking steps internally to alter our logic for which events should be ignored to ensure that no data is lost in the future.<br /><br />Thank you for your support and understanding.</p>tag:status.luckyorange.com,2005:Incident/99668482022-05-11T11:17:39-05:002022-05-11T11:17:39-05:00Realtime connection disruption<p><small>May <var data-var='date'>11</var>, <var data-var='time'>11:17</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved. Thank you for your patience as we worked through this issue.</p><p><small>May <var data-var='date'>11</var>, <var data-var='time'>09:06</var> CDT</small><br><strong>Investigating</strong> - We are investigating an issue where some customers may experiene a disruption in their realtime connect. This may affect live visitors, chat, online status, and realtime sync functionality.</p>tag:status.luckyorange.com,2005:Incident/98655912022-04-27T13:45:35-05:002022-04-27T13:45:36-05:00Delayed processing for 30 day retention<p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>13:45</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved and no data was lost. Thank you for your patience as we worked through this issue.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>11:57</var> CDT</small><br><strong>Monitoring</strong> - The system is once again processing data for customers on 30 day retention. You should start to see new data appear in the app.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>11:30</var> CDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>11:06</var> CDT</small><br><strong>Investigating</strong> - We are investigating an issue where new data is delayed for some customers.</p>tag:status.luckyorange.com,2005:Incident/95953682022-03-21T13:33:51-05:002022-03-21T13:33:51-05:00Data processing delay<p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>13:33</var> CDT</small><br><strong>Resolved</strong> - All delays have now been resolved and access to the app should be returning to normal. We have identified the cause of the delay and put in place protections against it from happening again. Thank you for your patience as we worked through this!</p><p><small>Mar <var data-var='date'>21</var>, <var data-var='time'>11:03</var> CDT</small><br><strong>Monitoring</strong> - We have identified a delay in new data being processed. No data has been lost and our system is currently working through the backlog. All events, sessions, and visitors should be visible within the app soon. Thank you for your patience.</p>tag:status.luckyorange.com,2005:Incident/92525912022-02-07T07:17:57-06:002022-02-07T07:17:57-06:00Web App - Downtime<p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>07:17</var> CST</small><br><strong>Resolved</strong> - All login issues have been resolved. We will continue to monitor to make sure all fixes that have been made continue to work as expected.</p><p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>05:20</var> CST</small><br><strong>Update</strong> - We have identified the issue with login for Classic Lucky Orange and are working hard to resolve it shortly. We will continue to provide updates as soon as this has been resolved.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>19:54</var> CST</small><br><strong>Monitoring</strong> - We have identified what was preventing some users from being able to login to Lucky Orange and have pushed a fix that seems to have resolved the issue for customers on Lucky Orange. We are still working through an issue with our service provider that may be causing some people trouble when trying to login to Classic Lucky Orange. We will continue to monitor and provide updates as soon as we can.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>17:39</var> CST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>17:33</var> CST</small><br><strong>Investigating</strong> - We're currently experiencing an outage with a service provider that is preventing some people from being able to login and use the various services of Lucky Orange. Our development team is working to resolve the issue and restore service as quickly as possible.</p>tag:status.luckyorange.com,2005:Incident/90572212022-01-11T18:11:06-06:002022-01-11T18:11:06-06:00Delayed processing for 30 day retention<p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>18:11</var> CST</small><br><strong>Resolved</strong> - This incident has been resolved and no data was lost. We identified that a configuration change with one of our databases caused it to crash. We have updated the database to the correct configuration to prevent this issue from happening again. We are also looking into solutions to prevent the impact of similar situations in the future. Thank you for your patience as we worked through this!</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>15:38</var> CST</small><br><strong>Monitoring</strong> - We have implemented a fix and the system is now working through the backlog of data. At this time, no data has been lost and all sessions and events will be visible in the dashboard soon.</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>14:15</var> CST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>13:51</var> CST</small><br><strong>Investigating</strong> - We are investigating an issue where customers storing data for 30 days are having an issue loading data and new data is delayed.</p>tag:status.luckyorange.com,2005:Incident/63887182021-02-24T09:47:14-06:002021-10-01T15:47:21-05:00API currently unavailable<p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>09:47</var> CST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>09:39</var> CST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.luckyorange.com,2005:Incident/60173512021-01-13T15:42:35-06:002021-10-01T15:47:21-05:00API currently unavailable<p><small>Jan <var data-var='date'>13</var>, <var data-var='time'>15:42</var> CST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>13</var>, <var data-var='time'>15:36</var> CST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>