tag:status.snapsheetclaims.com,2005:/historySnapsheet Status - Incident History2024-03-28T07:51:42-05:00Snapsheettag:status.snapsheetclaims.com,2005:Incident/199870632024-02-14T16:31:49-06:002024-02-14T16:31:49-06:00Twilio SMS Incident<p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>16:31</var> CST</small><br><strong>Resolved</strong> - Twilio Reported: All services impacted by the issues with the Twilio Rest API have recovered and are operating as intended. This incident has been resolved.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>15:59</var> CST</small><br><strong>Update</strong> - Twilio: We're seeing recovery in the Twilio Rest API, impacting multiple Twilio services.<br /><br />We will be monitoring all services to ensure a full recovery and provide another update in 30 minutes or as soon as more information becomes available.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>15:42</var> CST</small><br><strong>Monitoring</strong> - Twilio reported that they have identified and are working to fix an issue around SMS. We are monitoring the incident on our end and will post any updates here.<br /><br />Twilio Service Status: https://status.twilio.com/</p>tag:status.snapsheetclaims.com,2005:Incident/191088112023-12-09T22:00:03-06:002023-12-09T22:00:03-06:00Production Database Maintenance<p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>22:00</var> CST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>21:01</var> CST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>13</var>, <var data-var='time'>09:56</var> CST</small><br><strong>Update</strong> - We will be undergoing scheduled maintenance during this time.</p><p><small>Nov <var data-var='date'>13</var>, <var data-var='time'>09:55</var> CST</small><br><strong>Scheduled</strong> - We will be performing upgrades on one of our internal databases during the posted time. During this time, our platform will be inaccessible.</p>tag:status.snapsheetclaims.com,2005:Incident/190628662023-12-04T15:00:05-06:002023-12-04T15:00:06-06:00UAT Database Maintenance<p><small>Dec <var data-var='date'> 4</var>, <var data-var='time'>15:00</var> CST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 4</var>, <var data-var='time'>14:00</var> CST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>15:37</var> CST</small><br><strong>Scheduled</strong> - We will be performing upgrades on one of our internal databases during the posted time. During this time, our platform will be inaccessible.</p>tag:status.snapsheetclaims.com,2005:Incident/181538222023-08-22T10:15:07-05:002023-08-22T10:15:07-05:00Security Policy Updates<p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>10:15</var> CDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>10:00</var> CDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>16</var>, <var data-var='time'>10:41</var> CDT</small><br><strong>Update</strong> - In order to enable partners to prepare for this transition, we have applied this update our external facing UAT testing environments. This can be used to validate successful API connections in advance of next week's update.</p><p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>11:00</var> CDT</small><br><strong>Scheduled</strong> - At 10am Central Time on August 22nd, Snapsheet will perform a network upgrade to ensure the security of all communication of our applications and services. After this point, CBC ciphers will be excluded. If any integrations use libraries that connect to Snapsheet services use the following ciphers, they will begin to see TLS handshake failures:<br />ECDHE-RSA-AES128-SHA256 (IANA Name: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256)<br /><br />ECDHE-RSA-AES256-SHA384 (IANA Name: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384)<br /><br />After the update, Snapsheet will only support communication via the following TLS protocols and ciphers:<br /><br />Permitted Protocols<br />- TLSv1.2<br />- TLSv1.3<br /><br />Permitted Ciphers<br />- TLS-AES-128-GCM-SHA256<br />- TLS-AES-256-GCM-SHA384<br />- TLS-CHACHA20-POLY1305-SHA256<br />- ECDHE-ECDSA-AES128-GCM-SHA256<br />- ECDHE-RSA-AES128-GCM-SHA256<br />- ECDHE-ECDSA-AES256-GCM-SHA384<br />- ECDHE-RSA-AES256-GCM-SHA384</p>tag:status.snapsheetclaims.com,2005:Incident/181793622023-08-17T21:52:50-05:002023-08-17T21:52:50-05:00Increased SMS Error Rates<p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>21:52</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>19:38</var> CDT</small><br><strong>Monitoring</strong> - Our vendor has reported an issue and is currently seeing elevated error rates, causing SMS delivery delays. We will continue to monitor the issue and provide updates as they become available.</p>tag:status.snapsheetclaims.com,2005:Incident/179163082023-07-24T09:14:06-05:002023-07-24T09:14:06-05:00Increased SMS Error Rates<p><small>Jul <var data-var='date'>24</var>, <var data-var='time'>09:14</var> CDT</small><br><strong>Resolved</strong> - Our SMS provider has updated us stating this has been resolved. We will continue to monitor and provide any updates if there is a change.</p><p><small>Jul <var data-var='date'>21</var>, <var data-var='time'>13:32</var> CDT</small><br><strong>Update</strong> - We continue to see a small number of SMS delivery failures. We are continuing to engage with our SMS vendor to determine the resolution.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>21:04</var> CDT</small><br><strong>Monitoring</strong> - We continue to work with our vendor, who has identified the issue and pushed a fix to resolve the issues with SMS delivery. We will continue to monitor the situation and provide any updates as needed.</p><p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>16:50</var> CDT</small><br><strong>Investigating</strong> - We are seeing an increased error rate for SMS delivery. We are currently investigating and will provide additional updates once available.</p>tag:status.snapsheetclaims.com,2005:Incident/175646532023-06-13T19:21:40-05:002023-06-13T19:21:40-05:00Informational Announcement<p><small>Jun <var data-var='date'>13</var>, <var data-var='time'>19:21</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>13</var>, <var data-var='time'>15:38</var> CDT</small><br><strong>Monitoring</strong> - We are monitoring an ongoing AWS outage. Currently, none of our services are affected. <br /><br />We are posting this to raise awareness. We will continue to monitor and share updates as they become available.</p>tag:status.snapsheetclaims.com,2005:Incident/159766852023-01-30T13:18:07-06:002023-01-30T13:18:07-06:00Issues with Search Functionality<p><small>Jan <var data-var='date'>30</var>, <var data-var='time'>13:18</var> CST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>30</var>, <var data-var='time'>12:49</var> CST</small><br><strong>Monitoring</strong> - A fix has been implemented, and we are monitoring the results. The system appears to be operational.</p><p><small>Jan <var data-var='date'>30</var>, <var data-var='time'>11:59</var> CST</small><br><strong>Identified</strong> - We have identified an issue with the search functionality and are currently looking into a resolution.</p>tag:status.snapsheetclaims.com,2005:Incident/144211572022-12-07T10:09:48-06:002022-12-07T10:09:48-06:00Issue when trying to load photos and documents within a claim (AWS)<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>10:09</var> CST</small><br><strong>Resolved</strong> - Our workaround has resolved this issue, and we are continuing to work with AWS for an RCA</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>12:48</var> CST</small><br><strong>Monitoring</strong> - We’ve rolled out a workaround to mitigate this issue as we are still working with AWS.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>08:34</var> CST</small><br><strong>Identified</strong> - We have identified and are currently testing a workaround. We are still working with AWS to obtain an RCA and resolution to the original issue. We will continue to provide updates as they become available.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>22:00</var> CST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>21:58</var> CST</small><br><strong>Update</strong> - We spoke with a network specialist at AWS who will be working with his team to bring all eyes in AWS on this issue as we continue to try and identify the source of the disruption we are seeing when accessing files in vice-backend production. They will continue the investigation later tonight and also tomorrow morning.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>16:07</var> CST</small><br><strong>Update</strong> - We've found more information that suggests the problem is in the AWS network handling some IPv6 requests to files in vice-backend production. The AWS S3 team is helping to investigate.<br /><br />No ETA at this time.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>14:36</var> CST</small><br><strong>Update</strong> - The Snapsheet Team is still troubleshooting with AWS to resolve the issue. AWS has escalated the issue, and we are awaiting feedback from their Engineers.</p><p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>10:07</var> CST</small><br><strong>Investigating</strong> - Some users are unable to load photos and documents within a claim. We are currently investigating and following up with AWS.</p>tag:status.snapsheetclaims.com,2005:Incident/107118972022-07-28T17:13:31-05:002022-07-28T17:13:31-05:00SmartCommunications/AWS Issue<p><small>Jul <var data-var='date'>28</var>, <var data-var='time'>17:13</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'>28</var>, <var data-var='time'>14:09</var> CDT</small><br><strong>Monitoring</strong> - Summary of Issue:<br />- Time/Date Issue Identified --> Thursday 7/28/22 at approximately 1pm Central Time<br /><br />- Description of Issue --> Users are unable to access the SmartCommunications Draft Editor and are receiving a loading error message. <br /><br />- Next Steps --> This issue was caused by an AWS outage. Snapsheet and SmartCommunications are actively addressing this matter. SmartCommunications informed Snapsheet that the issue is being fixed, there has been improvement, and that the issue could be fully resolved in about 1 hour. Snapsheet will share another update once new information is available. We're terribly sorry for the issue. <br /><br />- Link Regarding AWS Outage --> https://seekingalpha.com/news/3862592-amazon-investigating-aws-outage-in-us-east-region?utm_source=feed_news_all&utm_medium=referralhttps://seekingalpha.com/news/3862592-amazon-investigating-aws-outage-in-us-east-region?utm_source=feed_news_all&utm_medium=referral<br /><br />- Resolution Reached --> This issue was fully resolved at approximately 5pm Central Time</p>tag:status.snapsheetclaims.com,2005:Incident/101324812022-06-02T12:57:50-05:002022-06-02T12:57:50-05:00Details Not Appearing in Claim File's "History" Section<p><small>Jun <var data-var='date'> 2</var>, <var data-var='time'>12:57</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 2</var>, <var data-var='time'>09:18</var> CDT</small><br><strong>Investigating</strong> - Summary of Issue:<br /><br />Time/Date Issue Identified --> Thursday 6/2 at 7:53am CT<br /><br />Description of Issue --> Upon visiting a claim file's "History" section, details aren't appearing in the default "Claim History" view. <br /><br />Workaround --> If you visit the exposure's "History" section, you could see the entries for notes, etc. <br /><br />Next Steps --> Snapsheet is actively investigating this matter and we'll share another update ASAP. We're terribly sorry for the issue.<br /><br />Resolution Reached --> At approximately 12:40pm CT the fix was deployed to the impacted environments.</p>tag:status.snapsheetclaims.com,2005:Incident/99231892022-05-05T17:19:15-05:002022-05-05T17:19:15-05:00Photos/Documents Not Appearing in "Photos and Documents" Section of Claim File<p><small>May <var data-var='date'> 5</var>, <var data-var='time'>17:19</var> CDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>14:04</var> CDT</small><br><strong>Identified</strong> - Summary of Issue:<br /><br />Time/Date Issue Identified --> On Thursday 5/5 at 9:18am CT<br /><br />Description of Issue --> Upon initially visiting the "Photos and Documents" section of the claim file, photos/documents aren't appearing in the default view. <br /><br />Workaround --> In the "Photos and Documents" section, if you change the view by clicking on the square icon that is below the "Download" button and off to the right, you should be able to view the photos/docs. Also, photos/docs could potentially be available in the exposure within the "Photos and Docs" tab.<br /><br />Next Steps --> Snapsheet is preparing a fix and planning on deploying the fix, today.<br /><br />Resolution Reached --> At approximately 4:40pm CT the fix was deployed to the impacted environments.</p>tag:status.snapsheetclaims.com,2005:Incident/92670472022-02-08T14:38:24-06:002022-02-08T14:38:24-06:00Snapsheet Claims - Slower Performance<p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>14:38</var> CST</small><br><strong>Resolved</strong> - This issue has been resolved and we're terribly sorry for the issue.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>11:54</var> CST</small><br><strong>Monitoring</strong> - Snapsheet deployed a fix and is monitoring the situation.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>10:38</var> CST</small><br><strong>Identified</strong> - Summary of Issue<br /><br />- Time/Date Issue Identified --> Tuesday 2/8 at 8:09am CT<br /><br />- Description of Issue --> When trying to access Snapsheet Claims, some users may encounter a blank screen and/or the page(s) load slowly. <br /><br />- Next Steps --> Snapsheet is actively addressing this matter and is preparing a fix for the issue. Snapsheet will provide updates as new details become available.</p>tag:status.snapsheetclaims.com,2005:Incident/91534142021-12-07T10:30:00-06:002022-01-24T10:40:36-06:00Issue Identified - AWS Outage<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>10:30</var> CST</small><br><strong>Resolved</strong> - Summary of Issue:<br />- Time/Date Issue Identified --> On Tuesday 12/7/21 at 10:30am CT<br /><br />- Description of Issue --> Snapsheet became aware that AWS experienced an outage which in turn caused an issue with the Snapsheet Claims Management System. Pages are loading slowly and some users may not be able to access the Claims Management System.<br /><br />- Next Steps --> Snapsheet is actively monitoring this situation and will post subsequent updates as new information becomes available. AWS hasn't shared when this issue will be resolved, yet. <br /><br />Resolution Reached:<br />- The morning of 12/8/21, AWS announced the issue was fully resolved.</p>