SERVICE STATUS UPDATE
On Friday September 8th, an engineer was fixing a bug on our online web viewing portal, view.wyze.com. In the process of deploying the fix, the wrong cloudfront caching setting was selected. Simply put, it crossed some wires in the backend and, for about 40 minutes, up to 2,300 users who logged in to the online web viewing portal may have seen cameras from one of the 10 affected users who had also logged in during that time.
When we discovered the incident, we immediately took down view.wyze.com to investigate and resolve the issue. View.wyze.com was back online a few hours later.
We want to make it absolutely clear that it did not affect the Wyze app or the 10M+ users who only access Wyze products through the Wyze app. The web portal view.wyze.com is a separate viewing experience behind a paywall.
Here’s what we’re doing to rectify the situation and prevent it from happening again. So far we’ve:
- Conducted a detailed investigation. Due to the low amount of traffic to this site we were able to analyze page traffic in detail and know exactly 10 users were affected.
- Provided as much detail as we could reliably confirm as it was unfolding in real time, including on Reddit, Facebook, Wyze Forum, core communities, our website and answering questions from the press.
- Notified the 10 users that their accounts were affected.
- Further limited account permissions, updated company policies, updated training for Wyze employees, and implemented other technical fixes including additional admin alerts so that this doesn’t happen again.
- Hiring an external security firm to do further penetration testing of Wyze systems and processes.
Security is a core focus for us here at Wyze. We have built a dedicated security team and continually invest millions of dollars into security to keep our customers safe. We made a mistake here and will take all the appropriate steps to make sure it doesn’t happen again. We especially apologize to the 10 affected users and any users who signed into the web portal during this time.
9/13/23 10:00 AM PT - We wanted to provide an update as we have continued to investigate the matter through the weekend. We have identified and notified the 10 users whose camera events may have been viewed by others who were logged into view.wyze.com during that brief period of time on Friday afternoon. We also adjusted the website so it no longer logs users out after 15 minutes of streaming and will stream as usual. We are continuing to investigate this issue and we have implemented multiple technological and policy measures in an effort to prevent this from occurring in the future. Again, this experience does not reflect our commitment to users or the investments we’ve made over the last few years to enhance security. We apologize for this incident.
This was a web caching issue and is now resolved. We continue to investigate and believe no more than 10 users were affected, and all will be notified.
For about 30 minutes on Friday afternoon, a small number of users who used a web browser to log in to their camera on view.wyze.com may have seen cameras of one of the 10 users who also logged in through view.wyze.com during that time frame. The issue DID NOT affect the Wyze app or users that did not log in to view.wyze.com during that time period.
Once we identified the issue, we shut down view.wyze.com for about an hour to investigate and fix the issue.
We have enacted numerous technical measures to prevent this from occurring in the future.
This experience does not reflect our commitment to users or the investments we’ve made over the last few years to enhance security. We are continuing to investigate this issue and will make efforts to ensure it doesn’t happen again. We’re also working to identify and notify affected users.
We will let you know if there are any further updates.
3/8/23 10:41 PM PT - The latency is back to normal and devices should all be functioning properly again. If you have a device that isn't connecting still, please try power cycling it. If you continue having difficulty logging in, using your Wyze devices after power cycling them, or loading Event Videos, please contact Wyze Customer Support.
Hey friends,
We successfully completed a large portion of the maintenance tonight. Your devices may have been offline between 12:00 AM and 12:15 AM Pacific. There is another part planned for tomorrow but the customer impact is expected to be minimal.
Your Wyze services and devices should be back online and working normally. Thank you for your patience during this process.
If you continue to experience issues, please contact our support team.
Best,
Your friends at Wyze
February 8th 12:00 AM PT Scheduled Maintenance
- Users will not be able to log in, sign up for accounts, or use the app.
- wyze.com, forums.wyze.com, view.wyze.com and services.wyze.com will go into maintenance mode at the same time.
- Event Videos will not be uploaded to the cloud during this time.
- Notifications from during that time will not be sent.
- Rules that would be triggered by Wyze Sense in the maintenance window will not work.
- Event Videos will not be uploaded to the cloud.
- AI tags will not be generated.
- Notifications from that time will not be sent.
- Event Videos will not be uploaded to the cloud.
- AI tags like Person Detection will not be generated.
- Notifications from that time will not be sent.
- Arming and disarming the system will not work.
- Alarms will not trigger the siren on the Wyze Cam.
- Noonlight will not be contacted.
- Event Videos will not be uploaded to the cloud.
- AI tags will not be generated.
- Notifications from that time will not be sent.
- Arming and disarming through the app will not work.
- The physical Wyze Sense Keypad will still be able to cancel alarms and arm/disarm your system.
- Alarms that occur during the maintenance window will trigger the Wyze Sense Hub siren.
- Noonlight will not be contacted.
- Keypads will still work without the cloud service.
- The fingerprint scanner will still work without the cloud service.
- Unlocking with the app and auto-unlock WILL NOT function during the maintenance window.
- The thermostat itself can work without internet but heating/cooling usage data and HVAC filter life data will be lost.
- Geofencing may not work.
- The router will continue to function.
- Some setting changes will be unavailable.
1/30/23 11:20 AM PT - We are now fully recovered and this outage is resolved. If you continue to have trouble with your Wyze device connecting, please try power cycling it. If that doesn't work, please reach out to Wyze Customer Support.
11:13 AM PT - We are unblocking the last bit of traffic and things are looking good. We're going to keep an eye on the metrics and make sure they're behaving as expected before we call this fully resolved.
10:49 AM PT - We're continuing to allow more traffic back to the servers and are watching our metrics. We are not fully recovered yet and appreciate your patience.
10:19 AM PT - We are entering recovery and are balancing the requests from the server side. Some people may see improvement now but the process is not yet complete.
9:25 AM PT - We are looking into reports of Wyze app not loading, cameras not showing, and Wyze Webview not loading. We will have more info when it becomes available.
1/24/23 5:48 PM PT - The setup and event issues impacting Wyze Cam OG and Wyze Video Doorbell Pro should be recovering. If you are still experiencing troubles please reach out to Wyze Customer Support.
4:35 PM PT - We are looking into reports of failed setup and event issues impacting Wyze Cam OG and Wyze Video Doorbell Pro. We will update this page with more info when it becomes available.
1/12/23 4:14 PM PT - We now see full recovery and we're keeping an eye on things. If your device is still offline, please try power cycling it. If you continue having difficulty connecting, please reach out to Wyze Customer Support.
3:00 PM PT - We have seen increasing server latency from our side and are hearing reports from customers that their apps aren't loading pages or controlling devices correctly. We are working on mitigation efforts and expect things to improve for affected customers soon.
1/11/23 4:00 PM PT - We have released iOS app version 2.38.4.3 to the public. This has the fix for Error Code -1. We're sorry that this bug escaped to the public and we've added new processes to prevent it from happening again. If you're still experiencing this, please check App Store for an update. If you continue to have trouble, please reach out to our Wyze Customer Support.
1/10/23 1:51 PM PT - Our QA team has verified the hotfix app version and we're working on App Store review. Ideally, we'll be releasing the fix for Error Code -1 tomorrow though it may release on Thursday instead. We'll keep you posted about any changes.
1/9/23 3:46 PM PT - We have received confirmation of a new app build that has a fix for the Error Code -1 iOS bug. We are currently waiting for it to go through code merge and QA review. Thank you for bearing with us as we work to resolve this problem.
1/8/23 4:21 PM PT - We have been making progress on a new app build to replace the problematic iOS app version. This is currently going through internal testing and looks promising for resolving the Error Code -1 issue. We appreciate your patience during this time. If needed, ungrouping your cameras can avoid the Error Code -1 issue while we work on the new app build release.
1/6/23 1:45 PM PT - We're seeing reports of grouped cameras giving a frequent Error Code -1 in the live stream when using iOS app version 2.38.3.3. We are looking into this. In the meantime, removing the cameras from the group can help. We apologize for the inconvenience.
1/5/23 6:00 PM PT - Our devices are back at their normal connection metrics. If you're still having trouble with your app, please try force closing the app and opening it again. If a device is not connecting, please try power cycling it. If those steps don't do the trick, our support team at support.wyze.com are here to help.
4:29 PM PT - Our recovery is still in progress and more devices are connecting to our servers. We're not there yet but our metrics continue to improve and we're hearing positive feedback from customers about their devices being back online.
3:06 PM PT - At this point, our metrics show that all devices except Wyze Cam v2 and Wyze Cam Pan have recovered. We are continuing to work on recovering the remaining devices.
2:39 PM PT - We have scaled our services and blocked repeat requests from devices sending too many requests to our servers. We are seeing improvement in app and device performance. We'll start slowly opening up access again with the goal of keeping up with the requests. We expect some devices such as Wyze Cam v2, Wyze Cam Pan, and some Wyze Cam v3 to still have difficulty connecting at this time but other devices are showing signs of improvement. Thank you for your patience.
1:52 PM PT - We are looking into issues with devices not reporting Events to the cloud and the app not being responsive. This can affect notifications due to the cloud not receiving the Event information. We'll update this page when we know more.
View Archived Service Advisory Events
View Software and Firmware Known Bugs