Infosec Insiders author Roy Dagan, CEO, SecuriThings
A crime occurs, police go to access video of the scene and then discover that crucial views are not available due to an outage or malfunction. This is precisely what the NYPD encountered in the recent subway shooting in New York City this past April when multiple surveillance cameras and video feeds failed at subway stations where the shooter was active. Should this be happening in 2022?
Unfortunately, outages of surveillance video are more common than you might think. There are many incidents that happen every year where having a working physical security system could have helped catch the perpetrators early.
But in many cities and enterprises there is a startling gap in maintenance of expensive security technology installed to protect the public. Equally surprising: this public safety problem is eminently fixable. In case after case, manual audits, planned periodic manual checks, and reliance on a variety of third parties simply don’t lead to accurate diagnostics or maintenance taking place.
The NYC case is not unique; it’s just the most recent high-profile example. The maintenance disconnect is seen in security systems nationwide. Investment in physical security for governments, transport systems, cities, hospitals, and many venues can reach millions of dollars per year. But what good is it to buy the equipment without organizing maintenance to be effective – to actually happen? This massive industry problem is, by extension, also a public safety and law enforcement issue.
Physical Security Doesn’t Get the Maintenance It Needs
It’s striking that hundreds – perhaps thousands – of cases of physical security system failure happen each year and remain in the shadows. What happened with the New York’s subway surveillance is most notable for one reason: it made news because it may have impeded identification of a terror suspect.
Cyber criminals routinely attempt to compromise IoT devices like networked cameras as entry points to attack IT networks. That’s just one reason cybersecurity must extend to connected devices used for physical security. But modern physical security infrastructure needs additional 24/7 automated protection and cyber management – beyond blocking hackers – to maintain 99% uptime and fulfill its physical security function.
In Atlanta, the most-surveilled U.S. city (49 cameras per 1,000 inhabitants), a 2021 murder in a park went unrecorded because nine cameras were down. Atlanta tries to have each of its 25,000-odd cameras checked twice a week, but is hampered by having different maintenance agreements with various providers and camera vendors for different makes. To its credit, Atlanta managed to drive down its not-working ratio down from 18% to 10%, but that is still twice its stated target.
Why Security Cameras Must Not Fail, and Why They Do
Every camera counts. One camera being down can mean no criminal identification. Multiple views are often necessary to secure arrests and convictions – making them far more important than 14-megapixel resolution. For identification purposes, a wide-angle view is rarely definitive. Conclusive identification usually comes from a narrow-angle view, so area coverage requires numerous cameras or mobile units.
Multiple cameras allow seamless tracking of a perpetrator from a crime scene, tracing a person from camera to camera to camera. That can lead to a vehicle or positive ID, or even enable police to establish stalking and intent, and identify collaborators. Knock one camera out of action, and the chain of identification can be broken.
There is another obvious reason physical safety depends on having a sufficient number of cameras up and running to catch multiple views: minutes and seconds of delay in identifying matter a great deal.
If a security device malfunctions when most needed, the cost to the organization can be very high. The price in human terms could even be higher. An assault on company property that goes unseen and unstopped by security guards because a camera fails as a result of insufficient maintenance, could directly lead to tragic outcomes for victims, and of course, lawsuits.
Video loss on surveillance security cameras is common. The causes can include severe weather, network issues, bad power supply, cabling problems, defective hardware, software bugs, IP address conflicts, and misconfiguration. And malicious hacking – or plain old vandalism.
Despite the wide range of root causes, the #1 cause of non-working video is simple lack of maintenance. Resource-constrained organizations may underestimate maintenance needs and costs. The fact is, many NVR and DVR systems fail within three years, so predictive maintenance is key. Since the unpredictable happens, it’s essential that system operators have fast, remote outage detection and diagnostic capability.
Manual maintenance checks are not scalable, and they are expensive. Atlanta recently estimated its cost of maintenance at $600 annually per camera, which is high. This shows the high cost of farming out maintenance to third parties that may do sporadic manual checks, rather than having automated, centralized control of devices and visibility into diagnostics.
In 2022, Maintenance of Physical Security Systems Should Not Be an Afterthought
It’s time for all facility operators, not just New York’s MTA, to adopt the technology that can manage and scale with their camera installations, and answer the urgency of central monitoring of their operation and security. NYC has 10,000 cameras in its subway system; the city of San Francisco has over 2,000 networked cameras overseen by neighborhood groups. Chicago has approximately 32,000 cameras.
Let’s picture this. After costly, laborious manual checks of surveillance cameras, a list of outages – often without accurate diagnostics – is handed upward, and is easily forgotten in a folder or desk drawer. By contrast, a centralized system that is constantly checking every camera, and doesn’t forget to alert management with accurate diagnostics when problems arise, is hard to ignore. Moreover, when it’s abundantly clear the system keeps track of just how long each camera was offline, quick action is incentivized. This type of management platform can also automate and carry out key security and maintenance operations such as firmware upgrades and password rotations that are essential to the health and uptime of each device.
These functions scale effectively when all the equipment is tracked and managed in one system; by contrast, a patchwork quilt of [manual] maintenance measures will have gaps and is proven ineffective..
For tasks like routine maintenance, full visibility is needed to know which cameras are down, and then determine why. Security staff and IT need remote diagnosis of malfunctioning units, to know whether an onsite technician visit is necessary. Given the technology available to manage and protect them today, all security cameras in public-facing and urban settings and transit systems should be properly secured, maintained, and managed – and they should work. It’s just too risky and costly to keep doing what does not work.
Roy Dagan is CEO and & Co-founder of SecuriThings—the provider of the first IoTOps solution designed to help organizations maximize their devices’ operational efficiency and security. He started the company after many years of building cyber security, risk management and intelligence systems. Prior to SecuriThings, Roy held multiple roles leading product management teams in a range of companies including RSA, The Security Division of EMC and NICE Systems.