Forensic search and analysis is a hugely powerful tool for security teams to better understand and investigate security footage. While there are several solutions on the market, Calipsa has been working hard behind the scenes to develop a forensic search solution with a difference.
Calipsa Investigate, part of Calipsa Pro Analytics, allows for site-wide, real-time investigation – without the need for live video streams. We use event-driven video alarm clips to analyse frames and identify certain activity before stitching together all relevant alarm images into a video loop, also known as a video summary.
We’ve asked two members of the Calipsa team to explain why forensic search is such an exciting new development with huge potential to improve security and video monitoring.
How and why does forensic search enhance security?
Toby Weatherall - Business Development Manager
In practice, this means if you see an incident in an alarm from one camera, you can then instantly pull alarms from all cameras on that site and review frames in real time to quickly investigate and put a stop to an ongoing incident. Let’s look at an example.
Imagine a man throws a brick through a shop window of a shopping centre. The internal control room can use Calipsa Investigate to identify all instances of that individual on alarms from other cameras in the shopping centre. A feature called Similarity Search would be incredibly useful in this scenario, and in many others.
Using this feature, the investigator could select the individual or vehicle they wish to find in the original alarm, in this instance it would be the man who threw the brick, and Calipsa will do the rest. We will pull up all alarms across the site that show matching or similar results to the individual. To do this, we use data points such as style, colour and shape of clothing or if it’s a vehicle we’ll be looking at shape, type and colour.
You don’t even need an alarm of the person or vehicle for this solution to work. Let’s imagine another example. Someone has reported seeing a white transit van acting suspiciously. All the security personnel would need to do in this instance is find an image of a generic white transit van online, paste it into Calipsa, and Calipsa will then display all events with a similar vehicle from cameras on that site.
With this tool at your fingertips, you can quickly identify trends and patterns in the vehicle movement as part of your investigation, before proactively implementing measures to prevent any crime from taking place.
The last thing that sets this feature apart is that you don’t need to have alarms sent to Calipsa to benefit. Our video uploader allows you to upload video footage from outside Calipsa, so that our forensic search tool can quickly and accurately comb through hours of video - saving hours if not weeks of manual human review. As Business Development Manager at Calipsa, I’m extremely excited about what this product can achieve and how we can help security teams to get more value out of their existing security systems with Calipsa Investigate.
Want to see how forensic search works in action? Check out our Calipsa Investigate product demo:
How does forensic search (or similarity search) work?
Ashwin D’Cruz - Senior Machine Learning Engineer
So, we want to link objects of the same identity together. For example: given a few different images of the same person, we want to be able to group them together. We'll focus on people in this case, but the process we go through works for vehicles too.
Linking instances of the same person can be tricky because they might not always look the same. The person could have a different pose, the lighting conditions could vary, or maybe there's something in the way that's blocking our view of the person. On the other hand, we might have images of other people which look very similar to the person in question (e.g. someone else wearing the same uniform). We need to ensure that we don't mistakenly label those other people with the same identity.
How do we use machine learning to solve this problem? We begin by collecting a dataset of images. Ideally this dataset will consist of many different identities with a variety of images for each identity. We then set up our algorithm to generate a summary for each image. Now, while humans might use descriptors like 'black hair', 'red shirt', or 'glasses' when describing a person, our algorithm instead generates a set of numbers. While this might seem strange, what's important about using a set of numbers is that we can now calculate distances between the different summaries.
Let's say we had 3 images. Two of these are different viewpoints of Person A while the third image shows a different person, Person B. We tell the machine that the summaries generated for Person A should have a smaller distance between them compared to the distance to the summary of Person B. By penalizing the machine when this isn't the case, it learns to update the way it generates summaries so that different identities are further apart from each other in this space of summaries. Once the machine is finished learning, it will be really good at generating these distinctive summaries.
Here, the algorithm was supplied with the image on the far left hand side, and has managed to find several very close summaries - even when the bag is not visible in the picture.
So with this machine that's good at generating summaries, we get it to process all the data that we already have and save these sets of summaries and the identities each is linked to. Now, when we see a new image, we create a new summary and measure the distance between this summary and the ones we have in our database.
If the new summary is close enough to any of the existing ones, we can assume that the new image is of the same person behind those summaries. Once we have that identity, we can pull all the older images of that person to look at the history of appearances for that person.
Thanks Toby and Ash for explaining the what, how and why of forensic search. Want to see Calipsa Investigate in action? Click on the button below where you can watch Toby's Investigate product demo:
No comments