Person of Interest - the workings of Video Analytics

Up to now, we have discussed quite a few interesting applications and points on visual analytics. But we have received some feedback asking to explain how it works. So here goes!

The basis of visual analytics, a practice of using computers to identify things or people of interest without an operator or a person having to view the video, are mathematical algorithms that monitor, analyze and manage big volumes of video. This means that video inputs are digitally analyzed and transformed into data that help people take decisions. It can be real time to help alert to specific incidents that could happen, such as an illegal suspect fleeing the country, and so on, or they can be post-event, looking incidents that have already occurred, for example a bombing or a fire in a store.

VA can happen in the centre, for example a central monitoring station, at the "edge", built into cameras which is ideal to locate live happenings, or as a combination of both. The optimal place to locate post-event analytics is on a central server as it is possible to search a large amount of recorded video for events, however this requires network capacity and processing power. But the main point on VA is that it works on motion detection and pattern recognition.

Motion Detection is basically examining each pixel in a given frame to detect the slightest movement. Pattern Recognition, on the other hand, recognizes specific patterns within a frame. These patterns or objects can be programmed and i.e, should a change happen or an object go missing the software immediately recognizes and sounds an alert.

The most common used types of video analytics used is violation of perimeters and recognition of license plates. Not only can this be used in surveillance, but in retail segment market analysis, to measure the number of people that pass in a given time (footfall), gender counting, shelf space effectiveness, and so on.

A fun and different way to see a little bit of VA in action, is watching a series called Person of Interest. It's just a TV show... but is it really?

Sources:
http://www.mistralsolutions.com/hs-downloads/tech-briefs/aug09-article-1.html
http://www.addic7ed.com/serie/Person_of_Interest/2/22/God_Mode
http://alexandra135.deviantart.com/art/Person-of-Interest-349555095

Visual Analytics: Challenges that lie ahead

It all started in 2005, when an international team of inter-disciplinary panelists from IEEE CG&A coined the definition of term visual analytics as, "Science of analytical reasoning facilitated by interactive visual interfaces". Visual Analytics and its applications in various fields of research, security and business have come a long way since then.

Today in 2013, as we take a look at various aspects of visual analytics through our blog, an obvious question that comes to our minds is, what are the potential challenges that lie ahead for this promising sector? Can these challenges be overcome?

Lets take a brief overview of all this and more in my current post.

First and the foremost of the challenges for visual analytics going forward is to develop in-situ analysis capabilities. With the volumes of data to be processed increasing by minute, the traditional approach of storing this data into conventional database and then processing it may be a major bottleneck. High Performance Computing of data, even while it is in the memory in the need of this day and visual analytics platforms need to incorporate this requirement while coming up with new upgrades.

Another major challenge comes in terms of limitations in terms of human interaction and user interfacing of visual analytics systems. The challenges in this regards are multiple, ranging from handling heterogeneous inputs to scalability to accurate representation of all data uncertainties and evidences, all the way up to limitations of human cognitive abilities to process the visual analytics data for accurate interpretations.

Speaking of human and system performance limitations, another major challenge is also of storage of huge data that visual analytics takes in. With all companies preferring cloud based storage, data required to be processed by visual analytics tools that are deployed by say Facebook is huge and requires not only huge cloud based storage, but also a capability to maintain this storage online at all times. Various algorithms which process this data in cloud, are also many a times based in the cloud and updating and maintaining these algorithms is another challenge that comes along with this.

With these huge scales of data that we are now talking about, when it comes to future and current challenges, there is also a regulatory aspect involved with various governments restricting the flow and access of data between countries and regions. These government regulations impose new constraints on efficient processing logic needed.

Well, well, well .. so much to talk about when it comes to challenges that lie ahead. So, can these be overcome?

Why not? With efficient combination of High Performance Computing [HPC] and parallel/multi-thread processing, many of the computing issues can atleast be nailed down. Government regulations cannot be helped and processing logic and storage and data retrieval mechanisms will have to find the most efficient way around.

When it comes to human cognitive skills though, who knows we may soon have to defrost the Super-Human "Kahn" and his crew that the Star Trek folks has recently succeeded to freeze into state of coma :-D

Facebook Tag Suggestions: A Value-Add or a Tresspass??

The debate continues. When Facebook first came up with a facility to allow tagging friends, it was a labor-some task to click every photo, then click on the "Tag Friends" option and then click on every face you see [or you want to tag], to type the name of that person.

Well, it takes less than a minute for one tag, but if this is a group photo of an entire class, or a wedding photo with all cousins, boy !!
It would take hours just to tag people in all the photos.

And why do we want to do that in the first place? Because, we always want to announce to the world that we were not alone when we did something crazy or something stupid or something great. We were with "Someone".
Being a part of "the group" is one of the needs, which is high up in Maslow's pyramid of needs.

And if the social networking corporations of today, would not have tapped into this desire of people and turned into a business value-add, it would have been a big surprise for me.

Anyways, even while people were shying away from tagging friends for the amount of work it involved, Faccebook came up with a smart way to tag friends. An excellent use of Visual Analytics - where all new faces are compared with the previous tags you have added, or the previous tags your friends have added, and even before you publish the album or photo, you are given a ready platter with all those smiley faces and a tag "suggestion" for each of this face.

The facial recognition software which most of the social network sites use is not very complex algorithm using too many biometrics, as we saw in the previous post by Sophie. They work on simple Smile recognition, a feature which is commonly available in many handy-cams, digi-cams and photo-editing software. The Analytics part comes into picture when the available database of tagged photos is processed to map with the new image under consideration. Output of running this analytics is a name of person - either your own name or name of one of your friend - commonly termed as a "suggestion".

All of this process gets completed within a few seconds, irrespective of whether you are posting a single photo or hundreds of photos. This speed of video analytics that facebook offers, is the best feature of it, as per my opinion.

This suggestion may sometimes be erroneous though, specially in case of newly added friends without any history of photo tags, but that's a very low percentage. All-in-all, if you like tagging your friends, this is a great feature available at your disposal.

On the fall-side, Every new feature has its own ill-effects and the major ill-effect of facebook tag suggestions is the tress-passing it does in someone's private space. I mean if I don't like being tagged, why should my face appear in my friends album, with a suggestion of my name?

Thankfully, after a ruckus in US about this privacy encroachment, Facebook withdrew the feature in US, and introduced it back, only after adding options [for the entire world], where-in you can deny being tagged by others; or have an authority to review and choose whether your name tag appears on a photo; or whether your name should appear as a tag suggestion.

Well, that takes care of personal privacy-trespassing issues. All-in-all there is no denying that visual analytics is the way ahead - not only for handling the security issues, but also for effectively using the social networks, considering the fast-paced lifestyle that we have.

On a more personal note, when I have enough time, I still prefer choosing whom to tag on a per-photo basis, as when I am browsing through those photos one by one and tagging people one after other, I can relive the memories associated with each picture and each person in that picture. But hey, that's just me :-)