Visual Analytics: the journey so far ..

A bunch of International MBA students : diverse in terms of countries of origin, diverse in terms of languages they speak, diverse in terms of professional backgrounds they come from - when this lot of ours came together in Group C and ventured into this exciting journey of exploring the latest happenings in an upcoming technology of Visual Analytics, we discovered so many things.

We understood how Visual Analytics works, we explored its applications in numerous sectors, we wondered on how it is helping many people from athletes to scientists alike, what are the possible challenges with this technology and what pros and cons it presents.

When we came across this information, we shared it on this blog, and we made every effort to make this blog accessible and visible to as many people as we can.

We added Plus-One's on Google-Plus, we shared the posts on our Facebook profiles, we created a facebook page [https://www.facebook.com/seeie] to promote it all at one place.

But, that was not the only thing we did.

We also played with the free Analytics tool made available by Google in form of Google Webmasters. In here, we submitted our blog website for indexing, and for Google bots to crawl around it.
This configuration gave us the Search keywords, which we used to label our posts with. This site also provided us with special tags to insert into the HTML of our blog for Search Engine Optimization [S.E.O.].

We think results of this entire effort by the group - to explore and post about different aspects of Visual Analytics, to publicize what we posted on Google Plus and Facebook, and to use modern day technology of web analytics and search engine optimization to make the blog more visible in Google searches - are paying off eventually. In around two months time, our viewership has already crossed 1700 mark and the count in on the rise everyday.

Before we conclude this journey so far, here are a couple of snapshots of the SEO and web analytics tool that we deployed. Who knows, may be some day our readers can try these settings themselves and benefit from these tools just like we did :-)

Person of Interest - the workings of Video Analytics

Up to now, we have discussed quite a few interesting applications and points on visual analytics. But we have received some feedback asking to explain how it works. So here goes!

The basis of visual analytics, a practice of using computers to identify things or people of interest without an operator or a person having to view the video, are mathematical algorithms that monitor, analyze and manage big volumes of video. This means that video inputs are digitally analyzed and transformed into data that help people take decisions. It can be real time to help alert to specific incidents that could happen, such as an illegal suspect fleeing the country, and so on, or they can be post-event, looking incidents that have already occurred, for example a bombing or a fire in a store.

VA can happen in the centre, for example a central monitoring station, at the "edge", built into cameras which is ideal to locate live happenings, or as a combination of both. The optimal place to locate post-event analytics is on a central server as it is possible to search a large amount of recorded video for events, however this requires network capacity and processing power. But the main point on VA is that it works on motion detection and pattern recognition.

Motion Detection is basically examining each pixel in a given frame to detect the slightest movement. Pattern Recognition, on the other hand, recognizes specific patterns within a frame. These patterns or objects can be programmed and i.e, should a change happen or an object go missing the software immediately recognizes and sounds an alert.

The most common used types of video analytics used is violation of perimeters and recognition of license plates. Not only can this be used in surveillance, but in retail segment market analysis, to measure the number of people that pass in a given time (footfall), gender counting, shelf space effectiveness, and so on.

A fun and different way to see a little bit of VA in action, is watching a series called Person of Interest. It's just a TV show... but is it really?

Sources:
http://www.mistralsolutions.com/hs-downloads/tech-briefs/aug09-article-1.html
http://www.addic7ed.com/serie/Person_of_Interest/2/22/God_Mode
http://alexandra135.deviantart.com/art/Person-of-Interest-349555095

Visual Analytics aid Supply Chain Management (SCM)

The changing marvels we see in today’s globalized world are brought about by major social and economic change agents like the web 2.0, the financial platforms utilizing Swift technology, and an increasing integration of world economies through outsourcing practices, major logistics industry overhauls and the barriers of language being minimized. Behind the scenes of this new world is a management discipline that continues to play a vital role in the success, growth, and versatility of a firm in the global economy. The field of supply chain management continues to increase in complexity as capabilities are shipped around the world and developed in more niche geographical locations like manufacturing personal computer chips in China, etc. 

Depending on the industry, supply chain management can be very complex. For example, industries like aerospace, automobile manufacturing, and medical device manufacturers can have numerous tiers of suppliers all over the world that are susceptible to risks including weather changes, government stability/regulation, and economic factors like shortages of key resources. Supply chain managers are beginning to find immense value in the use of Visual Analytics because it makes a complex discipline more visible, and thus makes it easier to analyze.

This is an exciting blend of practices and is being spearheaded by researchers at the Supply Chain Responsiveness SCRiP institute at Cranfield School of Management the UK. Last June, SCRiP hosted a talk showcasing the progressive practice of visual analytics in supply chain management using two case studies in the British American Tobacco Company and Shell’s lubricant division. To follow these developments you can access documents and research on the SCRiP website listed below.

http://www.som.cranfield.ac.uk/som/scrip

http://alumni.som.cranfield.ac.uk/public/Event.aspx?Id=740

 

Visual Analytics tells the Story Right !

In the United States, Florida, the Florida Department of Juvenile Justice ("DJJ") is using visual analytics to present a clearer and more precise picture of children in the justice system and the effectiveness of the state’s innovative reform efforts. Isn't it amazing to see that government authorities are currently going for real innovative solutions!

The state’s Juvenile Justice Information System is one of the largest juvenile databases in the country, generating millions of lines of data about juvenile arrests, recidivism and reform outcomes.

To provide this information to stakeholders, DJJ had published annual, lengthy reports or posted  Microsoft Excel files online.  Nonetheless, department officials could not correct errors or add information to the reports once they were published, prompting the search for an interactive presentation tool that was both easy to update and learn.

Additionally, the department wanted a tool to help tell its juvenile justice reform story with data visualizations that would illustrate how giving a “civil citation” to first offenders had reduced recidivism in areas where it was actively encouraged.

Visual Analytics: Integrating with every thread of our life !!

In today's world, all the application areas where large information spaces have to be processed and analyzed, visual analytics can play a pivotal role.

In line with this utility of visual analytics, major fields/sectors that are either using visual analytics  or are candidates for using it for their advantage include:
physics, astronomy, climate control, catastrophe and emergency management, terrorism informatics, border protection, investigation support, computer tomography in field of medicine, ultrasound imaging, sequencing of DNA and other biological data, human genome study, study of protein cells, combinatorial chemistry with tens of millions of compounds, financial data analysis, stock exchange trades analysis, credit card usage analysis, monitoring efficiency of prison systems, traffic light systems, airport traffic control; analysis of player data, games history, sports team analysis and much more.

The point here is that Visual Analytics is no longer constrained to being that "trendy new thing" out there for geeks to try; It is a integrating itself in every sphere of life as we know it - from DNA research to finance sector to climate control to sports to social media to crime control and monitoring. You think of a field/sector and I am sure you will find a news about how this field is getting benefited by visual analytics.

Just recently, TCS, a major technology company announced how it is using visual analytics and big data analysis to predict the future of IT industry and the trends of future.

And what do this integration of visual analytics with various aspects of our life mean? Well, to start with technological implications, remember the buzzword of 'High Performance Computing' that I had mentioned in one of my previous posts? Well, for starters, this would not be just the cool thing which is deployed only by major investment banks; it will be a necessity for every sector.

And to generalize, It is a mathematical fact that more samples you process for getting your regression model, the lower is the error value associated and no matter what sector you belong to, you have to admit that there is so much data now available to be processed and analyzed. In today's world, this 'big data' goes hand-in-hand with the various visual analytics tools, when it comes to using this data and putting it for use. I mean, along with all the other advantages that this association of visual analytics and the large volume of data will bring to your sector, you won't mind that 'added' accuracy in predictions of trends you care about most, would you? :-)

Visual analytics and sports ; " Taking the game to new levels"

http://www.sloansportsconference.com/?p=9994

Visual analytics are being used in applications to improve professional players's performance when playing their respective sports.
Coaches use the Data collected by the supporting staff and the assistant coaches in order to guide players on what optimal decisions to take and when exactly. These conclusions are based on studying trends that are repeated by opponent teams. Teams who use these tools to study trends and prepare accordingly have managed to merge technology and sports in a very creative way, and most importantly help improve player performance.

A Small Detour into Web Analytics :-)

Well, I know that the topic of this blog is Visual Analytics; But when I saw our blog crossing a viewership of one thousand, it's first thousand, I could not resist sharing some of the interesting Web Analytics Stats about how it has done so far, with our viewers. So, from the home page of our blog which hosts a counter of views, all the way until division of viewers as per the region, and other web analytics, we are herby posting it all.

Life is all about finding joy in little things that take place around us everyday. Do Join us in this celebration of a ton and enjoy reading posts on our blog, the ones posted earlier and the ones coming in near future!!

SAS is up to the Challenge!

More and more organizations are finding value in big data with SAS Visual Analytics. Global customers and industry watchers quickly recognized the advanced graphic solution's self-service data visualization as the key to yielding valuable insight.

SAS Visual Analytics combines industry-leading analytics with unique in-memory architecture, intuitive data exploration, and the option to deliver information to a wide variety of platforms, - including iPad and Android tablets.

I dare to say that so far SAS offers the only in-memory engine designed specifically for business visualization of big data on general-purpose, nonproprietary hardware.

SAS Visual Analytics, which runs on commodity hardware or database appliances from EMC Greenplum and Teradata, supports increasing demands for fast answers from big data.

Even, Employees with limited or no previous experience or skills in analytics can understand and benefit from the use of Visual Analytics, as all of the sophisticated, complex data is seamlessly integrated with a variety of user-friendly features such as on-the-fly predictions and scenarios, autocharting, 'what does it mean' pop-ups, and drag-and-drop capabilities.

Analytical Art

The human brain is limited in the ability to comprehend vastly large and complex truths. For example, it is impossible for the human brain to be able to visualize the size of our galaxy which is only a tiny portion of our universe. The number of zero's you have to multiply the size of the earth by attests to this fact. You can't think in these kinds of quantities or sizes. Its just not biologically wired into our little brains. Sizing down to compare the Earth to the size of Jupiter is a bit more feasible. If you were to compare the two you would be astonished, you can fit 1,321 Earths into Jupiter. Wow! When you see the visual of this you get the picture! And this is the point! Visual information can have profound effects on us, it can trigger emotions, feelings, and amazement. For the fun of visual analytics I just wanted to highlight a great source for presenting information in an artistic way. See the Link below for "Information is Beautiful."

http://www.informationisbeautiful.net/popular/


Also watch:

http://www.youtube.com/watch?v=eo4u4JJAPGk

seconds 2:20-2:46

Latest Visual Analytics deal!

Yesterday the 2nd of Sep 2013,  The German firm Software AG, which provides big data, integration and business process technologies, has announced the acquisition of US-based real-time visual analytics provider JackBe Corporation, and says it will shortly launch a new platform making use of JackBe’s technology.

AG is expected to launch the new web Methods Intelligent Business Operations Platform (IBO) for early access customers in Q4 2013. The platform will use JackBe’s mashup, analytic and visualization capabilities and according to AG, ‘will allow a comprehensive 360-degree view of operational processes by combining live, historical and transactional data with machine-to-machine (Industrial Internet) communications into real-time visualizations’. It also includes tools to put changes based on this intelligence rapidly into action, including dynamic and automated business processes.

Synchronizing Visual Analytics and Social Media

If you ever once uttered the words “It’s a small world!” it was likely in the context of realizing you were somehow connected to a random stranger through a random sphere of your life. It could have been a past job, an old class mate, or distant relative. This phenomenon is popularly recognized as the 6 degrees of separation. Essentially it is the theory that everyone and everything is six or fewer steps away from any other person in the world. How much closer do you think people are now with social media? It is a curiosity. Perhaps it is more fun to not know and have those connections surprise you from time to time.

With the help of Visual Analytics and Social Media we can explore all kinds of inter connections beyond the 6 degrees that cross  many various aspects of life, not just who we might bump into. The interdisciplinary approach is superior in solving complex issues. For example, the uploaded YouTube video contains an example of the capabilities to gather information from micro blog texts from various languages about symptoms of illnesses and visualize these over a geographical spread in time. With this information further analysis can be conducted utilizing keywords from micro blog accounts that people mindlessly publish. The end results and conclusion may surprise you! It is important to note that much of the information individuals publish on micro blogs like twitter and Facebook can be used by analysts to visualize trends in time and geographic space.

Will Facebook be sitting on the dock ?

Is it illegal for Facebook to use of facial recognition software that collects biometric data without user consent?? ( cf. July post by Prasad, "Facebook Tag Suggestions: A Value-Add or a Trepass??"). 

Facebook might be facing a legal action as it seems to not be complying with EU privacy laws according to the office of the Hamburg Commissioner for Data Protection and Freedom of Information.

The law prescribe that a company has to ask a user if they want their information stored or not, and then they are not allowed to store it if the person says no. Facebook does not search for approval, but goes ahead and stores the information.

Facebook’s defence is that users are allowed to change their settings to opt-out. However, Facebook state that “During our contentious dialogue with our supervisory authority in Europe, the Office of the Irish Data Protection Commissioner, we agreed to develop a best practice solution to notify people on Facebook about photo tag suggest.” Therefore, Facebook is currently working with the Office of the Irish Data Protection Commissioner to create a way to properly alert users to Facebook’s features and gaining their approval.

What is your opinion : Shall Facebook be oblige to ask for the prior approval of its users or a posterior notification shall be sufficient ?

ARE YOU READY???

In a recent commercial from a very well-known brand, the NBA star Kevin Durant becomes the Head of a recruiting agency where with the help of other NBA stars like Dirk Nowitzki and Marco Belinelli, they scout for young talents using the latest visual analytic's tools available in the market!

Visual Analytics: Challenges that lie ahead

It all started in 2005, when an international team of inter-disciplinary panelists from IEEE CG&A coined the definition of term visual analytics as, "Science of analytical reasoning facilitated by interactive visual interfaces". Visual Analytics and its applications in various fields of research, security and business have come a long way since then.

Today in 2013, as we take a look at various aspects of visual analytics through our blog, an obvious question that comes to our minds is, what are the potential challenges that lie ahead for this promising sector? Can these challenges be overcome?

Lets take a brief overview of all this and more in my current post.

First and the foremost of the challenges for visual analytics going forward is to develop in-situ analysis capabilities. With the volumes of data to be processed increasing by minute, the traditional approach of storing this data into conventional database and then processing it may be a major bottleneck. High Performance Computing of data, even while it is in the memory in the need of this day and visual analytics platforms need to incorporate this requirement while coming up with new upgrades.

Another major challenge comes in terms of limitations in terms of human interaction and user interfacing of visual analytics systems. The challenges in this regards are multiple, ranging from handling heterogeneous inputs to scalability to accurate representation of all data uncertainties and evidences, all the way up to limitations of human cognitive abilities to process the visual analytics data for accurate interpretations.

Speaking of human and system performance limitations, another major challenge is also of storage of huge data that visual analytics takes in. With all companies preferring cloud based storage, data required to be processed by visual analytics tools that are deployed by say Facebook is huge and requires not only huge cloud based storage, but also a capability to maintain this storage online at all times. Various algorithms which process this data in cloud, are also many a times based in the cloud and updating and maintaining these algorithms is another challenge that comes along with this.

With these huge scales of data that we are now talking about, when it comes to future and current challenges, there is also a regulatory aspect involved with various governments restricting the flow and access of data between countries and regions. These government regulations impose new constraints on efficient processing logic needed.

Well, well, well .. so much to talk about when it comes to challenges that lie ahead. So, can these be overcome?

Why not? With efficient combination of High Performance Computing [HPC] and parallel/multi-thread processing, many of the computing issues can atleast be nailed down. Government regulations cannot be helped and processing logic and storage and data retrieval mechanisms will have to find the most efficient way around.

When it comes to human cognitive skills though, who knows we may soon have to defrost the Super-Human "Kahn" and his crew that the Star Trek folks has recently succeeded to freeze into state of coma :-D

Shopping for Visual Analytics Platform??

Well, all this talk about how Visual Analytics is good or bad; and our readers must now be wondering - does this visual analytics make any business sense? is there any commercial application, or a platform, which businesses can deploy to use visual analytics for making their life better?

Answer is a big YES.
And, while there are a number of vendors currently providing Visual Analytics packages - both server based and cloud based, I found one vendor particularly interesting. Name of vendor is SAS.

In March 2012, SAS introduced its Big Data Visual Analytics Platform. This is how "Information Week" describes the SAS platform,
"It promises the speed-of-thought data-analysis capabilities of SAP Hana, the scalability of Hadoop, and the intuitive visual-analysis capabilities of Tableau. But what makes SAS Visual Analytics, a platform announced Thursday, truly stand out is the tie to the Cary, N.C.-based vendor's extensive predictive analytics portfolio.
SAS Visual Analytics is not an in-memory database. In fact, it liberates customers (and SAS) from dependence on an expensive third-party database because it holds data in memory on a rack of blades running the Hadoop Distributed File System. Customers won't have to know anything about configuring or running Hadoop, said SAS, because all the deployment, provisioning, and administration will be handled by the platform's SAS LASR Analytic Server. The platform has been tested with more than 20,000 columns and 1 billion rows of data, according to SAS, and to scale out, customers simply add more nodes."

Jim Davis, SAS' senior VP and chief marketing officer, elaborated further, "If you look at what SAS is doing versus others, we're not just exploring past activity, we're supporting analyses that are predictive, so people can see into the future of their business performance"

As an example, Davis said that predictive marketing campaign optimization efforts that now take eight to 10 hours in a conventional SAS environment can be completed in less than three minutes on the platform, and bank-risk calculations that formerly took 18 hours now take 15 minutes.

Moving a step ahead with current trends, on 1st May 2013, SAS took its visual analytics paltform to the cloud.
"Every IT organization is either building their own private cloud of virtualized systems on commodity hardware or looking to third-party providers," said SAS CTO Keith Collins, in a statement. "These enhancements will make it easier for users to access new SAS high-performance analytics capabilities in our current analytic product portfolio. Our goal is to decrease the time-to-value for SAS users while reducing the total cost of ownership for IT."

With these features and developments, SAS currently offers Business Analytics solutions for variety of industries as Banking, Communications, Education, Government, Insurance, Life Sciences, Manufacturing, Retail, Utilities, etc.

To discuss the example of its application in banking, it is interesting to know that SAS Visual Analytics can be used to visually explore data sets of any size to spot trends, patterns and hidden insights that you can use to design a strategy, confirm a hypothesis or identify a new idea. For example, you can:
- Create more personalized customer interactions and an improved customer experience based on a better understanding of customer needs and preferences.
- Quickly and accurately perform interactive regulatory data analysis to spot patterns and trends, answer regulatory questions and deliver the exact information requested during inquiries.

More information about business application of SAS Visual Analytics can be explored at their website:
http://www.sas.com/software/visual-analytics/overview.html

This post may sound like a promo for SAS, but based on my lookout, this seems to be one of the best ways to implement Visual Analytics for any business, no matter how small or big it is; and it is this practical utility of our discussed topic 'Visual Analytics' that makes all this information even more appealing to our group.

I am sure our readers will also enjoy reading this post and knowing more about a prominent Visual Analytics vendor.

Demographics & Video Analytics

Companies often segment their customers based on the information they gather through surveys, customer feedback forms, and information requests. Some grocery stores gather information through loyalty reward programs in which customers get special offers or discounts for being club members. In order to gain these benefits supermarkets require that the customers provide some basic demographic information to the grocery retailers that later gets uploaded to a data base and tracks their individual purchases. The demographic information is valuable because it gives the entire supply chain information as the grocery stores share the data with their suppliers and food manufacturers. This has been a great resource for larger companies but there are even more tools available made possible by Video Analytics that all retail stores can benefit from. Video Analytics is popularly known for its use as a security tool but it can be used for other capacities like marketing. 

Video Analytics provides an additional source of information that can help retailers gain more knowledge about their customer base. For example it can help businesses count how much foot traffic comes into its buildings and identify what kind of people are walking in. If you have 300 potential customers walk in to your store every day the video analytics program can help detect how many are male and how many are female. It can help distinguish what kinds of things they look at, where they spend their time and whether or not they purchase something. Even more, it can give an approximation of their age. The accuracy is high and reliable which makes the tool valuable because it can provide additional information about potential customers. Video Analytics can help businesses improve their marketing efforts as they learn more about the people who walk through their stores. With its use, we can expect that retailers will have a better idea of what to stock on the shelves not just based on what people have previously purchased but also based on what draws people in. 

See more here: 

http://www.3vr.com/products/videoanalytics/demographics