Watson Analytics

Interested in Watson Analytics? IBM Blueview has got you covered. Check out our Watson Analytics first look,  Natural Language Query review and overview of Watson Analytics Professional pricing and features.

Today we’re going to take a gander at the query and visualization component of Watson Analytics, “Explore”. Standalone it is a highly competitive visualization tool that puts Cognos Workspace Advanced to shame; forming like Voltron with Watson Analytics’ other features yields a powerful and unique data slaying super-robot. If you’re curious how Watson Analytics stacks up against Tableau this is the most relevant feature for direct comparison and I can give you a free TLDR by saying: Pretty damn well.

Side note: life doesn’t offer a ton of opportunities to make Voltron jokes, one must seize the lion by the mane… … … sorry!

Launching an Exploration Expedition

Starting the Explore feature requires that you’ve uploaded a data set to Watson Analytics. From there you click on the massive “Explore” tile at the top of the screen and select the set in which you are interested, or you can select the set itself and choose “Explore” from the listed options. This launches a starting point for our exploration:

1. The pane at the top is global across Watson Analytics and allows you to save, to quickly swap between open data sets and tools and to see my real name. Ooops.
2. Here you work the Natural Language magic we covered in our post on said topic, but briefly you type a question in English and will see suggested visualizations that (maybe) provide answers.
3. Watson Analytics assesses the data set and suggests a number of exploration starting points. You see a brief description of each above an image that tells you what sort of visualization it suggests. In our case “What is the trend of Receptions over Week by Position” is a line graph. Also notice the words in bold – those are column names from the data.
4. This little button brings up a window that lets you create custom dimensions, add new calculations and fields, and generally mess with the data.
5. Displayed across this bottom pane are each of the columns from the data set. Adding filters, groupings or calculations by clicking on an item will apply that change across all visualizations in your current “explore” session. Very handy!

If you are a Cognos guru or Tableau fanatic you might be saying to yourself, “This is kinda cool but can I skip all these suggestions and build it myself? As a guru fanatic I don’t need help from Doc Watson!” The answer to that is “No.” You are required to go through this screen to start. I wasn’t thrilled at first either but consider this – Tableau democratized visual storytelling and one-upped Cognos by empowering business experts to create compelling visualizations without IT: IBM fires back with Watson Analytics, which is a great visualization tool but also, crucially, is the expert. So no, they won’t let us skip the truly unique, potentially revolutionary features of a tool that analyzes the data automatically and can understand your questions.

Diving Into The Data with Watson Analytics Explore

Given that, let’s select “What is the contribution of Receptions over Week by Position?” Below you can see the results of a single click – a full featured visualization with a host of options for discovery and refinement. Let’s take a look at the interactive features:

1. Let’s start with the big one first – the visualization. Clicking on visualization elements brings up an explanation of their contents and provides a menu for refinement of the data – custom calculations, filters, etc can be accessed by simply clicking on a bar, point, bubble or box. Awesome.
2. Jumping to the top, here we find a number of other relationships Watson has uncovered automatically along with simplified visualizations representing the data. Clicking any of them will launch a full visualization like the one you see above.
3. The question you asked is displayed here. Here the words in bold from earlier are interactive – want to swap “Rushing Yards” for “Receptions?” Simply click and select the data you want. The visualization automatically updates.
4. The key is also interactive – clicking an element in the key opens a window to filter, calculate and otherwise manipulate that data set.
5. Anytime you see this icon in Watson Analytics you know you have options. Here we can make more advanced changes to the visualization as a whole.
6. You can create more than one visualization in an explore session – this is where they are managed. Create a new visualization, navigate through them, or display them all at once with these buttons.
7. The pin icon works like you’d expect it to – it pins the visualization for sharing with other users or the Assemble dashboarding tool. Next to that is the “change visualization type” button for swapping to different visualizations using your current data and filters. Finally, the visualization display options allow you to change the style and color of this visualization.

From here we have complete freedom to explore the data – expert users can breathe a sigh of relief. IBM has taken a huge leap forward in terms of design – you interact with each element by clicking on it rather than through hidden menus.  Almost everything reacts as you expect it to. Explore offers a variety of tools to shape the data however for more serious data quality tasks you’ll want to use Assemble, which we’ll cover in a future post.

Watson Analytics Explore In Action

This is without a doubt the easiest query/visualization tool in the IBM Analytics stack. Cognos Workspace Advanced and Report Studio require significant technical skill and education to use effectively, as well as a data modeler to clean and present the data. Watson Analytics Explore lacks many of the enterprise features of Cognos but is staggeringly more advanced when it comes to usability, agility and high fidelity presentation. Behold!

 

What you see above took approximately seven minutes of work. I can make something that looks very similar – but not as sharp – in Cognos in approximately two hours. However, clicking any of these charts will drill down to a large, fully interactive version which is easily refined. That functionality in Cognos is more like a consulting engagement. To top it off, the bottom pane is acting as a global filter – I currently have the NFC North selected, but the entire data set can be instantly filtered with the results rendered in all visualizations…instantly!

IBM’s Analytical Agility

“Agile” is, at this late stage, a methodology, buzzword and parody bundled neatly together, and its insights are eye-rolling obvious but still necessary, especially in Business Intelligence. A good BI deployment leads to new questions and a great one leads to new questions that must be answered urgently. If after go live the users politely agree that all requirements have been met and the system is complete you must immediately stow the champagne because you have failed.

This is where IBM and Cognos most often fail, due more to outmoded thinking than technical limitations in the product. Cognos does not require a six month workflow to make a new graph but the vestigial processes of the waterfall era weigh it down. In a sort of perverse way Tableau’s greatest strength is that you simply cannot do that much. The temptation to drown it in requirements, governance and process is ignored.

This is where Watson Analytics Explore fills a huge gap for Big Blue. It is free and easy to use. It looks great. It interacts with the much more complex features of Watson Analytics but doesn’t require them and thus offers a very low barrier to entry. And it is agile in the way a great BI project is agile; it answers questions with questions and does so beautifully. Like Tableau, it’s design makes obvious how it is to be used: quickly, joyfully and unburdened.

 

Very exciting news broke at Analytics Zone yesterday as IBM unveiled the pricing and feature set for Watson Analytics Professional Edition, presumably to help build hype for the upcoming Vision 2015 conference. Professional Edition joins Free Edition and Personal Edition as the third tier of Watson Analytics subscription available to the general public, and carries with it a substantial wish list of functionality heretofore missing from the tool. Let’s get to the big one first…

Watson Analytics Professional = Cognos Integration!

In my Watson Analytics Review: Natural Language Query article posted four days ago I wrote “… IBM is expected to announce Watson – Cognos integration in the coming months.” I like to believe someone over at Big Blue read that and said, “Oh yeah, we’ll show you!!!” Check this out:

Watson Analytics Professional Edition subscribers can browse the Public Folders and My Folders of their Cognos Business Intelligence environment and upload data sets from list container objects directly into Watson. Once completed Watson Analytics immediately identifies interesting relationships and presents them via suggested visualization and natural language queries as outlined in last weekend’s article. While narrow in scope this first step allows IBM’s enterprise BI customers an extra degree of confidence when using Watson Analytics.

IBM is expected to announce the ability to upload more complex reports and browse a Framework Manager package in the coming months… see what I did there?

More Data, More Users, More Money… Less Problems?

The remaining features are outlined at a very high level on Analytics Zone and in the Watson Analytics Expert Blog but the focus is clearly on expanding the volume and scope of data inputs and allowing collaboration within the tool. These feature sets are critical for IBM:WA’s claims to be a “big data” tool and for ushering along departmental deployments.

New Data Sources, Increased Storage and Data Refresh

Watson Analytics Professional Edition subscribers can look forward to the following enhancements:

•  Ability to upload data from DB2, DashDB and SQLDB housed on IBM’s Bluemix cloud platform.
• Ability to upload data from Dropbox and Box cloud storage services as well as Twitter.
• The cap on rows, columns and total storage have been dramatically increased: 500 columns, 10 million rows and 100GB total storage are the new limits.

They’ve also added the ability to refresh data sources, which is a really crucial feature for a tool like this and was sorely lacking. Finally you can now refine your data in the tool rather than relying on Excel.

Collaboration

Until now individuals using Watson Analytics had no means within the tool to share discoveries with collaborators, and the export options were limited to PDF and image files. While details in the written articles are sketchy at best we can glean some insight from the video linked above:

• The main screen of Watson Analytics now lists “Workspace: Rachel Pond”, continuing the fine IBM tradition of using the same two words  (workspace, insight) to christen virtually all tools and features in their analytics stack. The “Workspace” will presumably be your personal view.
• When you pin items in Watson Analytics, those items can be made available to the rest of your team.
• The sharing options listed include: Sharing via personal email, sharing via social media, sharing via a link, downloading to the desktop, printing and “create a story with this data.”
• Data sets can be shared and secured at the individual level for your team – no need to give Steve the ability to edit the data. You know you can’t trust that guy.

So what’s it going to cost me?

Watson Analytics Professional Edition will run you $80.00 per month, per user. That’s $50.00 more per month than Personal Edition, but for the feature set listed above I think that is a reasonable price. Consider a Tableau license starts at $1,000.00 for lower level desktop client license and goes up (quickly!!) from there.

Less Problems… Seriously?

Okay you got me, I was trying to be funny. But I can see there being “less problems” for IBM and its clients given these updates to the Watson Analytics feature set. And as a professional Cognos developer I’m excited to see the first step toward integrating these tools. If my users can construct a list report in Cognos Workspace Advanced and then utilize Watson Analytics for the visualizations I think we’re both going to be very happy.

Be sure to see my Watson Analytics First Look for a high level take on the tool and check out the upcoming Cognos Vs Watson series to understand how Natural Language Query compares to similar tools and features in Cognos.

In part one of my Watson Analytics review we’re going to take a deeper look at the Natural Language Query capabilities embedded in the tool. IBM has been been hyping Watson Analytics as the next generation, quantum leap, pick-your-superlative Big Advancement to put the BI startups back in their place and reclaim their spot as king atop Mt. Gartner. Read on to see if Big Blue has a shot at the crown.

From Ignorance to Insight, Faster

IBM’s approach with Watson Analytics is to minimize the effort necessary to turn raw data into insight and to experience those insights visually rather than as a chart or crosstab. The hope is to leapfrog current BI darling Tableau with a graphically modern UI that intelligently analyzes data, identifies interesting relationships and allows users to explore them using natural language queries and visualizations. I illustrate how this works in the gif below:

1. First I click on an uploaded data set, in this case offensive statistics from the 2014 NFL regular season.
2. Watson presents twelve visualizations based on relationships it identified in the data set. No human intelligence was involved in finding or visualizing these relationships!
3. I choose “What is the breakdown of Receiving TD by Location and Position” and boom! The Explore tool renders pretty tree chart showing exactly that.

Two clicks. Two clicks! That’s what it took to go from a freshly uploaded CSV of raw data to visual output. In Cognos Workspace Advanced that is probably one hundred twenty clicks and in Tableau, maybe sixteen? Working with Watson Analytics isn’t always that simple but fact that it can be is quite promising.

Ask And Ye Shall Receive (A Chart)

So what if Watson, in all his wisdom, fails to anticipate your question with zero input from you? Because Watson (kind of) understands English here are two options; free form a question in the text box, or click “How To Ask A Question” and use the just-launched query builder to guide you to a phrase Watson can understand. Take a look at the options below:

1. This is the data set I want to query. A single click will open the “Starting Points” window you see above.
2. These are interesting relationships Watson Analytics automatically identified. Clicking on any of these will launch the Explore tool with the visualization described.
3. This text box is where you ask Watson questions in plain English. When you submit a query the suggested visualizations below will update to reflect Watson’s best guess at the answer.
4. Here you can access the “Ask a question about your data” screen, which provides a guided Natural Language Query building experience.
5. Clicking “Create your own” will let you jump directly into the Explore, Predict and Assemble tools using this data set.

Ask Watson Analytics Yourself

Let’s take a look at the free form question option. Watson is best at interpreting language that contains column titles and specific data values from your data set in combination with a keyword telling him what sort of relationship you’re looking for – Compare, Trend, Average, Maximum, Count, Correlation, etc…

Here I type “Most Touchdowns By Player” into the text box. This is not precisely the kind of language outlined above but it is close to how actual human beings speak. None of these words are exact matches to what Watson expects – there are Total Touchdown, Passing Touchdown and Rushing Touchdown columns in the data but nothing called “Touchdowns”, and “Most” is not one of Watson’s listed keywords. Let’s see the results:

1. Clicking on my data set brings up Watson’s suggested visualization?
2. Liking none of them, I type in my desired query, “Most Touchdowns By Player”
3. “How do the values of Total Touchdowns compare by Player Name” seems to capture the spirit of it.
4. The Explore tool opens with the visualization Watson Analytics feels best represents the answer to our question.

Or Get A Little Help

The example above worked great, but believe me there are plenty of times Watson will have no idea what you are trying to ask him. IBM just launched a feature that allows you to construct your query sentence using column titles from your data set and the keywords Watson is designed to recognize. Instead of typing in the text box, click “How To Ask A Question?” to get to the screen below:

1. I choose the “Breakdown” Natural Language Query starting point because I want to see which players scored the most receiving touchdowns against each team in the NFL.
2. In the drop down boxes I select “Receiving TD”, “Opposition” and “Player Name”. Then I click “Ask”.
3. Watson returns to the “Starting Point” screen and displays the relationships it identified based on my question. I click on the one I am most interested in.
4. The Explore tool opens with the visualization I choose populated with data and ready for further analysis.

It’s important to note, no setup is required to use a feature like this. Watson figures out all of the data types when you upload a file and provides these query building starting points, no metadata modeling required. Using the dropdown lists you select from your data items and click “ask” to see what visualizations Watson suggests.

How Natural Is Natural Though?

These capabilities are impressive. Watson Analytics is targeting business users and executives who don’t have time to wait for someone like me to write a complex SQL statement for them, and also don’t have time to go back to college and learn to do it themselves. Allowing them to query the data in English really is the giant leap forward IBM purports it to be…

… for the most part. The thing is “Natural Language” means “Not SQL or another coding language” more than it means “How I talk about the NFL at the bar.” Let’s explore the limitations of Watson Analytics Natural Language Query as it exists today by trying to determine the best quarterback in the NFL.

“Who is the best quarterback?”

Straightforward, conversational and exactly how you’d phrase it over some wings and beer. Let’s see how Watson Analytics responds:

Watson has no idea what we’re asking. “Best” is not a term computers will understand for a looooong time, and even then our disagreement over the criteria for determining “best” is what makes the question fun to begin with. “Quarterback” Watson could understand but only if it’s spelled out in the “Position” column. It isn’t, it’s “QB” and so this sentence is complete nonsense to Watson.

“Who are the top ten QB?”

Here we’re providing Watson Analytics with the correct reference to the position (“QB” instead of “Quarterback”) and providing it with a specific set we’re concerned with, the top ten. Think we’ll have more success?

Well… sort of. Because we correctly used “QB” Watson Analytics has determined that we are talking about Position, and our use of “Top 10” is leading it to show breakdowns, counts and comparisons. It appears to be connecting “who” to “Player Name”. However we haven’t provided it a measure so it is comparing Player Name and Position.

That it is able to make these connections is impressive but we are no closer to identifying the best quarterback in the NFL. Let’s try again, this time with some more measurable criteria.

“What is the breakdown of player name by passing td and passing yards?”

Now we’re speaking Watson Analytics! Check out these results:

Here we used the keyword “Breakdown” and directly referenced the columns we are interested in comparing, Passing TD and Passing yards. By giving Watson Analytics the measures we want to see rather than a fuzzy human term like “best” we get the results we are seeking. Let’s choose “What is the relationship between Passing TD and Passing Yards by Player Name?” and see the results as rendered in the Explore tool:

Explore Your Options From Here

You may be wondering if Watson Analytics forces you to submit query after query until you get the phrasing just right to render the visualization you seek. The answer is thankfully, “No.” Once you get into the Explore tool pictured above you can tweak or completely change the visualization much like you would with Tableau. The Natural Language Query is the starting point of a conversation with your data that provides a first visualization from which to work. I will give more details of the Explore tool in an upcoming post.

Verdict: Not Perfect, But Perfect For The Right Users

As you can see, the Natural Language Query feature of Watson Analytics is a powerful tool for quickly identifying interesting relationships and generating modern looking visualizations without writing code or wrangling with chart axis and dimensional aggregates. BI professionals and advanced business users may wish they could skip it entirely and jump straight into the X – Y graph making with which they have expertise, but these people are not the target audience of Watson Analytics.

The relatively conversational nature of the Natural Language Queries Watson Analytics can interpret and the speed at which it presents and visualizes multiple possible relationships derived from a query makes this feature ideal for line of business analysts and executives who want a low cost, self service experience they can easily understand. That it requires no input from IT is either a blessing or curse depending on your perspective, but IBM is expected to unveil integration between Watson Analytics and Cognos Business Intelligence in the coming months.

There’s Much More To Watson Analytics

Natural Language Query is really just the gateway to Watson Analytics. There are three fully fuctional tools, Explore, Predict and Assemble that I will give the full treatment in standalone reviews before wrapping up with an comprehensive overview of Watson Analytics and how it fits into IBM’s portfolio and the BI marketplace. Overall I feel this tool shows great promise and I’m excited to see what IBM does with it going forward.

 

 

 

 

Today I’m going to take a first look at Watson Analytics, the newest BI offering from IBM. To say Watson Analytics is critical to IBM is an understatement. While IBM assures investors of their profitable future as a cloud analytics and services company, Gartner’s 2015 BI Magic Quadrant shows continued deterioration for IBM in the face of stiff competition from Tableau, Qlik and others.

The market clamors for easily deployed, departmental, user friendly BI tools and IBM’s existing champion Cognos is a slow moving behemoth tailored to IT managed enterprise deployments. Their first crack at meeting the new demands of analytics users, Cognos Insight, saw very limited adoption. So IBM went back to the drawing board and Watson Analytics was born.

A New Business Model for IBM Analytics?

Let’s get the most important question out of the way – how much does it cost? Watson Analytics is currently a cloud only app which can be accessed for free with .xlsx or .csv data sources of up to 500,00 rows and 50 columns with 500MB of total storage. 1,000,000 rows and 256 columns with 2GB total storage will run you $30.00 a month. This smartly removes the two biggest roadblocks to acquisition faced by Cognos, SPSS or TM1 – the IT Department and a few million dollars.

IBM has taken all the hard won software know-how of their existing analytics stack and applied it to Watson Analytics. You can see the fingerprints of Cognos and SPSS all over. SPSS handles the statistical magic happening under the hood while the Cognos RAVE visualization engine appears to drive data presentation. It’s wrapped in a mostly intuitive and very modern UI capable of competing with Tableau & Friends in the battle for eyeballs.

Who Is Your Watson? And What Does He Do??

There are currently three main features to Watson Analytics, Explore, Predict and Assemble. To help illustrate how they function I created some visualizations and predictions using a data set of 2014 NFL offensive stats. I will give each of these features the full treatment in future posts. Below is a quick overview and screenshot of the output.

Explore

Explore is kind of the “query tool” of Watson Analytics. Here you create visualizations and do guided data discovery. Swapping between visualizations is intuitive and there is a surprising amount of control hidden away in a few hard to find menus. Let’s take a look at the basic layout.

• The small blue summary charts at the top of the screen are interesting relationships identified automatically when Watson Analytics first analyzes the data set. Clicking on them will take you to a fully realized visualization of lower level detail.
• The visualization itself occupies the center of the screen and is fully interactive. For example, clicking on the “Calvin Johnson” tree map segment opens a menu that allows you to change, filter or exclude that element.
• To the right of the visualization is the key and a slide filter. These were generated automatically and provide an additional way to edit or filter the data elements on the visualization.
• The bottom of the screen lists all data elements and provides access to more advanced capabilities, including custom calculations and the ability to define dimensions for slice-and-dice analysis.

Predict

Predict is probably the part of Watson Analytics most aggressively marketed by IBM. Predict does exactly you think – applies statistical models to make a predictive analysis of the data set. It’s highly automated and you do not need to be a data science to understand the output, which perhaps a double edged sword because if you aren’t a data scientist there’s a big risk of misinterpreting the data.

• The top of the screen provides various details about the prediction, including a score of the data quality, top field associations, and additional statistically significant relationships.
• In the center we see the primary predictive relationships in the data – in the case, that receiving targets and yards are the primary predictors of total receptions. More interesting perhaps is the much weaker relationship between touchdowns and total receptions.
• You can go into much greater detail about the statistics behind all of these elements with just a click. I’ll explore this in detail in an upcoming post.

Assemble

Assemble is the dashboarding tool in Watson Analtyics. This is where you combine the visualizations from Explore and insights of Predict to tell the graphical story of the data. It provides an interactive environment and, I think, looks quite good compared to Cognos. But in its current form your options for sharing and collaboration are pretty sparse.

• The dashboard is highly interactive without any configuration, which should make Cognos developers accustomed to Active Reports party in the streets.
• New tabs and a variety of visualizations, images and external content can be added easily at the top of the screen.
• Total development time for a dashboard is reduced to minutes vs days or weeks in Cognos. I built this in about four minutes (and you can tell!)

And Now For The Secret Sauce

Looks cool so far but is that it? Well, no. Embedded in everything is a search box connected to a natural language processor that generates visualizations on the fly rather than crawling for pre-existing content…! To me this is the most exciting feature of the whole thing because so much of my job is hand crafting bespoke, locally sourced reports and visualizations which, popular as they may be in certain parts of Brooklyn, is a job better done by a machine.

The suggested visualizations you see above in response to my query are not something a human being was paid to create, and that seems to me to be the secret sauce of Watson Analytics. More than the predictive modeling and the cloud and the other buzzwords, this is the big improvement on IBM’s existing analytics platforms. Type in a question and get three days worth of the BI team’s best visualizations. Don’t like the results? Just ask another question.

Watson Analytics, Cognos and The Future

I am still thinking through the implications of IBM’s new direction for both the Cognos platform and Cognos developers. IBM has not outlined publicly their integration plans between the two platforms but it’s easy see this as the direction in which Cognos is headed in terms of functionality and aesthetics. As impressive as the above features are the infrastructure currently surrounding Watson Analytics for collaboration, metadata management, security, etc… is non-existent. I suspect that Cognos will be a primary vehicle bringing Watson to the enterprise.

Made it to the end? Man you must be as nerdy about this stuff as I. Check back soon for detailed explorations of Watson Analytics’ core features, as well as my thoughts on where all this is headed.