Category Archives: Systems

RStudio Keyboard Quick Reference by Freakalytics

RStudio-example-ggvis-interactive-graphs-300At Freakalytics, we frequently use R (often referred to as RStats) in our client projects and wanted to share our success using the RStudio Interactive Development Environment (IDE) with you. So, we created the RStudio® Keyboard Quick Reference by Freakalytics. It is available to you, compliments of Freakalytics, as a PDF and later in this article as a searchable data table.
 
 
The RStudio IDE was built by the team at RStudio to make you more productive in the R world. It is a free, open source application for Windows, Linux, Mac and UNIX desktop users. RStudio Desktop includes an interactive R console, a smart editor that supports direct code execution, graphing interfaces, code history, a debugger and project management for R code and related files.
 
Download the RStudio® Keyboard Quick Reference by Freakalytics. The reference card is available as a PDF download for your convenience. The PDF version is printable and usable in most e-book applications.
 
In addition the PDF version, we are pleased to share online access to the RStudio® Keyboard Quick Reference as a searchable data table (click here to access the searchable data table in a dedicated window.) This searchable data table has all the shortcuts from the PDF -and- advanced shortcuts not shown on the PDF version (which is one-page for newer users of R).
Continue reading

From Business Intelligence to Visual Analytics
Craft a Winning Data Strategy

Stephen McDaniel and Eileen McDaniel, Ph.D.
Freakalytics, LLC

Topics: Data Analysis, Visual Analytics and Business Intelligence

This was originally published in the
TDWI FlashPoint Newsletter in August of 2014
Italicized sections, images and their captions were not part of the TDWI version.

Until recently, visual analytics was considered a niche area. Those days are quickly passing; almost every major analytics and BI vendor is either launching or developing a product focused on visual analytics.

As a data professional, you’ll face challenges in integrating these products into your existing BI infrastructure. How can you successfully implement new visual analytics tools and keep your business customers engaged and happy? Step in front of the inevitable progression to visual analytics by crafting a winning data strategy.

We suggest starting in these three areas:

Freakalytics-growth-of-walmart
The growth of WalMart as an interactive dashboard

1. Learn the basics of the visual analytics tools used by the business analysts in your organization. Follow the process of how a real-world project is executed. Solving a typical business problem will give you a chance to experience firsthand what users are doing. You will be surprised at how the tools change your view of the data warehouse and “proper” data structures.

We’ve had many data professionals attend our analytics workshops. Even those with years of experience in the field tell us that managing code and databases is a completely different way of thinking about data compared to analyzing an issue, which has drastically different constraints and goals. Investigating a real problem that the business is facing should help you to see many possible ways that your data stores can be adjusted to enable successful analysis.

Check out our book on the principles of visual analytics, grounded in the scientific method, The Accidental Analyst.  Stephen Few called it, "... a wonderful book, filled with practical advice."

Tools to consider include Qlik Sense, SAP Lumira, Tibco Spotfire, Tableau and Microstrategy Analytics Desktop.  We have successfully used all of these tools in our work with various clients. They all have differing strengths, workflows and design philosophies.  Read more about these products and others in our Candid Quadrants report.

 

2. Find an ally in each of your key business areas, preferably one that is an expert analyst for a viewpoint from “the other side.” Leverage these analysts for invaluable knowledge to design better data structures in the form of tables, graphs, and system maps in your data systems. This is far more effective than decoding the whole process by yourself. When building data warehouses and downstream analytic data stores, we’ve discovered that expert analysts are often excited and motivated to collaborate on improving the efficiency and value of the data sources in their analyses.

Accidental_Analyst_in_confident
Unlike traditional BI projects, data projects are now a journey with many twists and unexpected turns. Working closely with business allies that understand the data teams and the business are key to success.

Read more about collaborating with business and data teams in our previous TDWI article.

 

3. Commit to the reality that self-service data management with desktop spreadsheets and databases among business users is not going away. Instead, it will only continue to accelerate over the next few years. Part of this reality is driven by the fact that the appropriate data structure is often dependent on the analysis problem at hand. Another reason driving this growth is that more data streams are flowing into organizations, often at a rate that is overwhelming for analysts and data teams alike.

In our experience, when we help business users improve their data management skills, they are less likely to make mistakes or inaccurate assumptions about the data. They also better comprehend and appreciate the hard work involved in maintaining central systems.

Seize the opportunity to be more successful in your career as a data professional by understanding and incorporating the new landscape of BI and visual analytics into your data warehouse and collaborating closely with business users to establish a strong environment for analytics. Ultimately, data warehouses are about making better decisions in a timely manner, and these suggestions can help you further the utility of your data warehouse.

Excel is a powerful ad-hoc data cleansing tool
Excel has been bashed by statisticians and data teams for years. However, it's a powerful tool for one-off data review and rapid cleansing of data for an urgent analysis. It's also ubiquitous, both in presence and knowledge amongst business analysts. Tread carefully if you think you can "take it away". You can definitely reduce reliance with better tools, training, support and evangelism.

Learn more about Excel for business analytics in our free, recorded webinar.

 

Stephen McDaniel is an Chief Data Scientist at Freakalytics, LLC and author of several books on analytic software. Eileen McDaniel, PhD, is author of The Accidental Analyst and Director of Analytic Communications at Freakalytics. Both work with clients on strategic analytic projects, teach courses on analytics and are on the faculty at INFORMS.

Join us at the Qlik 2014 World Conference

image
Click image for larger view

Eileen and I have been invited to attend the launch of Qlik® Sense at the Qlik 2014 conference in Orlando. As featured speakers, we are holding a session on The Accidental Analyst®, a reliable framework for building analyses to answer real-world business questions. I will also be on an expert data visualization panel with Alberto Cairo. We will discuss what makes data visualizations stand out to clearly inform decision-makers.

We had a great briefing with Qlik CTO Anthony Deighton about Qlik Sense.  So we dove right in and created a dashboard example from a data source that is included with Qlik Sense Desktop. We look forward to seeing what people are already doing with Qlik Sense and hearing about the future directions they may go with it.

Qlik-Sense-Exec-Dashboard-Freakalytics-201410
Click image for larger view

 

Great vs good analysts, software obsessions and informing executives

Small_packed_bubble_chart1.pngCharts and graphs to inform
Are your charts and graphs actually informing people?
Can I actually make a decision from your presentation?
If not, why should I or the executives care?

Cool software, but what about the business?
It's easier to obsess over the complex but oh so cool software features
than to truly understand the business issues and create substantive analysis.

Good analysts versus great analysts
Good analysts make data stores and useful analyses.
Great analysts speak the language of their audience & make informed recommendations.

Stephen McDaniel
Chief Data Scientist at Freakalytics

PowerTrip Analytics™ 2014
Candid Quadrants™
Based on three-year visual analytics cost estimates

PowerTrip_Analytics_Candid_Quadrants_2014_by_Freakalytics
Click for full page image

Detailed cost comparisons across all seven visual analytics offerings
at PowerTrip Analytics™ pricing review tables.

Read individual product details, highlights and examples:
Microstrategy Analytics Desktop,
SAP Lumira,
Tibco Spotfire,
Microsoft Power BI,
Tableau Desktop,
QlikView, and
SAS Visual Analytics.

Download data to predict gender using first name (US data)

Download US American first names and initials to predict gender sex 1Do you have data with just first names or even just first initials but no information on the person's gender/sex? If you would like better insights on your customers, based on whether they are likely male or female, then this data download is a great way to maximize your ROI! Download it today and begin using it to tailor your messaging and improve future communications.

There are three licenses available for this data- individual, corporate and corporate for multi-company consumers. The individual version is available free (with discount code) for a limited time. Simply select the Individual license for purchase and use discount code discfreepers at the checkout page- this will deduct $3.99 from your purchase price.

The primary table in this data download is First names by Freakalytics with 5164 rows (distinct names and common misspellings). You can use this data to guess if someone is a male or female based on their first name or find the probability that they are male or female based on their first name.

Here is the column information and simple summaries for this table:

Data Column Max Min Average Median Mode
Name mixed case Zulma Aaron N/A N/A James
Most likely gender Male Female N/A N/A Female
Rank Overall 4,019 1 2354 2397 4019
Male Probability 100% 0% 22% 0% 0%
Female Probability 100% 0% 78% 100% 100%
Count Either Gender 99,989 32 1,079 127 32
Male Count 99,671 0 524 0 0
Female Count 83,718 0 555 64 32
Male Probability Within 3.68% 0.00% 0.08% 0.01% 0.00%
Female Probability Within 2.92% 0.00% 0.02% 0.00% 0.00%
Male Rank 1,054 1 584 608 1,054
Female Rank 3,052 1 1,825 2,131 3,052
Name first initial Z A N/A N/A J
Name upper case ZULMA AARON N/A N/A JAMES

The top few rows from this table (as a snapshot of the data in Excel 2003 format and in text):

Download US American first names and initials to predict gender sex 1

Access this valuable data download here.

Top 5 Stories Worth Reading in Data, Data Warehousing, Analytics & BI

February 23rd, 2014
Stephen McDaniel
Chief Data Officer Advisor at Freakalytics, LLC

Finding it hard to make time to keep up with the rapidly changing world of data, data warehousing, analytics, data science, business intelligence and visual analytics? We understand! Each week, I read through hundreds of stories in this space and share the five (okay, this week I couldn't stop and chose six) most worthwhile articles with you. Each article includes a summary and link to the full story. Please note that inclusion of an article does not indicate that I agree with every point in the article, but I at least find it thought-provoking and useful for informed debate.

 

 

Recruit Better Data Analysts

In the big data talent wars, most companies feel they’re losing. Marketing leaders are finding it difficult to acquire the right analytical talent. In the latest CMO Survey, only 3.4% senior marketers believe they have the right talent. Business-to-business companies have a bigger gap than business-to-consumer companies, as do companies with a lower percentage of their sales coming from the internet.  And yet analytic skill is a must for effective marketing.

Results indicate that companies with above-average marketing analytics talent experienced significantly greater rates of marketing return on investment (MROI) than companies with below average analytics talent…

 

Big Data doesn’t lie. Or does it?

If you’ve seen any indication that humans are getting smarter and more sophisticated, please inform me, and you don’t have to read what follows. For everyone else, who sees no lack of stupidity and misinformation in both business and public life, read on about the joys big data will bring.

The insights offered by analysis of big data are only as good as the human beings that create the data, gather and assemble it, decide what questions should be asked and how the data is presented. And interpreted, especially that.

I am very concerned that big data, misapplied and misunderstood, will create big lies. Or more likely a combination of lies and truths that prove very difficult to sort out.

 

What is Data Mining?

One way businesses can turn the information into something useful is through data mining. Data mining is a process used to analyze raw information to try and find useful patterns and trends in it.

"Data mining applications help users discover correlations and connections within large data sets," Software Advice writes on its website. "These might have gone unnoticed without these algorithms."

 

Scientific dashboard with periodic table of elements
Includes history, photos, great filters, discoverers and more

There is nothing we love more than sharing great examples of what is possible with visual analytics and dashboards. It’s one of the best ways to inspire new analysts and expand people’s horizons of the nearly infinite number of ways that visual analytics tools like Tibco Spotfire can be used.

Continue reading

Top News – Analysis & Commentary
Data, Data Warehousing, Analytics & BI

Five Business Intelligence Predictions
from Paxata for 2014

January 26th, 2014
Stephen McDaniel
Chief Data Officer Advisor at Freakalytics, LLC

Paxata logo blue 201401Finding it hard to make time to keep up with the rapidly changing world of data, data warehousing, analytics, data science, business intelligence and visual analytics? We understand! Here’s a top new story worth reading and that we considered noteworthy enough to add commentary and analysis by Freakalytics (in purple).  A summary of the article and excerpts that I comment on are in black.

In this commentary and analysis, we cover the growth of Tableau and QlikView, the opportunities that exist for Microsoft to disrupt the second-generation business intelligence market and how self-service data integration will likely make data scientists & data enthusiasts much more productive- enabling wide swathes of Accidental Analysts to quickly answer tactical business questions.

Five Business Intelligence Predictions For 2014 (from the CEO of Paxata)
Summary
The dust is finally beginning to clear from the big data explosion, which is a good thing. One of the problems with big data is that it’s been led by technology, not business requirements. And business requirements will be the focus in the 2014 business intelligence (BI) ecosphere—to enable enterprises to achieve results with data mining and analytics and to prove those results.

Stephen
I found this article a fascinating glimpse into the strategic thoughts of a CEO of a promising, second-generation, cloud-based data integration company- Paxata.

Continue reading

Top 5 Stories Worth Reading in
Data, Data Warehousing, Analytics & BI

January 26th, 2014
Stephen McDaniel
Chief Data Officer Advisor at Freakalytics, LLC

Finding it hard to make time to keep up with the rapidly changing world of data, data warehousing, analytics, data science, business intelligence and visual analytics? We understand! Each week, I read through hundreds of stories in this space and share the five most worthwhile articles with you. Each article includes a summary and link to the full story.

 

1-Netflix-analyticsHow Netflix Got Analytics Wrong, Then Right
Two entertaining and informative articles about Netflix illustrate how to be smart about using analytics. In one instance we see where Netflix went wrong, and in another we see Netflix doing the right thing.

Netflix launched a high-profile crowdsourcing project in 2006 to develop a better recommendation engine, offering a $1 million prize to any person or team who could improve Netflix recommendations by a modest 10 percent.

 

2-BI-dashboardSigns That Your BI Dashboard Needs a Comeback

Everyone loves a good comeback. Stories about celebrities like Robert Downey Jr. and Britney Spears climbing back to the top after falling so far capture our collective imagination. Movies like Rocky and Cinderella Man – about underdogs making a comeback – inspire us to think we ourselves can rebound from any setback.

Is your business intelligence dashboard the underdog at your organization? Dashboards have been around for decades, with some companies not putting the time and effort into updating them regularly to keep pace with the innovations in BI and the growing expectations of users.

 

3-analytics-movies"Pitch Perfect" And How Analytics Are Transforming Movie Marketing

When Universal released the cult musical film Pitch Perfect in 2012, they did what any self-respecting studio would do: They commissioned marketing reports and forecasted ticket sales for the Anna Kendrick-starring movie. Among them was an analysis by a company called Fizziology which data-mines social media to see how the film would play out with audiences. Continue reading

2014 US growth forecasts of Business Intelligence vendors

There are many ways of measuring the growth of business intelligence vendors. One approach of interest in the era of self-service analytics is to measure the growth in web search volume. Derived from web search volume data from Google, the following analyses can serve as a useful reference to understand which companies/products are growing in popularity and which may be falling out of favor.

The estimates in all of the following analyses are based on simple web search volume indices from the United States through the end of November, 2013. Using historic search volume data, forecasts were built for each company/product and growth rates for 2014 were derived from these forecasts.

I would group these companies into three categories
fast growers- Tableau, PowerPivot, Qlikview, BIRST and GoodData;
the growers- Spotfire and Microstrategy,
and mature products- Oracle BI, SAS, Cognos, SPSS, SQL Server, Actuate and Business Objects.
Continue reading