This post was originally published on this site.
Data analysis has no shortage of AI tools, making it overwhelming to determine what’s available and what might help when it comes to analyzing huge datasets. Let’s try to sort out at least some of the confusion by examining some AI tools that work within existing data analysis workflows.
Chatbots and Virtual Assistants
Chatbots are ubiquitous to any type of programming work these days, especially the newest generative AI chatbots (such as ChatGPT). These tools can offer help with data analytics in several ways. For example, they can offer coding advice. If you’re writing some Python code, for instance, you can copy it into the chatbot and ask for help.
Chatbots can also be used for teaching and training, such as suggesting which tools are best for a particular job. Here are some general purpose chatbots popular among data people and coders:
- ChatGPT: Everyone knows this one, of course; while trained on a whole world of knowledge, ChatGPT also excels in coding and data analysis. The newest versions of ChatGPT (starting with 4) can perform data analysis by writing Python code and then running that code.
- Gemini: This is Google’s answer to ChatGPT, and it’s arguably every bit as good. You can provide it with data which it will analyze.
- Github Copilot: This is a plugin tool that adds AI chat and code suggestions for coding IDEs such as Visual Studio Code. Although based on the technology behind ChatGPT, Copilot has further training in different coding languages and technologies.
- Amazon Q: Amazon Q is a whole set of products, but at the heart of it is generative AI for assistance with various tasks. Amazon has integrated Q into many of its services, including its data-related ones. For example, Amazon’s Quicksite allows you to use natural (i.e., human) language to analyze data and even generate dashboards.
Data Cleaning Tools
One of the biggest problems data analysts confront on almost a daily basis is having to deal with messy data. As any data analyst can tell you, data can come from multiple sources in multiple formats, and it’s not always reliable.
With the help of statistical tools, data analysts become adept at “cleaning” the data by removing bad data or correcting it. For example, if the data analyst is analyzing gas prices, and they find one gas station that was charging $0.03 per gallon, likely that’s an error that needs to be either corrected or removed; otherwise, it will completely skew all the results.
Cleaning data can be a long, cumbersome process, even though it’s a regular part of the job. Thankfully, there are now AI tools that greatly help with locating data that’s problematic and automatically dealing with it. Here are some tools you will want to learn that use AI to clean data. (Note that these are premium tools, not free.)
- Trifacta: This is one of the bigger names in the business, and it was recently acquired by a company called Alteryx.
- Dataiku: This product uses generative AI to help you clean your data.
Generating Sample Datasets
When learning data analysis or testing out data apps, analysts need sample data to work with—ideally as realistic as possible. There are websites that provide sample data, but those datasets can be filled with redundant data that can throw off your tests. On the other hand, manually constructing hundreds of thousands of data rows can prove too time-consuming.
This is where AI data generation tools can help, including:
- Tonic.ai: This product bills itself with the name “real fake data.” On their website, they call it synthetic data.
- Mockaroo: This a free tool that uses AI to generate sample data.
Storytelling
After analyzing data, analysts typically present a story to the stakeholders explaining their findings. This storytelling includes written narratives, visuals such as charts and graphs, and recommendations. AI tools exist for all three of these aspects of storytelling.
Both PowerBI and Tableau, which we mention in the next section, now provide these features. But here are a couple more you’ll want to explore:
- Wordsmith: This is a product by Automated Insights that uses AI and starts with natural language generation.
- DataRobot: This is an entire suite of tools that includes a sophisticated system that uses AI for storytelling.
AI Add-ins to Tools
Data analysts have multiple tools they use on a regular basis. Let’s review some of the AI additions to these tools.
Power BI: Microsoft has added its Copilot AI features to PowerBI, and it can help in several different ways. For example, Copilot can help write DAX (Data Analysis Expressions) queries. (Pro tip: You’ll still want to learn as much DAX as you can, but Copilot can help you work faster here.) Copilot can also do an analysis of your data set and provide you with a quick summary of it, and even provide charts and graphs in the summary.
Tableau: Tableau now includes the Tableau Agent, which is a chatbot that will interact directly with Tableau via prompt. For example, you can ask it questions about the data and it will answer… and build charts and graphs as well. The marketing material for Tableau Agent says it’s good for new analysts; however, we would argue that experienced analysts will want to become familiar and comfortable with it, because such plugins are quickly becoming the norm for all levels, not just beginners.
Excel: Microsoft has added its Copilot AI technology to the entire Microsoft Office suite, which includes Excel. In this case, Copilot includes a chatbot that can interact with your sheets and manipulate them, as well. For example, you can ask it to create a bar graph based on data in a particular set of columns in a sheet.
Google Sheets: Google now includes its Gemini AI as an optional premium add-in to Google Workspace, including Gmail, Docs, and Sheets. Because we’re looking at data here, let’s focus just on Sheets. As you would expect, you can ask Gemini (via prompt) about information within your sheets, as well as use it to generate sheets. And as with Copilot and Excel, it’s probably more suited to less technical people.
Jupyter Notebooks (including Google Colab): Jupyter Notebooks now has an extension called Jupyter AI: the chatbot is called “Jupyternaut” and, like other tools mentioned here, can understand your prompts and interact with your notebooks. Because Jupyter is like an IDE, you can highlight code inside a notebook and ask Jupyternaut to analyze it for you. You can ask it questions about your code, and then have it modify your code with updates or even add comments. You can even have it create new notebooks for you that serve a particular purpose.
For example, in the blog linked above, there’s a demonstration of how you can ask it to create a notebook that shows how to use Matplotlib. Things like that can be useful for learning how to use Jupyter and the various Python libraries. You can also choose what LLM it should interact with; that’s an advanced feature, and an interesting one, especially if you work for a company that has developed its own LLMs.
Note: The Jupyter developers have gone to lengths to make sure that this plugin works across different Jupyter implementations, including Google Colab.
MATLAB: This one is a bit of an oddball, as you’ll see shortly, but we’re including it here for completeness. MATLAB, from Mathworks, is a computation and mathematics engine that was originally built back in the mid-1970s. MATLAB has continued to grow throughout the decades and is popular today in many fields, including data analytics.
Today, MATLAB offers AI tools and capabilities, including ones that help you create and manage AI models and integrate those models into your code, while also helping you develop data workflows.
Other Tools
While you’re at it, you might explore some lesser-known tools to see how they can help you. (And this helps the industry grow too by giving the smaller names a chance.) Here are a few we looked at:
- DataChat: This product is billed a “no code” analytics platform that uses AI to help you analyze your data. It lets you ask questions about your data, and it then uses generative AI for analysis and answers. It can also work closely with Google’s BigQuery data warehouse system, as well as HubSpot data.
- Zoho: Zoho has offered an online office suite for many years, including a spreadsheet tool. Recently they have added generative AI features, including an assistant with the clever name of Zia.
- Domo has been around since 2010, and they provide cloud-based tools for business intelligence and data visualization. Its new AI tools include a chatbot for questions about your data; it will provide quick answers and produce summaries of even the most complex datasets. Â
Conclusion
Today’s data analysts are needed more than ever. Data analysis positions are proliferating, and the specialists in these roles are tasked with incredibly complex projects. Given the pressures of the job, it’s critical for data analysts to learn and potentially master as many AI tools as possible, as this will allow them to become more productive.