How to Source Data for Analysis

What is data?

When it comes to data analysis, the first question to ask and answer is “what is data?” Data is structured information captured about people’s and system’s behaviors. 

People’s opinions are not data, they’re anecdotes. But even anecdotes can become data when you have 100s or 1000s of them. 

Every business today has data, whether they’re collecting and analyzing it or not. This includes things like:

• The price and quantity of purchases
• The frequency and day-part of purchases
• The content & resolution time of customer support tickets
• Financial metrics
•  The rate of product bugs or defects
• Customer & employee satisfaction scores
• And much more… 

Almost anything a business does, especially routine processes, can be instrumented to collect data. This could be as simple as making employees fill out a timesheet. Or it could be something like installing time tracking software or as complex as setting up lots of custom product events (click tracking, key logging, etc). 

Clearly, data-driven decision-making is relevant to every business. All you need to do to start down this path is to start with a question. 

How to start an analysis project

A great question is typically what kicks off a data analysis project. This could be a question you think of on your own or something that comes from an executive or a client or any other stakeholder. The questions can range from typical to highly complex:

• What’s our average contract worth?
• What’s our Net Promoter Score?
• What does it cost to acquire a new customer?
• How do our customers find out about us?
• Which of our product features is most valuable to our clients?
• Why do clients leave our firm?

Finding data sources

There are internal data sources and external data sources that you can rely on. 

On the internal side, most modern enterprise software products contain analytics suites from which you can export data. 

• If you want data on sales, you can pull information out of your Customer Relationship Management (CRM) tool, such as HubSpot or Salesforce
• If you want data on project management or task completion, you can pull that from whatever project management tools you use, such as Asana or ClickUp
• If you want to understand your customer service performance, you can grab that from a ticketing system such as Zendesk, Kustomer, or Front
• If you’re interested in email marketing performance, that data lives in tools like Mailchimp, ActiveCampaign, Customer.io, or Outreach.io
• If your company builds software, you probably collect product data in a proprietary database

To access such data, it’s just a matter of understanding which tools your company is using and for what purpose. In an ideal state, the company may have documentation on which tools it uses and what data is collected in each. More likely, this documentation partially exists or doesn’t exist at all. 

On the external side, it’s best to look at the press, government data, 3rd party data sources, and academic research. Some of the best sources are:

• General government data: https://www.data.gov/ 
• Bureau of Labor Statistics data: https://www.bls.gov/ 
• Public company filings: https://www.sec.gov/edgar/searchedgar/companysearch.html 
• Federal Reserve data: https://fred.stlouisfed.org/ 
• General business data: https://www.statista.com/ 
• Startup data: https://join.trends.vc/ 
• Venture capital & private equity data:
• https://pitchbook.com/
• https://www.cbinsights.com/ 
• https://www.crunchbase.com/ 
• Market reports:
• https://www.canvas8.com/ 
• https://www.stylus.com/
• App store data:
• https://sensortower.com/ 

There are many more sources of external 3rd party data, but this list will be a good start.

Collecting data

After starting with your question and identifying the right data sources, it’s time to pull together the data you need. 

If you’re pulling single data points, it’s best to collate them in a single spreadsheet like so:

If you’re pulling entire files, it’s best to collect them into a single folder in a cloud storage system like Dropbox, Google Drive, or similar. Set up a file naming convention that lets you identify the dataset in the future (e.g. {name} - {source} - {date pulled}) so something like “Contract Values - Salesforce - 10.16.2021”.

At this point, you’ll have the data you need to start doing some meaningful analysis!