Dealing with data: The Semrush approach to data science

Web-based technologies produce increasing amounts of data every day. This data can be utilised by businesses to create new efficiencies, develop trend recognition capabilities and attract new customers and clients, or it can overwhelm and frustrate marketing, business development and other vital teams.

Just a few years ago, businesses had the choice of utilising data in order to gain competitive advantage or finding an edge another way. Today, however, there is no longer an option. In business, data is everywhere and in everything. Managers must find a way to turn the data they’re receiving into business intelligence and insight.

To do this, many are turning to tools such as Semrush, which automate the most important data analysis functions, and more, to provide clear and tailored advice from big and unstructured data. Some of the sharpest minds in data science work at Semrush to keep the brands at the top of a highly competitive field.

We spoke with data scientists working at Semrush to find out how they manage their own data challenges, to shed light on how other businesses might do the same:

What role do data scientists play at Semrush, to ensure the platform does its required job of supporting business marketing activities?

Internet technologies produce more and more data every day and businesses using that data wisely can discover useful predictions, recommendations, insight and much more. That is what we strive for at Semrush. Every day we work on developing and implementing new, high-level data science models. Our main goal is to make it easy for customers to see the clear picture through what can sometimes be a confusing and overwhelming amount of unstructured data. To do this successfully, we need to understand our own data. So that is where we begin.

What benefits does such data analysis bring?

Data analysis is essentially about understanding the world around us. Businesses have huge technical capacities and can store terabytes of data. But this introduces a new challenge – how to benefit from this huge amount of data. For this, we need a highly professional, motivated and involved data analysis team. Great value then comes with the ability to automate particular routines. This is about the automation of data analysis and the offering of a constant feed of valuable insights for our customers and their businesses.

How does Semrush use data science for its own innovation?

We separate our data science specialisations into three streams:

First, there is our PR Data team. This, in itself, is an innovative concept. It is about using data analysis for PR and marketing purposes. The PR Data team carries out projects that involve finding out exactly what individuals, customers of ours and customers of our clients, are interested in hearing about at any specific time.

Next is our Data Lab, which is our main research team in the field of machine learning. Their research is aimed at finding data patterns that can reveal special insights within our business and for our customers, patterns that can tell a fascinating story. For instance, we’re looking into all of the possible reasons that a website might be ranked on Google in a particular position, as opposed to relying on the obvious ones. These insights can help our customers improve their online visibility.

Finally, our Analytics team is mostly focusing on internal tasks, particularly customer support. We use data science to research user behaviour so that we can suggest to customers which Semrush tools would be most useful for them. One project, for instance, is devoted to recommending to our user's specific Semrush tools that they have not yet tried, but from which they would likely benefit.

How has the analytics team improved user experience?

One of our latest projects was about churn prediction. We identified users who were the most likely to give up on our services. Armed with that information, our customer success managers contacted those users and provided them with valuable demonstrations and other useful information. Those users were better able to understand the value of the Semrush offering in solving their unique business problems.

How does the PR Data team discover what people are interested in hearing about?

We use mathematical statistics and machine learning techniques to analyse search trends, websites and social network content, domain statistics, SEO metrics, etc. We do both simple research (eg. Inhabitants of which city in the USA google ‘Black Friday’ the most?) and more complicated research (eg. Analysing website content to understand the hottest topics in a territory). The PR Data team doesn’t just work on online marketing but on broader news topics, too. They research the most popular celebrities for specific markets. They look into attitudes around presidential elections. They provide interesting and relevant data analysis and this is exactly what makes Semrush more visible than its competitors. In general, we can say that this team evolves data-driven journalism.

Those studying data science are taught that research is an important part of their role. Is this the case within your business?

Absolutely! Research is an essential part of any data scientist’s work. All of our data scientists and data engineers conduct research. They then cooperate with our software engineers to ensure that the results of their research affect the decisions made in the production process.

If a data scientist was to join the Semrush team, what would a typical day look like?

Professionals in the Data Lab begin with a customer’s problem, generate a hypothesis, look for data and pull it together to validate their hypothesis, apply different modelling methods, then check their results.

Those in our analytics team identify problems, prioritise them, develop possible ways of solving them and apply different models to get results.

Our PR data experts work very fast to keep up with trends. They collect ideas from PR teams and combine them with their own. They choose the most trending and interesting information of the time, develop ways of conducting research around those topics, and develop a thorough analysis as quickly as they can.

So there is varied work done at different speeds by data scientists working at Semrush?

The type of work and the deadlines depend on two things – what data is used and when the result is expected. For example, the PR Data team works at a very fast pace. Most of their research must be turned around within 24-hours to ensure we, and our clients, do not miss the opportunity of ‘newsjacking’. But our Data Lab and Analytics teams focus on long-term goals such as improving Semrush tools and customer support algorithms in order to make our product, and the customer experience, better.