Keyword research is the bread and butter of every SEO strategy, but if you ask any experienced SEO what their least favorite task is, chances are that this will be right up there.
Unfortunately, the less enjoyable AND more important a business process is, the more likely the output will not be ideal.
What I want to show you on this page is how recursive keyword research is done the right way and how frustratingly time-consuming and manual the process is.
Then, I’ll show you how you can achieve the same end result (and more) ten times faster.
Does a 10X process improvement sound like a bad idea?
Let’s see how that’s possible.
What Is Recursive Keyword Research?
First, let me give you a quick introduction to recursive keyword research. I learned this method from Matt Diggity, and it’s by far the best way to find all relevant keywords in your niche.
The process involves getting all the keywords competitors are ranking for, including all the subtopics in the niche, and creating a huge master list of keywords.
By using tools like SEMRush or Ahrefs, you reverse engineer your competitor’s pages by extracting organic search keyword reports. Using these tools, you get keyword reports for the top five or ten pages in the SERPs.
As you go through this process, you’ll find many keywords that you would target on a separate page, and for each one of those, you repeat the process of extracting the keyword lists for the top competing pages.
At a high level, the manual process involves extracting keywords for the top ten competing websites using tools like Google Keyword Planner, SEMRush, and Ahrefs, putting them all in one spreadsheet, deduplicating them, organizing them into separate silos and clusters and individual tabs.
All by hand for potentially tens of thousands of keywords.
More on the exact process shortly.
The goal of this process is to sweep up all of the keywords in a niche and organize them into topical clusters and silos to create your site’s overall architecture.
You’re not just going to get the high-volume ones with buyer intent but also the long-tail ones from much earlier in the sales cycle.
Why Is Recursive Keyword Research So Important?
When you look at the top ten SERPs for any search query, what you see is a representation of what Google believes are the most relevant pieces of content.
There will be some overlapping content on each of the sites, but imagine if you could map out all the keywords these sites rank for.
Now, imagine your site covered all those keywords in ideal clusters, resulting in hundreds (or even thousands) of pages.
That would make you stand out as an absolute topical authority on the overall subject.
The way you achieve that is with recursive keyword research, which essentially means reverse engineering the content from the top ten search results.
And here’s how you do that.
The Recursive Keyword Research Process
There are four steps that you need to follow.
1. Create A Master List Of Keywords
Let’s say your main target keyword is “best stuffed animals” for an affiliate website. You would enter that master seed keyword into these tools and then save the results into a CSV file:
- Google Keyword Planner
- Google Search Console
- Google Trends
As you can imagine, that’s a slow process, but it will result in a very long list.
As an extra step, you can use Ahrefs Keyword Explorer to export the keywords for the top ten SERPs in one go. It’s faster and is highly specific to the top ten SERPs.
2. Clean Up The Keyword List
In this step, you need to remove all the duplicates from the list. One way to do this is to use a tool like TextMechanic for deduplication.
It’s a critical step as you don’t want to spend more time on the next step than you have to.
Believe me, after about 30 minutes, you’ll be begging to watch paint dry instead.
3. Cluster Keywords Under Silos
Now, the “fun” part starts.
With all the keywords in a spreadsheet, you need to start going through the list and grouping those that belong together.
Using our “best stuffed animals” example above, you might have one page for “best stuffed poodle” with all the related and long-tail keywords.
Another cluster might be “best stuffed labrador” with its related keywords.
Each cluster goes onto a separate tab, and it’s a case of copy-pasting hundreds of keywords to different tabs.
Now imagine having 100+ tabs.
This can easily take a full day, even several days, depending on how many silos and clusters you end up with.
And it’s a mind-numbingly repetitive task where you can easily make mistakes.
While I loved the outcome of Matt Diggity’s process, I also knew that there was an easier and more effective way to do it.
And here’s how I implemented the process into Xagio.
How Does Xagio Improve The Process?
I’ve used recursive keyword research for a very long time on both my affiliate and client SEO sites. But the process seemed more painful than a root canal with a blind dentist.
That’s why I created specific features within Xagio to make it at least ten times faster.
Let me show you how it works.
1. Create A Master List Of Keywords
The Xagio development team has created three different ways to create a master keyword list.
Import Keywords
The first thing you can do is follow the manual step 1 above and use Google tools or Ahrefs to create a list of keywords for the top ten SERPs of your primary target keyword.
You can then import the CSV file directly into a new project planner, and Xagio will work on analyzing the keywords. There are also tools that allow you to manually refine how you want to cluster the keywords (more on those shortly).
But this is still quite a manual process that involves multiple different tools. That’s why I created the AI wizard.
AI Wizard
Xagio includes an AI project wizard that allows you to define a set of main keywords, and it will then do all the heavy lifting to create a master list of keywords.
You don’t need to use any Google tools, and there’s no need for an Ahrefs or SEMRush subscription either. Within minutes, you’ll have a huge list of the most relevant keywords.
And it gets better.
What Xagio will do is sort the keywords into related clusters using AI technology. That step alone can save you 5-6 hours of manual work for a relatively small number of silos and pages.
Website Audit
A third option Xagio provides is executing a website audit. You can enter one of the top-ranking websites for your target keyword, and Xagio will then pull in all the relevant keywords and cluster them in the same way that the target website has pages set up.
From there, you can use global or page clustering to further break down pages into separate clusters based on specific seed keywords you want to target.
All three options are highly effective, but the AI wizard is by far the most comprehensive and hands-off way to get a pre-clustered list of keywords.
2. Project Planner
Once you complete step 1, you’ll already have a new “Project” in the Project Planner section of the Xagio dashboard with pre-clustered keywords. The next step you can take is to review the clustering and either break up groups (clustering) or combine them based on a single word (seeding).
The first thing you’ll see is that all the keyword clusters are on one page. You can easily drag and drop single or multiple keywords between clusters if you spot any obvious mistakes.
Add a quick video showing how the drag-and-drop works
What Are Projects And Groups?
Projects and Groups in Xagio are what give the Project Planner its supreme organizational power. They allow you to organize large lists of keywords into manageable information that creates your site architecture.
Let me explain with the help of an example and visual aid.
My development team came up with a unique in-house technology to separate keywords into groups. Each group contains related keywords that become a separate topic for a page or post on your website.
Projects are folders within Xagio that contain any amount of groups within them. What’s powerful here is that you can create a project for each topical silo, which contains all the relevant groups for that topic.
This is how you might start logically breaking down our “best stuffed animals” topic:
PROJECT 1 (SILO 1): Stuffed Dogs
- KW Group 1: stuffed labradors (all related keywords)
- KW Group 2: stuffed collies (all related keywords)
PROJECT 2 (SILO 2): Stuffed Cats
- KW Group 1: stuffed Siamese (all related keywords)
- KW Group 2: stuffed Persians (all related keywords)
And this is what a visual representation looks like for projects and groups:
I’ll get to another part of Xagio called Silo Builder shortly, which provides even more functionality for managing groups and projects.
Managing Your Keywords In Projects And Groups
One way to manage your keyword clusters is to bundle them into projects that become a silo.
Returning to the keyword “best stuffed animals,” you can better manage all the pages you need to create by having one Xagio project for the dogs’ silo, one for the cats’ silo, one for the birds’ silo, etc.
Within each project, you can apply logic to either globally cluster or re-cluster all the keywords in every group within a project (how powerful is that) in a different way or apply logic to individual clusters that you think could be further broken down.
And if you find a keyword cluster that belongs in a different silo project, then you can simply move it to the most relevant project.
Add a video of the clustering logic screens with seed keywords
Comparison To Manual Recursive Keyword Research
Let’s revisit Matt Diggity’s recursive keyword research process outlined above and compare it to how you work in Xagio.
In the manual process, you start off with a spreadsheet that has one long list of keywords. This is where you can use the Xagio AI wizard or import a CSV file.
Then, you would use another tool to deduplicate the keywords, which Xagio can do for you as well.
You break those keywords down into silos of related keywords by creating new tabs on a spreadsheet. Xagio can do this for you using proprietary technology to bundle keywords into groups, and you can further define your own logic for creating groups of keywords.
Once you have refined your groups of keywords, you can then separate them into projects so that you have easily manageable and visible silos.
Rather than copy/paste individual keywords from one tab to another, you can simply select multiple keywords and drag and drop them to another group. Once you’re satisfied with the keyword groups, you can then select multiple groups and move them to different projects.
Remember, each project is a silo and is equivalent to a separate tab on the manual process spreadsheet. Xagio makes it so much easier to move groups of keywords to different projects without the need to copy and paste.
It’s an extremely fast and easy process that eliminates human error, copy-paste mistakes, and, more importantly, a massive amount of frustration.
Think of it this way.
Each project within Xagio is a separate spreadsheet in Matt Diggity’s process, and each group within a project is a separate tab. Now, consider that you can have as many projects and groups as you want, and you get to see the power and convenience that Xagio provides.
No more managing multiple spreadsheets and completely losing track of what topics you’re trying to cluster on each tab.
Yes, that’s how easy it is to create your clusters with your mouse while avoiding hand cramps from endless copying and pasting.
3. Xagio Silo Builder
This is where Xagio becomes really interesting and powerful.
Once you have defined all your keyword groups and created pages (even placeholder pages), you can jump into the Silo Builder and see all the bundles in a visual map. What this allows you to do is drag and drop pages into silos with a defined parent page.
Here is what a page silo looks like.
Once you’re happy with your silos, you click on save, and Xagio will go into your WordPress dashboard and create the parent-child relationships for you.
Yes, that means no more manually setting the parent on each silo page, which can save hours of work.
Silo Builder also works for published posts where you can see a visual map of how posts are linked to categories and tags. By simply adding and removing different links between posts and categories, you can map out the relationships.
Once you click save, all the changes will automatically update your posts, saving you hours of manually changing them.
4. Xagio AI Page Builder And Optimization
Another very powerful feature of Xagio is the ability to create pages for each keyword group in your projects using AI technology.
Let’s say your keyword research has revealed that you need to target the keyword “best stuffed poodle” on a separate page.
There are two things you can do to create page content and some on-page optimization with the click of a button.
First of all, you can use Xagio’s chatGPT integration to create an outline and content for the page using pre-defined parameters. It’s a great starting point that you can then further build out with human or AI content.
Secondly, you can use Xagio’s AI optimization feature to come up with a URL slug, Title tag, and H1 Page Title, aka, the 3 Kings, as Matt Diggity calls them.
I’ve created a separate page on the 3 Kings, but with this fourth step, you’ve taken recursive keyword research to a whole new level. You now have your keywords clustered into groups, separated into silos, and you have a page set up with the most important on-page optimization details.
And you can do all this in 10 times less time than manual research and clustering alone.
All Of This Happens In One Tool
The two most important things you need to take away from this is that Xagio will do all of the above within one tool, and it will save you at least 80% of your time and effort.
Imagine how much less frustrating your on-page SEO planning will be and how many more clients you could take on with all that extra time.
I’m so convinced that you will never want to go back to any other tool for recursive keyword research that I offer a 30-day full money-back guarantee.
Sign up today and see how fast you can complete the full keyword clustering process that would normally take you at least a full day.