google search council and google's own index give you a lot of information but they don't tell you everything. They don't for instance tell you if all of your title tags are unique or if you're using the appropriate rel canonical tag for that U. R. L. For this. We prefer using tools like screaming frog, deep crawl and site bulb. Now each of these tools attempts to imitate what google bought or google sees but they aren't perfect and in fact they tend to discover different problems. So if you have the luxury of buying all the tools I'd recommend it. Or if you can find a freelancer on fiber who has all these tools out of recommend them having them run all the audits with each tool with that said I'm gonna walk you through my personal favorite tool which is sight bulb and I'll show you how to set up a crawl, read the report and walking you through this, you'll be able to get the gist of how to use the other tools. The first thing I should mention right off the bat is that there's three di...

fferent types of crawls, Discovery crawls where you're searching for unknown unknowable problems. You didn't know what was wrong site map crawls where you check the same you or else to see if something happened each week each month and then there are Q. And a crawls where you cross subset of U. R. L. S. After the team has updated something. Or maybe you were curious about a folder. So you'll crawl just that folder for the first time you audit a site, I highly recommend starting with the discovery crawl rather than just crawling the site map as our goal is to discover the problems that we didn't know about finding those weird you RLS that our site accidentally creates and all that kind of stuff. So next we want to set up our crawl site bulb has to resources that walk you in detail for the whole set up of the crawl. I'll link to them in the comments but let me walk you through my quick setup, punching the domain in this case it's kids health dot org. Note that some sites may prevent you from crawling easily try health tap for instance and you'll see all these errors pop up and it might be hard for you to crawl it, but you can work with a specialist and they'll likely find a way around it. So check out some of the technical SEO specialists on fiber and they'll help you crawl different websites, change little euros here and there. Next we want to tick off the page speed even though we will run a separate tool for page speed. Finding page speed. Like this is very helpful because the other tool will use, although it is more robust in its analysis, it doesn't crawl every euro like site bulb will we also want to check off the page resources which will instruct the crawler to touch every asset on the site. It will get every image, every plug in, every CSS script, every everything on the site. And it's important because if we have a bunch of broken images on the site, this will catch that. If you verify the site, all writing google search council, then tick off google analytics and google search council. Then finally check crawl website and xml site maps to give yourself the broadest possible crawl. If you don't click xml site map, then the crawler will only start from the homepage and spider and touch every link that it discovers from the homepage. But if you have a whole bunch of links that aren't on the homepage, it may not reach them. So when you check xml site map and crawl website, it will start its crawl on every you are L and go from there. So you're likely to cover the entire site. And if you're crawling really, really large sites, you should check out these advanced options below. Um I would also mention that you should save html so that you can track the changes on a week to week basis. I use another tool for this that has a really great ui that I mentioned earlier called S. C. O radar. But if you could only have one tool for technical seO I'd recommend site bulb and just saving the html for every crawl that you do. Finally to mention before you start your audit, you may want to let your development team know about the crawl before you run it, it's possible your crawler could tie up important resources for their servers, which would cause the site to crash, especially if you're running during peak times. Um, It's happened a few times for me, um, and I owe it to my friends and engineering to mention this. So without further ado, let's press go.

Class Materials

Bonus Materials with Purchase

SEO Research Spreadsheet

buy this class

$00

Buy Class

Sale Ends Soon!

starting under

$13/month*

Unlock this classplus 2000+ more >

79.7K Students

99% Recommend this Class

00 Student Work

0 Video lessons in HD
00h 00m of class content
Exclusive bonus content
Lifetime access, anywhere, anytime
Streaming access on desktop and mobile browsers
Download and offline access with class purchase
Offline sync with our iPhone & iPad apps
100% satisfaction guarantee

Cleo Kirkland

An SEO expert, entrepreneur & business owner with 12+ years as an in-house specialist & consultant

2,000+ classes ranging from fundamentals to advanced techniques.

Setting up Crawler Tools

Setting up Crawler Tools

Lesson Info

11. Setting up Crawler Tools

Lessons

Intro- Course Overview

What is a Search Engine, and how do they work

What Kind of Work do SEO Practitioners Do

Popular Tools for SEO

The Evolution of Google's Algo

Quiz: Chapter 2: SEO 101

Analysis Introduction

Reporting your Findings

Checking Google's Index

Checking Google Search Console