This is part two of a five-part series filled with actionable tips that you can implement to optimize your website for search engines:
- Do your keyword research
- Make sure search engines can crawl your website
- Create content that search engines will love
- Build authority so search engines will trust your website
- Implement best practices for your website
Search engines discover content by “crawling” your website. It sounds a little creepy, but basically, the search engine goes to your website and then attempts to visit every page. That’s how it knows your website and all of its pages exist. As the search engine crawls your website, it will attempt to index every page so people can find them on the search engine.
WordPress Users Read This First!
Crawl Your Website with Google Search Console
You can use the Google Search Console to make sure search engines can crawl your site. You will need to verify that you own your website to use this tool. For that reason, I recommend setting up a URL Prefix property instead of Domain Name (for now), as that’s much easier to do for a beginner. Just follow the instructions they give you to verify the property. Then, take a moment to set up a property to check your website before you continue reading. Finally, join the Get Your Business Online Facebook group and ask for help if you get stuck.
After you set up your property, Google will attempt to crawl it. If it throws back any errors, you will need to address them because an error means search engines can’t crawl some or all of your website. One of the most common errors I see with my DIY clients is “Blocked by robots. txt.” That means a text file in the folder where your website lives is blocking the Google bot from crawling the website. If you have a WordPress website, you can go to Settings > Reading and uncheck the “Discourage search engines from indexing this site” option.
After Google Search Console crawls your website, you will get a Coverage report. It has filters for Errors, Valid with Warning, Valid, and Excluded. Click on a filter to see the relevant pages in the report. If you have a lot of “Excluded” pages, check the “type” to see the reason. As you can see in the image below, the STEM summer camp website has a lot of excluded pages, but they are excluded on purpose because they have a “noindex” tag assigned to them. If you find pages that you want to indexed in the excluded report, you might need to fix the page to make that happen.
Inspect a URL
Another nice feature of the Google Search Console is the URL inspector. You can enter any URL on your website in the search bar at the top of the page to see if Google currently indexes it. If it’s not, it will let you know, and you can request indexing of the page. Google will let you know if it has any trouble indexing the page.
When you search for your website on Google, you can use the URL inspector to re-index the page if something like a title comes up wrong. Just inspect the page with the problem and click the “Request Indexing” link to have Google index the page again. That should fix the problem.
One thing to note about the URL inspector is that it will let you know if a page is mobile-friendly. That’s very important to search engines. Mobile-friendly means that the pages on the website resize according to the screen’s dimensions. Have you ever had to pinch and drag to view a website on a phone? That website was not mobile-friendly! Suppose you have to do that with your website. In that case, you should consider a redesign very soon because no matter what other optimization you do, many search engines will rank your website lower because it’s not mobile-friendly.
Submit a Sitemap
You can make the search engine’s job easier by submitting a sitemap. The sitemap is an XML file that tells the search engine which pages on your website are important. Below you can see a screenshot of a sitemap XML file.
Websites don’t just come with a sitemap. Instead, you have to create it. But don’t worry, you don’t have to type out the XML file. You can if you want to, but I don’t recommend it. Some content management systems, like Wix, create them for you automatically. You can install a plugin like Yoast SEO to automatically create your sitemap if you’re using WordPress. If you don’t have a tool built into your website to generate the sitemap, you can use a tool like Slickplan to make it.
Take Action: Make Sure Your Site is Crawlable
Now that you know a bit more about making your website crawlable, I want you to take action and check your website. If you followed along with the text in the post, you might have already set up Google Search Console for your website, but if not, go ahead and do that now. Then I want you to inspect a few pages from your website to make sure Google can crawl them and see if you get any errors. After you finish, feel free to share your results on the Get Your Business Online Facebook page.
Need More Help?
Are you feeling overwhelmed? Pongos Interactive can help you with your SEO needs. Contact us to get started.