Technical SEO : Complete And Definitive Guide

Updated On 03, Sep, 2023

What Is Technical SEO?

Technical SEO is the process of technically optimizing your website to be easily accessible and discoverable from search engine crawlers. Also, Technical SEO involves optimizing the UX elements for better experience.

Among many other activities involved in technical SEO, we will discuss:

Creating and submitting XML Sitemap for your website
Crafting SEO-friendly slugs (URLS)
Optimizing Core Web Vitals
Mobile-Friendly website design
Spot and Fix duplicate content issues
Using Schema.org Markup
And many other aspects

In this definitive guide we will explore all the aspects whereas Technical SEO relies on.

Why Is Technical SEO Important?

Simply because Technical SEO can impact website performance in Google and major search engines.

Think of it, if you have the best content ever created. But your website can’t be technically reached? The result will be your pages won’t appear in search engine ranking pages. Because the website is not accessible.

Accordingly, this can result in a huge and tremendous loss in traffic which interprets into loss in revenue and leads too.

In addition to this, Google already confirmed that mobile-friendly websites and fast-loading websites are ranking factors.

If your website pages take time to load , users will bounce outside your website. And this can give a hint to Google that the website doesn’t have user experience signals. In a reply to that from Google’s end, your web pages will be negatively impacted in SERPs.

Now, let’s move to core concepts which are the cornerstone of Technical SEO.

Let’s Begin

What Is Crawling?

The initial step in Technically optimizing your website is to make sure search engine crawlers can easily crawl your website with no issues.

Crawling is an essential module whereas most modern search engines implement it in their algorithm.

What is Crawling? Answer is : Crawling occurs when search bots crawl a page it already knows about, to discover and follow links within this page for other pages it doesn’t know about.

Practical Example, whenever we publish a new article to our blog these articles automatically get added to the blog and their corresponding categories.

Google already knows about our blog page and so as category pages. From those known pages, Google will then discover the newly created article. This is crawling in essence.

In order to maintain an easy crawling process. There are few ways where this can be achievable.

Creating SEO-Friendly Site Structure

Site Structure or Site Architecture , defines how your web pages or URLs are linked together.

What differentiates a good website structure from a bad one, is how easily search crawlers can find website content easily and quickly.

At the end of the day, you should aim for a flat and clean website structure as Backlinko Proposed in his blog post , maximum three-clicks away from the homepage to reach the desired page.

Like This :

https://dbrandria.com/ => Homepage

https://dbrandria.com/services/ = > Services Page

https://dbrandria.com/services/online-brand-delivery/digital-marketing/seo/ => SEO Services Page

Like so, Our SEO Services page is easily accessible from the homepage in just a few clicks.

Brandria Main Menu, Links to Services Page and Services Page links to Service Category pages.

Also, remember that when linking all of your web pages using this way, you will leave no place for orphaned pages.

Submit Your XML Sitemap File To Google

Sitemap files are XML-Format files which help Google and other search engines find all of your web pages.

Search engines use XML Sitemaps to easily find and discover where your pages are hosted on your website and how to find them.

Sitemaps come in handy when the website has a big number of pages. Or there is no clear internal linking structure to define how those pages are linked.

This is How Digital Brandria Sitemap Index File Looks like :

Usually and in most of the cases your Sitemap file will hosted under two routes :

/sitemap.xml
/sitemap_index.xml (Like in our Case)

Once you locate where your Sitemap lives, take its URL and submit it to Google Search Console in the Indexing -> Sitemaps Like shown in the image below.

Click on Sitemaps and the image below should appear to you.

Then add your sitemap and click Submit

What Is Indexing?

After the Crawling phase, search engines begin to understand the content and context within each crawled resource or URL.

To answer the question, what is indexing? It’s how search engines store and organise data within their Search Index .

Right after, Search Engines store these data-sets within a big cluster of data called “Search Index”. Put simply, Search Index is a huge database storing billions or even trillions of webpages which is being crawled each day.

In order to appear in search results , your website must be indexed in the first place. And one of the easy methods to discover whether your website is indexed or not is to use the advanced search operator “site:” followed by your domain name :

Site:yourdomain.com

By plugging this query into Google search Box , the number of indexed pages from your website will be returned like this :

Querying our website using site: advanced search operator

We have in total 130 pages indexed for our agency website.

At the same time, the “site:” advanced search operator can also check a single URL from being indexed in Google or not. Like the image below :

Checking Single URL Indexed In Google Or not.

In the above image we have checked our Off-Page SEO Guide using “site:” advanced search operator and it returned that it’s already indexed from the publishing date.

Some factors may hold the website URLs from being indexed like “Noindex Tag”, “Canonicalization”, limited Server resources and other factors. Let’s delve into each one in detail to understand the Indexing Phase landscape from a wider angle.

What Is Noindex Tag?

“Noindex” tag is a HTML snippet which is being placed in the <head> part to tell search engines “Do Not Index this page”.

It’s looks like this in normal cases :

In normal cases, all of your important pages and assets need to be indexed. While excluding unnecessary pages using the “noindex” tag. Pages like :

Thank You Pages – Usually, used for conversion tracking
Landing Pages For PPC Campaigns – Most agencies create specific landing pages as per media buyer requirements. In order not to compete with main pages, they should be marked with a “noindex” tag.

And pages that can fall under these criteria or similarly, they should be excluded from search.

What Is a Canonical Tag?

When Google or other search engines find duplicate or near duplicate content within your website it confuses search engines. In that case, it doesn’t know whether to index this or that.

Canonical Tags come into the play to solve such a situation.

So what are canonical tags? Canonical tag (rel=”canonical”) helps search engines know which is the master version of any resource or URL and whether to index it or not.

Canonical tag is the same as “noindex” tag , usually will be found in the <head> section of your website and having the following HTML representation :

Limited Server Resources

One of the terms which is common in Technical SEO is “Crawl Budget”. Simply, this term defines how many pages are being crawled in a given timeframe. Crawl Budget, as a technical term in SEO, is strongly related to server resources. EG “Bandwidth”

Usually, people who host their websites on Shared hosting plan complaints about server outages sometimes. This can result in indexation errors, because the main goal didn’t occur which is “Accessibility”.

Accessibility is the core term where technical SEO is built upon. Because, Technical SEO is about ensuring that your website is technically accessible from Search Engines’ End.

Robots.txt File

Robots.txt file is a simple text file placed in the root of your website, which holds important instructions for search bots. Normally, robots.txt file can be accessible at yourdomain.com/robots.txt.

From this file you can control which parts of your website to include in the search index and parts to exclude from being indexed. Example : Sensitive directories, Password Reset Pages, Query Parameters to ignore, wildcard paths, and many.

So, any mess-up with this file will result in indexing issues. You can check which of your URLs are blocked by Robots.txt from Google’s Robots.txt Tester Tool .

Technical SEO Best Practices

As above mentioned, Technical SEO is about Discovery, Crawling and Indexing. So, creating friendly site-structure, flat navigation and Sitemap. And submitting Sitemap to Google will help your website get indexed faster.

In order to crush it to the maximum, follow these best practices in addition to the above recommendations for better results.

Use HTTPs Over HTTP

HTTPs is the secured version of the plain HTTP.

Connections made over HTTPs are secured and encrypted, because HTTPs transfer sensitive data to servers like : login credentials, credit cards, bank information and many others.

Google already declared HTTPs as a ranking factor in 2014 on their blog.

Also, you can use the digicert tool to check whether an SSL certificate is installed on your web server or not.

Or from your browser tab like the image below:

It shows that we already have SSL installed on our web server.

If you found that your connection is not safe which indicates that you have no SSL certificate installed on your web server.

In that case, you can get a free SSL certificate from Let’s Encrypt. And here is a handy guide to help you install Let’s Encrypt free ssl.

Important Note : After installing SSL on your web server, it’s very important to redirect from the old unsecured resources URLs to their corresponding secured resources. To avoid mixed Content issues.

Choose Your Preferred Version WWW or Non-WWW

Choosing your preferred version of your website will determine how it looks and how your technical SEO is doing. For example, in the world wide web there are only two versions of any website WWW and non-WWW version.

Having both versions accessible from user and search engines’ end, is a bad Technical SEO practice. As it creates duplicate versions of your website and confuses search bots.

And in the long term, may dilute and lessen your backlinks profile strength. As some links will be pointed to the WWW version and the other links will be pointed to the non-WWW version.

So, choosing a version for your website and redirecting the other version to this selected one, is a very-good practice to maintain a healthy and technically functioning website.

Improve Core Web Vitals Score

Core Web Vitals or CWT for short, are a set of metrics to measure how friendly is the user experience. Google announced about Core Web Vitals in their Google Search Central Blog titled as “Timing for bringing page experience to Google Search”.

These metrics are:

Largest Contentful Paint (LCP) : A Metric to calculate the time consumed to load the largest element in the web page for the user.
First Input Delay (FID) : Calculate and measure the time it takes to the first interaction from the user with the web page. Example : Clicking on any element that fires a specific event (Pop Up Form)
Cumulative Layout Shift (CLS) : Metric used to measure the shifts occurs in the layout of the web page after interaction from the user. Example : Web page after loading and during reading a smooth scroll may take you to the footer, then header. All these layout shifts consume time and result in high CLS Score.
Interaction To Next Paint (INP) : From 10, May 2023 Google declared the replacement of FID to be replaced with INP in March 2024. In essence, Interaction to Next Paint is a (pending metric). Will be used to measure responsiveness using the data fetched from the Time Event API.

In order to optimize for Core Web Vitals score, aim for the following scores for the above mentioned metrics:

LCP : 2.5 Sec or Lower
FID : 100 Ms or Lower
CLS : 0.1 or lower

You can even check for these metrics from Google Search Console > Experience > Core Web Vitals . Like Shown in the image below:

Check Core Web Vitals From Google Search Console

Structured Data Implementation

Structured data or HTML Microdata or JSON-LD all of these terms reference the same meaning. Which is a small snippet of code which is being added to your website to help search engines understand the context of your content.

By implementing the proper schema type to your website, your chance of being featured in rich results is high.

But first, What are rich results? Simply, Rich results are more appealing search results than the classic results. Example : Open Job vacancies in Google, Knowledge panels, Featured Snippet Box, People Also Ask, Local Maps Pack and So as Google Shop Tabs. All of those are good examples for Rich Results.

Reviews which appear below search results for commercial terms are another good example for Rich results. These rich results enhance your appearance in SERPs and increase your chance of winning more clicks than your competitors.

You can even check Google’s Official Structured Data gallery. Where they list down all the Schema markup types they support and display in their search results.

So, it’s better to search for a suitable Schema markup type that fits your content. For example, if you own an OTA (Online Travel Agency). There are plenty of Schema Types for such business, like:

There are many online tools to generate Schema markup for your website. Like

Technical SEO – Schema Markup Generator Tool
Or if you are a WordPress user, Schema plugin is a good plugin with minimal configurations.
[Advanced Way] : Installing Schema markup using Google Tag Manager refer to this guide for better guidance.

Hreflang Tags Implementation

Hreflang tags are small snippets of code usually placed in the <head> section, that helps search engines discover other language versions of your content.

If you serve your content in more than language, then hreflang tags come in handy to handle this properly and are SEO-friendly at the same time.

Hreflang Tag HTML Code Snippet:

You can even check your hreflang tags (If you serve your content in more than one language) or you can even create one using online tools. Like :

Enhance Page Speed Loading

Serving your website fast on mobile and desktop devices is a must.

You can check your website page speed score from 0-100 using PageSpeed Insights to check how fast your website loads for mobile and desktop.

You should aim for a 90-100 Score for mobile and desktop, this can be achievable in many ways. But the most important are :

Enhance Media Delivery : Compress your image files, serve your images in Web Express format (WebP) and scale your images to serve it in many sizes without affecting the quality of the image.
Use CDN (Content Delivery Network) : Cloudflare is the best free CDN provider with advanced caching layers to enhance your website loading speed.
Website Assets Minification : Minify all of your web assets HTML, CSS and Javascript, this can be done through tools that remove comments and whitespaces within your code which in return enhance speed loading of web pages.

Mobile-Friendliness Website

On November 04, 2016 Google announced the use of the Mobile-First index concept. Which means that Google looks for the mobile version of your website to index and rank its content.

You can Check Mobile Usability from within Google Search Console -> Experience -> Mobile Usability. Like in the image below:

Check Mobile Usability in Google Search Console

In case you don’t have Google Search Console setup and installed for your website, you can check the Mobile Usability for your website using Google’s Mobile-Friendly Test Tool

Fixing 404 and Broken Pages

404 Is a status code returned from a web server to indicate that the requested resource isn’t on the server.

In case you have migrated your website, changed URLs Structure or for any reasons. The technical audit result may show 404 errors and broken pages.

One of the simplest solutions to fix such an issue is “Redirection”. To redirect broken pages/not found pages to their corresponding resources.

By doing so, you are preserving the link juice to be passed to the new pages and the ranking data will be reassigned to those URLs again.

After implementing all the fixations needed, now you should revisit and reassess all of your internal links. To ensure that there is no resource on your blog is linking to a broken source within your website.

Find and Eliminate Duplicate Content

Finding and fixing duplicate content issues is considered one of the most important Technical SEO best practices. As it can make a lot of technical issues which can negatively impact your SEO.

Siteliner and Screaming Frog are my preferable two choices to spot and fix duplicate content issues which may be causes by:

Multiple pages sharing the same title and meta descriptions
Little variations in URL slugs can result in duplicate content issues. Eg Query Parameters, Search Filters.
Repetitive pieces of blocks within your website. Eg : Footer Disclaimer Text, Call To Actions.

Implementing Breadcrumbs Navigation

Breadcrumb trails or Breadcrumb navigation shows the user where they are in the website through a navigational schema usually placed above the fold or below title.

It usually looks like this :

Homepage / Blog / Current Post

Breadcrumb navigation enhances the Technical SEO, as it gives the user a clear hint where they stand in the website.

Why My Pages Aren’t Indexed?

Pages that aren’t indexed can’t be served on Google

In this section we will discuss the common errors which arises from Google Search Console Pages Report

What Is Page Indexing Report?

Page Indexing Report is a tab in Google Search Console which shows how many URLs from your website have been crawled and indexed by Google.

Here is how you can view your indexed or not indexed pages in Google Search Console:

Google Search Console -> Indexing -> Pages

Google Search Console' Page Indexing Report

You will see the following Columns in this page :

Reason : This column shows you the reason behind your pages being not indexed. Such as : Not Found (404), Page With Redirect, Excluded by ‘noindex’ tag, Alternate page with proper canonical tag.
Source : It tells you whether these errors arose from Google Systems’ End or from your website end.
Validation : It gives you a hint about the current validation status whether Validation Started, Validation Failed or Success.
Trend : Gives you a hint about the trend of increase in pages with specific errors or decreases in those page numbers.
Pages Or Affected Pages : It shows the number of the affected pages by a specific Reason. For example : X Number Of Pages giving status code Not found (404) and so on.

Like in the image below you can see it much clearer.

What Is Crawl Stats Report?

It’s a report which gives you statistics about the Crawling Budget history Google consumed against your website.

Like how many requests were being sent to your website?How did your web server respond? And with which status code it gave.

Crawl Stats Report for advanced SEOs, as it helps them to have a clear idea how their website responds to Google during the Crawling phase.

This is what you should see if you tried to access the Crawl Stats Report :

Now, let’s navigate and explore this section.

We will discuss the following terms in short, as there is not enough space for this Technical SEO guide to cover all of these aspects. Plus, it’s not necessary for websites with few pages of content.

Total Crawl Requests : Total number of crawling requests made by Googlebot to your website in a given timeframe. AKA (Crawling Budget)
Total Downloaded Size : Total size of downloaded web assets during the crawling phase. Example : HTML Files, CSS, Scripts, Javascript Files and Media.
Average Response Time : Average response time from a web page or resource to a given request regardless of the resources fetched (Scripts, Embedded Iframes Or Web Assets) in this request (Rendering The Page Contents).
Host Status : Host Status gives a summary about server failures in the past 90 days. If any server failures occurred during the Crawling phase, you should see these failures here.
- Robots.txt fetch : Robots.txt fetch tab within Host Status tab, shows you whether Googlebot encountered any issues or not while fetching your Robots.txt file.
- DNS Resolution : Domain Name System Resolution or DNS Resolution for short, gives you brief information about whether Googlebot encountered DNS issues while crawling your website. Example : Issues Related to your DNS Records, Conflict with your DNS Provider and Domain Name Registrar. The reasons are many.
- Server Connectivity : This describes whether Google had any server connectivity errors while crawling or not. Example : 500 Internal Server Error, 503 Service Unavailable, 504 Gateway Timeout and generally 5XX Errors

Frequently Asked Questions

Here are some common Technical SEO questions, which are commonly asked from the SEO Folks in communities and forums.

What Technical SEO Issue Can You Solve With An SSL Certificate?

It Can solve “Not Secure ” warnings which can negatively impact your SEO. Like we discussed above HTTPs is best practice over HTTP.

How To Conduct A Technical SEO Audit?

You can run a Technical audit SEO for your website using:

All of these tools can run Technical SEO audits for your website. Some of them are cloud-based , Saas Tools and Softwares To download.

They can help you spot and fix technical issues within your website.

Why Should You Ask For A Technical SEO Audit?

Because, it shows you the technical problems within your website that may hold your website back from unlocking its full potential in SERPs. And fixing Technical SEO issues to improve your website accessibility for search engines and users.

What Technical SEO Elements You Should Focus On?

Every element within Technical SEO concept is an important element, but the most important elements you should focus on in your Technical SEO:

Serving your website over HTTPs instead of HTTP
Create user-friendly structure to make it easy to crawl and index your website
Ensure your website is mobile responsive
Enhance your website loading speed and Core Web Vitals score
Enhance User Experience Elements

SEO