How to Add Custom Robots.txt File in Blogger (Blogspot)?

For better performance of our blog we need to optimize it for better SEO. A better SEO is also needed to get organic traffic from search engines. One great step towards a better SEO is adding robots.txt file to your blog. It tells the search engines crawlers about what page to crawl and what not to. In Blogger (Blogspot) we have the option to customize the robots.txt file according to our needs. But before adding it to your blog firstly as a beginner we need to know about what is robots.txt file. So lets start the tutorial. What is Robots.txt? Robots.txt is a file which tells the search engine crawlers what page to crawl and what not to. Simply, you can exempt your blog demo pages, label pages, archive pages and other which you don't consider as important. Search engine crawlers always scan the robots.txt file before crawling any blog. So lets start adding this great SEO improver to your blog. Also Check : How to Change Blogger (Blogspot) Favicon? Adding Robots.txt to Blogger (Blogspot) Blogs: To add Custom Robots.txt file to your blog, follow the basic steps mentioned below: Go to your blogger blog. Navigate to Settings >> Search Preferences ››  Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes Now paste the below robots.txt file code in the box. User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED Remember to change the highlighted url (example.blogspot.com) with your blog address. Click on Save Changes button. You are all done. How to Check Your Robots.txt File? Simply add /robots.txt after your blog address and you will be shown your robots.txt file. Take a look at the below example http://www.bbeginner.com/robots.txt All set! I hope you all easily added the custom robots.txt file in your blog. If you have any query regarding any of the blogspot tutorial, drop a comment below.

Tweet

26

For better performance of our blog we need to optimize it for better SEO. A better SEO is also needed to get organic traffic from search engines. One great step towards a better SEO is adding robots.txt file to your blog. It tells the search engines crawlers about what page to crawl and what not to. In Blogger (Blogspot) we have the option to customize the robots.txt file according to our needs. But before adding it to your blog firstly as a beginner we need to know about what is robots.txt file. So lets start the tutorial.

What is Robots.txt?

Robots.txt is a file which tells the search engine crawlers what page to crawl and what not to. Simply, you can exempt your blog demo pages, label pages, archive pages and other which you don't consider as important. Search engine crawlers always scan the robots.txt file before crawling any blog.

So lets start adding this great SEO improver to your blog.

• Also Check : How to Change Blogger (Blogspot) Favicon?

Adding Robots.txt to Blogger (Blogspot) Blogs:

To add Custom Robots.txt file to your blog, follow the basic steps mentioned below:

1. Go to your blogger blog.
2. Navigate to Settings >> Search Preferences ›› 
   
3. Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes
   
4. Now paste the below robots.txt file code in the box.

   User-agent: Mediapartners-Google
   Disallow:
   
   User-agent: *
   Disallow: /search
   Allow: /
   
   Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED

5. Remember to change the highlighted url (example.blogspot.com) with your blog address.
6. Click on Save Changes button.
   

You are all done.

How to Check Your Robots.txt File?

Simply add /robots.txt after your blog address and you will be shown your robots.txt file. Take a look at the below example

http://www.bbeginner.com/robots.txt

All set!
I hope you all easily added the custom robots.txt file in your blog. If you have any query regarding any of the blogspot tutorial, drop a comment below.