Talk with an Advisor
Talk with an Advisor

1 min read

Setting robots.txt in elcomCMS

Setting the robots.txt file is important for your web site's health - Google uses it as a guide for what to index and what to exclude.

I've previously included references to the robots.txt file in this post on going live with a new elcomCMS site (see point 6) but it's worth covering again here.

In elcomCMS, setting the robots.txt file is really easy.

You can access it from the Admin menu:

elcomCMS robots.txt

and then edit it directly:

elcomCMS robots.txt settings

Recommended robots.txt Settings

Staging Site

If a site is in staging, you'll ideally want to block Google from crawling it. Use the following to block the entire site:

User-agent: *
Disallow: /

Live site

Once the site is live, you only really want to block a few items eg blocking the login page is reasonably common. You can also block entire directories eg such as an FTP folder:

User-agent: *
Disallow: /cmlogin.aspx
Disallow: /ftp/

Be very careful when going live with a site - I often see sites go live (from staging) and keep their robots.txt file from staging - and in the process block their entire live site in Google! Make sure you check your robots.txt file when going live.

elcomCMS SEO Checklist - New Site Go-Live

We've been working with elcomCMS for many years now, and helped with a number of site go-lives. In this post I wanted to go through a simple...

Read More

Creating an XML Sitemap in elcomCMS

One of the best ways to guide Google in what it should crawl on your site, is to provide a listing of URLs that you deem important.

Read More

Intranet Software Trends: What's Up Next?

Today's intranet looks drastically different from the one your office might have had in the past. You don't have to keep up with every intranet...

Read More