SEO - Robots.txt

I’m guessing it’s good practice to turn these all off when developing a site or renewing a site

Q1 Could there be a master all off whilst in development checkbox

Q2 Google says no Robots.txt file exists for my site … is this necessary - if so - why don’t we produce one ?

Q3 Should we also provide a maintenance mode option like you have in WP .. for when you are updating / tinkering ?

You have a robots.txt file here:

https://www.peteryates.co.uk/robots.txt

It’s being generated by that www.seoptimer.com service, not by Elements.

Regarding maintenance mode, you can throw up an “Under Construction” home page temporarily. Elements is a static site generator (not dynamic with a database like WP), so a full on maintenance mode module seems a bit much.

1 Like

Why would you switch these off?

A robots.txt file is optional. Not having one is completely normal. It only becomes “necessary” if you actually want to disallow something. If the feature is needed later, Elements can generate one.

It’s not really a good idea to switch all of this off during development. Search engines aren’t going to index a half-built site unless you’ve already published the URL somewhere, and even then it’s rare they’ll crawl it immediately. Blocking everything just adds extra moving parts for no real gain.

1 Like

OK - my thinking … is that I’m reviewing my website … and whilst I do, it’s called xx/2026 - I’m don’t really want that being indexed until it’s ready (do I?) … When ready it will replace the old site at www.ODEChair.com

Looking for good practice

You could password protect the site during development, generally reputable search engine crawlers can’t get past that for indexing. :slightly_smiling_face: