# Blocking Google; remaking website on an empy website

**URL:** <https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184>\
**Category:** RapidWeaver Classic\
**Tags:** problem\
**Created:** [May 1, 2024, 9:51pm UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184 "2024-05-01T21:51:37Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![kim](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kim](https://forums.realmacsoftware.com/u/kim)\
**Post date:** [May 1, 2024, 9:51pm UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/1 "2024-05-01T21:51:37Z")

</div>

I am in the process of redoing my website.  
I prefer to redo my current website on a completely different, clean website because I have a mess on the original site.

I want to allow places like Screaming frog access my website.  
I do Not want Google or Bing to be able to access this hidden website.

Is this possible???  
Thank you!  
Kim

---

<div class="post-metadata">

**Author:** ![barchard](https://dub1.discourse-cdn.com/flex005/user_avatar/forums.realmacsoftware.com/barchard/32/2292_2.png) [@barchard](https://forums.realmacsoftware.com/u/barchard)\
**Post date:** [May 1, 2024, 11:19pm UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/2 "2024-05-01T23:19:19Z")

</div>

@kim do you want the site to be publicly viewable or private for you only? Do you want to maintain the current site live and make a second development site or is it okay to stop access to a live site?

---

<div class="post-metadata">

**Author:** ![kim](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kim](https://forums.realmacsoftware.com/u/kim)\
**Post date:** [May 2, 2024, 2:33am UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/3 "2024-05-02T02:33:30Z")

</div>

I really don’t care if others can see the development site. I just don’t want google or bing to index it and get penalized on my active site. I want the second to be a development site. I must keep my active site active.  
Thank you Barchard

---

<div class="post-metadata">

**Author:** ![Heroic\_Nonsense](https://dub1.discourse-cdn.com/flex005/user_avatar/forums.realmacsoftware.com/heroic_nonsense/32/20666_2.png) [@Heroic\_Nonsense](https://forums.realmacsoftware.com/u/Heroic_Nonsense)\
**Post date:** [May 2, 2024, 5:49am UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/4 "2024-05-02T05:49:42Z")

</div>

Hi @kim,

Assuming your new dev website is a directory called _ **/dev** _ of your primary website, you can setup your robots.txt as follows:

```auto
User-agent: Googlebot
Disallow: /dev/

```

This will tell Google to not index whatever is in the directory _ **/dev** _ on your webserver, so what you do in there doesn’t impact what Google thinks of the rest of your site.

You can set up robots.txt by putting a file with that name in the root of your website (where the main index file is located). The file should (in your case) only contain the example text that I gave you, but you can find numerous tutorials online which explain what else you can put in there.

Cheers,  
Erwin

---

<div class="post-metadata">

**Author:** ![kim](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kim](https://forums.realmacsoftware.com/u/kim)\
**Post date:** [May 2, 2024, 8:37pm UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/5 "2024-05-02T20:37:07Z")

</div>

Thank you Erwin  
That is great advice.  
Thank you for your help. 😄  
Kim

---

<div class="post-metadata">

**Author:** ![barchard](https://dub1.discourse-cdn.com/flex005/user_avatar/forums.realmacsoftware.com/barchard/32/2292_2.png) [@barchard](https://forums.realmacsoftware.com/u/barchard)\
**Post date:** [May 3, 2024, 2:36am UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/6 "2024-05-03T02:36:14Z")

</div>

Yes, what @Heroic_Nonsense said with a couple of caveats,

1. this only covers Google, you’ll want to expand it to include Bing. [Here](https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0) is a list of Bing’s UserAgents
2. [here](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers) are all of google’s user agent strings (for future reference), just the 1 should suffice.

---

<div class="post-metadata">

**Author:** ![kim](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kim](https://forums.realmacsoftware.com/u/kim)\
**Post date:** [May 8, 2024, 6:27pm UTC](https://forums.realmacsoftware.com/t/blocking-google-remaking-website-on-an-empy-website/43184/7 "2024-05-08T18:27:33Z")

</div>

Thank you Gregory!  
I needed this too. 😄  
Kim
