Hi all, my new site is here: https://www.jlordservicesdotcom/.
It’s been live for about a month.
For some reason I can’t see the site showing up on Google yet.
If I do a :www.jlordservices.com then nothing shows up, so I’m wondering if the sitemap.xml has been set up wrong?
I’ve submitted a sitemap to Google Search Console which is showing up green.
But it’s also saying ‘Submitted URL marked ‘noindex’’.
In Rapidweaver each page has ‘index this page’ ticked in the Meta tags area.
The Sitemap plus page has an area where it says’ disable search engines’. None of it is ticked except for the sitemap page itself, should that be unticked too?
When I look at your page source you have one spot where it says to index (see line 51 in the screenshot), which is usually the spot that RW puts that line.
But then on line 102, you’ve got another robots meta tag instructing to not index it. I see you have Joe’s SEO Helper, so I’m wondering if you have an noindex in that stack?
2 Likes
Holy cheesballs you’re right! I totally flipping forgot about that, and because I put it in a partial, it doesn’t show up on the page! SEO helper now deleted, thank you soo much!!!
Thanks so much for your help on this. It’s been a week now and I still don’t see the site showing up on Google. I went to the Google Webmaster tools and there is a new issue whereby it says ‘Submitted URL seems to be a Soft 404’. It’s referring to the 404 page I set up so any visitors who type in a slightly wrong url will be referred back to the homepage. But Google doesn’t seem to like it. Is that now what’s stopping the site from showing up?
How did you set up your 404?
That usually indicates that the page isn’t actually returning a 404 header
You can see what pages google has in its index using site:yourdomain
Have you done the request indexing in Search Console? In the old console, I believe you go to 'Fetch as Google, and then request it in the results. In the new one, it’s under the URL inspection. That may work more quickly than just submitting the sitemap…
I created a bespoke 404.php page. In my htaccess I have this instruction (see attached). It hasn’t been a problem on my other sites.
.
I’ve just requested indexing via your suggestion Jacob, but I’m not sure what I do next. Just wait for another email from Google?
Oh i just did the site:www.jlordservices(dot)com and I DO see the homepage showing up now, but that’s just one of 3 pages, shouldn’t the others show up too?
Thanks for your help on this.
Site: should return all,pages that are in the Google index
I’m not in my computer so I can’t check your return header for the 404 pages, but you can check in dev tools. Type a bogus page url a couple folders deep to test.
Like this;
Yoursite.com/foo/bar
Ok I did that and it successfully goes to my bespoke 404.php page.
Gabrielle
In the return header in the browser tools, you should get a 404 on the bad url then a 200 when it hits your actual 404 page, is that what happened?
Do you mean Firefox/web developer/browser console?
When I type in a url that doesn’t exist I see this code in that area (see attached).
I had a look for you, see screen shot. The bad page is returning a 302 before sending visitor to the 404 page. I would bet this is causing your soft 404 errors.
Thanks so much for looking. I’ve no idea what to do about this though, can you advise?
Gabrielle
Here is what I do.
I use two pages:
404.html is a page in my doc_root that has no content at all except in the Javascript panel, where it loads the 2nd page 404recovery.html. 404recovery has the actual 404 message, and includes the site style CSS etc. and it includes navigation to help the visitor recover.
The JS on the 404.html page is:
window.location = "https://cottsinc.com/404recovery.html";
Obviously, you would change this to load your recovery page.
my .htaccess contains the folowing:
RewriteEngine on
# Custom 404 errors
ErrorDocument 404 /404.html
#
you can see it in action by trying this bad url: https://cottsinc.com/foo/bar
1 Like
Thanks for that. I’ll have a go. But I’m just confused as to why it’s happening. This is the first time I used @joeworkman’s SEO helper to temporarily hide the webpages as I was testing them so I wonder if I’ve somehow caused the error when using that. Will try as you suggest. Then I have to resubmit the site to Google right?
That should fix your 404 page not returning 404…
You should not really need to resubmit your site, you don’t really want your 404 pages in Googles index anyway. (Mark them noindex).
Just make sure your sitemap.xml file is correct and Google knows about it. (Webmaster tools)
Are all of your pages now indexed? If not, which ones are missing?
What does webmaster tools return for those pages?
Before I do what you suggest (which may be tricky for me), would another option be to create a 302 page in the same way that I created the 404 page and then just add this code to the htaccess?
ErrorDocument 302 https://www.jlordservices.com/302.php
?
No,
The issue you have now is that your page isn’t returning 404 for a bad url.
What should happen is that the bad url page returns a 404. The recovery page (which does exist) should return a 200.
302 is for a page that exists but has temporarily moved.
301 is for a page that exists but has permanently moved.
1 Like
Ok, I’m setting things up the way you’ve suggested, I have 2 questions:
- On both my 404.html and my 404recovery.php: in the metatags column do I untick ‘robots index this page’ but keep the ‘follow links’ ticked?
- Do I amend my htaccess to the following? (see jpg attached).
-
Yes, you do NOT want your 404 pages indexed.
-
You just need your http->https part and that very last bit (I gave you). You can add the other error pages there Just like the 404 if you like.
If you move pages later, that is when you do a 301 redirect from the old page to the new page. (I highly doubt you will ever need a 302)
You can more than likely do this right in your Cpanel, and it will add the rule to your .htaccess for you.