noindex from google

[giacomosepe]

Hi! I have finished my personal website. And I couldn't be happier of the result. I have installed it using modules and I have set it up with two languages so that now I have EN on mydomain.com/en and IT on mydomain.com/it for every page. It works perfectly. I have deployed with Cloudlfare pages.
I have set "enableRobotsTXT = true" and HUGO_ENV=production on Cloudflare variables on deploy.

But after 24hrs Google still throws me a message of error and it can't index the site: " 'noindex' detected in 'robots' meta tag ". I could not find any meta tag in the page code source: giacomosepe.com

I have run out of resources and can't find solutions online. Any ideas of what I am doing wrong? any fixes? I am not a developer but can hustle around a little.

[jpanther]

This is not the default behaviour so I'm wondering if you're providing a robots parameter in your site configuration? If you are that may be the source of the problem. By default the site should always be indexable by search engines unless you specify otherwise.

[giacomosepe]

Hi!
I know, am sure I did something to trigger that. But can't figure out what. It seemed all configured correctly to me. Yes, I tried with both robots parameter activated and deactivated. And the result didn't change.

I seem to have solved the problem for now: I found out that when I have the two locales (en and it) activated and all pages split in subdirectories (content/en and content/it) hugo builds an index.html file in the root of the site (the public folder) with just a few lines of code, one of which is the robots meta tag set on noindex. That index.html file just stays in the folder and blocks the robots. And there is no way to prevent hugo from building it.

After I put the EN locale site files in the root content directory (content) it all worked and Google was able to index it. Not sure why is that. I also had to deactivate the 2 languages options in the config directory. and I am using translationKey to connect the pages in the 2 languages

[jpanther]

Thanks for the response. That sounds like it may potentially be a bug. When I have some time I'll try to look into this for you. I'll open an issue for it for now in case anyone else has a similar problem and can add any extra detail.

[wolfspyre]

i’ve experienced this too on a blowfish themed site that doesn’t have multi language enabled… I didn’t look into it (yet) as I didn’t really care at the time, but i’ll see if I can find any other commonalities..