Google: No Index Is A Directive, It Doesn’t Control Crawling | Industry News
Industry News – Google: No Index Is A Directive, It Doesn’t Control Crawling
There are many ways to control how Google crawls your web site but one of them is not the noindex tag. Google will still crawl your web site and web pages with the noindex tag on them. It needs to, in order to know what not to index. You can use robots.
from Pocket http://ift.tt/2qSxFjJ
Read the full story here – IFTTT