Post by afrina022 on Nov 22, 2023 23:30:13 GMT -5
Below is a list of the most popular of them in the CIS. Googlebot News searches for news Mediapartners Google AdSense service bot AdsBot Google checks the quality of the landing page AdsBot Google Mobile Apps – checks the quality of the landing page for applications on Android similar to AdsBot Googlebot Image for pictures Googlebot Video for video Googlebot Mobile for the mobile version. Disallow directive – ban on indexing This directive prohibits indexing files and pages that fall under the specified parameter.
The writing syntax is the same as for user agent. For example on the site there is a folder named users with files with user data and it needs to be closed from indexing then Country Email List we write like this Disallow users Thus we blocked both the users folder and all the contents in it from indexing. Note to specify parameters you do not need to specify the domain itself. For example we write not Disallow site.ru users but Disallow users . Allow directive – permission to index Using this directive we can open the pages and files necessary for indexing.
For example in the users folder we wrote about it above there is an images folder which contains the pictures necessary for indexing. Therefore we need to write the following line Allow users images As a result we get User agent * Commands are specified for all crawlers folder and all files Allow users images Opens the images subfolder and all its subfolders Prioritize directions If there is a conflict between Disallow and Allow the one with the longer parameter after the colon will be selected for execution. For example Disallow users Allow users images In fact the first Disallow command should close the images folder too but since after the colon the first directive has characters and the second has characters the Allow directive will take precedence. If the directives are completely identical then priority will be given to the one below.
The writing syntax is the same as for user agent. For example on the site there is a folder named users with files with user data and it needs to be closed from indexing then Country Email List we write like this Disallow users Thus we blocked both the users folder and all the contents in it from indexing. Note to specify parameters you do not need to specify the domain itself. For example we write not Disallow site.ru users but Disallow users . Allow directive – permission to index Using this directive we can open the pages and files necessary for indexing.
For example in the users folder we wrote about it above there is an images folder which contains the pictures necessary for indexing. Therefore we need to write the following line Allow users images As a result we get User agent * Commands are specified for all crawlers folder and all files Allow users images Opens the images subfolder and all its subfolders Prioritize directions If there is a conflict between Disallow and Allow the one with the longer parameter after the colon will be selected for execution. For example Disallow users Allow users images In fact the first Disallow command should close the images folder too but since after the colon the first directive has characters and the second has characters the Allow directive will take precedence. If the directives are completely identical then priority will be given to the one below.