For those website owners using Google Webmaster tools – you really should do – being both free, and a source of important information about how to improve your website.
Here it is: https://www.google.com/webmasters/tools
“Googlebot cannot access CSS and JS files on http://www.yourwebsitename.com
To: Webmaster of http://www.yourwebsitename.com/
So although this looks technical it is really quite straightforward.
A file in your website called “Robots.txt” tells the search engines which parts of the site to index, and which to ignore.
Here is an example of the file, from this website, so you can see what I mean: http://www.channeldigital.co.uk/robots.txt
In the case of a modern CMS website like Joomla, WordPress of Drupal, all of its workings are in subfolders in the web space, folders like: “administrator”, “bin”, “cache”, “components”, “libraries” and so on. Now none of the actual content of your website is in these folders so you don’t need Google to index them. In fact its best they don’t, so in the robots.txt file some lines say simply “don’t go there”, like this:
The change required is a fairly simple one for a web designer, clients will either need to have FTP or C-panel access to the web site files, or if they are on our servers just let us know and we’ll be able to make the changes.
To summarise however, the lines to add to the robots.txt file, rather than “Disallow” are simply “Allow” lines instructing the Googlebot that it CAN access the relevant folders. But which folders should you give it access to?
So by reading the list that Google gives us of “Blocked Resource s”, we are able to compile a comprehensive list of the folders that we must Allow access to.
Here it is for one particular website that we manage.
So these lines can be added to the end of the Robots.txt file, test again with the “Fetch as Google” feature, and before we know it the waring message from Google has gone away, and Google is developing the understanding that it needs of the mobile-friendliness of the website.
Now apologies to all you non-tecchie website owners out there, I appreciate that this can be terrifying. Do refer this to your web designer though, they will know what to do, and if Channel are involved in your website please give us a call, we will be very happy to make the required changes.