Indexing: A tale of two search engines

Indexing: A tale of two search engines

In Part 7 of our Messy SEO series, we look at the differences between Bing and Google’s indexing of MarTech’s content.

Messy SEO is a column covering the nitty-gritty, unpolished tasks involved in the auditing, planning, and optimization of websites, using MarTech’s new domain as a case study.

This installment of “Messy SEO” details my process of working with our team to analyze indexing patterns for MarTech’s pages. In Part 6, we discussed the necessity of creating pillar pages to establish a better site hierarchy and rank for our most relevant topics.

RELATED: Google lets you report an indexing issue has had many indexing issues since its creation last year. The most pressing one lately is that Google seems to be prioritizing outdated content in the SERPs — many of the (now redirected) Marketing Land and MarTech Today URLs continue to populate the index. As a result, the majority of MarTech’s top-performing pages are irrelevant to our brand as it exists today. 

One of the ways we’re addressing this issue is by creating pillar pages that center on the main industry topics we cover at MarTech. This will help us establish a hierarchy of relevant topics.

We’ve primarily focused on Google’s indexation throughout this process, neglecting to review the ways other search engines have treated our content. So, we decided to compare the MarTech, Marketing Land, and MarTech Today data from Google with that from Microsoft Bing – and the discrepancies were telling.

Indexing status almost a year after migration and consolidation

There have been a lot of changes to MarTech’s indexing since the migration, most notably the title change issues. Thankfully, these were largely resolved, but there are some other issues we found when comparing the content indexed on Google and that on Bing.

Google’s indexing

Despite many lingering indexing issues, Google has made some adjustments to MarTech’s indexation over the past year. The search engine removed virtually all of our duplicate URLs after we set up our redirects, and a good portion of Marketing Land and MarTech Today pages have been removed as well. However, we’ve recently noticed some interesting performance and indexing trends.

Performance. The majority of the top pages from the past three months in terms of interaction are legacy pages that have little to no relevance to our MarTech brand. Aside from the homepage, the “What is MarTech” page, and our CDP platform page, the top URLs are largely irrelevant to our target audience.

Page Impressions Clicks Avg. Position
/ 590170 13778 25.34
/content-marketing-done-right-8-examples-can-learn/ 529481 7429 30.13
/top-10-payment-processing-companies-world/ 717026 7259 32.48
/what-is-martech/ 278783 6773 11.33
/8-companies-social-media-right-marketers-can-learn/ 251427 5087 48.91
/martech-landscape-customer-data-platform/ 369856 3882 23.16
/100-questions-you-must-ask-when-developing-web-site/ 77152 3580 27.66
/10-steps-target-connect-potential-customers-effectively/ 170443 3353 20.48
MarTech’s top-performing pages on Google.

Granted, these articles have been live for years, building up authority on Marketing Land and MarTech Today domains. But, after almost a year of MarTech being live, it’s odd that there are so many old, less relevant pages sitting at the top of our performance lists – especially when our team has published so much good content since then.

Indexed pages. Google has roughly 29,000 MarTech URLs in its index. The majority of these are relevant links we’ve placed in our sitemaps. However, there are over 7,000 URLs in the “Indexed, not submitted in sitemap” category. Many of these URLs are irrelevant — a disconcerting number have parameters that look like either tracking code or, in some cases, spam.

URLs with parameters in the index.

The prevalence of URL parameters isn’t surprising, but it’s not clear why Google is including so many of these in the index. The more alarming trend, however, is the number of Marketing Land and MarTech Today URLs that are still in Google’s index as well.

Marketing Land URLs on Google.
MarTech Today URLs on Google.

We know that there are plenty of Marketing Land and MarTech Today URLs online, both in our older pieces of content and on other websites. But it’s strange to see so many still in Google’s index.

Get the daily newsletter search marketers rely on.

Processing…Please wait.

See terms.

Bing’s indexing

Bing’s indexing tells a different story. Though there are still plenty of irrelevant content pieces, they’re much less prominent in the SERPs.

Performance. MarTech’s top-performing pages on Bing look somewhat similar to those on Google. The homepage, “What is MarTech” page, and legacy pages are still there, but we also found one of our more recent news articles in the mix. The importance of the piece to our industry undoubtedly helped bring it to the forefront, but it’s peculiar that Google didn’t treat it the same way.

Page Impressions Clicks Avg. Position
/ 1.2k 119 4.75
/10-steps-target-connect-potential-customers-effectively/ 586 27 5.99
/100-questions-you-must-ask-when-developing-web-site/ 231 17 4.96
/whats-big-idea-3-fundamentals-successful-digital-creative/ 408 14 5.36
/what-is-martech/ 613 14 5.04
/top-10-payment-processing-companies-world/ 4.6k 11 7.51
/5-roles-need-marketing-team-2-roles-havent-thought/ 132 10 7.73
/google-to-end-universal-analytics-in-2023/ 40 8 3.55
MarTech’s top-performing pages on Bing.

This newer article’s numbers are encouraging, but, just like the results on Google, our more relevant topic pages are failing to perform well.

Indexed pages. Bing has indexed fewer of our MarTech pages (roughly 17,000 URLs), which isn’t surprising, given how much smaller it is than Google. However, after analyzing these URLs, we found the ratio of relevant content to irrelevant content to be much lower. We’re not seeing a huge number of indexed URLs with parameters.

The most glaring difference between the two search engines is their indexing of our old domain pages. While Google still retains over 2,000 URLs from Marketing Land and MarTech Today, there are only 143 of these URLs left in Bing’s index.

Marketing Land URLs on Bing.
MarTech Today URLs on Bing.

Yes, Bing had fewer of these pages to begin with, but the inconsistency is still shocking.

A discrepancy between Google and Bing’s indexing

Of the two search engines, it seems Bing is doing a better job of crawling our old URLs and adjusting its index accordingly. This makes sense — there are fewer pages indexed on Bing, so the search engine has less to clean up.

But why is Google holding on to so many of these old URLs? One possible explanation is that it simply hasn’t crawled all of the old URLs yet. This would mean it hasn’t found the 301 redirects we put in place, believing the old pages are still live.

This seems unlikely, however, as we migrated the site almost a year ago. Google has had plenty of time to crawl our pages. Yet, we’re still open to this possibility.

Another explanation could be that there’s a structural issue on the MarTech site that is somehow telling Google Marketing Land or MarTech Today is still live. We’re conducting some deep technical audits at the moment to determine if this is true. Until we know more, we’re going to continue to create good content and do all we can to help it rank higher than the less relevant pages.

Have you noticed discrepancies in indexing between Google and Bing? How are you addressing the issue? Email me at [email protected] with the subject line “Messy SEO Part 7” to let me know.

More Messy SEO

Read more about our new MarTech domain’s SEO case study.

  • Part 1: Navigating a site consolidation migration
  • Part 2: The importance of canonicalization
  • Part 3: How to find cached images and improve user experience
  • Part 4: When Google’s title change goes wrong
  • Part 5: Fixing site structure while a Google title change sinks clickthroughs
  • Part 6: Pillar pages and topic clusters

New on Search Engine Land

    The Great Marketing Reshuffle: 618,000 marketers changed jobs in 2021

    Indexing: A tale of two search engines

    Twitter will let you leave a conversation through ‘unmentioning’

    Search Engine Land seeks editor to cover paid search marketing

    The latest jobs in search marketing

Related Post

Leave a Reply