Dynamic websites that have ever-changing content, content that is live for a fixed period of time before being removed, often encounter issues with 404 errors in Google Search Console.
The types of websites that have this issue can include but are not limited to, estate agencies, job boards, auction websites, car sales, or limited run e-commerce websites. These dynamic websites tend to have pages, or adverts, live for a period of time before the job is filled, the house or car is sold or the product range expires. If that page is just turned off then it will create a 404 in Google Search Console the next time Google tries to crawl the website, creating an issue for SEO.
It can also make for an annoying user journey. If someone has saved a link and then the page that the link was pointing to disappears the user is cut off from the final destination.
Impact on SEO
404s are part of the internet ecosystem and a few across a website will not cause SEO issues. However, depending on the page they can have an impact on user experience. If key pages on the website are inaccessible and your users can not get the information they need you will lose them and they will go to your competitors.
However, this is not all.
Quality score issues
Managing 404s on smaller websites is a simple process but 404’s can rapidly get out of hand on dynamic websites and that puts them at risk of not only users moving on but the volume of 404s starting to cause quality issues with Google.
These issues arise as Google believes that your website is not being maintained correctly and therefore it will start to doubt the quality of your content and not want to rank your website over others.
Lost backlinks
Any website can link to any page on the internet and it’s sometimes surprising the seemingly random and deep inner pages that receive links. By letting pages turn into 404s there is a chance that your website will lose backlinks that are helping your website authority.
Throttled Link Juice
Site authority is not only passed through other websites linking to your own but also through internal linking. Broken internal links prevent the flow of “internal link juice” through the website which will hinder the performance of both individual pages as well as the website as a whole.
Lost Rankings
Pages that used to be live are likely to have content. Content that had the potential to rank in the search engines. By turning live pages into 404s there is a chance that you will lose content that may have been ranking.
Wasted crawl budget
Finally, we need to look at the crawl budget. While search engines can look at every page on your website every time they crawl in practice they don’t. GoogleBot will visit the main pages and their child pages but if there are too many pages under that then Google may limit the number of pages GoogleBot crawls. This is known as a crawl budget.
Generally, small websites do not need to worry about crawl budgets. However, if you have a dynamic website there is a strong chance that you will start encountering crawl budget limits. If GoogleBot encounters an excess of 404 pages your crawl budget will be limited and Google will simply not know that some of your pages exist.
Possible options
Leave the page live
The first option is to leave the page live but include an automatically generated message saying that the product, job, property etc has been taken and encourage people to use the search function to find something similar.
Indeed, to really give a good experience you can automatically generate suggested alternatives.
If you do take this approach you need to remember that this will lead to a significant number of expired pages on your website. To avoid these expired pages frustrating users it is best to to remove the page from any site search functions.
You can also no-index the expired pages to remove from the search engine results pages, however, you may want to keep them indexed to increase search engine presence. If you do leave them no-indexed just bear in mind that this can result in duplicate content or expired pages will fill your internal search functionality.
To avoid too many expired pages building up over time you can remove the expired pages and either 301 redirect the URL or have them return a 410. Two options we will explore below.
301 Redirect old pages
If you don’t want to keep the pages accessible to users then you can remove the expired pages and set up a 301 redirect back to the main search page. This ensures that you are not generating a significant number of 404s and that users who may have saved the old link are able to get to a useful page.
It’s important to use a 301 rather than 302 as this tells search engines that the expired page has been permanently retired and should therefore be redirected to the alternative URL.
However, if you have a very large number of expired pages over time this will bloat the .htaccess file and has the potential to slow the website down.
410 Response Code
Some cases may require some more drastic measures and this is where the 410 response code comes into play.
Unlike a 404 response code, which tells search engines that a page is temporarily broken, a 410 tells search engines that the page has gone and will not be coming back. This means that search engines will not keep revisiting the page, reporting on the 404 in Search Console, and means the .htaccess will not become bloated with expired page redirects
It’s not the ideal solution as unlike 404s users will not land on a dedicated “Broken Page” notice, instead, they will just see a “GONE” message from the server. This means that users who click on old links will not be able to navigate to another page.
If you are building a new website without a history of expired pages you can probably avoid the need for 410s but established dynamic websites that have a significant number of 404s in Search Console may find it more effective to draw a line under the existing 404s caused by expired pages and just 410 them.
However, as mentioned above, 410s can be used in conjunction with leaving expired pages live for a period of time. Once 18 months have passed the expired pages are not likely to be linked to any more and can therefore be 410ed to free up server space.
What’s best for SEO?
The best solution for SEO will depend on your specific circumstances but it is likely to involve more than one of the solutions detailed above.
Small websites
The 301 option is simple to implement but Webmasters should only 301 all expired pages if the website is small. Otherwise you run the risk of creating a very large .htaccess file while can start to slow the website down.
Larger Websites
For most dynamic websites it would be best to leave pages live for a period of time, with a message saying that the product/job/house is no longer available with links to alternatives. Then, after a period of time convert it to a 410.
This gives users the opportunity to update bookmarks, links or saved searches before the page is retired.
Get in Touch
If you’re having problems with expired pages on your dynamic website, feel free to get in touch with our talented team of SEO experts. We take pride in staying up to date with the latest industry insights and best practices to help give your business a competitive edge in the constantly evolving digital landscape.
