Page Rendering: how it happens and why search engines need it
The stages of discovery and analysis of web pages by search engines not only takes the form of the indexing process that we are already familiar with, but there is another decisive moment in which crawlers come into contact with what we publish: it is the rendering, the operation of interpreting and rendering the page graphically, which plays a crucial role in the way websites are viewed and indexed by search engines. This process may seem technical and complex, but it is important to understand it (at least superficially) because it is an essential component in ensuring that our website content is easily accessible and visible to both users and the search engines themselves. And so, with this quick guide we discover precisely how rendering works, why it is needed by search engines, and how it can affect SEO.
What is the rendering
Literally, rendering is the graphic encoding of the information on the page in HTML language, which are translated (rendered) so as to give an understandable form to the graphic elements that make up all the websites, each with its own characteristics.
In other words, in the Web context, the term rendering refers to the process by which search engines, such as Google, interpret and display the HTML code of a Web site in order to “see” the Web site in a manner similar to how a human visitor would see it, in order to then properly proceed to the stage of adding it to specific indexes.
To allow this process are the special rendering engines, rendering engines, which collect and digest the data packets coming from the server and transform the lines of code of the HTML language into graphical and functional elements, such as text blocks with hyperlinks, images, videos and other elements such as PDF files.
The keywords of rendering: definitions and clarifications
As web.dev experts explain, however, in order to best understand and use different architectures, it is important to first know the precise terminology to use when talking about them. In particular, there are some “key words” and acronyms we need to familiarize ourselves with, which also refer to specific technical approaches:
- Server Side Rendering (SSR): rendering of a client-side or universal HTML app on the server. This technique requires the server to generate the entire HTML page before sending it to the user’s browser, improving page load times and indexing by search engines, but also risking strain on server resources.
- SSG (Static Site Generation): HTML pages are generated in advance, during the build phase of the site, and saved as static files. This can offer very fast loading times and excellent SEO, but can be less flexible for sites with dynamic content.
- ISR (Incremental Static Regeneration): is a hybrid technique that combines SSR and SSG. With ISR, pages are generated statically during the build phase, but can be “regenerated” on an incremental basis when data changes. This provides a good balance between performance and flexibility.
- DSR (Distributed Static Rendering): DSR is another hybrid technique that combines elements of SSR and SSG. With DSR, pages are statically generated and distributed over a content delivery network (CDN) for fast loading. However, pages can also be dynamically regenerated in response to specific events or triggers.
- ESR (Edge Side Rendering): This technique involves rendering taking place “at the edge,” that is, as close to the end user as possible, usually on a server in a content distribution network (CDN). This can greatly improve page load times as content is rendered and served from a location geographically close to the user. ESR is particularly useful for sites with a high degree of user customization or dynamic content.
- PRPL (Push, Render, Pre-cache, Lazy-load): This is a performance optimization strategy to improve page loading. It consists of “pushing” critical resources as early as possible, “rendering” the initial UI, “pre-caching” the remaining resources, and “lazy-loading” other resources as they become necessary.
- Prerendering: running a client-side application at compile time to acquire its initial state as static HTML.
From a performance perspective, however, the metrics of interest (and interest) are:
- First Contentful Paint (FCP), the moment when the requested content (body of the article, etc.) becomes visible. Performance annoyances with First Paint typically occur when there is a big problem on the CSS and JS side of things of which are trying to load unnecessary scripts and actions to load the site in a fast past. All of this negatively impacts FCP, for example, while other problems can arise from the way the data is processed: SSR generally has a more consistent loading speed, both because it has repeatable hardware than CSR, and because there are a variety of devices that can affect how fast a page can retrieve and load content into the DOM.
- Interaction to Next Paint (INP), seen as a representative metric that assesses whether a page responds consistently and quickly to user input.
- Time to Interactive (TTI), which measures when a page becomes interactive: if something is heavily dependent on JS, it can further delay page speed and hinder performance.
- Total Blocking Time (TBT), a proxy metric for INP, which calculates the amount of time the main thread was blocked while the page was loading.
Rendering and SEO: why you need to be careful
Google cannot index what it cannot display: this simple consideration is enough to understand the crucial value of page interpretation and graphical rendering with respect to the fate of the site on the search engine.
In fact, when we think of page ranking, indexing immediately comes to mind, says Dave Davies on SEL pages, and specifically at the time when a search engine has already:
- Discovered a page by sitemap or crawling.
- Continued to visit the page for indexing.
- Collected all the content on the page.
- Started ranking the page for queries.
This is probably the most important stage of the process because these are the factors that influence the rankings, but it is not the final stage of the discovery process and, according to Davies, its weight may be diminishing over time as the final stage-that is, rendering-gains traction.
How and, more importantly, where the rendering process takes place can in fact have a significant impact on user experience, site performance and search engine optimization (SEO). As Google’s Martin Splitt said, Web page rendering is what happens between the browser and the Web page, the process of creating a Web page: an efficient rendering process results in high Core Web Vitals scores, while conversely, a less efficient rendering process can affect sales, advertising revenue, and even Web page crawling to some extent.
The importance of rendering for SEO
Rendering is closely related to SEO, as it affects whether and how search engines interpret and index a website’s content: taking it to extremes, good rendering can improve a site’s visibility in search results, while inefficient rendering can hide important content from search engines.
We must then also be aware of the negative effects of expensive rendering for users, because it may affect visitors browsing the site from older phones, causing problems viewing the page, but also on newer devices, which may have problems downloading a Web page if it has been active for days and RAM is spread across multiple open browser windows.
For these reasons, too, it is important to optimize website rendering for SEO, through the use of techniques such as server-side rendering or prerendering, which generate a static version of the page that can be easily indexed by search engines.
The advice coming from web.dev developers is to analyze the pros and cons and the fit of different rendering approaches to our projects and the types of sites we create, so that we make more informed choices about tools and technologies, rather than letting those choices dictate our approach.
How to optimize Rendering for SEO, basic tips
To optimize site rendering for SEO, it is important to make sure that crucial site content is easily accessible and indexable by Google.
A first method to achieve the goal is to understand, and possibly implement, techniques such as server-side rendering (SSR), static site generation (SSG), or prerendering, which generate a static version of the page that can be easily indexed by Google. In particular, static rendering techniques can be particularly effective because they generate a static version of the page during the site build phase, which can be easily indexed by search engines and offer very fast loading times and excellent SEO; the critical aspect is that it can be less flexible for sites with dynamic content.
In these cases, sites with dynamic content, hybrid rendering techniques, such as incremental static rendering (ISR) and distributed static rendering (DSR), should be evaluated and can offer a good balance between performance and flexibility. These techniques combine elements of SSR and SSG, generating static pages that can be easily indexed by search engines, but can also be dynamically regenerated in response to specific events or triggers.
Again, it is critical to monitor and test your site regularly to ensure that the rendering is optimized for SEO: tools such as Google Search Console can help you identify any rendering problems and get suggestions on how to fix them.
How rendering engines work
There are essentially three phases in which the work of a rendering engine is composed:
- Decoding of inputs – or input information.
- Data processing.
- Graphic representation of information.
Trying to simplify, the web rendering engine picks up the source code of the web pages requested by the user and scans all the HTML and CSS elements present; these information are then decoded to start building the web page to display.
In this process are identified the various parts of which the page is composed, such as the text block and its formatting, the background, the color, the content, the various multimedia elements and so on. Using the information in the source code of the resource and in the CSS file, the engine processes and tidies the elements and then, together with the GPU, transforms the source code into the elements actually displayed in the browser tab, finally allowing the user to access and consult correctly the desired resource.
Differences between rendering and indexing
The difference between rendering and indexing can be represented very easily with a comparison between two images, explains Davies: at the top we have the lines of HTML code of a page of our blog, while at the bottom there is the graphical representation of the same page as displayed in the browser.
Basically, it is the same content, first shown for how it looks during indexing (HTML) and then for how it is rendered in rendering (Chrome).
It is Jamie Indigo who takes these concepts further, explaining that rendering is the process in which Googlebot retrieves pages from our site, runs the code, and evaluates the content to understand the layout or structure of the site. All the information gathered by Google during the rendering process is then used to rank the quality and value of the site’s content against other sites and what people are searching for with Google Search.
Each web page essentially has two states, and a website be very different between the two states:
Why rendering is important
Essentially, the reason this process matters is that rendering provides the truth.
Through the code, a search engine can understand what a page is about and roughly what it contains. With rendering, they can understand the user experience and have much more information about which content should have priority.
During the graphical rendering phase the search engine can answer many relevant questions to properly understand a page and how it should be classified. For example, the article mentions issues such as:
- Is the content hidden behind a click?
- Is an ad filling the page?
- Is the content displayed at the bottom of the code actually displayed at the top or in the navigation?
- Is a page loading slow?
When the rendering actually happens
According to the information in our possession – and always in principle – the rendering occurs after indexing and may even take a few days or weeks, with a variable time sequence. This essentially means that search engines will understand the content and context of a page before they get a full understanding of how this should be prioritized.
This obviously does not mean that search engines are completely ignorant until rendering, because there are some solid rules and impressions that they have acquired over the years that allow them to quickly make assumptions about:
- Which elements are present.
- Where they are located.
- How important they are to the user.
But it is only after the graphic rendering of the pages that the engines will know that their assumptions are correct and can fully understand a page and its shape.
Problems with rendering
In summary, search engines send a crawler to the site, which will represent the page as a browser would.
If in Bing “they seem to be a little better at rendering, which is interesting”, at home Google Googlebot has a Web Rendering Service (WRS) component, updated in May 2019, on the occasion of the evergreen update of the crawler, which now also uses the latest version of Chrome for rendering.
In essence, this means that “when your page is rendered by Googlebot it is rendered more or less as you would see it in your browser“. However, it is wrong to think that it is enough to check on a browser if the page works properly, because the sense of rendering is other.
If we have a “basic site with predictable HTML and almost zero dynamic contents, there is really nothing to worry about” and probably there were no worries even with the old WRS configuration.
How Web Rendering Service works
The rendering life cycle follows this path:
- A page is discovered by sitemap, crawler, etc.
- The page is added to the list of pages on the site to be scanned when the crawl budget is available.
- The content of the page is scanned and indexed.
- The page is added to the list of pages to render on a site when budget rendering is available.
- The page is rendered.
Somehow, however, the search engine manages to figure out what where and how it appears on a page, even if it has no eyes to see it: when the process is completed in a positive way, “the rendered version will look like Googlebot and in the graphical browsers”, while “otherwise, the page is likely to be based on an unsupported function such as a user’s permission request”.
Rendering is still problematic
According to Davies, at the moment “we can count on the indexing capabilities of the engines, but the rendering side still has a long way to go” to bridge the gap between what search engines see and what a user’s browser does.
Based on the author’s experience, there is a possibility that in the short-to-medium term “the latency between indexing and rendering is drastically reduced, especially on sites that are based on this process”, and this could lead to the opening of a world for sites that need graphical rendering to be understood.
Google rendering: how it works and what value it has
To render web pages, Google uses a two-step or wave process designed to help the search engine better understand the content of a web page and index it effectively.
The first phase of Google’s rendering process is the analysis of a page’s raw HTML: during this phase, Google examines a page’s HTML to identify basic content and structural information, through a kind of “quick read” that allows the search engine to get a general view of the page’s content.
Even worse, of course, if content is not properly rendered, because in this case Google may not be able to “see” and thus index this content, resulting in reduced visibility of the site in search results. In addition, inefficient rendering can lead to longer loading times, which can negatively affect the user experience and, once again, the site’s ranking in search results.
Rendering according to Google: the recipe analogy
To better understand how Google handles rendering, we can think of what happens when we prepare a recipe, taking an analogy devised by the aforementioned Martin Splitt.
Google’s first wave of indexing is like reading the ingredient list of a recipe: it gives Google an overview of what the page contains. However, like reading a list of ingredients, it does not get the full picture of the finished dish.
To ensure that the “dish” is ready to be served as soon as Google reads the “ingredient list,” we can use techniques such as server-side rendering (SSR) or prerendering, which prepare the page in advance, generating a static version that can be easily “tasted” by Google during its first wave of indexing.
Furthermore, just as a cook would do taste tests while preparing a dish, we should regularly test the site with tools such as Google Search Console to ensure that Google is able to “taste” and index the content correctly.
Ultimately, understanding how Google “prepares” the site and how to optimize the dish for this process can make the difference between a site that is easily “tasted” and indexed by Google, and one that remains hidden in the “refrigerator.”