SEARCH ENGINE JOURNAL “THE IMPORTANCE OF DYNAMIC RENDERING” — SHOW NOTES

Podcast Description

Podcast Title:Search Engine Journal

Hosts:Brent Csutoras

Guest:Geoff Atkinson

Summary

In this episode, Brent and Geoff talk about the technical SEO topic of dynamic rendering. They cover everything you need to know — what it is, whether you need to use it or not, and how it can help your digital marketing efforts.

Click here to listen to the full episode

What You'll Learn in This Episode

Time Stamped Highlights

[1:36] An Introduction to Dynamic Rendering: What is it?

Brent:Yeah, I mean, this is one of those really interesting conversations that we're going to talk about today. And it really comes down to dynamic rendering, right, it is a very interesting topic for me, because I think that this is one of those things that like really high level, hardcore SEOs have been, you know, kind of knee-deep in for 5-6 years, I mean, for some time now, right? But it's also one of those things where, when you bring it up, a lot of people don't get it. They don't know what it is, they don't understand the point. And so I think this is a really valuable conversation to have, especially in this time period right now, with so much changing, technologically-wise with Google, kind of the landscape as a whole and AI and a lot of the learning elements, I think it's, it's really an interesting discussion.

So I wanted to kind of start it off a little bit by asking you, you know, what is dynamic rendering? And is it the same as pre-render? How is it different, you know, kind of give us a breakdown of what exactly this is we're talking about?

Geoff:Yeah, so it is pre-render. Huckabuy has a dynamic rendering product as well, that's solely based on SEO, but in my opinion, actually, dynamic rendering is probably the biggest change Google's made in maybe 10 years. I mean, for them to actually offer an option where they say, “We can crawl something that's different from what the user experiences now — content and data all need to match up,” that's a big change for them. 

Brent:Right. I mean, for years, they were like you have to have, you know, the user experience be the same thing. For context. Just real quick for context. I mean, anything that was different previously was considered cloaking. Right? So even Google has now said, very clearly, that pre-render, or dynamic rendering as a whole, right, any of these kind of like terms that you're used to using for this technology, dynamic rendering is not cloaking, as long as, like you said, they keep the information close.

Geoff:Correct. Yeah. So dynamic rendering. It's actually a pretty straightforward concept. It started with the difference between a mobile device and a desktop. All it means is that a URL will render differently or dynamically based on what calls it. So, if you call a web page from your mobile device, you're going to get one experience. If you call one from your desktop, you're going to get a slightly different one. Google’s big change was they said, “Well, now you can actually give a version for us.” 

Really, the reason they made that change, is because of the amount of JavaScript and front-end, dynamic technologies that have made it difficult for them to crawl and understand a site. They basically said, “Here you go, here's a way for us to keep it simple. Give us a simplified version, and we'll be able to crawl and index that much more efficiently than what the user is experiencing.” Google doesn't care about stuff like chat boxes, and so on. So yeah, it's kind of a big shift for them. It’s one that I think a lot of CEOs are trying to take advantage of.

[4:28] Why You Should Use Dynamic Rendering For Your Site

Brent:I think a lot of people are gonna ask the question like, “Well, why do I need to create something different for Google? Why not just let them crawl my site?” What is one or two things that will let people really visualize what this is?

Geoff:I'd say the most famous JavaScript thing that really makes Google get caught up while crawling is actually chat boxes. So chat boxes, personalization, tracking tags that are dynamic. As soon as they hit JavaScript, they simply can't crawl it with their HTML crawler. And so it goes to a rendering queue, and a rendering queue takes quite a bit more processing time. A rendering queue is literally the same technology as your Chrome browser. So it's just, you know, executing a page fully allowing them to come in and actually crawl that dynamic content takes more processing time. 

So, if you can, strip that stuff out in a dynamically rendered version. So for our Huckabuy Cloud dynamic rendering, for example, if we were to take a customer that's on our product and look at their actual page, in the Huckabuy Cloud, or that dynamically rendered version of the of the page, it almost looks identical, but it's like 20-40%, like the size of the previous page — it's wider, it's faster. It's flat HTML, it looks very similar, but you are going to see some of the dynamic stuff getting pulled out. So, chat boxes and things like that. 

The dynamically rendered version is really a simplified version that's queued up for them so that when they come and crawl with their HTML crawler, they can literally download the entire site really quickly and don't have to kick however many pages you have indexed over into this rendering queue — which takes processing time and a lot of money on their end. That’s why they're a fan of it. 

Brent: I think the other thing that makes a really big impact on SEO is the fact that it's not only a longer processing time, but you can wait weeks before they even visit to render your site. If they identify that you need a rendering crawl, I've actually dealt with a client that was having a hard time getting content to show up and they had a social platform that was running on pure custom Java, and they were like, “Nothing's getting indexed till six months later.” And I'm like, “Yeah, because you're running pure Java, and they're not even trying for another three to four months.” I think that's another big point.

[6:58] The SEO Benefits of Dynamic Rendering

Brent:There's obviously some basic benefits that would come from dynamic rendering. I mean, there's a clear benefit of like, you want to show Google what you want to show Google. But are there any other SEO benefits? What are some of the other things that dynamic rendering will do for somebody's website that they might not get otherwise?

Geoff: I think your example that you just made, where there's content that's not being indexed, and is taking, you know, 3-6 months for it to get into index provides good insight. You could have all the content resources in the world, but if Google can't see that actual content, what good is it doing? So, we see that a lot.

Companies have bigger indexation issues than they have any idea about, because it's kind of hard to know. You see the crawl stats, right? You're like, “Oh, they're crawling me. I'm good.” And you see that they're downloading information, but you don't really know exactly what they're downloading and how much they are actually accessing the stuff that you're working on. With dynamic rendering, all those problems just get eliminated. All the content’s being indexed, and content affects rankings and rankings affect traffic. So you get a pretty significant benefit. If the site is pretty heavy in JavaScript or difficult to crawl, all of a sudden, they're going to become privy to all this new information in a very short amount of time. And that's actually going to impact rankings and traffic and all those other good things.

[8:23] Similarities Between Dynamic Rendering and Mobile Optimization

Brent:I feel like it's really like mobile where the conversation with most people when mobile came out was like, “You can do all the best content. You can do all the best link building, you can do all the best SEO in the world. But if you're not included in a mobile search index, because you're not mobile friendly, it doesn't do any good.” And it kind of seems like that's the same place that we're at right now with web page indexation. Things have gotten so complicated with crawling, that we're at a place where it's like, if you do everything perfectly, but you're not crawlable then it's all for not. Is it that serious?

Geoff: You're spot on. It's exactly the same. And what's even more shocking is mobile because of the impact of page speed and all those things, that's still a huge problem for companies. They think that they're mobile friendly, but their page speed is slow, so slow that they just really can't get any rankings. And, you know, page speed I don't know how long Google's gonna have to, you know, be screaming about page speed for people to listen, but, you know, everybody is sort of guilty of bad page speed. But it's such a big factor. And you're exactly right, this dynamic rendering problem or sort of indexation problem is very similar, where it's like, you can be doing everything you want, but if you don't have the technical side, those sort of boxes checked, good luck, like it's just not gonna work.

[9:42] Why Doesn’t Everyone Embrace Dynamic Rendering?

Brent:I've been harping on this for years, there's a lot of experts like yourself, there's just a ton of people out there who are talking about this and have been for years — even Martin from Google — there's just a lot of conversation about this. Yet, very few people are really incorporating it or really taking it as a key thing. Why do you think the SEO community, as a whole, has not really embraced this or that it's not on every site?

Geoff:Yeah, I find that shocking. But we need to take a step back and look at marketing departments and their general skill set. Even SEO groups sometimes aren't the most technical. So if you think of like a marketing organization, whether it's, you know, Salesforce or whoever,

their skill set is really not technical SEO. That's the last thing they get to. They very rarely have developers working on SEO. It's a very technical problem. So you can throw tons of resources, at content, link building, and all those more straightforward tasks and not even fully understand, or fully recognize, the technical problems that you have, because you just don't have that skill set on the team. We see that almost happen everywhere. Even if they're working with an agency or whoever, that technical skill set is so rare. 

Within our little community, it's big, right? But for when you step into a big internal marketing team, you're like, "there's just no one there that speaks that language." We're educating them on this language. So, I think that's the reason is that it's such a different hat to wear as a marketer getting into technical SEO vs. managing your PPC spend or your content team or branding and messaging, or social. It's just a totally different skill set, and it's usually missing. I think that's kind of why it hasn't been adopted as quickly as we would like.

[11:49] SEOs, Developers, and CMOs Should All Care About Dynamic Rendering

Brent: I think a lot of SEOs deal with issues with development teams. I don't know if it's as bad of a problem as it was in the past, but it's like, the marketing team isn't always connected to the developing team; they're kind of two separate paths that are working in a company, and they have different goals. A lot of times, for developers and designers, their goal is to make a site functional. I've had a lot of pushback from people that just don't want to implement something because they just don't get it or like it. 

What would you suggest to somebody that might be listening who's dealing with that? How would they convince the developers? Are there certain aspects that they can talk about to a developer that will resonate more with the developers or is there a way of talking about it, that will make them more successful?

Geoff: Great question. Previous to this, I was the SVP of marketing at Overstock, and we had a crazy SEO run. We went from like zero to 300 million in just a few short years. We had, like 40 people working on SEO, and over half of them were developers. We just realized that you needed developers to change the site in a meaningful way, for you really to get rankings and you need to work on page speed and all these things. We had a laundry list of so many projects that we just started building out, you know, dev team after dev team, and eventually, it was a pretty massive team. But it is like an organizational mindset that you have to get in. For some reason, it was really straightforward for us there. Fortunately, I had the CEO Patrick Byrne was like, “Yeah, this totally makes sense. Do whatever you have to do. We'll spend whatever we need to spend to grow this because it's just growing so quickly,” but that’s a rare scenario within organizations. And so I feel really fortunate to have experienced that. 

But when I go into other organizations, now we look at what they're doing, it's almost impossible to get some of these things done. So, shameless plug, you can outsource it to Huckabuy. But there it is, it's a tough thing to do. And even as a CMO, like I have lots of friends that are CMOs, and they're like, “Good luck prioritizing a completely separate version of our site that no user will ever interact with, and it's just for Google as a dev priority within my organization.” 

Brent: Yeah. That's the hardest battle, trying to explain the outcome, right? 

I've seen situations where schema’s in the same place like a lot of people will harp on schema, and the company's like, “Okay, well, how is that going to increase our sales?” And you're like, “Well, it's potentially going to do A, B, and C,” and they're like, “Well, what does it actually mean from a numbers standpoint?” So I definitely think that that's a tough thing. I do think that companies like yours are extremely valuable in this space. There's just so many things that are happening, like right now, that, you know, are extremely difficult for an average SEO or an average person to even understand, let alone implement. And so I think it's things like these are really becoming solutions where you need a specialty, you know, you need a company that's focused on it and really in the weeds on it, and it sounds like that's what your company's kind of focusing on.

Geoff:Yeah, I'm always blown away about how slow CMOs are to adopt, like the ROI game when it comes to SEO. It's always like the best new customer driver. It's a huge revenue driver. You know, I think of it as like the oxygen of the internet. If you don't get you know, organic search traffic, you're just screwed. People still spend hundreds of thousands, if not millions of dollars a month on PPC and they won't be willing to invest, 10k a month in their SEO or whatever and it's just sort of shocking. When I was at Overstock, I thought of SEO just like any other channel, it's like, if I can spend $1 and get this much back, I'm going to do that. If I find an ATM where I put $1 in and gives me $2 back, I'm gonna use that ATM until I need to fall asleep at night. So yeah, it's a weird channel that they don't invest in. 

I mean, our average customer grows 61% in 12 months. So the ROI on that is just crazy. And these are some big companies like Salesforce and SAP. So yeah, it's always a fight. I'm always surprised. Really good CMOs and VPs of Marketing get it. The other weird thing is like B2B is such a different world than B2C. If you're not doing really good SEO in B2C, you have really no chance of surviving, but with B2B, the margins are so great and you can have sales-driven organizations that can generate enough business that they kind of get to SEO later in their lifecycle, even though the financial impact is insane if you crank up your SEO and inbound leads for these major software players.

[19:50] Does Everyone Need Dynamic Rendering?

Brent:My next question would be kind of like we obviously have established that there's a value in dynamic rendering, but do you feel like everybody needs to have dynamic rendering? There's industries in my mind where maybe it's not the biggest battle to win. Does everybody need to fight for this battle and win it, or only certain people?

Geoff:I'd say probably 60% of sites out there need it, which is a lot. And then there's 40%, where it's like, it'd be nice to have, but it's not going to blow your socks off. It's a really small site, you know, maybe there's only 100 pages indexed, so Google can get through it, the site doesn't change that much, there's just not as much upside as some of these larger sites that are more complicated where Google's really struggling to understand them. There are a good number of sites that don't necessarily need it. But everybody could benefit from it. 

What we find is that about 60% of the internet could use this solution, which is a large percentage. Think about the number of JavaScript things that are included by business owners on their websites without thinking at all about what this does for Google crawling. They're gonna be like, “Yeah, we want personalization, we want chat boxes” and they just throw it on there. And meanwhile, Google's like, “God, why did you do this? It makes our job almost impossible.” 

[21:35] Is Dynamic Rendering Here To Stay?

Brent: Now we're seeing a lot of conversation from Google and a lot of processes where now they're going to a single-crawl engine, right? The Java engine is supposed to be the single solution at some point. I don't want to start making statements like, “it's gonna happen now,” but that's, that's my understanding is this is kind of the direction they're going in. In that process, they actually said, they're going to need more help than ever with Java. And it's not because of their inability to read it, but it's more that everybody customizes and implements Java in different layers in different ways. And that becomes the challenge, right? Is it you know, they're trying to properly view your website when there's all these technologies going on. 

I guess my question is, this something that you see as a need for a long period of time? Is the single crawl going to solve the dynamic rendering requirement, or is it only going to make it even more important?

Geoff: So, yeah, what I think is going to happen is typically, Google works in an additive fashion. So very rarely do they completely eliminate something. So because their HTML crawler is so much more efficient than their rendering process, I would have a hard time saying that they're going to just get rid of it. For them to get rid of their HTML crawler is like them getting rid of the importance of backlinks, almost, it would be so hard for them to pull out of their systems. Now, maybe they will. And that would be really interesting if they got there. But I think it's more additive where they're like, “We have this rendering process. It's working. It's not the best and it takes a lot of processing time. So we're going to slowly make it better and better and better. And we're going to try to implement some standards to the internet that make our life easier.” But I think there's always going to be a pretty good chunk of the internet that's crawled by their other bots, because it's just so much more efficient. Now, I could be totally wrong, but I think that transition is going to take time, and it's going to take probably a lot of time. And, you know, they probably wouldn't have made dynamic rendering available, if they had a solution ready to roll. I do think it's going to be around for a while. They can move quickly, and they sometimes do some really impressive stuff like that, but I think it will take time, but, you know, your guess is as good as mine.

[25:40] Prediction: Sites Will Only Get More Complex, Making Dynamic Rendering and Structured Data Vital

Brent:Yeah. My honest opinion is, I think that where we're going is only going to be more complicated. I mean, we're starting to go into an era where, you know, complicated layers of data on, you know, whether it's an asset, a website, a mobile site, whatever is just getting more and more complicated and I think that being it to present Google with a standard single website that you actually control is very valuable. 

No single component and SEO today is like, sink or swim, but it all has an impact and so I think it's going to be stronger, stronger. 

Geoff:Well, I think you make a really good point about the future of this conversation between Google sites getting more complex. Like you look at just structured data and the adoption of structured data, like that's almost a shortcut, right, to be able to communicate what's happening on any given page with, you know, just a few kilobytes of information that super accurately describe what's happening on that page. So if you look at the future, you know, will they just get more sophisticated? Do they create and build up resources to crawl more complicated sites? Or do they push to simplify the communication between a website and a search engine? It could be that that method of communication evolves, where it's not just dynamic rendering, you have a way to sort of feed them what they need to be able to do their job, and then you are able to do all the complicated front-end stuff on your own and they just kind of ignore it. 

Brent:Even in two years you went from schema being four-five lines to being literally hundreds of lines per page now. Anybody out there who's really in the schema space, you realize now it's almost to a point where you can't even do it by hand. You know, and especially when you get into like the Gutenberg style content creation of the dynamic content, fundamentals as you kind of transition more and more into where we're taking content from being less of a single webpage to being almost like dynamic, in a sense, you know, where every element is its own element, you start seeing schema. So even in a couple years' time period, we've gone from something super simple to something extremely complicated.

Geoff:Yeah, Huckabuy sure has ridden that wave. Our first product was automated structured data. And it was pretty simple at first. And now to have automated structured data across all these sites, and the way that data down can communicate through this JSON-LD format — which is the structured data format that Google's really interested in — it just gets more and more complicated. So that's been a wild ride and one that we're really proud to have been on the frontlines of. 

Now with voice search is almost entirely powered by structured data and you just look at, you know, any given query on your mobile devices, you can see just how much of the above the fold is being captured by structured data. Yeah, it's gonna be kind of wild to see how that all plays out. But it's just getting more complicated and more technical, which is, you know, it's just interesting times, it's evolving super fast. And that's exciting from our end, because that's what we work on every day.

[29:40] Huckabuy’s Automated Software Solution

Brent:At Huckabuy, you guys still provide the automated structured data solution? 

Geoff:Oh, yeah, that’s our primary product. I mean, Huckabuy Cloud is only about a year old. So for most, I'd say 60% of our customers are literally just using the structured data product and then the rest are starting, you know, they're all kind of adopted. But yeah, that was our first product. 

Brent:Yeah, my friend John Alderson works with Yoast, and does the schema implementation. We were having a conversation about it, you know, even a year ago, when we were in Germany at Oktoberfest, and he was saying, “Look, honestly, at this point, you can't hand-code your schema effectively, you have to have an automated solution.” And so, I think that's really cool. I didn't know that was something you guys had built, but I think that's something valuable out there. Because I don't think there's a lot of people that provide that. I think that a lot of people don't realize how important that is going forward.

Geoff:Yeah, moves the needle a lot. That's for sure. So take a site from wherever it is to really good structured data. Boy, they get a huge jump. It's really about the setup of the structured data by page-type that gets the automation there. Once we know what to look for, the system optimizes itself. But the great part about the automation piece, especially for structured data, is that it's a moving target. Structured data is changing all the time in terms of what they want — like requirements and so forth. Also, websites are constantly changing. So to have those two variables just taken care of with an automated solution is really helpful.

[30:59] How To Implement Dynamic Rendering

Brent:It's very exclusionary. Either you are feeding Google what they want, or you're not included anymore. That's one of the big shifts in SEO that people haven't really come to grips with. It's not, “Can I do good enough anymore.” It's “If you don't do it, you're not there, period.” 

What does it look like to implement dynamic rendering? Is this something where you're building a whole other site? Or is it something like schema markup where you're putting a couple of pieces of code on there. Obviously, I think solutions like yours probably have something to make it a lot easier for people. But if you can answer in two parts. What is it like if someone is just trying to do it on their own, maybe if their a developer, and what does it look like when a company like yours is able to help with that solution?

Geoff:Sure. So to address the first piece of how to do it on your own, the crux of dynamic rendering is really the conversion of your dynamic content into flat HTML. So, if you can do that, that's sort of the technical challenge is to be able to do that. So if you have content being generated through JavaScript, that is important for your rankings, and you want Google to be aware of it, being able to convert that into flat HTML, and leveraging some sort of CDN like CloudFlare or CloudFront, or Akamai or whoever, to basically load that information up really quickly and eliminate literally all the JavaScript on the page, that's how you kind of have to go. It's doable. For sure. We actually see some companies doing it in house. It's kind of hard to do in-house, but we see it happening.

The second way to do it is to use Huckabuy. We think the automation of that is a really slick way to do it. We've built that converter so that all you have to do — like we don't actually have to have any developer look at your site, they don't have to, you know, log in and do a bunch of work, you literally make a DNS change. And then Huckabuy takes over the bot traffic and we create this dynamically rendered version through our Huckabuy Cloud, that's flat HTML, and we have a partnership with CloudFlare that allows us to keep all this information at edge. 

You're gonna hear that term being used now, “at edge SEO.” At edge basically means it's pre-cached and located all around the world in 200 different locations so that no matter where a bot’s coming from they get this really lightweight and cached version of a site. Meaning the average page speed is between 100-300 hundred milliseconds and it has the full structured data baked in and all the contents there.

It's just a DNS change, which is pretty straightforward. If you're already on CloudFlare, it's just an API token. So it's a quick, quick switchover. And usually, you know, you're live within five to seven days. And rockin’ 'and rollin’. It checks a big box vs. trying to knock it off in-house.

[36:44] How To Measure The Results of Dynamic Rendering

Brent:We don't try to pick winners, you know, we're just trying to talk about these conversations and give people some opportunities to find solutions. That's the end of it. What matters is people are able to solve their problems and make money and have success, right? So, with everything that people do in marketing, the first question you always get is, “Well, how do I know it's working? Like how do I know it was worth it?” 

What should people expect from making a change like this? What can they kind of expect, as far as measuring, whether they should continue to do it or whether it was a good choice or when and report to their higher ups and so forth?

Geoff: The first thing that moves is the crawl stats. So you'll see Google come in pretty quickly once they realize they can crawl through the site so fast. So your crawl stats are the first thing that's going to start to move. Then you'll probably see that you start ranking for new keywords because you're getting more content indexed, and all of a sudden, the site's getting associated with more mid-tail terms. Once that happens, you'll start to get more impressions as those links start to pop up in the top 10. Hopefully, you have good structured data and start capturing some rich cards. And then ultimately, what we care about is clicks and traffic, which generates revenue. That's kind of how we see it play out.

38:45 Examples of Dynamic Rendering Done Wrong

Brent:So there's normally some examples of people that are kind of doing this right or wrong. Is there any examples of not doing dynamic rendering and causing problems that, you know, off-hand? 

Geoff:There's a lot of examples of people not doing dynamic rendering with lots and lots of JavaScript. And resulting in stuff not getting indexed. I think SAP is a great example. They had a data page on sap.com, with over 100 JavaScript tags, which made it almost impossible for Google to ever crawl it. 

The other thing to just keep in mind with the rendering process, is it usually has about a five second timeout. Lighthouse uses a really slow connection speed. So you might think that you're under five seconds, but in fact, you're at 11 or 12. So they just leave after that amount of time, meaning they're not going to render it at all. So that presents a lot of problems. We see a lot of companies with that situation. I haven't seen a company try to do dynamic rendering and screw it up that just completely ruins their index. I haven't seen that happen yet, but it probably has just not out there actively looking for it. 

Brent: I've seen in the examples of people not implementing dynamic rendering right, is that it ends up just not working. You know, it's just like, still passing JavaScript through to the bot.

Geoff: Right. Yeah, that that does happen. And, you know, there's really nothing lost there. Or because that's just what they were experiencing previously. They've just put a lot of money into something that doesn't work. So yeah, that happens.

Additional Resources

Contact:

SEO Tools: