Transferring your domains to a new registrar isn’t something you do every day, and getting any step of the process wrong could mean downtime and disruption. That’s why this Speed Week we’ve prepared a domain transfer checklist. We want to empower anyone to quickly transfer their domains to Cloudflare Registrar, without worrying about missing any steps along the way or being left with any unanswered questions.
Domain Transfer Checklist
Confirm eligibility
Confirm you want to use Cloudflare’s nameservers: We built our registrar specifically for customers who want to use other Cloudflare products. This means domains registered with Cloudflare can only use our nameservers. If your domain requires non-Cloudflare nameservers then we’re not the right registrar for you.
Confirm your domain is not a premium domain or internationalized domain name (IDNs): Cloudflare currently does not support premium domains or internationalized domain names (Unicode).
Confirm your domain hasn’t been registered or transferred in the past 60 days: ICANN rules prohibit a domain from being transferred if it has been registered or previously transferred within the last 60 days.
Confirm your WHOIS Registrant contact information hasn’t been updated in the past 60 days: ICANN rules also prohibit a domain from being transferred if the WHOIS Registrant contact information was modified in the past 60 days.
Before you transfer
Gather your credentials for your current registrar: Make sure you have your credentials for your current registrar. It’s possible you haven’t logged in for many years and you may have to reset your password.
Make note of your current DNS settings: Make note of your current DNS settings: When transferring your domain, Cloudflare will automatically scan your DNS records, but you’ll want to capture your current settings in case there are any issues. If your current provider supports it, you could use the standard BIND Zone File format to export your records.
Remove WHOIS privacy (if necessary): In most cases, domains may be transferred even if WHOIS privacy services have been enabled. However, some registrars may prohibit the transfer if the WHOIS privacy service has been enabled.
Renew your domain if up for renewal in the next 15 days: If your domain is up for renewal, you’ll need to renew it with your current registrar before initiating a transfer to Cloudflare.
Unlock the domain: Registrars include a lightweight safeguard to prevent unauthorized users from starting domain transfers – often called a registrar or domain lock. This lock prevents any other registrar from attempting to initiate a transfer. Only the registrant can enable or disable this lock, typically through the administration interface of the registrar.
Sign up for Cloudflare: If you don’t already have a Cloudflare account, you can sign up here.
Add your domain to Cloudflare: You can add a new domain to your Cloudflare account by following these instructions.
Add a valid credit card to your Cloudflare account: If you haven’t already added a payment method into your Cloudflare dashboard billing profile, you’ll be prompted to add one when you add your domain.
Wait for your DNS changes to propagate: Registrars can take up to 24 hours to process nameserver updates. You will receive an email when Cloudflare has confirmed that these changes are in place. You can’t proceed with transferring your domain until this process is complete.
Initiating and confirming transfer process
Request an authorization code: Cloudflare needs to confirm with your old registrar that the transfer flow is authorized. To do that, your old registrar will provide an authorization code to you. This code is often referred to as an authorization code, auth code, authinfo code, or transfer code. You will need to input that code to complete your transfer to Cloudflare. We will use it to confirm the transfer is authentic.
Initiate your transfer to Cloudflare: Visit the Transfer Domains section of your Cloudflare dashboard. Here you’ll be presented with any domains available for transfer. If your domain isn’t showing, ensure you completed all the proceeding steps. If you have, review the list on this page to see if any apply to your domain.
Review the transfer price: When you transfer a domain, you are required by ICANN to pay to extend its registration by one year from the expiration date. You will not be billed at this step. Cloudflare will only bill your card when you input the auth code and confirm the contact information at the conclusion of your transfer request.
Input your authorization code: In the next page, input the authorization code for each domain you are transferring.
Confirm or input your contact information: In the final stage of the transfer process, input the contact information for your registration. Cloudflare Registrar redacts this information by default but is required to collect the authentic contact information for this registration.
Approve the transfer with Cloudflare: Once you have requested your transfer, Cloudflare will begin processing it, and send a Form of Authorization (FOA) email to the registrant, if the information is available in the public WHOIS database. The FOA is what authorizes the domain transfer.
Approve the transfer with your previous registrar: After this step, your previous registrar will also email you to confirm your request to transfer. Most registrars will include a link to confirm the transfer request. If you follow that link, you can accelerate the transfer operation. If you do not act on the email, the registrar can wait up to five days to process the transfer to Cloudflare. You may also be able to approve the transfer from within your current registrar dashboard.
Follow your transfer status in your Cloudflare dashboard: Your domain transfer status will be viewable under Account Home > Overview > Domain Registration for your domain.
After you transfer
Test your site and email: After the transfer is complete, you’ll want to test your site to ensure everything is working properly. If you encounter any issues or have any questions you can always talk with us on our community forums or Discord server.
Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.
To learn more about our mission to help build a better Internet, start here. If you’re looking for a new career direction, check out our open positions.
Internet connections are most often marketed and sold on the basis of “speed”, with providers touting the number of megabits or gigabits per second that their various service tiers are supposed to provide. This marketing has largely been successful, as most subscribers believe that “more is better”. Furthermore, many national broadband plans in countries around the world include specific target connection speeds. However, even with a high speed connection, gamers may encounter sluggish performance, while video conference participants may experience frozen video or audio dropouts. Speeds alone don’t tell the whole story when it comes to Internet connection quality.
Additional factors like latency, jitter, and packet loss can significantly impact end user experience, potentially leading to situations where higher speed connections actually deliver a worse user experience than lower speed connections. Connection performance and quality can also vary based on usage – measured average speed will differ from peak available capacity, and latency varies under loaded and idle conditions.
The new Cloudflare Radar Internet Quality page
A little more than three years ago, as residential Internet connections were strained because of the shift towards working and learning from home due to the COVID-19 pandemic, Cloudflare announced the speed.cloudflare.com speed test tool, which enabled users to test the performance and quality of their Internet connection. Within the tool, users can download the results of their individual test as a CSV, or share the results on social media. However, there was no aggregated insight into Cloudflare speed test results at a network or country level to provide a perspective on connectivity characteristics across a larger population.
Today, we are launching these long-missing aggregated connection performance and quality insights on Cloudflare Radar. The new Internet Quality page provides both country and network (autonomous system) level insight into Internet connection performance (bandwidth) and quality (latency, jitter) over time. (Your Internet service provider is likely an autonomous system with its own autonomous system number (ASN), and many large companies, online platforms, and educational institutions also have their own autonomous systems and associated ASNs.) The insights we are providing are presented across two sections: the Internet Quality Index (IQI), which estimates average Internet quality based on aggregated measurements against a set of Cloudflare & third-party targets, and Connection Quality, which presents peak/best case connection characteristics based on speed.cloudflare.com test results aggregated over the previous 90 days. (Details on our approach to the analysis of this data are presented below.)
Users may note that individual speed test results, as well as the aggregate speed test results presented on the Internet Quality page will likely differ from those presented by other speed test tools. This can be due to a number of factors including differences in test endpoint locations (considering both geographic and network distance), test content selection, the impact of “rate boosting” by some ISPs, and testing over a single connection vs. multiple parallel connections. Infrequent testing (on any speed test tool) by users seeking to confirm perceived poor performance or validate purchased speeds will also contribute to the differences seen in the results published by the various speed test platforms.
And as we announced in April, Cloudflare has partnered with Measurement Lab (M-Lab) to create a publicly-available, queryable repository for speed test results. M-Lab is a non-profit third-party organization dedicated to providing a representative picture of Internet quality around the world. M-Lab produces and hosts the Network Diagnostic Tool, which is a very popular network quality test that records millions of samples a day. Given their mission to provide a publicly viewable, representative picture of Internet quality, we chose to partner with them to provide an accurate view of your Internet experience and the experience of others around the world using openly available data.
Connection speed & quality data is important
While most advertisements for fixed broadband and mobile connectivity tend to focus on download speeds (and peak speeds at that), there’s more to an Internet connection, and the user’s experience with that Internet connection, than that single metric. In addition to download speeds, users should also understand the upload speeds that their connection is capable of, as well as the quality of the connection, as expressed through metrics known as latency and jitter. Getting insight into all of these metrics provides a more well-rounded view of a given Internet connection, or in aggregate, the state of Internet connectivity across a geography or network.
The concept of download speeds are fairly well understood as a measure of performance. However, it is important to note that the average download speeds experienced by a user during common Web browsing activities, which often involves the parallel retrieval of multiple smaller files from multiple hosts, can differ significantly from peak download speeds, where the user is downloading a single large file (such as a video or software update), which allows the connection to reach maximum performance. The bandwidth (speed) available for upload is sometimes mentioned in ISP advertisements, but doesn’t receive much attention. (And depending on the type of Internet connection, there’s often a significant difference between the available upload and download speeds.) However, the importance of upload came to the forefront in 2020 as video conferencing tools saw a surge in usage as both work meetings and school classes shifted to the Internet during the COVID-19 pandemic. To share your audio and video with other participants, you need sufficient upload bandwidth, and this issue was often compounded by multiple people sharing a single residential Internet connection.
Latency is the time it takes data to move through the Internet, and is measured in the number of milliseconds that it takes a packet of data to go from a client (such as your computer or mobile device) to a server, and then back to the client. In contrast to speed metrics, lower latency is preferable. This is especially true for use cases like online gaming where latency can make a difference between a character’s life and death in the game, as well as video conferencing, where higher latency can cause choppy audio and video experiences, but it also impacts web page performance. The latency metric can be further broken down into loaded and idle latency. The former measures latency on a loaded connection, where bandwidth is actively being consumed, while the latter measures latency on an “idle” connection, when there is no other network traffic present. (These specific loaded and idle definitions are from the device’s perspective, and more specifically, from the speed test application’s perspective. Unless the speed test is being performed directly from a router, the device/application doesn’t have insight into traffic on the rest of the network.) Jitter is the average variation found in consecutive latency measurements, and can be measured on both idle and loaded connections. A lower number means that the latency measurements are more consistent. As with latency, Internet connections should have minimal jitter, which helps provide more consistent performance.
Our approach to data analysis
The Internet Quality Index (IQI) and Connection Quality sections get their data from two different sources, providing two different (albeit related) perspectives. Under the hood they share some common principles, though.
IQI builds upon the mechanism we already use to regularly benchmark ourselves against other industry players. It is based on end user measurements against a set of Cloudflare and third-party targets, meant to represent a pattern that has become very common in the modern Internet, where most content is served from distribution networks with points of presence spread throughout the world. For this reason, and by design, IQI will show worse results for regions and Internet providers that rely on international (rather than peering) links for most content.
IQI is also designed to reflect the traffic load most commonly associated with web browsing, rather than more intensive use. This, and the chosen set of measurement targets, effectively biases the numbers towards what end users experience in practice (where latency plays an important role in how fast things can go).
For each metric covered by IQI, and for each ASN, we calculate the 25th percentile, median, and 75th percentile at 15 minute intervals. At the country level and above, the three calculated numbers for each ASN visible from that region are independently aggregated. This aggregation takes the estimated user population of each ASN into account, biasing the numbers away from networks that source a lot of automated traffic but have few end users.
The Connection Quality section gets its data from the Cloudflare Speed Test tool, which exercises a user’s connection in order to see how well it is able to perform. It measures against the closest Cloudflare location, providing a good balance of realistic results and network proximity to the end user. We have a presence in 285 cities around the world, allowing us to be pretty close to most users.
Similar to the IQI, we calculate the 25th percentile, median, and 75th percentile for each ASN. But here these three numbers are immediately combined using an operation called the trimean — a single number meant to balance the best connection quality that most users have, with the best quality available from that ASN (users may not subscribe to the best available plan for a number of reasons).
Because users may choose to run a speed test for different motives at different times, and also because we take privacy very seriously and don’t record any personally identifiable information along with test results, we aggregate at 90-day intervals to capture as much variability as we can.
At the country level and above, the calculated trimean for each ASN in that region is aggregated. This, again, takes the estimated user population of each ASN into account, biasing the numbers away from networks that have few end users but which may still have technicians using the Cloudflare Speed Test to assess the performance of their network.
Navigating the Internet Quality page
The new Internet Quality page includes three views: Global, country-level, and autonomous system (AS). In line with the other pages on Cloudflare Radar, the country-level and AS pages show the same data sets, differing only in their level of aggregation. Below, we highlight the various components of the Internet Quality page.
Global
The top section of the global (worldwide) view includes time series graphs of the Internet Quality Index metrics aggregated at a continent level. The time frame shown in the graphs is governed by the selection made in the time frame drop down at the upper right of the page, and at launch, data for only the last three months is available. For users interested in examining a specific continent, clicking on the other continent names in the legend removes them from the graph. Although continent-level aggregation is still rather coarse, it still provides some insight into regional Internet quality around the world.
Further down the page, the Connection Quality section presents a choropleth map, with countries shaded according to the values of the speed, latency, or jitter metric selected from the drop-down menu. Hovering over a country displays a label with the country’s name and metric value, and clicking on the country takes you to the country’s Internet Quality page. Note that in contrast to the IQI section, the Connection Quality section always displays data aggregated over the previous 90 days.
Country-level
Within the country-level page (using Canada as an example in the figures below), the country’s IQI metrics over the selected time frame are displayed. These time series graphs show the median bandwidth, latency, and DNS response time within a shaded band bounded at the 25th and 75th percentile and represent the average expected user experience across the country, as discussed in the Our approach to data analysis section above.
Below that is the Connection Quality section, which provides a summary view of the country’s measured upload and download speeds, as well as latency and jitter, over the previous 90 days. The colored wedges in the Performance Summary graph are intended to illustrate aggregate connection quality at a glance, with an “ideal” connection having larger upload and download wedges and smaller latency and jitter wedges. Hovering over the wedges displays the metric’s value, which is also shown in the table to the right of the graph.
Below that, the Bandwidth and Latency/Jitter histograms illustrate the bucketed distribution of upload and download speeds, and latency and jitter measurements. In some cases, the speed histograms may show a noticeable bar at 1 Gbps, or 1000 ms (1 second) on the latency/jitter histograms. The presence of such a bar indicates that there is a set of measurements with values greater than the 1 Gbps/1000 ms maximum histogram values.
Autonomous system level
Within the upper-right section of the country-level page, a list of the top five autonomous systems within the country is shown. Clicking on an ASN takes you to the Performance page for that autonomous system. For others not displayed in the top five list, you can use the search bar at the top of the page to search by autonomous system name or number. The graphs shown within the AS level view are identical to those shown at a country level, but obviously at a different level of aggregation. You can find the ASN that you are connected to from the My Connection page on Cloudflare Radar.
Exploring connection performance & quality data
Digging into the IQI and Connection Quality visualizations can surface some interesting observations, including characterizing Internet connections, and the impact of Internet disruptions, including shutdowns and network issues. We explore some examples below.
Characterizing Internet connections
Verizon FiOS is a residential fiber-based Internet service available to customers in the United States. Fiber-based Internet services (as opposed to cable-based, DSL, dial-up, or satellite) will generally offer symmetric upload and download speeds, and the FiOS plans page shows this to be the case, offering 300 Mbps (upload & download), 500 Mbps (upload & download), and “1 Gig” (Verizon claims average wired speeds between 750-940 Mbps download / 750-880 Mbps upload) plans. Verizon carries FiOS traffic on AS701 (labeled UUNET due to a historical acquisition), and in looking at the bandwidth histogram for AS701, several things stand out. The first is a rough symmetry in upload and download speeds. (A cable-based Internet service provider, in contrast, would generally show a wide spread of download speeds, but have upload speeds clustered at the lower end of the range.) Another is the peaks around 300 Mbps and 750 Mbps, suggesting that the 300 Mbps and “1 Gig” plans may be more popular than the 500 Mbps plan. It is also clear that there are a significant number of test results with speeds below 300 Mbps. This is due to several factors: one is that Verizon also carries lower speed non-FiOS traffic on AS701, while another is that erratic nature of in-home WiFi often means that the speeds achieved on a test will be lower than the purchased service level.
Traffic shifts drive latency shifts
On May 9, 2023, the government of Pakistan ordered the shutdown of mobile network services in the wake of protests following the arrest of former Prime Minister Imran Khan. Our blog post covering this shutdown looked at the impact from a traffic perspective. Within the post, we noted that autonomous systems associated with fixed broadband networks saw significant increases in traffic when the mobile networks were shut down – that is, some users shifted to using fixed networks (home broadband) when mobile networks were unavailable.
Examining IQI data after the blog post was published, we found that the impact of this traffic shift was also visible in our latency data. As can be seen in the shaded area of the graph below, the shutdown of the mobile networks resulted in the median latency dropping about 25% as usage shifted from higher latency mobile networks to lower latency fixed broadband networks. An increase in latency is visible in the graph when mobile connectivity was restored on May 12.
Bandwidth shifts as a potential early warning sign
On April 4, UK mobile operator Virgin Media suffered several brief outages. In examining the IQI bandwidth graph for AS5089, the ASN used by Virgin Media (formerly branded as NTL), indications of a potential problem are visible several days before the outages occurred, as median bandwidth dropped by about a third, from around 35 Mbps to around 23 Mbps. The outages are visible in the circled area in the graph below. Published reports indicate that the problems lasted into April 5, in line with the lower median bandwidth measured through mid-day.
Submarine cable issues cause slower browsing
On June 5, Philippine Internet provider PLDTTweeted an advisory that noted “One of our submarine cable partners confirms a loss in some of its internet bandwidth capacity, and thus causing slower Internet browsing.” IQI latency and bandwidth graphs for AS9299, a primary ASN used by PLDT, shows clear shifts starting around 06:45 UTC (14:45 local time). Median bandwidth dropped by half, from 17 Mbps to 8 Mbps, while median latency increased by 75% from 37 ms to around 65 ms. 75th percentile latency also saw a significant increase, nearly tripling from 63 ms to 180 ms coincident with the reported submarine cable issue.
Conclusion
Making network performance and quality insights available on Cloudflare Radar supports Cloudflare’s mission to help build a better Internet. However, we’re not done yet – we have more enhancements planned. These include making data available at a more granular geographical level (such as state and possibly city), incorporating AIM scores to help assess Internet quality for specific types of use cases, and embedding the Cloudflare speed test directly on Radar using the open source JavaScript module.
In the meantime, we invite you to use speed.cloudflare.com to test the performance and quality of your Internet connection, share any country or AS-level insights you discover on social media (tag @CloudflareRadar on Twitter or @radar@cloudflare.social on Mastodon), and explore the underlying data through the M-Lab repository or the Radar API.
Visit 1.1.1.1 from any device to get started with our free app that makes your Internet faster and safer.
To learn more about our mission to help build a better Internet, start here. If you’re looking for a new career direction, check out our open positions.
We know that plugins can be used to extend the functionality of WordPress. But what if you can do some smaller things in WordPress without installing them? Say, you dislike the admin bar at the top and wish to eliminate it? Yes, that can be accomplished by means of code snippets for WordPress.
Basically, code snippets for WordPress are used to do certain actions that might otherwise require a dedicated smaller plugin. Such code snippets are placed in one of the WordPress core or theme files (generally the functions.php file of your theme).
As you might have guessed, code snippets for WordPress, while really useful, tend to alter the default functionality. There can be a small margin of error with each snippet. Generally, such issues tend to arise due to incompatible plugins and/or themes and tend to disappear once you eliminate the said theme/plugin or decide not to use the said snippet.
However, to be on the safer side, be very sure to take proper backups of your WordPress website before making any changes by means of snippets. Also, if you encounter any error or performance issues, rollback your site and check for any plugins or incompatible theme issues.
Now, on to the code snippets for WordPress users!
1. Allow Contributors to Upload Images
By default, WordPress does not permit contributor accounts to upload images. You can, of course, promote that particular account to Author or Editor and this will give them the rights to upload and modify images, However, it will also grant them additional rights, such as the ability to publish their own articles (as opposed to submission for review).
This particular code snippet allows contributor accounts to upload images to their articles, without granting them any additional privileges or rights. Paste it in the functions.php file of your theme:
if ( current_user_can('contributor') && !current_user_can('upload_files') )
add_action('admin_init', 'allow_contributor_uploads');
function allow_contributor_uploads() {
$contributor = get_role('contributor');
$contributor->add_cap('upload_files');
}
2. Show Popular Posts Without Plugins
This one is a little trickier. However, if you are not too keen on installing an extra plugin to showcase popular posts (say, you have limited server memory or disk space), follow this snippet.
The search feature of WordPress has been around for a long time. However, if your website does not need it, or you do not want users to “search” through your website for some reason, you can use this code snippet.
Essentially, it is a custom function that simply nullifies the search feature. Not just the search bar in your sidebar or the menu, but the entire concept of native WP search is gone. Why can this be useful? Again, it can help if you are running your website on low spec server and do not have content that needs to be searched (probably you aren’t running a blog).
There are various ways to secure your website. You can install a security plugin, turn on a firewall or opt for a free feature such as Jetpack Protect that blocks brute force attacks on your website.
The following code snippet, once placed in your functions.php file, rejects all malicious URL requests:
global $user_ID; if($user_ID) {
if(!current_user_can('administrator')) {
if (strlen($_SERVER['REQUEST_URI']) > 255 ||
stripos($_SERVER['REQUEST_URI'], "eval(") ||
stripos($_SERVER['REQUEST_URI'], "CONCAT") ||
stripos($_SERVER['REQUEST_URI'], "UNION+SELECT") ||
stripos($_SERVER['REQUEST_URI'], "base64")) {
@header("HTTP/1.1 414 Request-URI Too Long");
@header("Status: 414 Request-URI Too Long");
@header("Connection: Close");
@exit;
}
}
}
5. Paginate Your Site Without Plugins
Good pagination is very useful for allowing users to browse through your website. Rather than “previous” or “next” links. This is where another one of our code snippets for WordPress comes into play – it adds good pagination to your content.
In functions.php:
global $wp_query;
$total = $wp_query->max_num_pages;
// only bother with the rest if we have more than 1 page!
if ( $total > 1 ) {
// get the current page
if ( !$current_page = get_query_var('paged') )
$current_page = 1;
// structure of "format" depends on whether we're using pretty permalinks
$format = empty( get_option('permalink_structure') ) ? '&page=%#%' : 'page/%#%/';
echo paginate_links(array(
'base' => get_pagenum_link(1) . '%_%',
'format' => $format,
'current' => $current_page,
'total' => $total,
'mid_size' => 4,
'type' => 'list'
));
}
6. Disable the Admin Bar
The WordPress Admin Bar provides handy links to several key functions such as the ability to add new posts and pages, etc. However, if you find no use for it and wish to remove it, simply paste the following code snippet to your functions.php file:
// Remove the admin bar from the front end
add_filter( 'show_admin_bar', '__return_false' );
7. Show Post Thumbnails in RSS Feed
If you wish to show post thumbnail images in your blog’s RSS feed, the following code snippet for WordPress can be useful.
Place it in your functions.php file:
// Put post thumbnails into rss feed
function wpfme_feed_post_thumbnail($content) {
global $post;
if(has_post_thumbnail($post->ID)) {
$content = '' . $content;
}
return $content;
}
add_filter('the_excerpt_rss', 'wpfme_feed_post_thumbnail');
add_filter('the_content_feed', 'wpfme_feed_post_thumbnail');
8. Change the Author Permalink Structure
By default, WordPress shows author profiles as yoursite.com/author/name. However, you can change it to anything that you like, such as yoursite.com/writer/name
The following code snippet needs to be pasted in the functions.php file. Then, it changes the author permalink structure to “/profile/name”:
add_action('init', 'cng_author_base');
function cng_author_base() {
global $wp_rewrite;
$author_slug = 'profile'; // change slug name
$wp_rewrite->author_base = $author_slug;
}
9. Automatically Link to Twitter Usernames in Content
This is especially useful if you are running a website that focuses a lot on Twitter (probably a viral content site, etc.) The following code snippet for functions.php converts all @ mentions in your content to their respective Twitter profiles.
For example, an @happy mention in your content will be converted to a link to the Twitter account “twitter.com/happy” (“happy” being the username):
If you are using the PayPal Donate function to accept donations from your website’s visitors, you can use this code snippet to create a shortcode, and thus make donating easier. First, paste the following in your functions.php file:
function donate_shortcode( $atts, $content = null) {
global $post;extract(shortcode_atts(array(
'account' => 'your-paypal-email-address',
'for' => $post->post_title,
'onHover' => '',
), $atts));
if(empty($content)) $content='Make A Donation';
return '<a href="https://www.paypal.com/cgi-bin/webscr?cmd=_xclick&business='.$account.'&item_name=Donation for '.$for.'" title="'.$onHover.'">'.$content.'</a>';
}
add_shortcode('donate', 'donate_shortcode');
As mentioned with each code snippet, you just need to add the said snippet to the required file. Mostly, you would only need to add code snippets to the functions.php file (in some cases, it can differ).
However, what if you are just not comfortable editing your theme’s files? If that is the case, have no fear. The Code Snippets plugin can help you out!
It is a simple plugin that lets you add code snippets to your functions.php without any manual file editing. It treats code snippets as individual plugins of their own – you add the code and hit save … and the rest is handled by the Code Snippets plugin.
Once you activate the plugin, you will find a Snippets menu right under “Plugins.” Head to Snippets » Add New:
Add a name for your snippet, paste the snippet in the code area, and then provide a description for your own reference. Once done, activate the snippet and you’re good to go! Even if you change the theme, the code snippet remains functional.
There’s a powerful tool hiding in your browser: Inspect Element.
Right-click on any webpage, click Inspect, and you’ll see the innards of that site: its source code, the images and CSS that form its design, the fonts and icons it uses, the JavaScript code that powers animations, and more. You can see how long the site takes to load, how much bandwidth it used to download, and the exact color in its text.
Or, you could use it to change anything you want on the page.
Inspect Element is a perfect way to learn what makes the web tick, figure out what’s broken on your sites, mock up what a color and font change would look like, and keep yourself from having to Photoshop out private details in screenshots. Here’s how to use Inspect Element—your browser’s secret superpower—to do all the above and more.
Most web browsers include an Inspect Element tool, while Microsoft’s Edge browser includes a similar set of Developer Tools. This tutorial focuses on Inspect Element tools for Google Chrome, Mozilla Firefox, and Apple Safari, but most of the features work the same in other browsers like Brave.
If you’ve never peeked at a website’s code out of curiosity, you might wonder why you should learn how to use Inspect Element. Below are just a few reasons why different roles can benefit from learning this trick of the trade.
Designer: Want to preview how a site design would look on mobile? Or want to see how a different shade of green would look on a sign-up button? You can do both in seconds with Inspect Element.
Marketer: Curious what keywords competitors use in their site headers, or want to see if your site’s loading too slow for Google’s PageSpeed test? Inspect Element can show both.
Writer: Tired of blurring out your name and email in screenshots? With Inspect Element, you can change any text on a webpage in a second.
Support agent: Need a better way to tell developers what needs to be fixed on a site? Inspect Element lets you make a quick example change to show what you’re talking about.
Web developer: Need to look for broken code, compare layouts, or make live edits to a page? Inspect Element does that, too.
For these and dozens of other use cases, Inspect Element is a handy tool to keep around. For now, let’s see how to use the main Elements tab to tweak a webpage on your own.
How to inspect elements with Google Chrome
There are a few ways to access Google Chrome Inspect Element. Just open a website you want to try editing (to follow along with this tutorial, open the Zapier blog post What is AI?), then open the Inspect Element tool in one of these three ways:
Method 1: Right-click anywhere on the webpage, and at the very bottom of the menu that pops up, click Inspect.
Method 2: Click the hamburger menu (the icon with three stacked dots) on the far-right of your Google Chrome toolbar, click More Tools, then select Developer Tools.
Method 3: Prefer keyboard shortcuts? Press command + option + I on a Mac, or Ctrl + Shift + Con a PC to open Inspect Element without clicking anything.
Once you take your preferred route to opening the Developer Tools pane, by default, it will show the Elements tab—that’s the famed Inspect Element tool we’ve been looking for.
If you want to change the orientation of the Inspect Element pane, click the three vertical dots on the top-right side of the Inspect Element pane near the “X” (which you’d click to close the pane). Now, you’ll see options to move the pane to the bottom, left, or right side of your browser or to open the pane in a completely separate window (undock view).
For this tutorial, let’s dock the pane on the right side of our browser window to give us more space to work. You can make the Developer Tools panel wider or narrower by hovering over the left-side border. Once the ↔ cursor appears, drag the pane left to widen it or right to narrow it.
How to inspect elements with Firefox
To get to Inspect Element on Firefox, like Chrome, you have three options.
Method 1: Right-click anywhere on the page and click Inspect at the bottom of the menu.
Method 2: Click the hamburger menu(three horizontal lines at the top-right corner of the window), select More tools, then click Web Developer Tools.
Method 3: The keyboard shortcut on Firefox is command + option + I for Macs and Control + Shift + C for PCs.
The Element pane in Firefox likes to pop up at the bottom of the window, which doesn’t give you much room to work with. To move that pane to the side and free up more room, click the hamburger menu (three horizontal dots, next to the “X” in the top-right corner) and click Dock to Right (or left, if you prefer).
If you like, you can also move the pane into a separate window in this menu. You can also expand the pane further or narrow it by hovering over the edge until your cursor changes, and then drag it to the left or right.
How to inspect elements with Safari
To launch Inspect Element with Safari, you’ll need to activate the developer capabilities in the advanced settings first. Here’s how.
Click the Safari dropdownin the top navigation bar above the Safari window, and then click Preferences.
Navigate to Advanced,and check the box at the bottom of the window by Show Develop menu in the menu bar. Close the window.
Now, you should be able to right-click anywhere on the page and click Inspect Elementto open the Elements pane.
The pane should appear along the bottom of your window. To move it to a side alignment and give yourself a little more space to look at the code, click the Dock to right of window (or left of window) option on the top-left corner of the pane, next to the “X.”
I prefer right, but you can easily switch this to the other side or detach the pane into its own separate window if you prefer. To make the pane wider or narrower, just hover over the edge until the cursor changes to the dragger, then drag to move the edge.
Tools you can access through Inspect Element (+ tutorials)
Now that we’re in Inspect Element, there’s an array of useful tools at our fingertips that we can use to make any site look exactly how we want. For this tutorial, we’ll focus on the Search, Elements, and Emulation tabs. These aren’t the only useful tools Inspect Element opens up—not by a long shot—but they’re extremely helpful ones that beginners can start putting to use right away.
Note that, for simplicity, I’ll be using Chrome to demonstrate, but the instructions should be essentially the same for all three browsers.
Find anything on a site with Inspect Element Search
Wondering what goes into your favorite sites? Search is your best tool for that, aside from reading a site’s entire source code.
You can open the default Elements view, press Ctrl + F or command + F, and search through the source code. But the full Search tool will also let you search through every file on a page, helping you find text inside CSS and JavaScript files or locate an icon image you need for an article.
To get started, open Zapier’s blog article on “What is AI?” in Chrome, then open Inspect Element, click the hamburger menu, and select Search. The Search tab will appear on the bottom half of the Developer Tools pane.
In the search field, you can type anything—anything—that you want to find on this webpage, and it will appear in this pane. Let’s see how we can use this.
Type meta name into the search field, press Enter, and you’ll immediately see every occurrence of “meta name” in the code on this page. Now, you can see this page’s metadata, the SEO keywords it’s targeting, and whether or not it’s configured to let Google index it for search. That’s an easy way to see what your competitors are targeting—and to make sure you didn’t mess anything up on your site.
Search is an effective tool for designers as well since you can search by color, too. Type #ff4a00 into the search field and press Enter (and make sure to check the box beside Ignore case to see all of the results). You should now see every time the color #ff4a00, Zapier’s shade of orange, appears in this site’s CSS and HTML files. Then, just click the line that reads “color: #ff4a00;” to jump to that line in the site’s HTML and tweak it on your own (something we’ll look at in the next section).
This is a handy way for designers to make sure a site is following their brand’s style guide. With the Search tool, designers can easily check the CSS of a webpage to see if a color is applied to the wrong element, if an incorrect font family is used on a webpage, or if you’re still using your old color somewhere on your site.
The Search tool is also the perfect way to communicate with developers better since you can show them exactly where you’ve found a mistake or exactly what needs changing. Just tell them the line number where the problem exists, and you’ll get your fix that much quicker.
Or you can change the webpage yourself with Elements, the core part of Chrome’s Developer Tools.
Change anything with Elements
Front-end developers use the Inspect Element tool every day to modify the appearance of a webpage and experiment with new ideas—and you can, too. Inspect Element lets you tweak the appearance and content of a webpage by adding temporary edits to the site’s CSS and HTML files.
Once you close or reload the page, your changes will be gone; you’ll only see the changes on your computer and aren’t actually editing the real website itself. That way, you can feel free to experiment and change anything—and then copy and save the very best changes to use later.
Let’s see what we can do with it.
Click the Elements tab in the Developer Tools pane—and if you want more room, tap your Esc key to close the search box you had open before. You should see the HTML for this page—now you know how the sausage gets made.
In the top-left corner of the Developer pane, you’ll see an icon of a mouse on top of a square. Click it, then you can select any element on the page you want to change. So let’s change some things!
Change the text on a webpage
Ever wanted to change text on a site? Perhaps to see how a new tagline would look on your homepage or to take your email address off a Gmail screenshot? Now you can.
Click the icon of a mouse cursor on a square in the top-left corner of the pane.
Click any text on the page (like the copy on our “What is AI?” blog), which will correspond with a blue highlight over the related code.
Double-click the highlighted text in the Developer Tools pane (not the text on the live page) to turn it into an editable text field.
Type anything you want in this text field (“Auri is a genius” should work just fine), and press Enter.
Voila! You’ve just (temporarily) changed the text on the webpage.
Refresh the page, and everything will go back to normal.
Fun? Let’s try another way to change some things on this page by closing out of the Developer pane altogether. You can then highlight any part of the live webpage you want to edit, then right-click it and hit Inspect.
When your Developer Tools pane opens, it should automatically highlight that sentence. Pretty neat, huh? It’s the little things that count.
Now that we’ve selected a sentence to change on our blog, let’s change how it looks.
Change the color and font of elements
On the lower half of the Developer Tools pane, you’ll see a sub-pane with a few additional tabs that allow you to change how this text looks on the page. Let’s get started on the Styles tab.
You may notice that some things are crossed out. This means that these styles are not active for the element we’ve selected, so changing these values will have no effect.
Let’s try changing something.
Look through the code for the “font-size” field and click into it. Let’s change it from 34px to 42px.
Now scroll down to “color” and change it to Zapier’s signature #ff4a00.
This will look a bit cramped, so let’s finish by changing the “line-height” to 44px.
Now check the blog post to see the difference.
Now let’s try something really cool.
Change element states
Want to see how a button or link will look once someone interacts with it? Inspect Element can show that, too, with force element state tools. You can see how the element will look once a visitor hovers over the element (hover state), selects the element (focus state), and/or has clicked that link (visited state).
As with the other examples, you’ll need to click the mouse cursor/box icon. For this example, we’ll select the “Artificial Intelligence (AI)” tag on the “What is AI” article to try a color change.
In the Developer Tools pane, right-click on that code in the Elements tab, hover over Force state, and click the :active: option. Do this one more time, but click the :hover: option this time.
That will change the button’s background to black, which is what happens when you hover over the button on the live site.
Now, change the “background-color” value to #ff4a00.
You should instantly be able to see what the new hover color will look like.
Try experimenting—change the :hover: color, then uncheck :hover: in the right-click menu and drag your mouse over the button to see the new button color.
Change images
You can easily change images on a webpage with Inspect Element, too. Using the same “What is AI?” blog post as an example, let’s replace the orange solid color background on the “Power your automation with AI” button with a dramatic photo of a solar flare from NASA.
First, copy this link to the image: https://c1.staticflickr.com/9/8314/7931831962_7652860bae_b.jpg
Open Inspect Element on the orange background of the “Power your automation with AI” button and look for the “background-color” code in the pane.
Click “background-color” and replace color with image—this should cause an error. Just replace the color code with url and then paste the URL you copied into the parentheses.
This should automatically replace that boring single-color background with a flashy new image.
Note: You can also change a photo to a GIF or a video—all you need is a link to the file, and you can add it in.
Editing text is handy, swapping out images is fun, and changing colors and styles just might help you quickly mock up the changes you want made to your site. But how will that new tagline and button design look on mobile?
That’s where Emulation comes in—it’s where everything we’ve reviewed so far can be applied even further. Let’s see how.
Test a site on any device with Emulation
Everything has to be responsive today. Websites are no longer only viewed on computers—they’re more likely than ever to be viewed on a phone, tablet, TV, or just about any other type of screen. You should always keep that in mind when creating new content and designs.
Emulation is a great tool to approximate how websites will look to users across various devices, browsers, and even locations. Though this does not replace actual testing on a variety of devices and browsers, it’s a great start.
In the Developer Tools pane, you’ll notice a little phone icon in the top-left corner. Click it. This should change the page into a tiny, phone-styled page with a menu at the top to change the size.
Resize the small browser to see how things look if you were browsing on a tablet, phone, or even smaller screen. Or, click the menu at the top to select default device sizes like Surface Duo or iPhone 12 Pro—let’s go ahead and select the latter.
The webpage screen should shrink down to the device’s size, and you can zoom in a bit by clicking the percentage dropdown next to the dimensions.
If you change the device preset to “Responsive,” you can enlarge the view by dragging the right edge of the webpage emulation right. See what happens? Dragging the screen along the grid allows you to see how the webpage will change as the screen size changes. You can even toggle portrait and landscape views by clicking the little rotation icon at the end of the top menu.
Play around with the other devices to see how the webpage and screen resolution changes. All of the other developer tools we’ve gone over so far will also react to the device view.
Emulate mobile device sensors
When you start interacting with a device preview, you may notice that your mouse now appears as a little circle on the webpage. This allows you to interact with the page as if you’re on your mobile device.
If you click while dragging the page down, it doesn’t highlight text like it normally would in your browser—it drags the screen down like you’re on a touchscreen device. Using this view, you can see how large touch zones are on a webpage. This means you can see which buttons, icons, links, or other elements are easily touchable with the finger.
You can even make your browser act like a phone. Press your Esc key to open the Search pane in Inspect Element again, and this time click the hamburger menu on the top-right. Select More toolsand then Sensors to get four new tools: Location, Orientation, Touch, and Emulate Idle Detector state.
Touch lets you choose whether the circle selector that acts more like a finger than a normal mouse cursor is forced or device-specific.
Orientation lets you interact with motion-sensitive websites, such as online games that let you move things by moving your phone.
Location lets you pretend you’re in a different location.
Emulate Idle Detector state allows you to toggle between different idle user conditions.
Let’s try viewing this site from Berlin. Just click the dropdown and select the city—nothing changes, right?
This is because there isn’t content on this page that changes based on your location. If you change the coordinates on a site like Groupon.com that uses your location to show localized content, though, you would get different results. Go to Google.com in a different location, and you’ll perhaps see a new Google logo for a holiday in another country, or at least will get the results in a different language.
Emulation is a great way to put yourself in your user’s shoes and consider what the user may be seeing on your webpage—and it’s a fun way to explore the international web.
Emulate mobile networks
You can also see what it’s like to browse a site on different networks—perhaps to see if your site will load even if your users are on a slower 3G network.
To give it a try, click the hamburger menu in the top-right corner of the pane, hover over More tools, and select Network conditions.
There, you can choose from fast or slow 3G, or offline to see how the page works without internet. Or, click Add… to include your own testing (perhaps add 56Kbps to test dial-up internet). Now, reload the page, and you’ll see just how long it’d take for the site to load on a slow connection—and how the site looks while it’s loading. That’ll show why you should improve your site to load faster on slow connections.
You can also change your user agent—uncheck Use browser default in the User agent field and select Firefox — Mac perhaps to see if the site changes its rendering for other browsers on different devices. That’s also a handy hack to make webpages load even if they claim they only work in a different browser.
This is by no means a complete list of things you can do with Inspect Element. As you start exploring, you’ll see many more features. My advice: click all the buttons.
CDNs help keep Microsoft 365 fast and reliable for end users. Cloud services like Microsoft 365 use CDNs to cache static assets closer to the browsers requesting them to speed up downloads and reduce perceived end user latency. The information in this topic will help you learn about Content Delivery Networks (CDNs) and how they’re used by Microsoft 365.
What exactly is a CDN?
A CDN is a geographically distributed network consisting of proxy and file servers in datacenters connected by high-speed backbone networks. CDNs are used to reduce latency and load times for a specified set of files and objects in a web site or service. A CDN may have many thousands of endpoints for optimal servicing of incoming requests from any location.
CDNs are commonly used to provide faster downloads of generic content for a web site or service such as Javascript files, icons and images, and can also provide private access to user content such as files in SharePoint Online document libraries, streaming media files, and custom code.
CDNs are used by most enterprise cloud services. Cloud services like Microsoft 365 have millions of customers downloading a mix of proprietary content (such as emails) and generic content (such as icons) at one time. It’s more efficient to put images everyone uses, like icons, as close to the user’s computer as possible. It isn’t practical for every cloud service to build CDN datacenters that store this generic content in every metropolitan area, or even in every major Internet hub around the world, so some of these CDNs are shared.
How do CDNs make services work faster?
Downloading common objects like site images and icons over and over again can take up network bandwidth that can be better used for downloading important personal content, like email or documents. Because Microsoft 365 uses an architecture that includes CDNs, the icons, scripts, and other generic content can be downloaded from servers closer to client computers, making the downloads faster. This means faster access to your personal content, which is securely stored in Microsoft 365 datacenters.
CDNs help to improve cloud service performance in several ways:
CDNs shift part of the network and file download burden away from the cloud service, freeing up cloud service resources for serving user content and other services by reducing the need to serve requests for static assets.
CDNs are purpose built to provide low-latency file access by implementing high performance networks and file servers, and by leveraging updated network protocols such as HTTP/2 with highly efficient compression and request multiplexing.
CDN networks use many globally distributed endpoints to make content available as close as possible to users.
The Microsoft 365 CDN
The built-in Microsoft 365 Content Delivery Network (CDN) allows Microsoft 365 administrators to provide better performance for their organization’s SharePoint Online pages by caching static assets closer to the browsers requesting them, which helps to speed up downloads and reduce latency. The Microsoft 365 CDN uses the HTTP/2 protocol for improved compression and download speeds.
Note
The Microsoft 365 CDN is only available to tenants in the Production (worldwide) cloud. Tenants in the US Government, China and Germany clouds do not currently support the Microsoft 365 CDN.
The Microsoft 365 CDN is composed of multiple CDNs that allow you to host static assets in multiple locations, or origins, and serve them from global high-speed networks. Depending on the kind of content you want to host in the Microsoft 365 CDN, you can add public origins, private origins or both.
Content in public origins within the Microsoft 365 CDN is accessible anonymously, and can be accessed by anyone who has URLs to hosted assets. Because access to content in public origins is anonymous, you should only use them to cache non-sensitive generic content such as Javascript files, scripts, icons and images. The Microsoft 365 CDN is used by default for downloading generic resource assets like the Microsoft 365 client applications from a public origin.
Private origins within the Microsoft 365 CDN provide private access to user content such as SharePoint Online document libraries, sites and proprietary images. Access to content in private origins is secured with dynamically generated tokens so it can only be accessed by users with permissions to the original document library or storage location. Private origins in the Microsoft 365 CDN can only be used for SharePoint Online content, and you can only access assets through redirection from your SharePoint Online tenant.
The Microsoft 365 CDN service is included as part of your SharePoint Online subscription.
Although not a part of the Microsoft 365 CDN, you can use these CDNs in your Microsoft 365 tenant for access to SharePoint development libraries, custom code and other purposes that fall outside the scope of the Microsoft 365 CDN.
Azure CDN
Note
Beginning in Q3 2020, SharePoint Online will begin caching videos on the Azure CDN to support improved video playback and reliability. Popular videos will be streamed from the CDN endpoint closest to the user. This data will remain within the Microsoft Purview boundary. This is a free service for all tenants and it does not require any customer action to configure.
You can use the Azure CDN to deploy your own CDN instance for hosting custom web parts, libraries and other resource assets, which allows you to apply access keys to your CDN storage and exert greater control over your CDN configuration. Use of the Azure CDN isn’t free, and requires an Azure subscription.
Microsoft’s Ajax CDN is a read-only CDN that offers many popular development libraries including jQuery (and all of its other libraries), ASP.NET Ajax, Bootstrap, Knockout.js, and others.
To include these scripts in your project, simply replace any references to these publicly available libraries with references to the CDN address instead of including it in your project itself. For example, use the following code to link to jQuery:
For more information about how to use the Microsoft Ajax CDN, see Microsoft Ajax CDN.
How does Microsoft 365 use content from a CDN?
Regardless of what CDN you configure for your Microsoft 365 tenant, the basic data retrieval process is the same.
Your client (a browser or Office client application) requests data from Microsoft 365.
Microsoft 365 either returns the data directly to your client or, if the data is part of a set of content hosted by the CDN, redirects your client to the CDN URL.a. If the data is already cached in a public origin, your client downloads the data directly from the nearest CDN location to your client.b. If the data is already cached in a private origin, the CDN service checks your Microsoft 365 user account’s permissions on the origin. If you have permissions, SharePoint Online dynamically generates a custom URL composed of the path to the asset in the CDN and two access tokens, and returns the custom URL to your client. Your client then downloads the data directly from the nearest CDN location to your client using the custom URL.
If the data isn’t cached at the CDN, the CDN node requests the data from Microsoft 365 and then caches the data for time after your client downloads the data.
The CDN figures out the closest datacenter to the user’s browser and, using redirection, downloads the requested data from there. CDN redirection is quick, and can save users a lot of download time.
How should I set up my network so that CDNs work best with Microsoft 365?
Minimizing latency between clients on your network and CDN endpoints is the key consideration for ensuring optimal performance. You can use the best practices outlined in Managing Microsoft 365 endpoints to ensure that your network configuration permits client browsers to access the CDN directly rather than routing CDN traffic through central proxies to avoid introducing unnecessary latency.
Is there a list of all the CDNs that Microsoft 365 uses?
The CDNs in use by Microsoft 365 are always subject to change and in many cases there are multiple CDN partners configured in the event one is unavailable. The primary CDNs used by Microsoft 365 are:
CDN
Company
Usage
Link
Microsoft 365 CDN
Microsoft Azure
Generic assets in public origins, SharePoint user content in private origins
There are many factors involved in measuring specific differences in performance between data downloaded directly from Microsoft 365 and data downloaded from a specific CDN, such as your location relative to your tenant and to the nearest CDN endpoint, the number of assets on a page that are served by the CDN, and transient changes in network latency and bandwidth. However, a simple A/B test can help to show the difference in download time for a specific file.
The following screenshots illustrate the difference in download speed between the native file location in Microsoft 365 and the same file hosted on the Microsoft Ajax Content Delivery Network. These screenshots are from the Network tab in the Internet Explorer 11 developer tools. These screenshots show the latency on the popular library jQuery. To bring up this screen, in Internet Explorer, press F12 and select the Network tab, which is symbolized with a Wi-Fi icon.
This screenshot shows the library uploaded to the master page gallery on the SharePoint Online site itself. The time it took to upload the library is 1.51 seconds.
The second screenshot shows the same file delivered by Microsoft’s CDN. This time the latency is around 496 milliseconds. This is a large improvement and shows that a whole second is shaved off the total time to download the object.
Is my data safe?
We take great care to protect the data that runs your business. Data stored in the Microsoft 365 CDN is encrypted both in transit and at rest, and access to data in the Microsoft 365 SharePoint CDN is secured by Microsoft 365 user permissions and token authorization. Requests for data in the Microsoft 365 SharePoint CDN must be referred (redirected) from your Microsoft 365 tenant or an authorization token won’t be generated.
To ensure that your data remains secure, we recommend that you never store user content or other sensitive data in a public CDN. Because access to data in a public CDN is anonymous, public CDNs should only be used to host generic content such as web script files, icons, images and other non-sensitive assets.
Note
3rd party CDN providers may have privacy and compliance standards that differ from the commitments outlined by the Microsoft 365 Trust Center. Data cached through the CDN service may not conform to the Microsoft Data Processing Terms (DPT), and may be outside of the Microsoft 365 Trust Center compliance boundaries.
For in-depth information about privacy and data protection for Microsoft 365 CDN providers, visit the following:
Learn more about Microsoft 365 privacy and data protection at the Microsoft Trust Center
Learn more about Azure privacy and data protection at the Azure Trust Center
How can I secure my network with all these 3rd party services?
Using an extensive set of partner services allows Microsoft 365 to scale and meet availability requirements and enhance the user experience when using Microsoft 365. The 3rd party services Microsoft 365 leverages include both certificate revocation lists; such as crl.microsoft.com or sa.symcb.com, and CDNs; such as r3.res.outlook.com. Every CDN FQDN generated by Microsoft 365 is a custom FQDN for Microsoft 365. If you’re sent to a FQDN at the request of Microsoft 365, you can be assured that the CDN provider controls the FQDN and the underlying content at that location.
For customers that want to segregate requests destined for a Microsoft 365 datacenter from requests that are destined for a 3rd party, we’ve written up guidance on Managing Microsoft 365 endpoints.
Is there a list of all the FQDNs that leverage CDNs?
The list of FQDNs and how they leverage CDNs change over time. Refer to our published Microsoft 365 URLs and IP address ranges page to get up to date on the latest FQDNs that leverage CDNs.
Can I use my own CDN and cache content on my local network?
We’re continually looking for new ways to support our customers’ needs and are currently exploring the use of caching proxy solutions and other on-premises CDN solutions.
Although it isn’t a part of the Microsoft 365 CDN, you can also use the Azure CDN for hosting custom web parts, libraries and other resource assets, which allows you to apply access keys to your CDN storage and exert greater control over your CDN configuration. Use of the Azure CDN isn’t free, and requires an Azure subscription. For more information on how to configure an Azure CDN instance, see Quickstart: Integrate an Azure storage account with Azure CDN.
I’m using Azure ExpressRoute for Microsoft 365, does that change things?
Azure ExpressRoute for Microsoft 365 provides a dedicated connection to Microsoft 365 infrastructure that is segregated from the public internet. This means that clients will still need to connect over non-ExpressRoute connections to connect to CDNs and other Microsoft infrastructure that isn’t explicitly included in the list of services supported by ExpressRoute. For more information about how to route specific traffic such as requests destined for CDNs, see Implementing ExpressRoute for Microsoft 365.
Can I use CDNs with SharePoint Server on-premises?
Using CDNs only makes sense in a SharePoint Online context and should be avoided with SharePoint Server. This is because all of the advantages around geographic location don’t hold true if the server is located on-premises or geographically close anyway. Additionally, if there’s a network connection to the servers where it’s hosted, then the site may be used without an Internet connection and therefore can’t retrieve the CDN files. Otherwise, you should use a CDN if there’s one available and stable for the library and files you need for your site.
Your site needs to have a defined structure because, without it, it’ll just be a random collection of pages and blog posts. Your users need this structure to navigate on your site, to click from one page to another. Google also uses the structure of your site to determine what content is important and what is less relevant. This guide tells you everything you need to know about site structure.
Site structure refers to organizing and arranging a website’s pages and content. It defines the information hierarchy within the site and serves as a roadmap for search engine crawlers. A well-structured site facilitates easy navigation, enhances user experience, and helps search engines like Google understand and effectively index the site’s content. This, in turn, can improve the site’s performance by making it easier for users to find and engage with the content. Ultimately, an optimized site structure helps achieve higher rankings, more traffic, and better conversion rates.
Importance for usability
The structure of your website significantly impacts the experience for your visitors (UX). If visitors can’t find the products and information they’re looking for, they’ll not likely become regular visitors or customers. In other words, you should help them navigate your site. A good site structure will help with this.
Navigating should be easy. You need to categorize and link your posts and products so they are easy to find. New visitors should be able to grasp what you’re writing about or selling instantly.
Importance of your site structure for SEO
A solid site structure vastly improves your chances of ranking in search engines. There are three main reasons for this:
a. It helps Google ‘understand’ your site
The way you structure your site will give Google vital clues about where to find the most valuable content on your site. It helps search engines understand what your site is mainly about or what you’re selling. A decent site structure also enables search engines to find and index content quickly. A good structure should, therefore, lead to a higher ranking in Google.
b. It prevents you from competing with yourself
On your site, you might have blog posts that are quite similar. If, for example, you write a lot about SEO, you could have multiple blog posts about site structure, each covering a different aspect. Consequently, Google won’t be able to tell which of these pages is the most important, so you’ll be competing with your content for high rankings. You should let Google know which page you think is most important. You need a good internal linking and taxonomy structure to do this, so all those pages can work for you instead of against you.
c. It deals with changes on your website
The products you sell in your shop will likely evolve. So does the content you’re writing. You probably add new product lines as old stock sells out. Or you write new articles that make old ones redundant. You don’t want Google to show outdated products or deleted blog posts, so you need to deal with these kinds of changes in the structure of your site.
Are you struggling with setting up your site’s structure? Don’t know the best strategy to link from one post to another? Check out our Site structure training, part of the Yoast SEO academy. Access to Yoast SEO academy is included in the price of Yoast SEO Premium. Before you know it, you’ll be able to improve your rankings by creating the best structure for your site!
How to set up the structure of your site
So, how do you construct a solid site structure? First, we’ll look at an ideal site structure and then explain how to achieve this for your site.
What’s an ideal site structure?
Let’s start by looking at an ideal situation: How should you organize your site if you’re starting from scratch? We think a well-organized website looks like a pyramid with several levels:
Homepage
Categories (or sections)
Subcategories (only for larger sites)
Individual pages and posts
The homepage should be at the top. Then, you have some sections or category pages beneath it. You should be able to file your content under one of these categories. You can divide these sections or categories into subcategories if your site is larger. Beneath your categories or subcategories are your pages and posts.
An ideal site structure looks like a pyramid. On top, you’ll find the homepage and, right below, the main sections or categories, possibly followed by subcategories. On the ground, you’ll find all the individual posts and pages.
Your homepage
On top of the pyramid is the homepage. Your homepage should act as a navigation hub for your visitors. This means, amongst others, that you should link to your most important pages from your homepage. By doing this:
Your visitors are more likely to end up on the pages you want them to end up on;
You show Google that these pages are important.
Further down this article, we’ll help you determine which pages are essential to your business.
Beware not to link too many pages from your homepage, which will cause clutter. And a cluttered homepage doesn’t guide your visitors anywhere. If you want to optimize your homepage further, you can do many other things. Read our article on homepage SEO to find out what.
Navigation
In addition to having a well-structured homepage, it’s also important to create a clear navigation path on your site. Your site-wide navigation consists of two main elements: the menu and the breadcrumbs.
The menu
First, let’s take a look at the menu. The website menu is the most common aid for navigation on your website, and you want to make the best possible use of it. Visitors use your menu to find things on your website. It helps them understand the structure of your website. That’s why the main categories on your site should all have a place in the menu on your homepage.
Furthermore, putting everything in just one menu is not always necessary. If you have a big site with lots of categories, this may clutter your website and makes your main menu a poor reflection of the rest of your site. Where it makes sense, creating a second menu is perfectly fine.
For instance, eBay has one menu at the top of the page – also called the top bar menu – and, in addition to that, a main menu. This top bar menu links to important pages that aren’t categories in the shop, like pages that relate to the visitor’s account on the site. The main menu reflects the most important product categories on eBay.
eBay has multiple ways to start navigating from the homepage
Finally, just like on your homepage, you shouldn’t add too many links to your menu. They will become less valuable for your users and search engines if you do.
Adding breadcrumbs to your pages can make your site’s structure even clearer. Breadcrumbs are clickable links, usually at the top of a page or post. Breadcrumbs reflect the structure of your site. They help visitors determine where they are on your site. They improve your site’s user experience and SEO, as you can read in our guide on breadcrumbs.
You can use one of the many breadcrumb plugins for your WordPress site. You can also use our Yoast SEO plugin, as we’ve implemented a breadcrumb functionality in our plugin as well.
Taxonomies
WordPress uses so-called taxonomies to group content; other CMSs have similar systems. The word ‘taxonomy’ is a fancy term for a group of things — website pages, in this case — that have something in common. This is convenient because people looking for more information on the same topic can find similar articles more easily. You can group content in different ways. The default taxonomies in WordPress are categories and tags.
Categories
You should divide your site’s blog posts or products into several categories. If these categories grow too big, you should divide these categories into subcategories to clear things up again. For example, if you have a clothing store and sell shoes, you can divide this category into subcategories: ‘boots’, ‘heels’, and ‘flats’. These subcategories contain products, in this case, shoes, of that specific type.
Adding this hierarchy and categorizing your pages helps your user and Google make sense of every page you write. Add your main categories to your site’s menu when implementing your category structure.
Your site’s structure will also benefit from adding tags. The difference between a category and a tag mostly concerns structure. Categories are hierarchical: you can have subcategories and even sub-subcategories. Tags, however, don’t have that hierarchy. Tags say: “Hey, this article or product has a certain property that might interest a visitor.” Think of it like this: categories are the table of contents of your website, and tags are the index. A tag for the online clothing store mentioned above could be a brand, for instance, Timberlands.
Try not to create too many tags. You’re not structuring anything if you add a new unique tag to every post or article. Ensure each tag is used at least twice, and your tags group articles that genuinely belong together.
Some WordPress themes display tags with each post, but some don’t. Ensure your tags are available to visitors somewhere, preferably at the bottom of your article or in the sidebar. Google isn’t the only one that likes tags: they are useful for visitors wanting to read more about the same topic.
Site structure is all about grouping and linking the content on your site. Until now, we mostly discussed so-called classifying links: links on your homepage, navigation, and taxonomies. On the other hand, contextual links are internal links within the copy on your pages that refer to other pages within your site. For a link to be contextual, the page you link to should be relevant for someone reading the current page. If you look at the previous paragraph, for instance, we link to a post about tagging, so people can learn more about it if they’re interested.
Your most important pages are often very relevant to mention on several pages across your site, so you’ll link to them most often. Just remember that not only the page you’re linking to is relevant, the context of the link is important as well.
Google uses the context of your links to gather information about the page you’re linking to. It always uses the anchor text (or link text) to understand what the page you’re linking to is about. But the anchor text isn’t the only thing Google looks at. Nowadays, it also considers the content around the link to gather extra information. Google is becoming better at recognizing related words and concepts. Adding links from a meaningful context allows Google to value and rank your pages properly. Yoast SEO Premium makes internal linking a breeze by automatically suggesting relevant content from your site to link to.
Contextual linking for blogs
For blogs, you should write extensively on the topics you want to rank for. You should write some main articles — your cornerstone articles — and write various posts about subtopics of that topic. Then link from these related posts to your cornerstone articles and from the cornerstone articles back to related posts. In this way, you’ll ensure that your most important pages have both the most and most relevant links.
The following metaphor might help you understand this principle:
Imagine you’re looking at a map of a state or country. You’ll probably see many small towns and some bigger cities. All towns and cities will be interconnected somehow. You’ll notice that small towns often have roads leading to the big cities. Those cities are your cornerstones, receiving the most links. The small towns are your posts on more specific topics. Some roads (links) lead to these smaller towns, but not as much as the big cities.
Contextual internal linking works differently on an online store with very few to no pages that are exclusively meant to inform. You don’t explore a specific topic on your product pages: you’re selling a product. Therefore, on product pages, you mostly want to keep people on a page and convince them to buy the product. Consequently, contextual linking is far less prominent in this context. You generally shouldn’t add contextual links to your product descriptions because it could lead to people clicking away from the page.
There are just a couple of meaningful ways of adding contextual links to the product pages for your ecommerce SEO:
link from a product bundle page to the individual products
a ‘related items’ or ‘compare with similar items’ section
a ‘customers also bought’ section
a ‘product bundles’ or ‘frequently bought together’ section.
Landing pages are the pages you want your audience to find when they search for specific keywords you’ve optimized for. For instance, we want people who search for ‘free SEO training’ to end up on the page about our free training called ‘SEO for beginners’. You need to approach the content of your most important landing pages differently than your regular pages.
Here, we’ll discuss two types of landing pages: cornerstone pages and product landing pages. They’re both pages you’d like people to land on from the search engines, but they require quite a different approach. But first, we’ll shortly go into search intent because you have to know what your audience is really looking for.
Search intent
When setting up your site structure, you must consider search intent. It’s about what you think people are looking for when they enter a query into a search engine. What do people want to find? And: what do they expect to find?
Consider different possibilities in search intent, as you might want to cater to different types on your site. Are people just looking for an answer to a question or a definition? Are they comparing products before purchase? Or are they intending to buy something right away? This is often reflected in the type of query they make. You can also use Google’s search results to create great content that fits someone’s needs.
When you have an idea of the search intent, ensuring your landing page fits your audience’s search intent is essential. Pages can answer multiple search intents, but you need a clear view of at least your most important pages.
Cornerstone articles are the most important informational articles on your website. Their focus is to provide the best and most complete information on a particular topic; their main goal is not to sell products.
Because of this focus, we usually think of blogs when discussing cornerstone content. Of course, that doesn’t mean it can only be a blog post. All different kinds of websites have cornerstone articles! Rule of thumb: if an article brings everything you know about a broad topic together, it’s a cornerstone content article.
Product landing pages significantly differ from cornerstone articles. The latter are lengthy, whereas product landing pages shouldn’t be that long. Rather than complete articles, they should be focused. These pages only need to show what your visitors need to know to be convinced. They don’t need to hold all the information.
You want to rank with these pages, meaning they need content. Enough content for Google to understand what the page is about and what keyword it should rank for. Where cornerstone articles could be made up of thousands of words, a couple of hundred could be enough for product landing pages. The main focus of the content should be on your products.
Structuring or restructuring your content doesn’t always have high priority in everything you have to do. Especially when you blog a lot or add other content regularly, it might feel like a chore. Although it isn’t always fun, you must do it, or your website might become messy. To prevent that from happening, you need to fix your site structure and keep an eye on it while adding new content. Site structure should be part of your long-term SEO strategy.
Evaluate your menu
When your business goal or website changes, your menu must also change. Planning things visually will pay off when you start thinking about restructuring your site. Make a flowchart.
Start with your new menu one or two levels deep and see if you can fit in more pages you have created over the years. You’ll find that some pages are still valid but don’t seem relevant to your menu anymore. No problem, just be sure to link to them on related pages and in your sitemaps so that Google and your visitors can still find these pages. The flowchart will also show you any gaps in the site structure.
Creating an overview of your categories, subcategories, and products or posts will also help you to rethink your site’s taxonomy. This could be a simple spreadsheet, but you can use more visual tools like LucidChart or MindNode.
Do your product categories and subcategories provide a logical overview of your product range or your posts and pages? Perhaps you’ve noticed somewhere down the line that one category has been far more successful than others, or you wrote many blog posts on one subject and very few on others.
If one category grows much larger than others, your site’s pyramid could be thrown off balance. Think about splitting this category into different categories. But, if some product lines end up much smaller than others, you might want to merge them. Don’t forget to redirect the ones you delete.
If you have built your HTML sitemap manually, update that sitemap after changing your site structure. In the far more likely event you have an XML sitemap, re-submit it to Google Search Console.
You might be able to update and republish some outdated articles to make them relevant again. If an article is outdated, but no one reads it anyway, you might delete it. This could clean up your site nicely.
What you should know, in that case, is that you should never delete a page or article without thinking. If Google cannot find the page, it serves your user a 404 error page. Both the search engine and your visitor will see this error message saying the page doesn’t exist, and that is a bad experience and, thus, bad for your SEO.
Be smart about this! You need to redirect the URL of the page you’re deleting properly so your user (and Google) lands on a different page that is relevant to them. That could even improve your SEO!
Got some old content to clean up on your site? Sort out hidden pages and dead ends in four easy steps with our orphaned content SEO workout, available in Yoast SEO Premium.
Avoid keyword cannibalization
Your website is about a specific topic, which could be quite broad or rather specific. While adding content, you should be aware of keyword cannibalization. If you optimize your articles for keywords that are all too similar, you’ll be devouring your chances of ranking in Google. If you optimize different articles for similar key terms, you’ll be competing with yourself, making both pages rank lower.
You’ll have some work to do if you suffer from keyword cannibalization. In short, you should research the performance of your content and probably merge and redirect some of it. When merging posts, we recommend creating a new draft by cloning one of the original posts with the free Yoast Duplicate Post plugin. This allows you to work on your merged post without making these changes to a live post. Read the guide by Joost to learn more about keyword cannibalization and how to fix it.
Internal linking with Yoast SEO
Feeling a bit overwhelmed by all this advice? Yoast SEO has some handy tools to make internal linking so much easier.
Yoast SEO’s text link counter visualizes your links so you can optimize them. It shows the internal links in a post and the internal links to a post. This tool can enhance your site structure by improving the links between your related posts. Make sure your cornerstones get the most (relevant) links! You can identify your cornerstones by finding them in the column with the pyramid icon.
Quickly see which posts have internal links pointing to them with the text link counter in Yoast SEO
Yoast SEO Premium helps you with your internal linking as well. Our internal linking suggestions tool will show you which articles are related to the one you’re writing, so you can easily link to them: just by dragging the link into your editor!
The internal linking suggestions even include other content types
Moreover, our tool allows you to indicate which articles you consider cornerstone content on your site. Those articles will be shown at the top of the internal linking suggestions. You’ll never forget to link to them again.
As we have seen, there are several reasons why site structure is important. A good site structure helps both your visitors and Google navigate your site. It makes it easier to implement changes and prevents competing with your content. So use the tips and pointers in this guide to check and improve your site structure. That way, you’ll stay on top and keep your website from growing out of control!
Want to improve your site structure but don’t know where to start? Get Yoast SEO Premium and get loads of helpful tools and guidance, including free access to Yoast SEO Academy, our Site structure training, and our SEO workouts!
Marieke was head of strategy and former CEO at Yoast. After the sale of Yoast to Newfold Digital in 2021 she is no longer active at Yoast in 2023. Marieke, together with her husband Joost, actively invests in and advises several startups through their company Emilia Capital.
In SEO, there are on-page factors and off-page factors. On-page SEO factors are aspects of your website that you can optimize for better search rankings. It’s about improving things like your technical set-up, your content, and how user-friendly your site is. In this post, we’ll explain all about on-page SEO and how it differs from off-page SEO, and we’ll talk about some on-page optimizations that can help you to rank better.
On-page and off-page SEO: what’s the difference?
Every SEO strategy is focused on ranking as high as possible in the search engines. To do this, we all try to design and develop a website that Google’s algorithm — and people! — will love. That’s basically what SEO is about. The factors in Google’s algorithm can be divided into two categories that will determine the ranking of your website: on-page factors and off-page factors.
On-page SEO factors all have to do with elements on your website. For instance, things you work on to improve your E-E-A-T also fall in this category. Some of the most important on-page SEO factors are:
Your site set-up and technical features, site speed in particular
The quality of your content and use of keywords
How do you use additional media, such as images and videos
Your site structure and internal linking
Structured data and search appearance
Your URL structure
User experience
Meanwhile, off-page SEO looks at what happens away from your website. Some off-page SEO factors include:
Relevant links from other websites leading to your site
Social media activity
Business and map listings
External marketing activities
Pro tip: Find out more about on-page SEO with our front-end SEO inspector! You can use the front-end inspector tool in Yoast SEO Premium to explore the SEO data, metadata and schema output for pages on your site. It’s a great way to get to grips with your on-page SEO.
Importance of on-page SEO
On-page SEO consists of all the elements of SEO that you have control over. If you own a website, you can control the technical issues and the quality of your content. We believe you should be able to tackle all of these factors as they’re in your own hands. Remember: if you create an excellent website, it will start ranking.
Focusing on on-page SEO will also increase the chance that your off-page SEO strategy will be successful. Link building with a crappy site is very tough. Nobody wants to link to poorly written articles or sites that don’t work correctly.
How to optimize on-page SEO factors
1. Make sure search engines can crawl and index your site
If you’re unfamiliar with crawlability and indexing, here’s a quick explanation of what it is and what it has to do with Google. To show your page in the search results, Google must first know about that page. It has to be indexed by Google, meaning that this page has been stored in their index. And for that to be possible, you must ensure you’re not blocking Google from indexing your post or your whole site. So check if you’re not unintentionally doing that (we still see this happening!), and ensure your site is indexed.
Although this isn’t technically a ranking factor, getting your site into the search results requires it, so we thought it should be included here.
2. Invest time in creating quality content based on solid keyword research
Why do you think people visit your site? Most likely because it contains the information they’re looking for. Therefore it’s essential to write excellent content that corresponds with their needs. Search engines like Google also read your text. Which site ranks highest for a specific search term is primarily based on a website’s content. That’s why your content should be informative, easy to read, and focused on the right keywords that your audience uses.
Aside from creating quality content, you must remove or remedy low-quality pages. So-called thin content can harm your SEO. Take time to find these pages and do something with them occasionally to keep your content in good shape.
A significant on-page ranking factor is site speed. Users don’t want to wait for pages to load, so Google tends to rank fast-loading sites higher. If you’re unsure how fast (or slow) your site is, check out your Core Web Vital scores using the report in Google Search Console. This helpful tool will point out areas where your site speed can be improved so you know what to work on.
If you’re tech-savvy, you can probably handle this on your own. If you’re unsure where to start, our Technical SEO training can help you.
4. Get your site structure and internal linking right
A good site structure helps Google (and users) understand your site and navigate your content. And when it comes to making that site structure, internal linking is the way to do it. Firstly, you must channel many internal links to your most important content. We call those pages cornerstone content. Secondly, you should tidy up pages that aren’t getting many (if any) internal links. Those pages are what we refer to as orphaned content. It would be best if you decided whether to improve those pages and add more internal links pointing to them or remove them altogether.
Yoast SEO Premium has two SEO workouts to help you improve your site structure and internal linking. Using the workouts can help you to make big improvements quickly, so give them a go!
5. Optimize your use of images and videos
You’ll want to include images on your site to make it attractive, and maybe some videos too. Doing that wrong can harm your SEO, but doing it correctly comes with some SEO benefits.
High-quality images are usually large files that can slow your site down, and that’s a problem. Using smaller image files and giving them descriptive names, captions, and alt tags will favor your SEO. Plus, there are additional benefits. For one, you’ll make your site more accessible, so it’s helpful for a wider audience. And for another thing, you’ll have a chance of your images ranking in the Google Image search results. Read more about these topics in our posts about image SEO and alt tags.
Adding videos to your site is a bit more complicated than images. And ranking your videos on Google (or YouTube) comes with its own set of challenges. We’ve got a great series of posts all about video SEO, if you’d like to learn more about optimizing in this area. There’s also a dedicated Yoast Video SEO plugin, if you’re serious about getting your videos ranking.
6. Create a persuasive search appearance
How your site looks in the search results is vital for SEO. While the search results aren’t part of your site, the things you do to optimize your search appearance are. Therefore, we consider SEO titles, meta descriptions, and structured data part of on-page SEO.
Optimize your SEO title and meta description, and then add structured data for results that stand out
Optimizing the text for your search snippets is fairly straightforward. Adding structured data can be trickier. Good to know: Yoast SEO can help with all these tasks. With checks and previews to help you, getting your SEO titles and meta descriptions right couldn’t be easier. And when it comes to structured data, Yoast does all the hard work for you — all you need to do is select the content type and fill in the blanks.
7. Make your URLs SEO-friendly
A well-crafted URL structure helps your on-page SEO — it’s like giving your web pages a good road map. Think of it as a friendly address that guides search engines and invites users to explore your content. Creating SEO-friendly URLs makes it easier for humans and search engines to understand what your page is all about. Opt for concise and descriptive URLs that include relevant keywords, as they provide a clear signpost. A clean and organized URL structure enhances navigation, making it easier for everyone to understand your website. Don’t forget to keep it short and readable.
8. Design an excellent user experience
The last thing we want to mention is user experience. Simply put, users need to understand your website easily. They should be able to find what they want in a heartbeat. They should know where to click and how to navigate through your site. And your site should be fast! A beautifully designed website is nice, but you should make it your top priority to create a user-friendly website first.
If you want to learn more about combining SEO and UX to get more people to your site, we’d advise you to look at our other articles on user experience. Or check out our all-around SEO training course.
To conclude
We’ve talked about the most important on-page SEO factors. First, ensure that your website works correctly and that your technical SEO is up to par. Secondly, create content that is user-centered and focused on the right keywords. Thirdly, work on the usability and speed of your site to help users and search engines around your website.
As these factors are all a part of your site, you can work on them to ensure your on-page SEO is top-notch! That being said, do remember to also work on your off-page SEO. Although you may not have total control over these factors, you can still put some effort into creating that exposure on other sites too!
This post explains everything you need to know about cornerstone content – or evergreen content, as it’s also known. You’ll learn what it is, why it’s important for SEO, how to write this kind of content and how you should link from your posts to your cornerstone articles.
What is cornerstone content?
Cornerstone content is the core of your website. It consists of the best, most important articles on your site; the pages or posts you want to rank highest in the search engines. Cornerstone articles are usually relatively long, informative articles, combining insights from different blog posts and covering everything that’s important about a certain topic.
Their focus is to provide the best and most complete information on a particular topic, rather than to sell products. Still, they should reflect your business or communicate your mission perfectly.
Cornerstone content can be either a blog post or a page. But whichever they are, you should make sure they’re very well written, update them often, and aim to get them to rank for your most competitive keywords.
Why are cornerstone articles so important for SEO?
Cornerstone content plays a significant role in any SEO strategy. It can be hard to rank for search terms that are very popular, but a cornerstone approach can help you tackle those competitive search terms. If you write a lot of pages on similar subjects, you need to tell Google which of them is the most important. If you don’t, you’ll be eating away your own chances to rank well in the search results. Providing the correct internal link structure between your posts tells Google which article is the most important.
Link structure for cornerstones
Cornerstone articles should have a prominent place on your website. Ideally, someone should be able to click straight from your homepage to your cornerstone articles. Also, all your other posts about similar topics should link back to their corresponding cornerstone article, so its importance is clear from your site structure. As your site develops, you will write tons of new blog posts approaching that topic from other angles, each one linking back to your cornerstone article. This internal linking structure will increase the chance of your cornerstone content pages ranking in Google searches.
The following metaphor might help you understand this principle: imagine you’re looking at a map of a state or country. Small towns and big cities will all be interconnected somehow. But the big cities will have many more roads leading towards them than the small towns. Those cities are your cornerstones, receiving the most links. The small towns are your posts on more specific topics. There are some roads (links) leading to them, but not as many as to the big cities.
A more concrete example: at Yoast, we write a lot of different posts about SEO copywriting, each looking at a different aspect of SEO copywriting. The cornerstone article for this topic is the ultimate guide to SEO Copywriting, and whenever we write a new post on SEO copywriting, we add a link to that cornerstone article. In doing so, I’ll make clear to Google that the ultimate guide is the most important article about SEO Copywriting on our site, thereby increasing its chances to rank.
Which articles are my cornerstones?
Choose your cornerstones carefully. Think of the four or five pages you would like someone to read when they first visit your website. These articles should be the cornerstones of your site. Which articles are most important to you? Which are the most complete and authoritative? Do these target the keywords you most want to rank for?
It might be tempting to think of your homepage as a cornerstone article, but that’s not really what your homepage does. Although it does have lots of links leading back to it, content-wise it doesn’t really do what a cornerstone should do. A cornerstone article should target a specific topic, with lots of in-depth content talking about that topic. Your homepage will be much more general than this, so it doesn’t give you the same opportunity to rank for your target keywords. That being said, of course, it is important to spend time optimizing your homepage.
If your website is enormous, you’ll have more cornerstones than if your website is small. You’ll probably write about more than one topic, so be sure to choose a cornerstone article from each category.
Give your cornerstones extra attention
The concept of cornerstone content is so important, that our Yoast SEO for WordPress plugins and Yoast SEO for Shopify app include an option to indicate whether or not a page is cornerstone content. If you mark pages as cornerstone content, Yoast SEO helps you write kick-ass content and build a solid internal linking structure.
In the classic editor, the Yoast meta box has a Cornerstone content toggle. The block editor has one in the sidebar
Marking your cornerstone articles means you can create a list of them in your post overview, so you can easily work on improving them. And, most importantly, the link suggestion tool in Yoast SEO Premium will give priority to the articles that you mark as cornerstone content, so you’ll never forget to link to your best article on a certain topic if you write about something related.
Optimizing your cornerstones with Yoast SEO
Yoast SEO offers a specific cornerstone content analysis that helps you optimize your cornerstone content. Yoast SEO for WordPress also includes a text link counter, and – in Premium – even internal linking suggestions, in which cornerstone articles get priority over other posts.
If you really want to make your cornerstone articles great, you need specific content analysis for cornerstones. Content marked as cornerstone will be judged more strictly than usual on SEO and readability in the content analysis, as you want this article to be longer, have excellent content, keep the reader’s attention and rank high. Read how this analysis helps you optimize your cornerstones in Yoast SEO for WordPress and Yoast SEO for Shopify.
Text link counter
With Yoast SEO for WordPress you can filter your cornerstone articles in the post overview to see how many internal links a post has pointing to it and how many posts it links to. This text link counter is extremely useful because you can see at a glance if your cornerstone content has enough links from other, related posts:
Internal linking suggestions
Yoast SEO Premium has an internal linking feature. We analyze what you write and use the most prominent words in your text to determine which articles are related – and therefore which you should link to. Cornerstone articles are treated differently in our calculation of internal linking suggestions because they are more important and have a higher value. To give these articles more prominence, we place the cornerstone articles at the top of the list of the internal linking suggestions. That makes it much easier for you to link to your critical articles.
5 steps to a killer cornerstone approach
Ideally, you should do extensive keyword research, which will help you to produce really awesome, long, informative and beautifully written cornerstone articles. But what if you don’t have that much time? And what if you’ve already written tons of articles? Follow these five steps to make killer cornerstone content.
Step 1: Think about your keywords
You have to decide on the essential keywords you want to rank for. Your cornerstone articles should be optimized for the ‘head’ or most competitive keywords, so be sure to carry out some keyword research.
Step 2: Choose the best post
Go through the posts that are optimized for keywords surrounding the most important keywords. Which post do you think is the best? That’ll be your cornerstone from now on!
Step 3: Rewrite it
Rewrite your cornerstone article. Make it awesome and SEO-friendly. As cornerstone articles are usually lengthy, pay extra attention to readability. Make sure you use plenty of headings. An index at the beginning of a long cornerstone article is also a great idea. Expand your article and make sure it’s totally up to date. And don’t forget to rewrite and update that article regularly.
Step 4: Optimize your other posts on long tail variants
The other blog posts about similar topics as your cornerstone article should be optimized for long tail variants of the ‘head’ keyword you’re attacking in your cornerstone article.
Step 5: Linking from tail to head
You have to tell Google that your new cornerstone article is the most important article on that topic on your site. Don’t forget to link from all the long tail articles to your cornerstone article!
Quick wins!
In real life, perhaps you do not have the time to develop such an elaborate structure. Still, writing an article about a specific topic often leads to inspiration to write an article about a similar topic. You should then try to optimize this second post for a slightly different focus keyword and link these posts internally.
Every time you write a post, you should think about similar posts you have written and link to these. Do you already have an important article about this, even though it is not entirely awesome yet? If you use Yoast SEO Premium, keep an eye on the internal linking tool to see what comes up.
Also, make sure you set up the right linking structure. This does not take a lot of your time and could really help to rank your most important article. Take a look at all the posts on the topic you have already written about and add links to your most important article from all of your (less awesome) posts about that specific topic. You can use the Yoast text link counter to regularly check if there are enough internal links to your important articles.
Do our SEO workout: the cornerstone content approach
Want to build a fantastic cornerstone content strategy in WordPress? Do our SEO workout: the cornerstone content approach to quickly power up your internal linking. With the workout, you’ll get all the guidance you need to make sure that your most important articles have enough internal links pointing to them. You can find the tools you need right there in the workout — all you need to do is follow the steps. Give it a go!
The first step in the cornerstone content SEO workout in Yoast SEO Premium
Yoast’s plans for cornerstone content
Site structure is important for SEO. Having a solid site structure means both search engines and visitors can easily navigate your site to find what they want. To help you achieve this, we are continually working on many more features in Yoast SEO that’ll improve the structure of your website.
On your site, you’ll probably have a few articles that are most dear to your heart. Articles you desperately want people to read. Articles you want people to find in Google. At Yoast, we call these articles your cornerstone articles. How does the Yoast SEO plugin help you set up a cornerstone content strategy? We’ll tell you all about that in this blog post. Plus, find out how our new SEO workout can make the whole process much easier!
What is a cornerstone content strategy?
Cornerstone content consists of those articles that you’re most proud of, that are most important to you. The posts that make people come back to your site or buy your stuff. The articles that reflect the mission of your company perfectly, and the ones you definitely want to rank highest. In general, cornerstone articles are lengthy, and they tend to be informative.
In a nutshell, a cornerstone content strategy means choosing your best content and channeling the most internal links towards it from other relevant pages on your site.
What does Yoast SEO do with cornerstone content?
There are three aspects to a successful cornerstone content approach:
Cornerstone content should be lengthy, well-written, and well-optimized.
Cornerstone articles should have a prominent place in your site’s structure.
You should keep your cornerstones fresh and up to date.
Without a doubt, the most common question we are asked is: “how do I make my site rank for keyword X?”. What most people don’t realize, is that they’re asking the wrong question. You see, sites don’t rank: individual pages rank. If you want to rank for a particular keyword, you’ll need to determine which specific page you want to rank for that keyword.
Adding a keyword to the title of every page is not helpful; you should use a focus keyphrase only once. What also won’t work is writing 200 articles around variations of a keyphrase without giving these a proper linking structure. You need one page that is the center of the content about that topic – a “hub” page if you will. That’s where cornerstone content comes in. But how do you make sure your cornerstone content articles start ranking in the search engines?
To rank with these articles, you need to make sure they’re the best articles you can write. You also need a kickass internal linking structure. Luckily, Yoast is there to help! In this post, we’ll explain just what cornerstone content is and how to rank with these articles. Are you struggling with implementing cornerstone content? Check out our Internal linking SEO workout: the cornerstone approach!
How to set up a cornerstone content strategy with Yoast SEO
Step 1: Choose which pages will be your cornerstones
Your cornerstone content pages will need to be 100% awesome in every way. You need to think about keyword research, headlines, first-class content and more. This article just covers what to do with those pages once they’re created, so if your pages aren’t ready, go and make some now! Not sure where to begin? Check out our detailed post on what type of articles should be your cornerstone content.
Which keywords to target with cornerstone articles?
Your cornerstone articles should be optimized for your most ‘head’ or most competitive keywords. Of course, you should still be realistic when determining these head keywords. But, your internal linking structure will help your cornerstone pages rank (more on that below), which is why these articles should aim to rank for your most competitive keywords.
Positioning that new cornerstone content on your site
Now let’s talk about where to place that content on your site. Important content deserves a place within your core site structure, not a news item or blog post drifting around somewhere. It should be easily found in a few clicks. This also means you should not create other pages within your site that target the exact same keyword! And you really don’t have to, as there are many ways to use keyword variations for these other pages and use these in your site structure.
Step 2: Mark those pages as cornerstone content using the toggle button
Once you’ve put together a list of the pages that will be your cornerstone content, you need to go to each of those pages and make sure this button is toggled to ‘On’:
Alternatively, you can head straight over to our SEO workout: the cornerstone content approach and take a look at the overview in the first step. If there are any missing that you want to add, just look them up using the search box and click to add them to the list.
Our SEO analysis will help you optimize your blog post for the search engines. For cornerstone content, you have to go the extra mile, so indicating that an article is cornerstone content will make the SEO analysis and the readability analysis a bit more strict. For example, if a post is cornerstone content, we urge you to write at least 900 words, instead of the 300 words for a normal post.
Make sure you use your focus keyphrase enough, mention it in a few headings, and optimize your images. Readability is equally important, though. Our readability analysis helps you to, for instance, use enough headings and to write in short, easy-to-read sentences and paragraphs.
Step 3: Check that all your cornerstone content pages are marked
You’ll want to make sure all of your chosen pages are marked as cornerstone content at this point (otherwise you might need to double back later and redo some steps). It can help you to keep track if you make a note next to each page on your list once you’ve toggled the cornerstone content button to ‘On’.
If you’re a Yoast SEO Premium user you can access our new Internal linking SEO workout and quickly check which pages are marked as ‘cornerstone content’ in the overview provided:
Again, if anything is missing from this overview, you can easily add it using the search box below the list.
Step 4: Make sure your cornerstone content pages have the most incoming internal links
You have to link to your cornerstone articles to make them rank high in the search engines. By linking to your favorite articles the most often, you’ll tell Google that these are the ones that are most important. Think of it as a map: big cities have considerably more roads leading towards them than small towns. Those cities are your cornerstones. They should receive the most links. The small towns are your posts on more specific topics. If you build your site structure like this, you won’t be competing with your own content for a place in the search engines.
The text link counter allows you to see all the internal links you’ve put in a post and all internal links to a post from your other pages. This tool provides you with a clear overview of the distribution of your internal links. Make sure to check (and keep checking) if your cornerstone articles receive enough internal links!
If you’re using the Premium Internal linking SEO workout to set up your cornerstone content strategy, you can see all the incoming links of your posts and pages in one simple overview, instead. If any are lacking enough internal links, you can click to add them to the next step of the workout where you will add more.
Step 5: Add text links (in context) from relevant pages on your website
When you’re adding links to your cornerstones, use the keyword you’re targeting as the anchor text for that link, if possible. But most importantly, link from within the content. Don’t just add some site-wide sidebar/footer links. The reason for this is simple: links from within content are way more valuable than links from sidebars.
In addition to that, you need to make sure that you’re linking to your cornerstones from pages that actually are about related topics. Contextual links are the ones that’ll help you rank. Adding hand-picked, relevant links that are useful for someone visiting your website is the best way of achieving this. Automation will not give you quality results. That means that building a decent linking strategy can be a lot of work, especially if your site is large.
If you use our Premium plugin, you can use our internal linking tool. This tool will make linking suggestions for other posts based on the words you’re using in your post. The posts you’ve marked as cornerstone content articles – as described previously – will always appear on top of our list of suggestions. That way, whenever you’re writing about a specific topic, you’ll find the right cornerstone article to link to.
Using our internal linking tool will remind you to link to your cornerstones whenever you’re writing a new post. As a result, your cornerstones will stay on top of your linking structure. And that’s what they need to get ranking!
Once again, the Internal linking SEO workout can make this step a whole lot easier. We’ve added a tool that shows you your cornerstone articles (with a link you can copy) as well as relevant pages from your site that you can visit to add that link right away!
This means you can manage the whole thing from one page — plus you can immediately see whether you’ve added enough links or not. If you still need more after you’ve added all the relevant suggested links, just refresh the SEO workout and you’ll get more suggestions to add!
Step 6: Monitor and maintain!
Don’t start neglecting your cornerstone content strategy once everything is set up — as you add more pages to your website, you’ll need to keep making sure your cornerstone content is getting enough links. You probably don’t need to check this weekly, but if you leave it for more than a year then your website could change a lot in that time. Keep on top of your links to keep your cornerstone content strategy healthy and effective.
Regularly updating your cornerstone content is important for your cornerstone strategy, too. After all, your cornerstones should be timeless, and therefore, always contain the latest insights. If you have Yoast SEO Premium installed, you’ll have an additional feature to help you keep your cornerstones up to date. The stale cornerstone content filter allows you to see at a glance which of your cornerstones need updating. It works in both your post overview, and your pages overview. Neat, right?
Of course, at Yoast, we practice what we preach, so you’ll find no stale content here 😉
Don’t forget to promote your cornerstone content
If well-written, your cornerstone content should be something to be proud of! Something that others willingly share and thereby also something that will attract links. Don’t be afraid to reach out to other people who have written about related topics: show them what you have created and that it might be worthwhile for their visitors to see. You might even want to offer to write a guest post for them on the topic, linking back to your article.
Cornerstone content strategy made simple with Yoast SEO
The cornerstone content approach is a powerful strategy that channels your internal links toward the content that matters. Your cornerstone articles deserve special attention. They need to be written carefully, to be the most complete and authoritative. They should also be easy to find on your site! Cornerstones need many contextual links pointing towards them to make Google see that they are the most important articles. That’ll make them rank in the search engines. That’ll get them the traffic they’re worthy of!
Our Yoast SEO plugin comes with an array of tools to help you set up and maintain your cornerstone content and your internal links. Upgrade to Yoast SEO Premium and the benefits get even better. Try our Internal linking SEO workout today and give your cornerstone content the treatment it deserves!
At Fastly we often highlight our powerful POPs and modern architecture when asked how we’re different, and better than the competition. Today we’re excited to give you another peek under the hood at the kind of innovation we can achieve on a modern network that is fully software-defined.
This past February, Fastly delivered a new record of 81.9 Tbps of traffic during the Super Bowl, and absolutely no one had to do anything with egress policies to manage that traffic over the course of the event thanks to Autopilot. Autopilot is our new zero-touch egress traffic engineering automation system, and because it was running, no manual interventions were required even for this record-breaking day of service. This means that for the first time ever at Fastly we set a new traffic record for the Fastly network while reducing the number of people who were needed to manage it. (And we notably reduced that number all the way to zero.) It took a lot of people across different Fastly teams, working incredibly hard, to improve the self-managing capabilities of our network, and the result is a network with complete automation that can react quickly and more frequently to failures, congestion, and performance degradation with zero manual intervention.
Autopilot brings many benefits to Fastly, but it is even better for our customers who can now be even more confident in our ability to manage events like network provider failures or DDoS attacks and unexpected traffic spikes — all while maintaining a seamless and unimpacted experience for their end users. Let’s look at how we got here, and just how well Autopilot works. (Oh, but if you’re not a customer yet, get in touch or get started with our free tier. This is the network you want to be on.)
Getting to this result required a lot of effort over several years. Exactly three years ago, we shared how we managed the traffic during the 2020 Super Bowl. At that time, an earlier generation of our traffic engineering automation would route traffic around common capacity bottlenecks while requiring operators to deal with only the most complex cases. That approach served us well for the traffic and network footprint we had three years ago, but it still limited our ability to scale our traffic and network footprint because, while we had reduced human involvement, people were still required to deal reactively with capacity. This translates to hiring and onboarding becoming a bottleneck of its own as we would need to scale the number of network operators at least at the same rate of the expansion of our network. On top of that, while we can prepare and be effective during a planned event like a Super Bowl, human neurophysiology is not always at its peak performance when woken up in the middle of the night to deal with unexpected internet weather events.
Achieving Complete automation with Autopilot and Precision Path
The only way forward was to remove humans from the picture entirely. This single improvement allows us to scale easily while also greatly improving our handling of capacity and performance issues. Manual interventions have a cost. They require a human to reason about the problem at hand and make a decision. This cannot be performed infinite times, so that requires us to preserve energy and only act when the problem is large enough to impact customer performance. It also means that when a human-driven action is taken, it normally moves a larger amount of traffic to avoid having to deal with the same issue again soon, and to minimize the amount of human interventions needed.
A modern CDN gives you huge improvements in caching, SEO, performance, conversions, & more.
With complete automation the cost of making an action is virtually 0, allowing very frequent micro-optimizations whenever small issues occur, or are about to occur. The additional precision and reactivity provided by full automation makes it possible to safely run links at higher utilization and rapidly move traffic around as necessary.
Figure: Egress interface traffic demand over capacity. Multiple interfaces had a demand that exceeded three times the physical capacity available during the Super Bowl, triggering automated traffic engineering overrides, which enabled continued efficient delivery without negative consequences to the network.
The graph above shows an example where Autopilot detected traffic demand exceeding physical link capacity. During the Super Bowl this demand exceeded 3 times the available capacity in some cases. Without Autopilot the peaks in traffic demand would have overwhelmed those links, requiring a lot of human intervention to prevent failure, but then to manage all of the downstream impacts of those interventions in order to get the network operating at top efficiency again. With Autopilot the network deflected traffic onto secondary paths automatically and we were able to deliver the excess demand without any performance degradation.
This post sheds light on the systems we built to scale handling large traffic events without any operator intervention.
Technical problem
Figure – Fastly POP is interconnected to the Internet via multiple peers and transit providers
The Fastly network of Points of Presence (POPs) is distributed across the world. Each POP is “multihomed”, i.e., it is interconnected to the Internet via a number of different networks, which are either peers or transit providers, for capacity and reliability purposes. With multiple routing options available, the challenge is how to select the best available path. We need to ensure that we pick the best performing route (in any given moment), and quickly move traffic away from paths experiencing failures or congestion.
Network providers use a protocol called Border Gateway Protocol (BGP) to exchange information about the reachability of Internet destinations. Fastly consumes BGP updates from its neighbors, and learns which neighbor can be used to deliver traffic to a given destination. However, BGP has several limitations. First, it is not capacity or performance aware: it can only be used to communicate whether an Internet destination can be reached or not, but not whether there is enough capacity to deliver the desired amount of traffic or what the throughput or latency would be for that delivery. Second, BGP is slow at reacting to remote failures: if a failure on a remote path occurs, it typically takes minutes for updates to be propagated, during which time blackholes and loops may occur.
Solving these problems without creating new ones is challenging, especially when operating at the scale of tens of Terabits per second (Tbps) of traffic. In fact, while it is desirable to rapidly route around failures, we need to be careful in those processes as well because rerouting large amounts of traffic erroneously can move traffic away from a well performing path onto a worse performing one and create congestion downstream as a result of our action, resulting in poor user experience. In other words, if decisions are not made carefully, some actions that are taken to reduce congestion will actually increase it instead – sometimes significantly.
Fastly’s solution to the problem is to use two different control systems that operate at different timescales to ensure we rapidly route around failures while keeping traffic on most performing paths.
The first system, which operates at a timescale of tens of milliseconds (to make a few round trips), monitors the performance of each TCP connection between Fastly and end users. If the connection fails to make forward progress for a few round trip times it reroutes that individual connection onto alternate paths until it resumes progress. This is the system underlying our Precision Path product for protecting connections between Fastly and end users, and it makes sure we rapidly react to network failures by surgically rerouting individual flows that are experiencing issues on these smaller timescales.
The second system, internally named Autopilot, operates over a longer timescale. Every minute it estimates the residual capacity of our links and the performance of network paths collected via network measurements. It uses that information to ensure traffic is allocated to links in order to optimize performance and prevent links from becoming congested. This system has a slower reaction time, but makes a more informed decision based on several minutes of high resolution network telemetry data. Autopilot ensures that large amounts of traffic can be moved confidently without downstream negative effects.
These two systems working together, make it possible to rapidly reroute struggling flows onto working paths and periodically adjust our overall routing configuration with enough data to make safe decisions. These systems operate 24/7 but had a particularly prominent role during the Super Bowl where they rerouted respectively 300 Gbps and 9 Tbps of traffic which would have otherwise been delivered over faulty, congested or underperforming paths.
This approach to egress traffic engineering using systems operating at different timescales to balance reactivity, accuracy, and safety of routing decisions is the first of its type in the industry to the best of our knowledge. In the remainder of this blog post, we are going to cover how both systems work but we’ll need to first make a small digression to explain how we route traffic out of our POPs, which is unusual and another approach where we’re also industry leaders.
Figure – Amount of traffic (absolute and percentage of total traffic) delivered by Precision Path and Autopilot respectively during the Super Bowl
Fastly network architecture
Figure – Fastly POP architecture
A typical Fastly POP comprises a layer of servers that are interconnected with all peers and transit providers via a tier of network switches. The typical approach to build an edge cloud POP consists in using network routers, which have a large enough memory to store the entire Internet routing table. In contrast, Fastly started designing a routing architecture that pushed all routes to end hosts in order to build a more cost-effective network, but we quickly realized and embraced the powerful capabilities that this architecture made possible. Endpoints that have visibility into the performance of flows now also have the means to influence their routing. This is one of the key reasons Fastly’s networking capabilities, programmability, flexibility, and ease of use continue to exceed the competition.
Here’s how our routing architecture works: Both switches and servers run routing daemons, which are instances of the BIRD Internet Routing Daemon with some proprietary patches applied to it. The daemons running on switches learn all routes advertised by our transits and peers. However, instead of injecting those routes in the routing table of the switches, they propagate them down to the servers which will then inject them into their routing tables. To make it possible for servers to then route traffic to the desired transit or peer, we use the Multiprotocol Label Switching (MPLS) protocol. We populate each switch with an entry in their MPLS lookup table (Label Forwarding Information Base [LFIB]) per each egress port and we tag all BGP route announcements propagated down to the servers with a community encoding the MPLS label that is used to route that traffic. The servers use this information to populate their routing table and use the appropriate label to route traffic out of the POP. We discuss this more at length in a scientific paper we published at USENIX NSDI ‘21.
Quickly routing around failures with Precision Path
Our approach of pushing all routes to the servers, giving endpoints the ability to reroute based on transport and application-layer metrics, made it possible to build Precision Path. Precision Path works on a timeframe of tens of milliseconds to reroute individual flows in cases of path failures and severe congestion. It’s great at quickly routing away from failures happening right in the moment, but it’s not aware or able to make decisions about proactively selecting the best path. Precision Path is good at steering away from trouble, but not zooming out and getting a better overall picture to select an optimized new route. The technology behind our precision path product is discussed in this blog post and, more extensively in this peer-reviewed scientific paper, but here’s a brief explanation.
Figure – Precision path rerouting decision logic for connections being established (left) and connections already established (right).
This system is a Linux kernel patch that monitors the health status of individual TCP connections. When a connection fails to make forward progress for some Round Trip Time (RTT), indicating a potential path failure, it is rerouted onto a randomly chosen alternate path until it resumes forward progress. Being able to make per-flow rerouting decisions is made possible by our host-based routing architecture where servers select routes of outgoing traffic by applying MPLS labels. End hosts can move traffic rapidly on a per-flow granularity because they have both visibility into the progress of connections, and the means to change network route selection. This system is remarkably effective at rapidly addressing short-lived failures and performance degradation that operators or any other telemetry-driven traffic engineering would be too slow to address. The downside is that this system only reacts to severe performance degradations that are already visible in the data plane and moves traffic onto randomly selected alternate paths, just to select non-failing paths, but they might not be the best and most optimal paths.
Making more informed long-term routing decision with Autopilot
Autopilot complements the limitations of Precision Path because it’s not great at responding as quickly, but it makes more informed decisions based on knowledge of which paths are able to perform better, or are currently less congested. Rather than just moving traffic away from a failed path (like Precision Path), it moves larger amounts of traffic *toward* better parts of a network. Autopilot has not been presented before today, and we are excited to detail it extensively in this post.
Autopilot is a controller that receives network telemetry signals from our network such as packet samples, link capacity, RTT, packet loss measurements, and availability of routes for each given destination. Every minute, the Autopilot controller collects network telemetry, uses it to project per-egress interface traffic demand without override paths, and makes decisions to reroute traffic onto alternate paths if one or more links are about to reach full capacity or if the currently used path for a given destination is underperforming its alternatives.
Figure – Autopilot architecture diagram
Autopilot’s architecture is comprised of three components (shown above):
A route manager, which peers with each switch within a POP and receives all route updates the switch received from its neighbors over a BGP peering session. The route manager provides an API that allows consumers to know what routes are available for a given destination prefix. The route manager also offers the ability to inject route overrides via its API. This is executed by announcing a BGP route update to the switch with a higher local preference value than routes learned from other peers and transit providers. This new route announcement will win the BGP tie-breaking mechanism and be inserted into servers’ routing tables and used to route traffic.
A telemetry collector, which receives sFlow packet samples from all the switches of a POP which allow an estimation of the volume of traffic broken down by destination interface and destination prefix as well as latency and packet loss measurements for all the traffic between Fastly POPs over all available providers from servers.
A controller, which consumes (every minute) the latest telemetry data (traffic volumes and performance) as well as all routes available for the prefixes currently served by the POP, and then computes whether to inject a BGP route override to steer traffic over alternate paths.
Making Precision Path and Autopilot work together
One challenge of having multiple control systems operating on the same inputs and outputs is having them work collaboratively to select the overall best options rather than compete with each other. Trying to select the best option from the limited vantage point of each separate optimization process could actually lead to additional disruption and do more harm than good. To the best of our knowledge, we are the first in the industry using this multi-timescale approach to traffic engineering.
The key challenge here is that once a flow is being rerouted by Precision Path, it no longer responds to BGP routing changes, including those triggered by Autopilot. As a result, Autopilot needs to account for the amount of traffic currently controlled by Precision Path in its decisions. We addressed this problem in two ways: first we tuned Precision Path to minimize the amount of traffic it reroutes, and by making that traffic observable by Autopilot so that it can be factored into Autopilot decisions.
When we first deployed Precision Path, we fine-tuned its configuration to minimize false positives. False positives would result in traffic being rerouted away from an optimal path that is temporarily experiencing a small hiccup, and onto longer paths with worse performance, which could in turn lead to a worse degradation by impacting the performance of affected TCP connections. We reported extensively on our tuning experiments in this paper. However, this is not enough, because even if we make the right decision at the time of rerouting a connection, the originally preferred path may recover a few minutes after the reroute, and this is typically what happens when BGP eventually catches up with the failure and withdraws routes through the failed path. To make sure we reroute connections back onto the preferred path when recovered, Precision Path probes the original path every five minutes after the first reroute, and if the preferred path is functional, it moves the connection back onto it. This mechanism is particularly helpful for long-lived connections, such as video streaming, which would otherwise be stuck on a backup path for their entire lifetime. This also minimizes the amount of traffic that Autopilot cannot control, giving it more room to maneuver.
The problem of making the amount of traffic routed by Precision Path visible to Autopilot is trickier. As we discuss earlier in this post, Autopilot learns the volume of traffic sent over each interface from sFlow packet samples emitted by switches. These samples report, among other things, over what interface the packets were sent to and which MPLS label it carried but do not report any information about how that MPLS label was applied. Our solution was to create a new set of alternate MPLS labels for our egress ports and allocate them for exclusive usage by Precision Path. This way, by looking up an MPLS label in our IP address management database, we can quickly find out if that packet was routed according to BGP path selection or according to Precision Path rerouting. We expose this information to the Autopilot controller which treats Precision Path as “uncontrollable”, i.e., traffic that will not move away from its current path even if the preferred route for its destination prefix is updated.
Making automation safe
Customers trust us with their business to occupy a position as a middleman between their services and their users, and we take that responsibility very seriously. While automating network operations allows for a more seamless experience for our customers, we also want to provide assurances to its reliability. We design all our automation with safety and operability at its core. Our systems fail gracefully when issues occur and are built so that network operators can always step in and override their behaviors using routing policy adjustments. The last aspect is particularly important because it allows operators to use tools and techniques learned in environments without automation and apply them here. Minimizing cognitive overhead by successfully automating more and more of the problem is particularly important to reduce the amount of time needed to solve problems when operating under duress. These are some of the approaches we used to make our automation safe and operable:
Standard operator tooling: both Precision Path and Autopilot can be controlled using standard network operator tools and techniques.
Precision Path can be disabled on individual routes by injecting a specific BGP community on an individual route announcement, which is a very common task that network engineers typically perform for a variety of reasons. Precision Path can also be disabled on an individual TCP session by setting a specific forwarding mark on the socket, which makes it possible to run active measurements without Precision Path kicking in and polluting results.
Autopilot route reselection is based on BGP best path selection, i.e., it will try to reroute traffic onto the second best path according to BGP best path selection. As a result, operators can influence which path Autopilot will fail over to by applying BGP policy changes such as altering MED or local pref values, and this is also a very common technique.
Finally, data about whether connections were routed on paths selected by precision path or autopilot is collected by our network telemetry systems, which allows us to reconstruct what happens
Data quality auditing: We audit the quality of data fed into our automation and have configured our systems to avoid executing any change if input data is inconsistent. In the case of Autopilot, for example, we compare egress flow estimation collected via packet samples against an estimation collected via interface counters, and if they diverge beyond a given threshold it means at least one of the estimations must be wrong, and we do not apply any change. The graph below shows the difference between those two estimations during the Super Bowl on one North American POP.
Figure – Difference between link utilization estimates obtained via interface counters and packet samples. The +/- 5% thresholds represent the acceptable margins of error
What-if analysis and control groups: in addition to monitoring input data we also audit the decisions made by systems and step in to correct them if they misbehave. Precision Path uses treatment and control groups. We randomly select a small percentage of connections to be part of a control group for which Precision Path is disabled and then monitor their performance compared to the others where precision path is enabled. If control connections perform better than treatment connections our engineering team is alerted, and steps in to investigate and remediate. Similarly, in Autopilot, before deploying a configuration change to our algorithm, we run it in “shadow” mode where the new algorithm makes decisions, but they are not applied to the network. The new algorithm will only be deployed if it performs at least as well as the one that is currently running.
Fail-static: when a failure occurs at any component of our systems, rather than failing close or open, they fail static, i.e., leave the network in the last known working configuration and alert our engineering team to investigate the problem.
Conclusions
This blog post is a view into how Fastly automates egress traffic engineering to make sure our customers’ traffic reaches their end users reliably. We continue to innovate and push the boundaries of what is possible while maintaining a focus on performance that is unrivaled. If you are thinking that you want your traffic to be handled by people who are not only experts, but also care this much, now is a great time to get in touch. Or if you’re thinking you want to be a part of innovation like this, check out our open listings here: https://www.fastly.com/about/careers/current-openings.
Open Source Software
The automation built into our network was made possible by open source technology. Open source is a part of Fastly’s heritage — we’re built on it, contribute to it, and open source our own projects whenever we can. What’s more, we’ve committed $50 million in free services to Fast Forward, to give back to the projects that make the internet, and our products, work. To make our large network automation possible, we used:
goBGP – BGP routing daemon library, used to build the Autopilot route collector/injector
BIRD – BGP routing daemon running on our switches and servers.
We did our best to contribute back to the community by submitting to their maintainers improvements and bug fixes that we implemented as part of our work. We are sending our deepest gratitude to the people that created these projects. If you’re an open source maintainer or contributor and would like to explore joining Fast Forward, reach out here.
Lorenzo Saino is a director of engineering at Fastly, where he leads the teams responsible for building the systems that control and optimize Fastly’s network infrastructure. During his tenure at Fastly, he built systems solving problems related to load balancing, distributed health checking, routing resilience, traffic engineering and network telemetry. Before joining Fastly he received a PhD from University College London. His thesis investigated design issues in networked caching systems.
Jeremiah Millay is a Principal Engineer on the Network Systems team at Fastly where he spends most of his time focused on network automation and writing software with the goal of improving network operations at Fastly. Prior to Fastly he spent a number of years as a Network Engineer for various regional internet service providers.
Paolo Alvarado is a Senior Manager of Technical Operations at Fastly. Paolo has over 10 years of experience working with content delivery networks in customer-facing and behind-the-scenes roles. Paolo joined Fastly to help build out the Fastly Tokyo office before moving into network operations. Currently, he manages a team of Network and System Operation engineers to meet the challenges of building and running a large scale network.
VP of Engineering leading Network Systems Organization
Hossein Lotfi is VP of Engineering leading Network Systems Organization at Fastly. Hossein has over 20 years of experience building networks and large-scale systems ranging from startups to hyper-scale cloud infrastructure. He has scaled multiple engineering organizations geared towards rapid, novel innovation development and innovations that are informed and inspired by deep involvement with the operational challenges of global scale systems. At Fastly, Hossein is responsible for building reliable, cost-effective, and low-latency systems to connect Fastly with end-users and customer infrastructures. The Network Systems Organization teams include Kernel, DataPath (XDP), L7 Load Balancing, TLS Termination, DDoS Defence, Network Architecture, Network Modeling and Provisioning Systems, Traffic Engineering, Network Telemetry, DNS, Hardware Engineering, Pre-Production Testing and Fastly’s Edge Delivery platform.
