{"id":1800,"date":"2019-12-12T13:46:29","date_gmt":"2019-12-12T13:46:29","guid":{"rendered":"https:\/\/www.seotesteronline.com\/?p=1800"},"modified":"2021-05-19T13:13:54","modified_gmt":"2021-05-19T13:13:54","slug":"crawling-indexing-ranking","status":"publish","type":"post","link":"https:\/\/www.seotesteronline.com\/blog\/seo-basics\/crawling-indexing-ranking\/","title":{"rendered":"Crawling, Indexing and Ranking: how do Search Engines work?"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Search engines have evolved into answer engines. And very sophisticated ones.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Their job is to discover, interpret, and organize content on the internet; to show users results that are relevant to them and their needs.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To be sure that your content appears to as many users as possible, you must understand how to make it visible first.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Visibility is the most critical aspect of SEO; the &#8216;zero moment&#8217;<\/span><i><span style=\"font-weight: 400;\">\u00a0<\/span><\/i><span style=\"font-weight: 400;\">when everything starts. If the search engine does not index your website, it cannot show it among its results.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">But, how do search engines work? They do three things: crawling, indexing, and ranking.<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\"><strong>Crawling<\/strong>:\u00a0<\/span><\/i><span style=\"font-weight: 400;\">is the analysis of the webpages on the internet. The search engine scans the code and the content of every URL it finds.<\/span><\/li>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\"><strong>Indexing<\/strong>:\u00a0<\/span><\/i><span style=\"font-weight: 400;\">is the gathering of contents found during the crawling process. If a page is on this list, it means that the search engine indexed it, and it can appear among the search results for the related queries.<\/span><\/li>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\"><strong>Ranking<\/strong>:\u00a0<\/span><\/i><span style=\"font-weight: 400;\">is the order in which the indexed results appear on the result page (SERP). The list goes from the most relevant content to the least.<\/span><\/li>\n<\/ul>\n<h2><span style=\"font-weight: 400;\">Crawling<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Crawling is the process through which the search engine rummages the content on the world wide web: websites old and new, articles, product sheets, images, links, etc.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Search engines use crawlers (also called bots or spiders) that, through specific algorithms, decide which websites to scan and how often, distributing the so-called\u00a0<\/span><i><span style=\"font-weight: 400;\">crawl budget.<\/span><\/i><\/p>\n<p><span style=\"font-weight: 400;\">The crawler discovers new content by identifying and recording every link it finds on the scanned pages and then puts them in a list of to-be-crawled URLs.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This action is critical for SEO because this is the moment when the search engine discovers the number and the quality of connections of a page, both inbound and outbound.\u00a0<\/span><\/p>\n<p>Do you wanna know more about crawling? Learn how Search Engines&#8217; crawlers work <a href=\"https:\/\/www.seotesteronline.com\/blog\/seo-basics\/crawling\/\">reading our article<\/a>.<\/p>\n<h2><span style=\"font-weight: 400;\">Indexing<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">Once it has extracted the page content, the crawler will put it on the index of the visited pages and organizes all the information. The information is then used to measure the relevance of the page compared to similar ones.\u00a0<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Ranking<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">While the two previous activities are the &#8220;behind-the-scenes&#8221; of the work of search engines, ranking is the most evident one.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">SERP is where an SEO consultant can show its customers the results of her work. Once a user enters a keyword on the search bar, the search engine will search on its index the pages that match the search query.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Then it gives to these pages a score calculated out of 200 different ranking factors.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">If you did a good SEO job, the page, blog post, or product sheet you worked on would get a better positioning than your competitors.<\/span><\/p>\n<h2><span style=\"font-weight: 400;\">Ranking factors<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">A page ranking process includes different aspects.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Putting technicalities aside, an &#8216;answer engine&#8217; like Google continually learns how to provide information in the best way possible, putting into relation different factors:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\">Website typology<\/span><\/i><span style=\"font-weight: 400;\">: a simple ranking, made by the search engine to tell a search query from another;<\/span><\/li>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\">Context:<\/span><\/i><span style=\"font-weight: 400;\">\u00a0related to each search query;<\/span><\/li>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\">Time:<\/span><\/i><span style=\"font-weight: 400;\">\u00a0The importance given to a single factor varies with the query;<\/span><\/li>\n<li style=\"font-weight: 400;\"><i><span style=\"font-weight: 400;\">Layout:\u00a0<\/span><\/i><span style=\"font-weight: 400;\">SERP will show different results depending on the search intent.<\/span><i><span style=\"font-weight: 400;\">\u00a0<\/span><\/i><\/li>\n<\/ul>\n<h2>Website typology<\/h2>\n<p><span style=\"font-weight: 400;\">As soon as the user types a query, the first thing the search engine does is to categorize it so that it can return the best typology for that query.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For example:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Your Money, Your life (YMYL) websites;<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Local businesses&#8217; websites;<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Adults websites;<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">News websites.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Etc.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">It is an-ever changing ranking, but it helps determine to which &#8216;place&#8217; a query belongs.<\/span><\/p>\n<h3>Context<\/h3>\n<p><span style=\"font-weight: 400;\">The search engine considers context, too. It extracts any relevant information form the user who types a query, understanding the social, historical, and environmental factors in play and providing the most useful answer. These factors are, for example, position, time, query type.\u00a0<\/span><\/p>\n<h3>Time<\/h3>\n<p><span style=\"font-weight: 400;\">Another aspect that you must keep into consideration is the relation between the &#8216;weight&#8217; given to a query, the time it has been performed, and the time the content has been indexed.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">This because every search query has a different intent, and the search engine knows it.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For this reason, results for, say, &#8220;First World War&#8221; are more influenced by the source authority, while for &#8220;movies out now&#8221; the search engine prioritizes the content freshness.<\/span><\/p>\n<h3>Results layout<\/h3>\n<p><span style=\"font-weight: 400;\">The choice to show the knowledge graph rather than videos or related results depends on the query.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">If the researched content is mainly video, videos are what Google will show on the SERP.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-2996\" src=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-example.png\" alt=\"Google SERP typing &quot;how to make pizza&quot;\" width=\"900\" height=\"599\" srcset=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-example.png 900w, https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-example-300x200.png 300w, https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-example-768x511.png 768w\" sizes=\"(max-width: 900px) 100vw, 900px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">If instead, the search intent is about a topic with many relevant queries, the &#8220;people also search for&#8221; box appears.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-2997\" src=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-people-also-ask-box.png\" alt=\"People also ask box\" width=\"869\" height=\"629\" srcset=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-people-also-ask-box.png 869w, https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-people-also-ask-box-300x217.png 300w, https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/ranking-google-serp-people-also-ask-box-768x556.png 768w\" sizes=\"(max-width: 869px) 100vw, 869px\" \/><\/p>\n<p><span style=\"font-weight: 400;\">And this also goes for other SERP elements, such as topics and related searches.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-2998\" src=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/google-searches-related-dog-food.png\" alt=\"Google searches related to dog food\" width=\"633\" height=\"383\" srcset=\"https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/google-searches-related-dog-food.png 633w, https:\/\/www.seotesteronline.com\/wp-content\/uploads\/2019\/12\/google-searches-related-dog-food-300x182.png 300w\" sizes=\"(max-width: 633px) 100vw, 633px\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What factors can affect ranking? What does it mean to index a site? Read the guide and find out!<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[131],"tags":[],"acf":[],"lang":"en","translations":{"en":1800,"it":1778},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/posts\/1800"}],"collection":[{"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/comments?post=1800"}],"version-history":[{"count":18,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/posts\/1800\/revisions"}],"predecessor-version":[{"id":4557,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/posts\/1800\/revisions\/4557"}],"wp:attachment":[{"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/media?parent=1800"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/categories?post=1800"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.seotesteronline.com\/wp-json\/wp\/v2\/tags?post=1800"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}