Read on Twitter
Log File Integration 
is a difficult topic in relation to SEO as it involves quite a lot of work (from multiple teams). But let’s dive into it as for many companies 
, it could help them get additional SEO insights 
. Time for a thread:
is a difficult topic in relation to SEO as it involves quite a lot of work (from multiple teams). But let’s dive into it as for many companies , it could help them get additional SEO insights . Time for a thread:
Every request to a server is saved in a log file 
. Long text documents where every line represents a request. Lines include info on the type of request (POST, GET, etc.), User Agents, and depending on the setup, other info. 
. Long text documents where every line represents a request. Lines include info on the type of request (POST, GET, etc.), User Agents, and depending on the setup, other info.
Those lines can be parsed to understand better what specific User Agents (including search engines) are looking at. For example Googlebot - “Mozilla/5.0 (compatible; Googlebot/2.1; + http://www.google.com/bot.html ”
This provides you with an overview of all requests 
that a search engine has made and could you for example help identify the patterns in which way they crawl a site. Or to discover the most crawled pages on your site.
that a search engine has made and could you for example help identify the patterns in which way they crawl a site. Or to discover the most crawled pages on your site.
All this will provide you with insights 
into crawl behavior. In many cases, this could help after launching improvements to measure impact. For example, internal linking, footer 
structures, etc.
into crawl behavior. In many cases, this could help after launching improvements to measure impact. For example, internal linking, footer structures, etc.
Crawl behavior is often a first signal for starting SEO; you first need a crawler to see your pages, after which you can focus on getting them to index & rank the page. Tracking the way it's being crawled is an important matter for that reason.
