Spidering - Web Crawl

Spidering or crawling is when Google or any other search engine has system follow links from page to page finding new information.  Theoretically any website can be found by following links from one website to another. It is this theory that search engines have used to build information about the internet and the sites it contains.

We have provided a small example of how spider might work to learn about your website.