Tryhackme-Google Droking

Google Dorking

Task1 Ye Ol’ Search Engine

Roger dodger!

Task2 Let’s Learn About Crawlers

1.Name the key term of what a “Crawler” is used to do

index 索引

2.What is the name of the technique that “Search Engines” use to retrieve this information about websites?

crawling

3.What is an example of the type of contents that could be gathered from a website?

keywords

Task3 Enter: Search Engine Optimisation

Use the same SEO checkup tool and other online alternatives to see how their results compare for https://tryhackme.com and http://googledorking.cmnatic.co.uk

Task4 Beepboop - Robots.txt

1.Where would “robots.txt” be located on the domain “ablog.com”

ablog.com/robots.txt

2.If a website was to have a sitemap, where would that be located?

/sitemap.xml

3.How would we only allow “Bingbot” to index the website?

User-agent:Bingbot

4.How would we prevent a “Crawler” from indexing the directory “/dont-index-me/”?

disallow:/dont-index-me/

5.What is the extension of a Unix/Linux system configuration file that we might want to hide from “Crawlers”?

.conf

Task5 Sitemaps

1.What is the typical file structure of a “Sitemap”?

xml

2.What real life example can “Sitemaps” be compared to?

map

3.Name the keyword for the path taken for content on a website

route

Task6 What is Google Dorking?

1.What would be the format used to query the site bbc.co.uk about flood defences

site:bbc.co.uk flood defences

2.What term would you use to search by file type?

filetype

3.What term can we use to look for login pages?

type?

filetype

3.What term can we use to look for login pages?

intitle:login