Some people might want to skip that because they think that they've already been crawled, which is a possibility and you can apply a manual exclusion to any results you think are going to be returned on Google for your search query. So things like YouTube, you don't want to crawl, then you can add them in there and it will ignore them if they returned in the search results. Okay, so that's yeah! That's pretty much the search setting query settings here. So your search terms go here. You can have as many as you like just enter them in their line separated.
1. Some domain names don’t go to auction and are instead held by the registrar in their own portfolios (you called this “warehousing” by the registrar). This is probably less than 0.001% of the domain names….actually, probably even less. I suspect it would only be premium domain names with massive exact match local search quantities. Not surprisingly, I cannot find any information from registrars about their procedure, quantities of domains their keep, or ICANN’s rules (if any) about this.
Okay. So, let's move on to our next tab crawl from search query, something look at the settings. So if you want to limit the amount of results that you cruel per search query, you can tick that and specify, so if you only want to crawl the top three or ten results from each search query, you can enter that there. He'll just go to the end of the search. Next setting, results on Google skip the domain if it's in the majestic million, so the majestic million is just something from majestic, it's a free resource that majestic SEO makes available that shows the most popular hundred domains.
Okay, so let's run it. I'm just going to copy those in there to give it a bit of a kick start, but I'm on modern fiber UK fiber, so that might actually be too slow still, but yeah just trial and error with the amoount of crawlng threads, because it depends what kind of resources your PC has, and what kind of connection you're on. Okay, so we'll come back in a minute, oh by the way also, you can get the DA and the PA for each result for any of the crawl types. As long as you put in your MOZ key in the settings, so I just paused the video because I want to play around with the settings, because I haven't used this on my home fibre and I found on my UK fibre, these things worked pretty well but, like I said it will vary from connection to connection and machines to machine as well. So because I want some results to show you pretty quickly. I swapped it for website list and we're going to crawl some directories because they're always really good for finding tons of expired domains.
The following TLDs are now fully supported with a deleted list and a droplist (pending delete list). .hn, .ht, .rw, .cr, .gl, .gy, .gs, .om, .qa, .ir, .pe, .com.pe, .tc, .ky, .th, .co.th, .ke, .co.ke, .uz, .kz, .pm, .re, .tf, .wf, .yt, .sx, .gd, .gi, .kg, .mu, .im, .ug, .sb, .tl, .st, .sm, .so, .sn, .pf, .nyc, .cloud, link, .ms, .mk, .af, .bi, .pr, .com.pr, .nf, .nc, .dm, .bo, .ki, .iq, .bj, .am, .as, .ax, .fo, .mg