If it contains any of those, then I am going to crawl it. Otherwise, I'm not going to because it's probably just something like an Amazon listing. Okay lets move on to the endless cruel. So this is the endless crawl. Basically, here you put your seed websites, one will do if it's a big website, because there's probably loads of domains on it. If it's a tiny, tiny a website, then you might want to stick a few more in so, as I was saying before, it will crawl on all the pages on these websites and then for each external domain that it finds on that website. It will check whether it's expired if it's not expired, it'll try and crawl it and then it will start the loop again, it'll try and take all the domains from there check that they are expired.
By using GoDaddy auctions section, you can also use keywords to search the domain names that you are interested in. For instance, if you type the keyword something like “fitness”, it will list you out all the domain names that matches with the keyword “Fitness” so you can easily make a list of all the domain names with your desired keyword selection.
.ac .ae .af .ag .am .ar .as .at .au .aw .ax .be .bg .bi .bj .bn .bo .br .bw .by .bz .ca .cc .ch .ci .cl .cn .co .cr .cx .cz .de .dk .dm .do .ee .es .fi .fm .fo .fr .gd .gg .gi .gl .gs .gy .hk .hn .hr .ht .hu .id .ie .il .im .in .io .iq .ir .is .it .je .jp .ke .kg .ki .kr .ky .kz .la .lc .li .lt .lu .lv .ly .ma .md .me .mg .mk .mn .ms .mu .mx .nl .nc .nf .ng .no .nu .nz .om .pe .pf .pl .pm .pr .pt .pw .qa .re .ro .rs .ru .rw .sb .sc .se .sg .sh .si .sk .sm .sn .so .st .su .sx .tc .tf .th .tl .tn .to .tr .tv .tw .ua .ug .uk .us .uy .uz .vc .wf .yt .za
×