wish to be discovered. Onion internet sites are “crawled” and added into the listing delivered their “robots.txt” file permits it, and if it is not on their blacklist of web sites with abuse materials. Site operators might also post their very own .onion sites for indexing.