A purported leak of 2,500 pages of internal documentation from Google sheds light on how Search, the most powerful arbiter of the internet, operates....
Surprisingly, it's very doable, requires basic technical knowledge and relatively minimal computing resources (runs in the background on your computer).
I have tampermonkey script that sends yacy to crawl any websites that I visit, and it's keeping up relatively good index for personal use of the visited websites. Combine yacy with ~300gb of Kiwix databases, add searxng as a frontend and you have pretty strong self hosted search engine.
Of course you need to supplement your searches from other search engines, as yacy does not crawl the whole web, just what you tell it to.
I encourage anyone who's even slightly interested on this stuff to try Yacy, it's ancient piece of software, but it still works very well and is not an abandoned project yet!
--
I personally use Yacy mostly on private mode, but it does have the distributed network there as well. Yacy current freeworld status
Google won’t comment on a potentially massive leak of its search algorithm documentation ( www.theverge.com )
A purported leak of 2,500 pages of internal documentation from Google sheds light on how Search, the most powerful arbiter of the internet, operates....