conciselyverbose , 6 days ago It won't have any relevance at all. Either scraping to transform the information in the page is fair use, and consent isn't necessary, or it is not fair use, and the absence of a robots.txt doesn't constitute consent. There's no middle ground where a robots.txt can mean anything.
It won't have any relevance at all.
Either scraping to transform the information in the page is fair use, and consent isn't necessary, or it is not fair use, and the absence of a robots.txt doesn't constitute consent. There's no middle ground where a robots.txt can mean anything.