I'm having fun distracting myself with the robots.txt of major websites thanks to Oggy.
https://ognjen.io/using-robots-txt-to-discover-hidden-content/
Like why does the verge not want to index the iPhone 15 event?
@brozek see the GitHub one... they listed some specific users