codesections is a user on fosstodon.org. You can follow them or interact with them if you have an account anywhere in the fediverse.
If you don't, you can sign up here.
@lord @drwho It sounds like you're describing implementing a trie in the file system.
https://en.wikipedia.org/wiki/Trie
I think I'd end up with…billions…of empty folders though
@lord @codesections I had no idea. That is really, really cool.
@drwho @codesections Yes, when you need to grep big files containing hashes it's an easy way to split files into smaller chunks to parse them really fast.
Each time you can't process you file fast enough, you split them a bit more.
It needs a little bit more storage but it speeds up the processing.
@codesections @drwho Hoo i didn't know this word.
Thanks.
This is employed in some rainbow tables to do fast lookup.