Conversation

Personal inconvenience aside, this just isn't a good way to run an archive. They are a single datacenter fire, server compromise, lawsuit or organizational fuckup away from loosing everything.

Making it impossible for anyone to mirror their most useful data is terrible for long term preservation. (I know you can bulk download collections, but I don't want a terabyte of pirated movies: the wayback machine is far more valuable and is not something that exists anywhere else)

Did I mention the rate limit isn't documented anywhere? If you try say, a two-second delay between requests, the server IP bans you for 10 minutes. Actually getting a 429 is quite tricky.