https://project.datalad.org/products/extensions/datalad-crawler/index.html This is a DataLad extension that allows you to crawl external web resources into an automated data distribution. It provides functionality for tracking data on a website and make its files available on a local machine, as well as for querying for potential updates to the website and obtaining any changes. Hugo en-us