All Versions
3
Latest Version
Avg Release Cycle
114 days
Latest Release
1321 days ago

Changelog History

  • v0.3.0 Changes

    August 17, 2019

    👌 Improved support for in-page Robots configuration (#7)
    ➕ Added fragment support for crawl data (#7)
    ➕ Added support for request timeouts (#7)
    ➕ Added support for max pages to crawl (#7)
    Obey no-follow on anchors (#7)
    🛠 Various other minor tweaks and fixes (#7)

  • v0.2.0 Changes

    March 15, 2019

    Replaced crawl contents stream with a string, allowing easier access to contents.

  • v0.1.0 Changes

    January 01, 2019

    🎉 Initial release with core functionality