MVN

io.github.jasperroel : SiteCrawler

Maven & Gradle

Jul 30, 2018
22 stars

SiteCrawler · This project provides a simple WebCrawler with retry-capabilities, functionality to distinguish between http/https sites. It biggest feature is that it allows for plugins (or CrawlerActions), which allows you to hook your scripts into the crawling process. It also allow for setting "blocked" URLs. Those URLs or patterns will not be crawled.

<dependency>
    <groupId>io.github.jasperroel</groupId>
    <artifactId>SiteCrawler</artifactId>
    <version>1.0.0</version>
</dependency>
Copy
pom.xml
Table Of Contents

Latest Version

Choose a version of io.github.jasperroel : SiteCrawler to add to Maven or Gradle - Latest Versions:

  • Latest Stable: 1.0.0

All Versions

Choose a version of io.github.jasperroel : SiteCrawler to add to Maven or Gradle - All Versions:

Version Vulnerabilities Updated
1.0.x
SiteCrawler-1.0.0

How to add a dependency to Maven

Add the following io.github.jasperroel : SiteCrawler maven dependency to the pom.xml file with your favorite IDE (IntelliJ / Eclipse / Netbeans):

<dependency>
    <groupId>io.github.jasperroel</groupId>
    <artifactId>SiteCrawler</artifactId>
    <version>1.0.0</version>
</dependency>

How to add a dependency to Gradle

Gradle Groovy DSL: Add the following io.github.jasperroel : SiteCrawler gradle dependency to your build.gradle file:

implementation 'io.github.jasperroel:SiteCrawler:1.0.0'

Gradle Kotlin DSL: Add the following io.github.jasperroel : SiteCrawler gradle kotlin dependency to your build.gradle.kts file:

implementation("io.github.jasperroel:SiteCrawler:1.0.0")

How to add a dependency to SBT Scala

SBT Scala: Add the following io.github.jasperroel : SiteCrawler sbt scala dependency to your build.sbt file:

libraryDependencies += "io.github.jasperroel" % "SiteCrawler" % "1.0.0"

Advertisement

Dependencies from Group

Jul 30, 2018
22 stars

Discover Dependencies

Sep 26, 2018
Feb 27, 2023
1 usages
110 stars
Jan 27, 2020
1 usages
Aug 15, 2023
2 usages
0 stars
Mar 09, 2023
2 usages
Aug 02, 2018
Jan 19, 2023
3 usages
12k stars
mln
Jul 14, 2020
1 usages
Aug 06, 2018
1 usages
3 stars