Skip to content

buildhappy/webCrawler

Repository files navigation

crawler

##Intro Crawl the infomations of apps from different channel.MyCrawler exposes the restful interface to the users. Users send the trigger infomations(e.g.taksId,channelId and so on),then the crawler can perform perfectly the task and response the apps infos to the user. ##Fetures

  • based on the webmagic0.5.2
  • uniform rest api
  • concurrent crawling approach
  • simpified models
  • high-efficiency HTML parser JSOUP

About

Dynamic web crawler,just for single machine

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages