#

scraper

Here are 5,049 public repositories matching this topic...

huginn / huginn

Star

Create agents that monitor and act on your behalf. Your agents are standing by!

notifications agent rss scraper automation twitter monitoring huginn feed feedgenerator webscraping twitter-streaming

Updated Jul 31, 2021
Ruby

cheeriojs / cheerio

Sponsor Star

Open

.attr(name) should lowercase name property

1

psayre23 commented Oct 21, 2014

When looking up an attribute with .attr(), the name of the attribute should be lowercased before looking up in .attribs object.

Read more

❌ Bug good first issue

iawia002 / annie

Star

👾 Fast and simple video download library and CLI tool written in Go

go golang crawler scraper downloader youtube video download tumblr bilibili qq youku iqiyi

Updated Sep 5, 2021
Go

gocolly / colly

Star

Elegant Scraper and Crawler Framework for Golang

go golang crawler scraper framework spider scraping crawling

Updated Sep 7, 2021
Go

codelucas / newspaper

Star

News, full-text, and article metadata extraction in Python 3. Advanced docs:

python crawler scraper news crawling news-aggregator

Updated Sep 15, 2021
Python

pwxcoo / chinese-xinhua

Star

📙 中华新华字典数据库。包括歇后语，成语，词语，汉字。

json data scraper json-data python3 chinese chinese-nlp chinese-characters chinese-simplified chinese-traditional json-dataset chinese-language

Updated Nov 9, 2020
Python

guyueyingmu / avbook

Star

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

crawler scraper laravel database spider magnet-link guzzlehttp magnet adult javbus javlibrary avmoo adult-video

Updated Aug 11, 2021
PHP

arc298 / instagram-scraper

Star

Scrapes an instagram user's photos and videos

python instagram-client instagram scraper instagram-scraper instagram-user-photos instagram-api

Updated Aug 21, 2021
Python

yujiosaka / headless-chrome-crawler

Star

Distributed crawler powered by Headless Chrome

jquery crawler chrome scraper promise scraping crawling chromium headless-chrome puppeteer

Updated Aug 31, 2021
JavaScript

BruceDone / awesome-crawler

Star

A collection of awesome web crawler,spider in different languages

crawler scraper awesome spider web-crawler web-scraper node-crawler

Updated Aug 26, 2021

ferret

MontFerret / ferret

Star

Open

Update e2e tests

1

ziflex commented Apr 8, 2021

It's been awhile since I updated e2e tests and there are some of them that are filing (most of them are related to examples).

Also, we need to add e2e tests that cover headers and cookies for both drivers.

Read more

help wanted good first issue area/drivers type/maintenance

Open

Documentation: add examples of queries made using `XPATH`

3

Open

Add support of synthetic navigation to HTTP driver

Find more good first issues →

autoscraper

alirezamika / autoscraper

Sponsor Star

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

python crawler machine-learning scraper automation ai scraping artificial-intelligence web-scraping scrape webscraping webautomation

Updated Feb 3, 2021
Python

IonicaBizau / scrape-it

Star

🔮 A Node.js scraper for humans.

scraper node-scraper hacktoberfest

Updated Sep 7, 2021
JavaScript

fent / node-ytdl-core

Star

YouTube video downloader in javascript.

scraper youtube node video-downloader youtube-downloader

Updated Sep 12, 2021
JavaScript

DIGITALCRIMINAL / OnlyFans

Star

Scrape all the media from an OnlyFans account - Updated regularly

scraper archive datascraping onlyfans

Updated Aug 25, 2021
Python

jae-jae / QueryList

Star

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

crawler scraper spider querylist

Updated Aug 8, 2021
PHP

jinfagang / weibo_terminater

Star

Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator

scraper chatbot corpus chinese weibo sina

Updated Oct 25, 2019
Python

realsirjoe / instagram-scraper

Star

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

python bot instagram crawler scraper scrape ig igramscraper

Updated Sep 5, 2021
Python

go-rod / rod

Star

A Devtools driver for web automation and scraping

testing go golang scraper automation web chrome-devtools headless devtools web-scraping cdp chrome-headless rod chrome-devtools-protocol devtools-protocol gorod

Updated Sep 12, 2021
Go

facundoolano / google-play-scraper

Star

Open

scoreText is broken

2

facundoolano commented Dec 3, 2018

Tested in search and list methods, scoreText shows ' Rated 4.3 stars out of five stars ' instead of 4.3

Read more

bug good first issue

Open

Don't mutate options objects

Open

updated does not match the date displayed in browser

3

Find more good first issues →

geziyor / geziyor

Star

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

go crawler scraper spider scraping

Updated Sep 1, 2021
Go

JobFunnel

PaulMcInnis / JobFunnel

Star

Scrape job websites into a single spreadsheet with no duplicates.

python search yaml scraper csv job jobs waterloo monster beautifulsoup automated international tfidf glassdoor indeed beautifulsoup4

Updated Sep 5, 2021
Python

fabienvauchelles / scrapoxy

Star

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

nodejs angularjs crawler scraper cloud proxy scrapy blacklisting

Updated Dec 18, 2020
JavaScript

Emby.Plugins.JavScraper

JavScraper / Emby.Plugins.JavScraper

Star

Emby/Jellyfin 的一个日本电影刮削器插件，可以从某些网站抓取影片信息。

plugin metadata scraper japanese jav synology emby adult javbus jellyfin fc2 jsproxy jav-scraper fanart-poster

Updated Jun 22, 2021
C#

felipecsl / wombat

Sponsor Star

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

ruby crawler scraper dsl

Updated Feb 3, 2021
Ruby

Adyzng / jd-autobuy

Star

Python爬虫，京东自动登录，在线抢购商品

python crawler scraper jingdong

Updated Aug 19, 2019
Python

aliparlakci / bulk-downloader-for-reddit

Star

Downloads and archives content from reddit

python scraper downloader reddit imgur archive gfycat

Updated Sep 14, 2021
Python

holgerd77 / django-dynamic-scraper

Star

Creating Scrapy scrapers via the Django admin interface

python scraper django spider scraping scrapy webscraping

Updated Jun 27, 2021
Python

website-scraper / node-website-scraper

Star

Download website to local directory (including all css, images, js, etc.)

nodejs javascript scraper hacktoberfest website-scraper

Updated Jun 30, 2021
JavaScript

vesche / scanless

Star

online port scan scraper

scraper command-line pentesting port-scanner scanning

Updated Dec 9, 2020
Python

Improve this page

Add a description, image, and links to the scraper topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scraper topic, visit your repo's landing page and select "manage topics."