#

scraping

Here are 1,936 public repositories matching this topic...

scrapy / scrapy

Star

Scrapy, a fast high-level web crawling & scraping framework for Python.

python crawler framework scraping crawling

Updated Jul 27, 2020
Python

gocolly / colly

Star

Elegant Scraper and Crawler Framework for Golang

go golang crawler scraper framework spider scraping crawling

Updated Jul 25, 2020
Go

psf / requests-html

Star

Pythonic HTML Parsing for Humans™

python html http scraping requests kennethreitz beautifulsoup lxml css-selectors pyquery

Updated Jul 17, 2020
Python

code4craft / webmagic

Star

A scalable web crawler framework for Java.

java crawler framework scraping

Updated Jun 24, 2020
Java

tabulapdf / tabula

Star

Tabula is a tool for liberating data tables trapped inside PDF files

pdf csv excel scraping tables

Updated Jul 26, 2020
CSS

yujiosaka / headless-chrome-crawler

Star

Distributed crawler powered by Headless Chrome

jquery crawler chrome scraper promise scraping crawling chromium headless-chrome puppeteer

Updated Jul 7, 2020
JavaScript

ferret

MontFerret / ferret

Star

Declarative web scraping

go cli golang crawler chrome data-mining scraper library tool dsl scraping crawling query-language scraping-websites cdp

Updated Jul 24, 2020
Go

emadehsan / thal

Star

Getting started with Puppeteer and Chrome Headless for Web Scraping

nodejs mongodb mongoose scraping chrome-headless puppeteer

Updated Jan 28, 2020
JavaScript

NikolaiT / GoogleScraper

Star

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

python search-engine crawler scraping search-engines search-engine-optimization

Updated May 11, 2020
HTML

panther

symfony / panther

Star

A browser testing and web crawling library for PHP and Symfony

php webdriver symfony scraping selenium chromedriver selenium-webdriver e2e-testing

Updated Jul 15, 2020
PHP

oscarotero / Embed

Star

Get info from any web service or page

embeds oembed twitter-cards scraping opengraph

Updated Jul 6, 2020
PHP

transitive-bullshit / awesome-puppeteer

Star

A curated list of awesome puppeteer resources.

automation awesome scraping crawling awesome-list headless-chrome puppeteer

Updated Jul 11, 2020

geziyor / geziyor

Star

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

go crawler scraper spider scraping

Updated Dec 12, 2019
Go

medialab / artoo

Star

artoo.js - the client-side scraping companion.

Updated Jun 6, 2020
JavaScript

holgerd77 / django-dynamic-scraper

Star

Creating Scrapy scrapers via the Django admin interface

python scraper django spider scraping scrapy webscraping

Updated Mar 25, 2020
Python

meetmangukiya / instagram-scraper

Star

Scrape the Instagram frontend. Inspired from twitter-scraper by @kennethreitz.

javascript client instagram scraping python-3-6 instagram-scraper no-authentication requests-html

Updated Jun 29, 2018
Python

istresearch / scrapy-cluster

Star

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

python redis kafka scraping distributed scrapy

Updated Jul 20, 2020
Python

iawia002 / Lulu

Star

[Unmaintained] A simple and clean video/music/image downloader 👾

python crawler scraper downloader video scraping crawling python3

Updated Oct 18, 2019
Python

sananth12 / ImageScraper

Star

✂️ High performance, multi-threaded image scraper

python scraper terminal command-line pypi scraping commandline-tool

Updated Jan 4, 2018
Python

speed / newcrawler

Star

Free Web Scraping Tool with Java

docker crawler spider scraping

Updated May 9, 2020
JavaScript

scrapy / parsel

Star

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

css python xml scraping selectors xpath lxml

Updated Jun 18, 2020
Python

MorvanZhou / easy-scraping-tutorial

Star

Simple but useful Python web scraping tutorial code.

crawler regex scraping crawling requests asyncio scrapy beautifulsoup distributed-scraper urllib

Updated Oct 22, 2019
Jupyter Notebook

Lackoftactics / facebook_data_analyzer

Star

Analyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, contacts, friends added statistics and more

ruby ruby-gem data-science facebook statistics script scraping data-visualization conversation facebook-data english-language facebook-data-analyzer

Updated Jul 28, 2020
Ruby

phantombuster / nickjs

Star

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)

automation browser phantomjs scraping deprecated casperjs headless-chrome

Updated Jun 19, 2020
JavaScript

AlexMathew / scrapple

Star

A framework for creating semi-automatic web content extractors

python crawler tutorial extractor scraping web-scraper selector css-selector web-scraping scrapy scrapers beautifulsoup xpath-expression lxml selector-expression

Updated Oct 12, 2019
Python

online-judge-tools / oj

Star

Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.

automation scraping competitive-programming codeforces atcoder programming-contests

Updated Jul 26, 2020
Python

slotix / dataflowkit

Star

Extract structured data from web sites. Web sites scraping.

go golang scraper headless scraping crawling golang-library extract-data scraping-websites cdp chrome-fetcher

Updated Jun 12, 2020
Go

dufferzafar / geeksforgeeks.pdf

Star

Topic wise PDFs of Geeks for Geeks articles. (Last updated in October 2018)

pdf download scraping geeksforgeeks

Updated Oct 22, 2019
Python

covidatlas / coronadatascraper

Star

COVID-19 Coronavirus data scraped from government and curated data sources.

scraping coronavirus covid-19

Updated Jul 28, 2020
HTML

jekyll

programminghistorian / jekyll

Star

Jekyll-based static site for The Programming Historian

python api data-mining mapping scraping text-analysis digital-humanities data-management data-manipulation exhibits pedagogy network-analysis linked-open-data programming-historian dh

Updated Jul 28, 2020
HTML

Improve this page

Add a description, image, and links to the scraping topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scraping topic, visit your repo's landing page and select "manage topics."

You can’t perform that action at this time.