Skip to content
#

web-scraping

Here are 2,626 public repositories matching this topic...

Language: All
Filter by language
All 2,626 Python 1,311 Jupyter Notebook 496 JavaScript 250 HTML 133 R 74 Go 46 Java 45 PHP 35 TypeScript 34 Ruby 30
Sort: Best match
Sort options
Best match Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated
apify-js

apify / apify-js

Open

Update main examples to include DOM manipulation

1
mtrunkat
mtrunkat commented Sep 17, 2019

Main examples at Apify SDK webpage, Github repo and CLI templates should demonstrate how to manipulate with DOM and retrieve data from it.

Also add one example of scraping with Apify SDK + jQuery to https://sdk.apify.com/docs/examples/basiccrawler

Feedback from: https://medium.com/better-programming/do-i-need-python-scrapy-to-build-a-web-scraper-7cc7cac2081d

I lost an hour trying to make

Read more
good first issue
Find more good first issues

adbar / trafilatura

Open

Test trafilatura on further web pages and report bugs

1
adbar
adbar commented Jan 9, 2020

I have mostly tested trafilatura on a set of English, German and French web pages I had run into by surfing or during web crawls. There are definitely further web pages and cases in other languages for which the extraction doesn't work so far.

Corresponding bug reports can either be filed as a list in an issue like this one or in the code as XPath expressions in [xpaths.py](https://github.com

Read more
good first issue up for grabs

austinoboyle / scrape-linkedin-selenium

Open

Certifications return empty []

2
anntdiv
anntdiv commented Jun 10, 2021

Hello,
Thanks for new update in personal_info section,
I found out that the attribute 'certifications' return empty list []
Test url: https://www.linkedin.com/in/an-nguyen-9b3248122/
Results:
`{'personal_info': {'name': 'An Nguyen',
'headline': 'Data Scientist/Machine Learning Engineer',
'company': 'PERSOL PROCESS & TECHNOLOGY CO., LTD.',
'school': 'National Chiao Tung University',

Read more
help wanted good first issue
Find more good first issues
web-scraping

je-suis-tm / web-scraping

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

web-scraper web-scraping newsletter reuters bloomberg futures web-scrapers scrapper financial-data news-websites data-scraping news-scraper futures-historical-data data-scraper sraping python-web-scraper financial-times options-data wall-street-journal wallstreetbets
  • Updated Jun 28, 2021
  • Python

Improve this page

Add a description, image, and links to the web-scraping topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the web-scraping topic, visit your repo's landing page and select "manage topics."

Learn more