Follow + Star, Learn New Python Skills Every Day

Source: Internet

If you are looking for the most powerful Python scraping tool, look no further! This one line of code will help you get started immediately.ScrapeasyScrapeasy is a Python library that makes it easy to scrape web pages and extract data from them. It can be used to scrape data from a single page or multiple pages. It can also extract data from PDF and HTML tables.Scrapeasy allows you to scrape websites with just one line of code in Python, making it very user-friendly and handling everything for you. You just need to specify the website to scrape and what kind of data you want to receive, and the rest is taken care of by Scrapeasy.The Scrapeasy Python scraper is designed for quick use. It offers the following main features:

• One-click scraping of websites—not just a single page.

• The most common scraping activities (receiving links, images, or videos) are already implemented.

• Receive special file types from the scraped website, such as .php or .pdf data.

How to Use ScrapeasyDownload via pip

$ pip install scrapeasy

Using It

Scrapeasy is designed with ease of use in mind. First, import Website and Page from Scrapeasy

from scrapeasy import Website, Page

Initialize the Website

First, let’s create a new website object. For this method, just provide the URL of the homepage. I will use the URL of a website I created years ago:

web = Website("https://tikocash.com/solange/index.php/2022/04/13/how-do-you-control-irrational-fear-and-overthinking/")

Get Links to All Subpages

Now that our website is initialized, we are interested in all subpages that exist on tikocash.com. To find this out, let the Web object receive links to all subpages.

links = web.getSubpagesLinks()

Depending on your local internet connection and the server speed of the website you are scraping, this request may take some time. Make sure not to use this very large method to scrape an entire webpage.

But back to link retrieval: by calling .getSubpagesLinks(), you request all subpages as links and will receive a list of URLs.

links2 = web.getSubpagesLinks()

You may have noticed the absence of the typical http://www.-stuff. This is intentional and makes your life easier when further using the links. But make sure—when you actually want to call them in a browser or via requests—to prepend http://www. to each link.

Find Media

Let’s try to find all image links placed on fahrschule-liechti.com on their website.

We do this by calling the .getImages() method.

images = web.getImages()

The response will include links to all available images.

Download Media

Now let’s do something more advanced. We like the images on tikocash.com, so let’s download them all to our local disk. Sounds like a lot of work? Actually, it’s quite simple!

web.download("img", "fahrschule/images")

First, we define to download all image media by the keyword img. Next, we define the output folder where the images should be saved. That’s it! Run the code and see what happens. In seconds, you will have all the images from Tikocash.com.

Get Links

Next, let’s find out which pages tikocash.com links to. To get an overall view, let’s find out which other websites it links to, for this reason, we specify to only get domain links.

domains = web.getLinks(intern=False, extern=False, domain=True)

Thus, we get a list of all links that are linked on tikocash.com.

Okay, but now we want to learn more about these links, how do we do that?

Get Link Domains

Well, more detailed links are simply external links, so we made the same request but this time including externals, but excluding domains.

domains = web.getLinks(intern=False, extern=True, domain=False)

Here, we will get detailed information about all external links.

Initialize the Page

So far, we have seen a lot about the website, but we haven’t discovered what Page does.

Well, as mentioned, the Page is just a site within the website, let’s try a different example by initializing the W3schools page.

w3 = Page("https://www.w3schools.com/html/html5_video.asp")

If you haven’t guessed yet, you will soon understand why I chose this page.

Download Video

Yes, you heard it right. Scrapeasy allows you to download videos from web pages in seconds, let’s see how.

w3.download("video", "w3/videos")

That’s all there is to it. Just specify the output folder w3/videos to download all video media, and you’re good to go. Of course, you can also just receive the links to the videos and then download them, but that would be less cool.

video_links = w3.getVideos()

Download Other File Types (like pdf or images)

Now let’s talk more generally about downloading special file types, such as .pdf, .php, or .ico? Use the generic .get() method to receive links, or use the .download() method with the file type as a parameter.

calendar_links = Page("https://tikocash.com").get("php")

That’s it.

Now let’s download some PDFs.

Page("http://mathcourses.ch/mat182.html").download("pdf", "mathcourses/pdf-files")

In summary, Python is a versatile language that allows you to scrape content from any website in seconds with just one line of code.

This makes it a powerful tool for web scraping and data mining.

Therefore, if you need to extract data from a website, Python is the right tool for you.

Long press or scan the QR code below to get free access to Python public courses and hundreds of gigabytes of learning materials compiled by experts, including but not limited to Python eBooks, tutorials, project orders, source code, etc.

Recommended Reading
Stop using loc/iloc in pandas loops!
Using `print` for Python debugging? You’re OUT!
These 15 habits made my Python performance soar
See how I perform Python object injection exploitation
Click to read the original text for more information.