Skip to main content

Scrape SERP Data using SerpStack API with Python

Updated on: webdamn Python Tutorial, Resources & Reviews, Reviews

Search Engine Results Pages known as SERP is the web pages that’s displayed by Search Engine. When a user enter keywords to search something and search engine make search for that and returned relevant search result as web pages.

Generally we make manual search for keywords to get search results from few web pages. But sometimes we need to search for many keywords and get search results from thousands of web pages and it’s not possible through manual search. We can use SERP APIs to search for many keywords and get result from number of search result pages.

So if you’re looking for implementing functionality to get SERP data, then you’re here at right tutorial. In this tutorial, you will learn how to Integrate SerpStack API with Python to get SERP data from search engine like Google etc.

Also, read:

We will cover this tutorial step by step to integrate SerpStack API using Python with example. So let’s proceed.

Step1: Get API Access Key

You need to create an account to get free API Access key. There are API key access needs to pass with API and search terms to make HTTP GET request get search results data.

https://api.serpstack.com/search?access_key = YOUR_ACCESS_KEY

Step2: Import Python Request Library

As we will get SERP data using SerpStack API with Python, so we need to install Request Python library to make HTTP request. We will run the following command to install Request library.

$ pip install requests

After installation, we will use Request library in our Python file like below.

import requests

Step3: Build Search Query with Params

We will build search query with Access key and search terms to pass with search URLs to make HTTP request.

searchQuery = {
  'access_key': 'YOUR_ACCESS_KEY',
  'query': 'KFC restaurant'
}

Step4: Make HTTP GET Request and Get SERP Data

We will make HTTP GET request through requests.get() method and pass search query to get search data.

searchData= requests.get('https://api.serpstack.com/search', searchQuery)

Step5: Decode SERP JSON Data and Get Result Data

We will decode the result search JSON data and display search results records.

searchResults = searchData.json()
print "Total results: ", searchResults['search_information']['total_results']
for number, result in enumerate(searchResults['organic_results'], start=1):
    print "%s. %s" % (number, result['title'])

Step6: Complete Python Code To Get SERP Data using SerpStack API

import requests

searchQuery = {
  'access_key': 'YOUR_ACCESS_KEY',
  'query': 'KFC restaurant'
}

searchData= requests.get('https://api.serpstack.com/search', searchQuery)

searchResults = searchData.json()

print "Total results: ", searchResults['search_information']['total_results']

for number, result in enumerate(searchResults['organic_results'], start=1):
    print "%s. %s" % (number, result['title'])

Step7: Conclusion

Here in this tutorial you have learned how to integrate SerpStack API using Python with default options. You can go through documentation for more details to use API with more options. You can also integrate the API in other programming languages like PHP, Nodejs, jQuery, Go and Ruby.

You may also like: