Python get data from api json

Hello, readers! In this article, we will be focusing on How to Pull data from an API in Python.

So, let us get started!

Steps to pull data from an API using Python

Let us now focus on the steps that we need to follow in order to pull out the particular data from an API.

You can check out the article on Connecting to an API to know more about the API and the response status codes, etc.

Let us begin!

Example 1: Pulling data from an Open source COVID API

In this example, we would be connecting to an Open source COVID API just to extract and parse the json information in an customized manner.

1. Connect to an API

At first, we need to connect to an API and make a secure connection as shown below–

In this article, we have used the COVID19-India API to fetch the data of the cases from the state-wise list.

import requests
import json
response_API = requests.get['//api.covid19india.org/state_district_wise.json']
#print[response_API.status_code]

As we are pulling the data from an API, we have used the get[] function to get the information from the API.

2. Get the data from API

After making a healthy connection with the API, the next task is to pull the data from the API. Look at the below code!

The requests.get[api_path].text helps us pull the data from the mentioned API.

3. Parse the data into JSON format

Having extracted the data, its now the time to convert and decode the data into proper JSON format as shown below–

The json.loads[] function parses the data into a JSON format.

4. Extract the data and print it

The JSON format contains data into a key-value format which resembles a Python dict. Thus, we can pull out and print the data using the key values as shown–

parse_json['Andaman and Nicobar Islands']['districtData']['South Andaman']['active']

You can find the entire code below!

import requests
import json
response_API = requests.get['//api.covid19india.org/state_district_wise.json']
#print[response_API.status_code]
data = response_API.text
parse_json = json.loads[data]
active_case = parse_json['Andaman and Nicobar Islands']['districtData']['South Andaman']['active']
print["Active cases in South Andaman:", active_case]

Output:

Active cases in South Andaman: 19

Example 2: Pulling data from an Open Source GMAIL API

Now, let us connect and pull data from GMAIL API. This API represents the generic structure and information which we can fetch from the API.

So, let us begin!

Have a look at the below code!

Example:

import requests
import json
response_API = requests.get['//gmail.googleapis.com/$discovery/rest?version=v1']
#print[response_API.status_code]
data = response_API.text
parse_json = json.loads[data]
info = parse_json['description']
print["Info about API:\n", info]
key = parse_json['parameters']['key']['description']
print["\nDescription about the key:\n",key]

Output:

Info about API:
 The Gmail API lets you view and manage Gmail mailbox data like threads, messages, and labels.

Description about the key:
 API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.

Explanation:

  • At first, we have connected to the generic GMAIL API using the get[] function.
  • After forming a healthy connection with the API, we get the data from the API using response_object.text
  • Now, we parse the data into JSON format using json.loads[] function.
  • Finally, we extract the data from the JSON object such as the description of the API an the description of the key.
  • You can cross check the values by visiting the API link mentioned in the example.

Conclusion

By this, we have come to the end of this topic. Feel free to comment below in case you come across any question.

For more such posts related to Python, Stay tuned and till then, Happy Learning!! 🙂

An Introduction to JSON and APIs using Python

Photo by Sean Lim on Unsplash

Introduction

In a different tutorial, we discussed how to web scrape with python. The goal of web scraping was to access data from a website or webpage. Well, sometimes a website can make it easier for a user to have direct access to their data with the use of an API [Application Programming Interface]. This basically means that the company has made a set of dedicated URLs that provide this data in a pure form [meaning without any presentation formatting]. This pure data is often in a JSON [JavaScript Object Notation] format, which we can then parse through and extract what we need using python.

For this tutorial, we will use the free API found at covid19api.com that provides data on the coronavirus. We will find the total number of confirmed cases in each country and then we will create a pandas dataframe that contains that information. So let’s begin!

Inspecting the API

If you go to the documentation page of the API, this is what you’ll see:

This shows us the different URLs in the API, the information they provide, and example requests/responses of those URLs on the right.

We can see that the information we are seeking is in the summary page. We can click on view more on the right so we can see what the response would be from that URL:

This is a JSON object! As you can see, it is very similar to a python dictionary and is made up of key-value pairs. In fact, in order for us to parse through this and extract what we want from it, we will eventually turn it into a python dictionary object. Upon inspection, we can see that it looks like a nested dictionary. The outer dictionary has the keys ‘Global’ [with a value of a dictionary] and ‘Countries’ [with a value of a list that is made up of dictionaries, with each dictionary corresponding to a specific country].

Making an HTTP Request from the API

So let’s open a jupyter notebook and request the information from that URL. We will use the requests library to make an HTTP request from that URL and save the response object’s text under the variable response:

response = requests.get[‘//api.covid19api.com/summary’].text

This shows what the response is to our HTTP request from the API. As you can see, it is a long python string that is in JSON format.

Creating a Python Dictionary

Since the response is in JSON format, we can load this string into python and convert it into a python dictionary. We first need to import the json library, and then we can use the loads method from the json library and pass it our string:

response_info = json.loads[response]

Note how the type of our response_info variable is now a python dictionary!

Now that our response is in the form of a python dictionary, we can use what we know about python dictionaries to parse it and extract the information we need!

Also note: The requests library has a built-in JSON decoder that we could have used instead of the json module that would have converted our JSON object to a python dictionary. However, I used the above method to introduce the json module in this tutorial. Here’s what the code would have looked like if we instead used the JSON decoder within the requests module:

requests.get[‘//api.covid19api.com/summary’].json[]

Parsing the Dictionary

As previously mentioned, we would like to make a pandas dataframe that has two columns: countries, and the number of total confirmed cases for that country. We can do so by looping through the values of the ‘Countries’ key of our outer dictionary:

As you can see, the value of our ‘Countries’ key is just a list of dictionaries, with each dictionary containing key-value pairs corresponding to a specific country. So we need to loop through this list of dictionaries, extracting the values of the ‘Country’ and ‘TotalConfirmed’ keys from each dictionary and then appending them to a new list as follows:

country_list = []for country_info in response_info[‘Countries’]:
country_list.append[[country_info[‘Country’], country_info[‘TotalConfirmed’]]]

This will loop through the list of dictionaries, extracting the values from the ‘Country’ and ‘TotalConfirmed’ keys from each dictionary into a list, and then adding this resulting list to our country_list. We will end up with a list of lists, with each list or element in the outer list containing the country name and the total confirmed cases for that specific country.

Creating a Pandas DataFrame

We will now create a pandas dataframe using this country_list and the pandas DataFrame constructor:

country_df = pd.DataFrame[data=country_list, columns=[‘Country’, ‘Total_Confirmed’]]

Success! We now have a dataframe that contains two columns: Country and Total_Confirmed!

Conclusion

In this tutorial, we had a brief introduction to what APIs and JSON are. We then made an HTTP request to a Coronavirus COVID19 API to get information on the number of total confirmed coronavirus cases in each country. We then converted this JSON response to our request into a python dictionary. We then parsed through this dictionary, extracting the information we were seeking, and then created a pandas dataframe containing this information.

How does Python fetch JSON data from API?

To use this library in python and fetch JSON response we have to import the json and urllib in our code, The json..
Import required modules..
Assign URL..
Get the response of the URL using urlopen[]..
Convert it to a JSON response using json. loads[]..
Display the generated JSON response..

How do I get data from JSON to Python?

Converting JSON file to Python object Apart from JSON, Python's native open[] function will also be required. Instead of the JSON loads method, which reads JSON strings, the method used to read JSON data in files is load[]. The load[] method takes up a file object and returns the JSON data parsed into a Python object.

How do I retrieve data from API?

What Does an API Do?.
An API is how two computers talk to each other. ... .
Finally, you will likely need an API key. ... .
The easiest way to start using an API is by finding an HTTP client online, like REST-Client, Postman, or Paw. ... .
The next best way to pull data from an API is by building a URL from existing documentation..

How do I get an API response in Python?

r = requests.get[url = URL, params = PARAMS] Here we create a response object 'r' which will store the request-response. We use requests. ... .
data = r.json[] Now, in order to retrieve the data from the response object, we need to convert the raw response content into a JSON type data structure..

Chủ Đề