Exploratory Community

How to make an HTTP request with custom HTTP headers to get the data


When you want to get data from the REST API, sometimes you need to set custom HTTP headers such as Authorization header to pass information such as API Key. You can do this on Exploratory by using “R Script Data Source” and “curl” R package. Here is a sample code for Github API to get the list of repositories of a certain user.

In the code, I create a new handle object by new_handle and call handle_setheaders to set the HTTP headers to the handle. Here I add a Cache-Control HTTP header but you can customize it to what you want. Then, I call curl_fetch_memory to get the data with the URL and the handle.

Note that you need to extract the data from the response and construct a data frame by yourself. This example shows how to convert the JSON data into a data frame since Github API returns JSON data.

# Create a new handle.
h <- curl::new_handle()
# Set custom http headers to the handle.
curl::handle_setheaders(h,  "Cache-Control" = "no-cache")
# Get the data from the URL.
data <- curl::curl_fetch_memory("https://api.github.com/users/hadley/repos", handle=h)
# Convert the response data to a data frame.