Restful API – Handling large amounts of data

posted in: API, Data | 0

Summary:

  • You can change your API to include additional parameters to limit the scope of data returned by your application.¬†You could add limit and offset parameters to fetch just a little part.

myapi.co.uk/messages?limit=10&offset=20

  • Another way to decrease the payload would be to only ask for certain parts of your resources’ representation. Asking for only the required columns for reporting on data for example would be a good idea when working with massive data sets

/joe.smith/friends?fields=id,name,picture

 

Thanks to:

http://stackoverflow.com/questions/11725078/restful-api-handling-large-amounts-of-data


After our workshop/meeting with Andre Venter about API’s server-side vs client-side data processing we found as a summary that we can have 3 types of data to display and maintain:

  • List data – Can get raw data from API
  • Detailed data – Can get raw data from the API
  • Summary data – Store processed summary in local storage or in other table and retrieve only this summary/answer to draw the summary (Pie chart for example). Process the data once a week/month/10min to store the summary object