Skip to main content

Syncronization using Polling

In this use case, we suppose that you have a system that you want to keep in sync with the Fatture in Cloud API, in particular retrieving the list of the products from our API and storing them in some way. In this example, we'll adopt a polling approach to retrieve the current state of the data stored in Fatture in Cloud.

🏈  Let's start!

To keep the code as simple as possible, we decided to write the data to a JSON lines file, that could be then sent as input for an external system; of course, it is possible to use the retrieved elements as needed: for example, to update a Database status or to perform API calls towards a third-party system.

Schedule the execution!

To keep two systems in sync using polling, it is necessary to reiterate the extraction of the data from our APIs, scheduling the code execution.

In this example, we deliberately omitted to add the code to repeat the execution of our code. For example, you could decide to add a Cron Library to your application or to execute the script using a Job Scheduler on your operating system. This is up to you.

Or execute it once...

Even if we created this code to keep two systems synchronized periodically, it can also be used for a one-off extraction of the data from the Fatture in Cloud API. Just execute it as it is...

🧱  Libraries

In our examples, we'll use the Fatture in Cloud SDKs; of course, you can just replace our SDKs with a simple HTTP Client if you prefer: check the dedicated page for further info. Additionally, for some languages, we added some other libraries to perform common tasks such as writing to the file system or performing the exponential backoff.

1️⃣  Initialize our SDK

In the first part of this example, we import our SDK and initialize it, using the Access Token requested to perform the API call. For simplicity, we didn't implement the code to retrieve the access token in these examples, but you can find more information in the Quickstarts or in the Authentication Implementation pages.

Since we're trying to collect the list of products for a certain company, the products:r scope will be required; if you need to use another API method please select the appropriate scopes.

Once the token is provided to our SDK, it is possible to start using it to interrogate our APIs.

2️⃣  Implement the Exponential Backoff

The polling strategy implies the exigency to perform a potentially huge amount of requests in a really short period. As explained here, our APIs are protected by a set of limits that could result in an error if too many requests are performed in a certain interval.

This is why in our examples we wrapped the API call with an Exponential Backoff method: it manages retries and time distance between two consecutive attempts, to avoid your script failing for a temporary quota-related issue.

3️⃣  The API Method

The API method that we're using is the List Products method, which returns the list of all the possible products for a certain company. The List methods provide Sorting and Customizing functionalities, but in this example we'll not use them.

Each response of the List methods will be Paginated: to avoid returning enormous amounts of data with a single call, the list of products will be split into different pages, that can be retrieved consecutively. Each response contains a set of dedicated pagination parameters, that are meant to make the pages navigation task easier for you.

In our code, we'll perform the first List request on the first page provided by our API, with a page size of five elements. Then we'll use the pagination parameter last_page to retrieve the subsequent pages, performing the needed number of API calls to export all the needed elements from the APIs.

Be aware of the index! :dog2:

Please, notice that the pages on our APIs use one-based numbering: the first page has index 1.

4️⃣  Manage the response

Each response contains a data parameter, that is a JSON array representing one page (e.g. one subset of the list of products). Our SDKs parse the JSON array, so you can just use the elements contained in the array to perform the requested operations.

In this example, we're just trying to populate a JSON Lines file, so for each product retrieved we obtain the related JSON representation and append it to the text file.

💻  Code Examples

Here you can find the code described above.

// The following dependencies are required
// dotnet add package It.FattureInCloud.Sdk
// dotnet add package Polly
// dotnet add package Polly.Contrib.WaitAndRetry

using System;
using System.IO;
using System.Linq;
using Newtonsoft.Json;
using System.Collections.Generic;
using Polly;
using It.FattureInCloud.Sdk.Api;
using It.FattureInCloud.Sdk.Model;
using It.FattureInCloud.Sdk.Client;
using Polly.Contrib.WaitAndRetry;

namespace poll
{
class Program
{
public static ProductsApi apiInstance;

static void Main(string[] args)
{
// This code should be executed periodically using a cron library or job scheduler.
// For example: https://www.quartz-scheduler.net/
SyncProducts();
}

private static void SyncProducts()
{
// Here we init the Fatture in Cloud SDK
// The Access Token is retrieved using the "GetToken" method
Configuration config = new Configuration();
config.AccessToken = GetToken();

// In this example we're using the Products API
apiInstance = new ProductsApi(config);

// The ID of the controlled company.
var companyId = 2;

// Here we setup the exponential backoff config
var maxRetryAttempts = 5;
var pauseBetweenFailures =
Backoff.ExponentialBackoff(TimeSpan.FromSeconds(2), retryCount: maxRetryAttempts);

var retryPolicy = Policy
.Handle<ApiException>()
.WaitAndRetry(pauseBetweenFailures);

try
{
// In this example we suppose to export the data to a JSON Lines file.
// First, we cancel the content of the destination file
File.WriteAllText("products.jsonl", String.Empty);

// List Products
var perPage = 5;

// We perform the first request
ListProductsResponse result =
ListProductsWithBackoff(companyId, 1, perPage, retryPolicy, apiInstance);
// We use the first response to extract the last page index
var lastPage = result.LastPage;
// We append the products obtained with the first request top the output file
// Data contains an array of products
AppendProductsToFile(result.Data);

// For the missing pages (we already requested the first one)
for (var i = 2; i <= lastPage; i++)
{
// We require the page to the API
result = ListProductsWithBackoff(companyId, i, perPage, retryPolicy, apiInstance);
// And append all the retrieved products
AppendProductsToFile(result.Data);
}
}
catch (ApiException ex)
{
Console.WriteLine("Exception when calling ProductsApi.ListProducts: " + ex.Message);
Console.WriteLine("Status Code: " + ex.ErrorCode);
Console.WriteLine(ex.StackTrace);
}
}

// In this function we append the products in the JSON Lines file.
// You can replace this function to perform the operations you need.
// For example, you can build SQL queries or call a third-party API using the retrieved products.
private static void AppendProductsToFile(List<Product> products)
{
StreamWriter sw = File.AppendText("products.jsonl");
// For each product in the list
foreach (Product p in products)
{
// We write the product to the file
sw.WriteLine(JsonConvert.SerializeObject(p, Formatting.None) + "\n");
}
sw.Close();
}

// Here we wrap the SDK method with an exponential backoff
// This is to manage the quota exceeded issue
private static ListProductsResponse ListProductsWithBackoff(int companyId, int currentPage, int perPage, Policy retryPolicy, ProductsApi apiInstance)
{
return retryPolicy.Execute(() =>
{
// The actual SDK method is executed here
return apiInstance.ListProducts(companyId, null, "detailed", null, currentPage, 5);
});
}

// This is just a mock: this function should contain the code to retrieve the Access Token
private static string GetToken() {
return "YOUR_TOKEN";
}
}
}

📚  Additional resources