Serverless for dummies - pragmatic encounter on azure
July 16, 2020
Here I am without the server
Serverless in on the rise. Gaining a lot of traction to push new abilities, features and there is more and more hosting/deployment options.
It is not only bound to public cloud providers but it’s also getting more popular on on-premise environments where you can tune it to your needs. This is backed up by frameworks like OpenFAAS, Kubeless, which are often build in open source world.
Main idea of serverless is to not worry about infrastructure technicalities (no server) and instead focus on solving real problems/writing code.
Serverless code generally is short-living and needs to be stateless, meaning state should be put into/handled by external system, database, etc. Thus it is harder/trickier to do small stateful things like connection pooling (i.e. connections pooling to sql databases).
When thinking about it in context of cloud platforms (azure, aws), payment model is very interesting:
- pay as you go
- scale as you go
On other hand there are some limitations:
- limited message size
- limited active connection time(only request reply)
- limited number of concurrent connections
But What can I build ?
Typical question is what I can build it with it and answer to that is typical
it depends
oreverything
.
Most of the time serverless is used as a additional layer on top of typical server-full apps.
It fullfils role to solve little problems, threat serverless function as single purpose endpoint that solves one problem.
Then compose those little problem-solvers (functions) together to create something bigger
.
Not another hello world example
Let’s create something that might be useful.
Function that will accept as a parameter
id of spotify playlist
and return out of it list of songs together with youtube link.
Here is the final result:
You can experience that by executing, although it might return 500 if youtube api limits were exceeded:
https://spotifytranslatorfunctionapp.azurewebsites.net/api/map?playlist=2GK6j1Wh1NfmIPcFVXC97t
Where:
playlist=
is a id of spotify playlist
Please note i will focus on local development, thus i won’t go through azure portal
and creating/managing/deploying stuff there.
Never the less this of course can be deployed to cloud without a problem.
What we need ?
We will use azure, below are prerequisites:
- Setup
.net core
(version 3.1 >= required). - Install
Azure Functions Core Tools
from here. - We will need to connect to api of spotify and youtube. Since only limited usage of these api’s is free, we need to create
youtube app
andspotify app
so our usage can be tracked.
Scaffolding
mkdir SpotifyToYoutubeTranslator && cd $_
func init --worker-runtime dotnet
func new --language C# --name SpotifyToYoutubeTranslator
Code walkthrough
Entire code is available here. It’s splitted into 3 major sections:
Before we begin
Youtube/spotify application keys need to be stored somewhere safe. Those are secrets that authenticate our app, that can not be compromised.
We will use simple approach, which is local.settings.json
.
Cloud version is using azure application settings
, as secrets from local.settings.json
are of course not commited.
Then in code these will be read as follow.
private static readonly string SPOTIFY_APP_KEY =
System.Environment.GetEnvironmentVariable("SPOTIFY_APP_KEY");
private static readonly string YOUTUBE_APP_KEY =
System.Environment.GetEnvironmentVariable("YOUTUBE_APP_KEY");
If something more advanced is needed, have a look at azure key-vault
.
It is ment to store secrets but with more features on top of it :)
Section #1
First we need to know what playlist we want to take from spotify.
This is easy as getting playlist id
from incoming request query parameters.
var playlist = req.Query["playlist"];
If there is no cached spotify token or it expired, a new one is obtained and cached.
private static async Task<SpotifyItems> GetSpotifyItems(HttpRequest req, string playlist)
{
if (_spotifyToken == null || _spotifyToken.ExpiryDate < DateTime.UtcNow)
{
var fullToken = await GetSpotifyAccessToken();
_spotifyToken = new SpotifyToken(fullToken["access_token"].ToString(), Int32.Parse(fullToken["expires_in"].ToString()));
}
var token = _spotifyToken.Token;
To obtain fresh token, a POST
request with spotify app key has to be send to spotify api.
private static async Task<dynamic> GetSpotifyAccessToken()
{
var tokenHeaders = new FormUrlEncodedContent(new[]{
new KeyValuePair<string, string>("grant_type", "client_credentials")
});
var tokenRequest = new HttpRequestMessage(HttpMethod.Post, "https://accounts.spotify.com/api/token");
tokenRequest.Content = tokenHeaders;
tokenRequest.Headers.Add("Authorization", SPOTIFY_APP_KEY);
var tokenRequestResponse = await _httpClient.SendAsync(tokenRequest);
return await tokenRequestResponse.Content.ReadAsAsync<dynamic>();
}
Section #2
Secondly, We need actual playlist of songs, given above playlist id
.
For that, GET
request is executed. In the header We pass spotify app token from previous section.
Query parameters contain items, that are going to be returned back by spotify api. In this case we would have:
- name of a track
- artist/s
- name of an album
private static async Task<SpotifyItems> GetSpotifyItems(HttpRequest req, string playlist)
{
if (_spotifyToken == null || _spotifyToken.ExpiryDate < DateTime.UtcNow)
{
var fullToken = await GetSpotifyAccessToken();
_spotifyToken = new SpotifyToken(fullToken["access_token"].ToString(), Int32.Parse(fullToken["expires_in"].ToString()));
}
var token = _spotifyToken.Token;
var playlistQuery = HttpUtility.ParseQueryString(string.Empty);
playlistQuery["fields"] = "items(track(name,artists, album(name))), next";
string playlistQueryString = playlistQuery.ToString();
var playlistRequest = new HttpRequestMessage(HttpMethod.Get, $"https://api.spotify.com/v1/playlists/{playlist}/tracks?{playlistQueryString}");
playlistRequest.Headers.Add("Authorization", $"Bearer {token}");
var playlistRequestResponse = await _httpClient.SendAsync(playlistRequest);
playlistRequestResponse.EnsureSuccessStatusCode();
var spotifyItems = await playlistRequestResponse.Content.ReadAsAsync<SpotifyItems>();
return spotifyItems;
}
Section #3
Finally, for each song from the playlist, we need to have link to youtube video and return it as a final response.
To do so we call youtube with GET
request.
For this request no token is needed, we just set the app key
in query parameters.
Also in query params we provide:
q
, search phrase{artist name - track name}
type
of a resource, hardcoded tovideo
part
, we want to get as a result, hardcoded toid
maxResults
, we are interested only in the first video matching
var resultItems = spotifyItems.Items.Select(async track =>
{
var youtubeItems = await GetYoutubeItems(track);
return new ResultItem(
track.Track.Name,
track.Track.Artists[0].Name,
track.Track.Album.Name,
youtubeItems.Items[0].Id.VideoId
);
});
return (ActionResult)new OkObjectResult(await Task.WhenAll(resultItems));
private static async Task<YoutubeItems> GetYoutubeItems(SpotifyItem track)
{
var youtubeRequest =
new HttpRequestMessage(HttpMethod.Get, $"https://www.googleapis.com/youtube/v3/search?q={track.Track.Artists[0].Name}-{track.Track.Name}&type=video&part=id&maxResults=1&key={YOUTUBE_APP_KEY}");
var youtubeRequestResponse = await _httpClient.SendAsync(youtubeRequest);
var youtubeItems = await youtubeRequestResponse.Content.ReadAsAsync<YoutubeItems>();
return youtubeItems;
}
Final result is composed, by taking ids
returned by api and putting them as v
query param into youtube video link.
public ResultItem(string name, string artist, string album, string ytId)
{
Name = name;
Artist = artist;
Album = album;
Url = $"https://www.youtube.com/watch?v={ytId}";
}
Important note - HttpClient is in a static field
We don’t want to create HttpClient
for each request. Rather it should be long-lived resource, this will lower memory footprint and it will benefit from reusing tcp connections.
Important thing to remember is that, there won’t be one and only instance of HttpClient
, instead each instance of a function runtime will have single HttpClient
instance.
For more info about HttpClient
check this blog post.
Running locally
You can run it locally by cloning repo and creating config file local.settings.json
.
"Values": {
"AzureWebJobsStorage": "",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"SPOTIFY_APP_KEY": "Basic YourSpotifyAppKey",
"YOUTUBE_APP_KEY": "YourYoutubeAppKey"
}
Then just run func start
and call http://localhost:7071/api/map?playlist=YourSpotifyPlaylistId
Have fun :)