I have a use case where my service talks to a server which has multiple fallback servers. I am trying to develop a mechanism in which I talk to the primary server first, but it doesn't respond in x seconds, I try another fallback server and so on and so forth until any one of them responds.
With my investigation I found I could possibly use Timer() or LinkedCancellationToken but none of them are quite fit for my use case.
Timer() would schedule something for later, but I want to schedule it only if primary server doesn't respond within limit.
LinkedCancellationToken would let me cancel the call if any of the linked cancellationTokens are cancelled, but that's not my use case.
Any ideas on how I could possibly implement this?
This is a fairly straightforward use of promises + a loop.
public async Task<Connection> ConnectAsync()
{
foreach (var srv in Servers)
{
var cts = new CancellationTokenSource(ConnectionTimeout);
try
{
return await srv.ConnectAsync(cts.Token);
}
catch (OperationCancelledException)
{
// no-op
}
}
throw new NoServerRespondedException();
}
Start each attempt, if it times out you move on to the next. If all timeout, fail.
If you don't want to cancel the earlier requests, then Task.WhenAny is your friend.
public async Task<Connection> ConnectAsync()
{
var cts = new CancellationTokenSource();
var inProgress = new List<Task<Connection>>();
foreach (var srv in Servers)
{
// Ensure task does not complete before success
inProgress.Add(async () =>
{
try
{
return await srv.ConnectAsync(cts.Token)).ConfigureAsync(false);
}
catch (OperationCancelledException) { /* no-op */}
catch (Exception ex)
{
// TODO: Log connection failure
// prevent this task from completing before cancellation
var cancellationTask = new TaskCompletionSource<bool>();
cts.Token.Register(() => cancellationTask.TrySetResult(true));
await cancellationTask.Task;
}
});
var completedTask = Task.WhenAny(inProgress.Concat(new []{ Task.Delay(ConnectionInterval) });
if (completedTask is Task<Connection> connectedTask)
{
cts.Cancel();
return await connectedTask;
}
}
cts.Cancel();
throw new NoServerRespondedException();
}
The above will also return on failure of any task. If you want to ignore failures of
In principle, this is pretty simple: you can await Task.WhenAny(...), passing in the tasks you've created plus another "delay" task that will end when you're tired of waiting and ready to try another server.
Of course, best practice says you should never leave any tasks unawaited, and you should cancel the other requests once you've received a good request. That can make things a little more complicated:
async Task<Response> GetFirstServerResponse(IEnumerable<string> serverNames, CancellationToken cancellationToken)
{
var linkedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
var tasks = new List<Task<Response>>();
try
{
foreach (var serverName in serverNames)
{
tasks.Add(TalkToServer(serverName, linkedTokenSource.Token));
await Task.WhenAny(tasks.Append(Task.Delay(delay)));
var completedTask = tasks.FirstOrDefault(t => t.IsCompleted);
if(completedTask != null)
{
return await completedTask;
}
}
var firstCompleted = await Task.WhenAny(tasks);
return await firstCompleted;
}
finally
{
// Make all the incomplete tasks stop trying.
linkedTokenSource.Cancel();
try
{
// Never leave tasks unawaited.
// Since we cancelled these, they should all finish very quickly now.
await Task.WhenAll(tasks);
}
catch (OperationCanceledException) // ignore tasks that got cancelled.
{
}
}
}
Play around with it in this dotnetfiddle.
Depending on your requirements, you may need to tweak this. For example, I'm assuming if a request ends in error, then you want to bail and surface that error. If you expect certain types of intermittent errors may occur from one server but not from others, and you want to ignore errors as long as any request succeeds, you'll need to make some changes to this code.