-
Notifications
You must be signed in to change notification settings - Fork 141
Open
Description
We're getting an exception when attempting to retrieve the list of fetched jobs via the Monitoring API.
This line of code:
JobStorage.Current.GetMonitoringApi().FetchedJobs("default", 0, int.MaxValue)
Causes the following exception:
System.ArgumentException: An item with the same key has already been added. Key: 24014
at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
at System.Collections.Generic.Dictionary`2.Add(TKey key, TValue value)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](List`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
at Hangfire.PostgreSql.PostgreSqlMonitoringApi.FetchedJobs(IEnumerable`1 jobIds)
at Hangfire.PostgreSql.PostgreSqlMonitoringApi.FetchedJobs(String queue, Int32 from, Int32 perPage)
at Somewhere.In.My.Code
The issue seems to occur in the following situation. This is purely based on logs from our environments, I have not had the time to attempt to reproduce it locally:
- A hangfire server is currently processing jobs.
- The hangfire server disappears from the active server list (such as when the server gets restarted or the application crashes)
- The other hangfire servers (or the 'new' server if it was restarted) notices that the jobs from step 1 might have been aborted, because they're marked as being processed by a server that is not in the active server list.
If you attempt to get the list of fetched jobs after step 3 has occurred, it seems to (sometimes?) crash. This eventually resolves itself, presumably when the job gets successfully processed.
Versions:
Hangfire.PostgreSql: 1.20.8
Hangfire: 1.8.7
.NET version 8.0.202
Metadata
Metadata
Assignees
Labels
No labels