Skip to content

Conversation

@fische
Copy link
Contributor

@fische fische commented Sep 23, 2024

This PR adds some debug logs to flair, so it's easier for us to see which client the load is coming from.

@fische fische self-assigned this Sep 23, 2024
@peterebden
Copy link
Collaborator

I'm not sure how this will identify the client? In our usual setup the client of Flair is running on the same machine as it.
I also think it'll be very noisy when those clients are active...

@fische
Copy link
Contributor Author

fische commented Sep 23, 2024

I'm not sure how this will identify the client? In our usual setup the client of Flair is running on the same machine as it. I also think it'll be very noisy when those clients are active...

Yeah actually you're right it's going to be noisy. We probably don't need the logs on reads, only on Execute, since Reads can come from the workers; this should lower the noise.
RE how to identify the client, we can do that with the logs we collect on Kibana as we label them with the hostname, IP, and so on.

@fische
Copy link
Contributor Author

fische commented Sep 23, 2024

Actually I'm wondering if we could collect those as metrics instead of logs. That would be much more useful and wouldn't pollute the logs.

@Tatskaari
Copy link
Contributor

Tatskaari commented Sep 23, 2024

Let's have a think about what we want to know. Maybe something like:

  1. What machine is generated what actions for which targets (from flair)
  2. What actions are downloading the most data (from the mettle workers)

I think we may want to make flair use structured logging so we can record metadata against these loglines that we can analyse in logstash too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants