Unstable behavior from mr.roboto

[mauritsvanrees]

See also chat in release-team channel on Discord. mr.roboto is having some problems. Some changelog checks go fine, for example this log line:

2025-05-09 11:13:51,123 INFO PR plone/plone.z3cform#30: changelog entry: success

Others give an error, even when everything seems fine to me:

2025-05-09 09:44:04,001 INFO PR plone/plone.app.users#141: changelog entry: error

Also, when I add the comment to ask to run the Jenkins jobs, this is not always done.

See for example this PR: plone/plone.app.users#141
And check the roboto log at https://jenkins.plone.org/roboto/log?token=XXX (you need a different token, known to the admins team).

Timeline:

PR is created:

2025-05-09 09:43:58,447 INFO PR plone/plone.app.users#141: Contributors Agreement report: success

Mr.roboto runs some actions:

Contributors Agreement report: success
changelog entry: error
created pending status for plone 6.0 on python 3.9
...

Alex adds a comment comment, but nothing special:

2025-05-09 09:51:51,396 INFO COMMENT plone/plone.app.users#141-2865522988: with action created on pull request plone/plone.app.users#141

Then I add a comment, including the command to create Jenkins jobs. Strangely, I don't see this in the roboto log.
Also, most comments have a title like "name commented X minutes ago", and it is clickable (it has an #anchor), but for my comment I see "mauritsvanrees left a comment" and it is not clickable.

Ah, that could be because it is part of a PR review. When I click the three dots to the right of my comment and copy the link, it is this:

plone/plone.app.users#141 (review)

I add another comment with the command, and then it works:

2025-05-09 12:06:33,413 INFO COMMENT plone/plone.app.users#141-2865964423: with action created on pull request plone/plone.app.users#141

This is followed by "Triggered jenkins job for PR 6.0-3.9" plus five more such jobs.

There is one more thing, there is this line:

2025-05-09 11:40:42,313 ERROR Exception while serving *

This is followed by a traceback (shown upside down because of how the log is shown by Jenkins:

AssertionError: URL fragments must start with / or http:// (you gave '*')
    assert (not url or url.startswith('/')
  File "/srv/mr.roboto/lib/python3.11/site-packages/paste/urlmap.py", line 125, in normalize_url
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    path_info = self.normalize_url(path_info, False)[1]
  File "/srv/mr.roboto/lib/python3.11/site-packages/paste/urlmap.py", line 203, in __call__
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return app(environ, start_response)
  File "/srv/mr.roboto/lib/python3.11/site-packages/waitress/proxy_headers.py", line 64, in translate_proxy_headers
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    app_iter = self.channel.server.application(environ, start_response)
  File "/srv/mr.roboto/lib/python3.11/site-packages/waitress/task.py", line 436, in execute
    self.execute()
  File "/srv/mr.roboto/lib/python3.11/site-packages/waitress/task.py", line 168, in service
    task.service()
  File "/srv/mr.roboto/lib/python3.11/site-packages/waitress/channel.py", line 428, in service
Traceback (most recent call last):

So:

1. Something is flaky about the changelog check.
2. The 'start Jenkins jobs' command is ignored when it is in a PR review. I guess that would be a feature request. Or just something we need to remember. I did not notice this before, but maybe I did not pay enough attention.
3. There is a traceback while serving '*'. No idea if that has anything to do with points 1 or 2.

[gforcada]

I've seen the while serving '*' for quite a while, but I always thought it was either spam or some malicious attacks.

The jenkins job on a PR review, I can imagine it is not being handled properly, as I'm expecting the payload to be quite different.

Finally the changelog check seems to still be flaky without any relevant traceback 😅

[stevepiercy]

Here's another example plone/volto#7055

[rohnsha0]

with my experience in plone/Products.CMFPlone#4168, once it worked just fine and showed green but when i pushed another commit it again failed!

[gforcada]

I added some debugging code on mr.roboto to see what's going on. Ping again whenever you notice that it goes wrong, then we can look at the log files and try to understand what goes wrong there.

For the curious, the code is very small and simple: https://github.com/plone/mr.roboto/blob/master/src/mr.roboto/src/mr/roboto/subscriber.py#L307-L353

[davisagli]

@gforcada It just happened again here: plone/plone.restapi#1921

[mauritsvanrees]

One theory in icalendar is: "It seems to me that it only checks the first commit of a PR."

[rohnsha0]

One theory in icalendar is: "It seems to me that it only checks the first commit of a PR."

@mauritsvanrees As far I know, it checks during each commits... Because I never add changelogs in the first commit bt in the later commits... However it used to detect it... Before flakiness started since few weeks

[stevepiercy]

This PR worked, then failed, and now works again. It might be a good one to scan for in the logs.

plone/volto#7075

[stevepiercy]

Another false positive: plone/volto#7084

[stevepiercy]

This time a false positive: plone/volto#7126

[fredvd]

@gforcada . We (@erral , @fredvd) want to add a fix to mr.roboto for the weblate blocking language update PR's for missing commits/authors. #169

I assume the local changes/patch on. current mr.roboto production git checkout without any comment refering to this issue is from you? ;-) (Can you please add a # with link to the issue next time?)

I assume I should keep the patch for further debugging this issue, I'll add it to main branch.