feat: new filter for meaningful elements

[dicagno]

Please ensure your pull request adheres to the following guidelines:

• make sure to link the related issues in this description
• when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes
• If data sources for any opportunity has been updated/added, please update the wiki for same opportunity.

Related Issues

Thanks for contributing!

[hannessolo]

This probably works, but I'm wondering if we shouldn't just change the approach completely.

Eg. parse the html -> strip header/footer completely -> convert page content to markdown -> run readability for each paragraph of the markdown content

I think that would be more reliable than what we're doing here, eg. what if we have a p inside the header, there just seem many edge cases where this can still give werid results.

Or what happens when we have a li inside a p tag here? Does it run twice for that content?

[github-actions]

This PR will trigger a minor release when merged.

[hannessolo]

Given time pressure this is ok, but as I said, I think long term a solution extracting the text more generally, eg though md conversion, would be more robust.

[dicagno]

Given time pressure this is ok, but as I said, I think long term a solution extracting the text more generally, eg though md conversion, would be more robust.

agree, as discussed. i would add that using md would be beneficial to preserve "rich text" elements and include them properly in the "auto fix" stage

[dicagno]

moved to #1668