Skip to content

Conversation

@lun-4
Copy link

@lun-4 lun-4 commented May 28, 2025

if a post has replyCount>0 but no replies, that means the bluesky appview is restricting the depth of the thread.

not everyone wants to recurse over all posts, so leave it as -o expand=1 like the twitter extractor.

testing like this under an extremely long root thread: env/bin/gallery-dl --verbose -o expand=1 -J 'https://bsky.app/profile/blackle.bsky.social/post/3lpmnsdic722n'

if a post has replyCount>0 but no replies, that means
the bluesky appview is restricting the depth of the thread

not everyone wants to recurse over all posts, so leave it
as '-o expand=1' like the twitter extractor.
@lun-4 lun-4 force-pushed the bluesky-expand branch from 6621e0c to 4c8b95a Compare May 28, 2025 17:24
@mikf
Copy link
Owner

mikf commented Jun 2, 2025

Any particular reason why you are collecting all posts first, then expanding them, and only then returning any results instead of doing "lazy" expansions for each returned post as is the case for Twitter?

@lun-4
Copy link
Author

lun-4 commented Jun 2, 2025

there wasn't any intentional technical reason other than not being super familiar with the codebase so I wanted to isolate all the logic away, but I do need access to bluesky's threadViewPost so I can fetch the "replies" field and I saw some other bluesky functions strip it (via post = post["post"]), so if there's a way to access replies with "lazy" expansions I'm interested

@lun-4
Copy link
Author

lun-4 commented Aug 5, 2025

regarding lazy expansion, do you mean this part of logic on the twitter extractor?

    def _expand_tweets(self, tweets):
        seen = set()
        for tweet in tweets:
            obj = tweet["legacy"] if "legacy" in tweet else tweet
            cid = obj.get("conversation_id_str")
            if not cid:
                tid = obj["id_str"]
                self.log.warning(
                    "Unable to expand %s (no 'conversation_id')", tid)
                continue
            if cid in seen:
                self.log.debug(
                    "Skipping expansion of %s (previously seen)", cid)
                continue
            seen.add(cid)
            try:
                yield from self.api.tweet_detail(cid)
            except Exception:
                yield tweet

specifically yield from ... and how I don't use that in bluesky expansion logic, or something else entirely?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants