Google Drive "New or Modified Files" polling source #19070

GTFalcao · 2025-11-14T22:19:40Z

Created as an alternative to existing webhook-based sources

Supports shared drives, and provides optional filters for specific files, specific folders, and "new files only".

Summary by CodeRabbit

New Features
- Introduced a new polling source to monitor new and modified files in Google Drive with configurable drive selection, file and folder filtering, and an option to emit only newly created files.
Chores
- Version bump to 1.3.0

vercel · 2025-11-14T22:19:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Preview	Comments	Updated (UTC)
pipedream-docs	Ignored			Nov 14, 2025 10:19pm
pipedream-docs-redirect-do-not-edit	Ignored			Nov 14, 2025 10:19pm

coderabbitai · 2025-11-14T22:19:58Z

Walkthrough

Version 1.3.0 of the Google Drive component adds a new polling source that monitors Google Drive for new or modified files. The source includes configuration for drive selection, optional file and folder filters, deployment-time state initialization with sample events, and a polling loop that streams changes and updates persistent state.

Changes

Cohort / File(s)	Summary
Version Management `components/google_drive/package.json`	Bump version from 1.2.0 to 1.3.0
New Polling Source `components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs`	Implement new polling source with key, name, description, version, and polling timer; configure with googleDrive app, drive selection, optional file/folder filters, and newFilesOnly toggle; deploy hook initializes page token and last run timestamp with sample events; run loop streams changed files via listChanges, applies filtering, and emits events; include helper methods for drive ID retrieval, filtering logic, event metadata generation, and state storage
Test Fixture `components/google_drive/sources/new-or-modified-files-polling/test-event.mjs`	Export mock Google Drive file metadata object for testing event handling and polling workflows

Sequence Diagram(s)

sequenceDiagram
    participant Deploy as Deployment
    participant Source as Polling Source
    participant Drive as Google Drive API
    participant DB as State Storage
    participant Emit as Event Emitter

    rect rgba(100, 150, 255, 0.3)
    Note over Deploy,Emit: Deploy Phase (Bootstrap)
    Deploy->>Source: Initialize source
    Source->>DB: Set initial pageToken and lastRunTimestamp
    Source->>Drive: List recent modified files (last 30 days)
    Drive-->>Source: File metadata (up to 5 non-folders)
    Source->>Emit: Emit sample events
    end

    rect rgba(100, 200, 100, 0.3)
    Note over Deploy,Emit: Polling Loop (Recurring)
    loop Each poll interval
        Source->>DB: Retrieve pageToken, driveId, lastRunTimestamp
        Source->>Drive: listChanges(pageToken, driveId)
        Drive-->>Source: Changed files page
        Source->>Source: shouldProcess: filter folders,<br/>apply newFilesOnly,<br/>match file/folder IDs
        Source->>Emit: Emit qualified events with metadata
        Source->>DB: Update pageToken
    end
    Source->>DB: Update lastRunTimestamp after all pages
    end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–30 minutes

new-or-modified-files-polling.mjs: Requires careful review of multi-layered filtering logic (newFilesOnly, file/folder matching, folder exclusion), streaming/pagination patterns, state management (pageToken and lastRunTimestamp storage), and integration points with the Google Drive API
Filtering and state update logic: Verify shouldProcess filter order and edge cases, and ensure pageToken and lastRunTimestamp are updated correctly to prevent event duplication or data loss

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description lacks a required WHY section as specified in the template. It provides context about the polling approach and features but does not explain the motivation behind this change.	Add a WHY section explaining the motivation for introducing a polling-based alternative to webhook-based sources.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately and concisely describes the main change: adding a new polling source for Google Drive file detection, which is the primary purpose of this pull request.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch drive-polling-source

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 4

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1f97c22 and 399f192.

📒 Files selected for processing (3)

components/google_drive/package.json (1 hunks)
components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs (1 hunks)
components/google_drive/sources/new-or-modified-files-polling/test-event.mjs (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs (1)

components/google_drive/google_drive.app.mjs (1)

changedFiles (424-426)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

GitHub Check: pnpm publish
GitHub Check: Lint Code Base
GitHub Check: Verify TypeScript components
GitHub Check: Publish TypeScript components

🔇 Additional comments (2)

components/google_drive/package.json (1)

3-3: Version bump looks appropriate

The version update to 1.3.0 is consistent with adding a new source and has no functional risk by itself.
components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs (1)
177-218: Confirm listChanges token semantics to avoid reprocessing or 4xx errors

The run loop is structured well, but its correctness depends on how googleDrive.listChanges handles tokens:
const changedFilesStream = this.googleDrive.listChanges(pageToken, driveId);

for await (const changedFilesPage of changedFilesStream) {
  const { changedFiles, nextPageToken } = changedFilesPage;
  ...
  this._setPageToken(nextPageToken);
}
...
this._setLastRunTimestamp(currentRunTimestamp);
Please verify that:

nextPageToken is always the correct token to resume from on the next run (e.g., it already accounts for newStartPageToken from the Drive API), and

It’s safe to persist nextPageToken even on the last page (i.e., it does not end up undefined in a way that breaks subsequent calls or causes reprocessing).

If listChanges already normalizes this, great; if not, you may want to guard the _setPageToken call or explicitly handle a final newStartPageToken.

components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs

components/google_drive/sources/new-or-modified-files-polling/test-event.mjs

michelle0927

Looks good. Left a couple comments. I'll leave it up to you, but I wonder if it would be simpler to just have one of either files or folders.

michelle0927 · 2025-11-14T22:57:24Z

components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs

+    files: {
+      type: "string[]",
+      label: "Files",
+      description: "The specific file(s) you want to watch for changes. Leave blank to watch all files in the Drive. Note that events will only be emitted for files directly in these folders (not in their subfolders, unless also selected here)",
+      optional: true,
+      default: [],
+      options({ prevContext }) {
+        const { nextPageToken } = prevContext;
+        return this.googleDrive.listFilesOptions(nextPageToken, this.getListFilesOpts());
+      },
+    },
+    folders: {
+      type: "string[]",
+      label: "Folders",
+      description: "The specific folder(s) to watch for changes. Leave blank to watch all folders in the Drive.",
+      optional: true,
+      default: [],
+      options({ prevContext }) {


Is there any reason not to use the fileId & folderId propDefinitions from the app file?

michelle0927 · 2025-11-14T22:57:34Z

components/google_drive/sources/new-or-modified-files-polling/new-or-modified-files-polling.mjs

+      // Check if specific files are being watched
+      if (this.files?.length > 0) {
+        if (!this.files.includes(file.id)) {
+          return false;
+        }
+      }
+
+      // Check if specific folders are being watched
+      if (this.folders?.length > 0) {
+        const watchedFolders = new Set(this.folders);
+        if (!file.parents || !file.parents.some((p) => watchedFolders.has(p))) {
+          return false;
+        }
+      }


The user can specify both files and folders. But if the changed file is in a watched folder, but isn't one of the selected files, it will return early.

vunguyenhung · 2025-11-14T23:31:35Z

For Integration QA:

Source Test Report: https://www.notion.so/Google-Drive-New-or-Modified-Files-polling-source-Sources-2abbf548bb5e81f69ff1d71c44597401

vunguyenhung · 2025-11-14T23:52:17Z

Hi everyone, all test cases are passed! Ready for release!

Test reports

Google Drive "New or Modified Files" polling source - Sources: https://www.notion.so/Google-Drive-New-or-Modified-Files-polling-source-Sources-2abbf548bb5e81f69ff1d71c44597401

GTFalcao added 3 commits November 14, 2025 19:08

Adding "New or Modified Files (Polling)" source

57f4dd4

Adding "new files only" option

56cfd6a

Version bump and missing newFilesOnly stuff

399f192

coderabbitai bot reviewed Nov 14, 2025

View reviewed changes

GTFalcao added this to Component (Source and Action) Backlog Nov 14, 2025

GTFalcao moved this to Ready for PR Review in Component (Source and Action) Backlog Nov 14, 2025

pipedream-component-development requested a review from michelle0927 November 14, 2025 22:33

michelle0927 requested changes Nov 14, 2025

View reviewed changes

michelle0927 moved this from Ready for PR Review to Changes Required in Component (Source and Action) Backlog Nov 14, 2025

vunguyenhung moved this from Changes Required to Ready for QA in Component (Source and Action) Backlog Nov 14, 2025

vunguyenhung moved this from Ready for QA to Ready for Release in Component (Source and Action) Backlog Nov 14, 2025

michelle0927 approved these changes Nov 15, 2025

View reviewed changes

michelle0927 merged commit 8d9de72 into master Nov 15, 2025
10 checks passed

michelle0927 deleted the drive-polling-source branch November 15, 2025 16:46

github-project-automation bot moved this from Ready for Release to Done in Component (Source and Action) Backlog Nov 15, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Google Drive "New or Modified Files" polling source #19070

Google Drive "New or Modified Files" polling source #19070

Uh oh!

GTFalcao commented Nov 14, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

vercel bot commented Nov 14, 2025

Uh oh!

coderabbitai bot commented Nov 14, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michelle0927 left a comment

Uh oh!

michelle0927 Nov 14, 2025

Uh oh!

michelle0927 Nov 14, 2025

Uh oh!

vunguyenhung commented Nov 14, 2025

Uh oh!

vunguyenhung commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Google Drive "New or Modified Files" polling source #19070

Google Drive "New or Modified Files" polling source #19070

Uh oh!

Conversation

GTFalcao commented Nov 14, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

vercel bot commented Nov 14, 2025

Uh oh!

coderabbitai bot commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michelle0927 left a comment

Choose a reason for hiding this comment

Uh oh!

michelle0927 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

michelle0927 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

vunguyenhung commented Nov 14, 2025

For Integration QA:

Uh oh!

vunguyenhung commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

GTFalcao commented Nov 14, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Nov 14, 2025 •

edited

Loading