Skip to content

Conversation

@riseandignite
Copy link

Motivation:

Users often need to discuss specific documents, images, or data files with the AI. Currently, there is no mechanism to upload and reference these files directly within the chat conversation, limiting the agent's contextual understanding. This pr enables it.

Implementation Details:

  1. Frontend (app.tsx):

    • Added a file input button () allowing users to select one or more files.
    • Displays previews/icons of attached files below the input area before sending.
    • Implemented an uploadFiles function that sends selected files to a new backend endpoint (/api/upload) via multipart/form-data.
    • Added loading indicators during file upload.
    • On successful upload, the file metadata (name, contentType, public URL) received from the backend is passed to handleAgentSubmit using the experimental_attachments option in useChat.
    • The user's attached file previews are cleared upon successful submission.
    • Messages now render associated experimental_attachments if present, displaying images or providing download links using the URL returned by the server.
  2. Backend (server.ts):

    • Added a /api/upload route to handle POST requests with file data.
    • This route uploads the received files to a configured R2 bucket (FILE_BUCKET), generating a unique key for each (uploads/{id}/{filename}).
    • It returns a JSON response containing an array of attachments, where each object includes the file's name, contentType, and a dynamically generated public url pointing to the file retrieval endpoint.
    • Added a GET route matching /uploads/{id}/{filename} to serve the corresponding file directly from the R2 bucket.

Benefits:

  • Allows users to provide richer context to the AI agent by uploading relevant files.
  • Enables multi-modal interactions (e.g., asking questions about an uploaded image or document).
  • Improves the agent's utility for tasks requiring specific file analysis or discussion.

Testing:

  • Verified locally that files can be selected and attached in the UI.
  • Confirmed files are uploaded to the /api/upload endpoint.
  • Confirmed the backend stores files in R2 (requires local R2 setup or preview deployment).
  • Confirmed the backend returns correct metadata including a functional URL.
  • Verified files can be retrieved via the /uploads/... URL.
  • Confirmed attachments are passed via experimental_attachments and visible in agent messages.

CleanShot 2025-04-25 at 15 58 28@2x
CleanShot 2025-04-25 at 15 59 43@2x
CleanShot 2025-04-25 at 16 00 15@2x
CleanShot 2025-04-25 at 16 00 43@2x

@JonathanVelkeneers
Copy link

It might not be a good idea to implement this using the experimental_attachments workaround in an official starter repo.
It's good as a quick patch fix, but I don't think it should be a final solution.

It might be better to wait for the upstream issue cloudflare/agents#154 to be fixed using the correct FileUIPart from the AI SDK than to rely on an experimental feature, depending on their timeline.

@riseandignite
Copy link
Author

Why it's not a good idea? Why it's better to wait?

@JonathanVelkeneers
Copy link

Why it's not a good idea? Why it's better to wait?

I don't think experimental features should make its way into an official example codebase for a GA product, and definetly not when there's a non-experimental alternative (soon to be) available.
But that's just my opinion, the maintainers might feel different about it 😬.

@riseandignite
Copy link
Author

Understand your concerns, but here's why I believe using the experimental feature is a good call in this context:

  1. It’s not “unstable” in a broken sense. Vercel labels it “experimental” mainly to reserve the right to tweak the API, not because it's buggy or risky. In practice, it works reliably, and is already used across their own demos and playgrounds.

  2. It solves a real need now. The stable alternative isn’t available yet. Waiting means blocking key functionality just to avoid a name prefix, it's not ideal for shipping value fast.

  3. We’ve version-pinned ai sdk. If the API does change, we can adapt quickly without breaking anything downstream.

So unless there's a hard blocker or major risk, using this now feels like a pragmatic move.

@threepointone
Copy link
Collaborator

The approach is good, I don't mind using the ai sdk experimental api. I'll try to review this over the weekend. Thank you!

@Gyurmatag
Copy link
Collaborator

@riseandignite It looks great to me! Can you please make sure you update the branch with the latest master and resolve the conflicts? I've added some things since you opened this PR. Thank you very much!

@riseandignite riseandignite force-pushed the feature/attachments branch from f1f7a0f to 6d1d32f Compare May 6, 2025 12:13
@riseandignite
Copy link
Author

@Gyurmatag thank you! Done.

@Gyurmatag
Copy link
Collaborator

Okay, great! Now it's up to @threepointone to review it! :)

@whoiskatrin
Copy link
Contributor

@riseandignite love this PR! can we get it up to date and we will kindly ask @threepointone to review it?

@riseandignite
Copy link
Author

@whoiskatrin thanks! I've updated it.

@whoiskatrin
Copy link
Contributor

@whoiskatrin thanks! I've updated it.

seems like your build is failing with some syntax errors

@riseandignite
Copy link
Author

Supported v5
CleanShot 2025-09-29 at 22 44 15@2x

@whoiskatrin
Copy link
Contributor

Supported v5 CleanShot 2025-09-29 at 22 44 15@2x

awesome! thanks a ton! @threepointone could you please review this?

1 similar comment
@whoiskatrin
Copy link
Contributor

Supported v5 CleanShot 2025-09-29 at 22 44 15@2x

awesome! thanks a ton! @threepointone could you please review this?

@whoiskatrin
Copy link
Contributor

@riseandignite fix the build again please :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants