Interest in "Diffusion Models for Generative Lens Design" & Follow-Up Plans

[JasonG98]

Hi there,
I’m really interested in the "Diffusion Models for Generative Lens Design" part of your repo—it’s right up the alley of what I’m focusing on lately!
Just wanted to ask: do you have any follow-up plans for this work (like updates, expansions, etc.)?

Thanks!

[HarrisonKramer]

Hi @JasonG98,

Thanks for opening this, and great to hear you’re interested in that part of the project!

I’ve started some initial work on the diffusion-based generative lens design idea. The goal was to create a minimal example for the repo - something like using a diffusion model to generate simple, image-forming doublets. With Optiland, building a dataset of a few thousand well-performing lenses is relatively straightforward, and I already have a prototype script that does this.

I also ran a few early training experiments to test feasibility, but the work is incomplete. I wanted to refine the dataset and potentially the model/notebooks before publishing something concrete.

If you’re exploring similar ideas or would like to collaborate, I’d be very open to that! We could, e.g., co-develop a small proof-of-concept notebook here, or compare approaches if you already have something in progress.

Let me know. I can already push some of the (older) scripts or datasets I have, if interesting. I'd also be interested to hear more about what you're working on.

Best,
Kramer

[JasonG98]

Hi @HarrisonKramer

Thanks so much for your detailed reply—it’s really helpful to hear about your progress with the diffusion-based lens design work!

To be honest, I don’t have concrete progress on this front yet. I’m not a professional optical designer, and right now I’m stuck at the dataset building stage. My colleague recommended I use the Zemax API for this, so I’ve been spending a lot of time learning how it works, but I’m still figuring things out.

That’s why I’d be incredibly grateful if you’re willing to share those older scripts or datasets you mentioned. Having a reference to learn from would save me so much time and help me get past this current roadblock.

If there’s any feedback I can provide after going through them, or if you’d like to chat more about the technical details later, I’m more than happy to do that too.

Thanks again for being open to sharing—this means a lot!

[HarrisonKramer]

Hi @JasonG98,

Sure, happy to share. I can push a new branch soon with the draft notebooks and dataset-building scripts so you can take a look. It’s all still in an exploratory stage, but it should serve as a useful starting point or reference.

Using Zemax is of course perfectly fine. It’s the industry standard and a reliable option for dataset generation. One advantage of Optiland, though, is that it can run on a PyTorch backend, which allows gradients to be propagated through the full optical calculation(s). That’s not required for a “vanilla” diffusion model, but it could be valuable in some hybrid or physics-guided setups. I’m planning to include a few examples of this in the repo later on (probably for models unrelated to diffusion).

I’d be very interested to hear about any progress on your end if you’re open to sharing. Any form of collaboration or feedback is appreciated.

I’ll post an update here once the branch with the drafts is available.

Best,
Kramer

[JasonG98]

Sure, happy to share. Thanks aaaaagain.

[HarrisonKramer]

Added draft notebooks here: https://github.com/HarrisonKramer/LensAI/tree/feat/doublet_diffusion/notebooks/Doublet_Diffusion

The primary diffusion model script is not yet very robust - it can (sometimes) generate reasonable doublets. More work to be done there. Feel free to build and improve on it, and share if so.