This internal (not-yet-released) experimental/research project is called, "The Living Garden" and it functions as a kind of online social game. Players (or users) get assigned real plants to start with, and by prompting the LLM to perform miraculous "cross-pollination" between unlikely parents, a plausible - but fanciful and chaotic - process of evolution unfolds over time. Oaks get combined with pansies, coconut trees with cactii, etc.

Integration: OpenAI API

The "AI" part is handled by OpenAI's API - specifically, ChatGPT for text generation and DallE for images. A little bit of prompt engineering helps to prepare the "plants" (described with some text and JSON) for their cross-pollination, and the LLM returns some plausible new plant species complete with characteristics that are reminiscent of one or both parents.

The real challenge for this project was working this generative AI pipeline into a sensible interface, and building a custom web application around it.

The tech stack

Early on in the project, I expected to work as the sole developer, so I picked a framework I was interested in learning more about: SvelteKit, the full stack complement to the frontend-focussed Svelte. Most of my frontend development in the last few years has been done in React, but Svelte has proved an interesting alternative due to its approach (no runtime!) and its supreme performance. SvelteKit suited my NodeJS/TypeScript experience, anyway.

The creative director on this project (not me!) kept expanding the scope of this playful "experiment", to the point that it became clear a database would be needed to store "users", each with their own (persisted) "seed banks" and also the new "plants" generated by the AI system. I chose "good old" Postgresql as it was familiar, but also took the opportunity to try out a new ORM that could work well with Svelte (and TypeScript) - in this case, Drizzle, which provides a nice frontend interface to the database (see below). User authentication was handled fairly simply using Lucia.

Empowering others to continue the work

Suddenly, I was far too busy on other client projects to give my full-time attention to this one, and the only developers available to continue the work were three interns with almost no web development experience. I had to quickly train them up on the basic of version control with Git, Agile project management (with daily standups), TypeScript and Svelte (I initially let them focus on the frontend). I also helped them set up development environments with local Postgresql database instances running on Docker, introduced them to Tailwind so that the custom CSS didn't quickly get out of control, and set up some scripts to make development easier.

We ended up deploying the production version using a Postgresql database hosted on AWS, media/CDN (for generated images) on Amazon S3 and the main application running on Netlify. The image generation requests to OpenAI could take much longer than the 10 second timeout allowed on Netlify, so I also had to set up two "background" functions which allowed the longer-running processes to be spawned asynchronously.

It turned out to be a great opportunity not only to explore LLMs and AI image generation, but also try out a few new modern libraries and frameworks for full stack development. I was also successful in training the junior developers "on the job" and setting things up in a way that allowed them to be very productive within a matter of a few weeks.