AI art and quilting

In January 2023, the news was filled with AI and court cases about AI. I got interested in the following question: what exactly is AI?

In my art practice, I was interested in modifying photos (taken by me) to make them more amenable to quilting or overstitching. I decided to explore the topic of AI and do some research.

The US copyright office ruled in February 2023 that AI generated artwork is not legally protected as explained in this article.

The decision by the Copyright Office decision hinges on whether the AI is an “assisting instrument”. However, in the case cited by the article, the Midjourney AI is stated to turn words and phrases into tokens and then generated the AI artwork. The crux of the matter appears to be that since the program does not understand words or phrases then it follows a different process for creation.

As an artist/programmer, I find this decision a bit confusing and likely to be open to legal debate in the future. AI (artificial intelligence) covers a large number of machine learning techniques that span images (image classification) and language (nlp or natural language processing). In particular, the decision does not the fact that an artist could write a AI model with the specific goal of creating artwork. AI can undoubtably be used to plagiarize, but it also opens the possibly of new methods of artwork.

Working with AI

In my work, I decided to focus on two specific types of AI models: 1) neural style transfer and 2) clustering algorithms. These algorithms offered the opportunity to modify photographs and achieve my artistic goal of creating images that amenable to overstitching.

In neural style transfer, two images are selected and the style of one image is transferred to the other. This AI technique relies on an pre-built image classification model. I used the VGG-19 model which has the ability to recognize 1000 images. However, I did not directly use the classifier directly. I instead utilized the filters that are used to calculate the probability of each image class. In this context, the “filter” is essentially a sequence of matrix multiplication operations that output a matrix.

I used this filter in an algorithm that changes the underlaying rgb codes of the source image towards the rgb codes of the style image.

We can see the technique applied to the style and source image below. The source image is one that I love, but the center of the flower is not amenable to my style of stitch - so my goal is smooth regions of the image as I accomplished in the yellow flower image.

Style Image

Source Image

AI image

To see the full technical details and the python code base, you can read my tech article here.

I’m not sure the US Copyright’s office fully addresses the issues of using AI in artwork. What is a token? It turns out in this case that a token or filter is really just a matrix of numbers that I can manipulate.

What else to explore?

I haven’t fully investigated how much of the source and style image are recoverable from the filters in the VGG-19 model.

I also haven’t finished writing my clustering AI model, another technique that will allow me to alter my images.

This technique reduces the number of color regions in an image, smoothing the regions and pulling the remaining regions closer to selected colors. These types of techniques are widely used in pixelated or mosaic artwork.