Daily sketches in 2017
I started the process of doing daily code sketches Jan 1st, 2016. These are small animations I make with code and post on instagram. I wrote about the year-long process in 2016 here. Also, this talk I gave at the AIGA has an overview about daily sketching (as well as a snippet of live coding!).
I kept going with the daily sketching last year and I thought for 2017 I would offer some updates and talk about how the process of sketching went this year, focusing on some broad themes and lessons I’ve learned.
Overall I had a really fun year and went in all kinds of directions, which makes write up like these kind of hard — I don’t know exactly what to focus on. Anyway, I’ll try my best to sum it up the year and things I’ve learned. Where possible, I’ll link to the underlying sketch.
Process
My process is relatively similar to what I wrote about last year (coding in openframeworks → capture with quicktime → ffmpeg to compress → airdrop to phone →post), but this year I started to experiment with augmented reality (AR) a bit more, which meant more on device recording and changed some of the workflow. I write a bit about AR further down the write up.
I had some simple rules which I kept from last year to this year:
- Avoid using “over-used” computational algorithms like delaunay triangulation, particle systems, HSB color spaces, etc.
- Keep sketches really simple
- Prioritize iteration over novelty — how can I push an old sketch to be new?
- Create visuals that are somewhat ambiguous — 3d that looks like 2d, 2d that looks like 3d. I like making images that ask your brain to work a little bit more.
I also tried to mix things up a bit more this year, and I experimented with:
- Using images and video, including more webcam related work
- Using data sets such as Google’s quick draw data, brushes images, videos, etc.
I still spend about 30 mins to an hour or so per day. Sometimes I do things really quickly, other times I like to meditate on a sketch and keep pushing it throughout the day. To me sketching is a form of mediation, I often use it as a way of taking a break from work.
Another thing I want to stress is how important references / inspirations are — I spent alot of time this year studying Sophie Tauber Arp’s curves, Ruth Asawa’s sculptures, Armin Hoffmann diagrams, etc. I feel like having a collection of images that inspire you can be really helpful. I often would go to these when I felt like I didn’t know what I wanted to make.
Motivation
What I wrote last year still totally feels like a great intro to what I am doing, but I feel like I want to add a few more points. The first is that I feel like doing daily sketching is essentially about iteration, not necessarily about developing new ideas, but really finding new ways to say older things. What is the smallest change to make a new thing? I think if every day I felt like I have to say something new it would be really hard, so by reframing it to change something every day reduces the mental burden. Often times after I post a sketch, I think, there must be a better way to say this.
The act of doing something over and over again forces you to make up shortcuts. I really like this image to explain what I mean:
Here, a student had to write, “I will make better choices.” over and over again. You can see that at the start, they write each letter one at a time. After some time they connect all the stems of the letters to save time. The act of doing something over and over again forces you to develop shortcuts. Those shortcuts actually become your style — how you get faster to a new thing.
I feel like there’s a larger lesson about creativity. In the past my mental image of creativity was an image of a painter starting with a blank canvas or a writer starting with a blank piece of paper, waiting for a lightning bulb shaped insight above their head. But now I feel like creativity is more about tedium, doing the same thing again and again, almost like a meditation, and that what matters are the shortcuts, the breaks, the changes, the alterations you make. I don’t know exactly how to articulate this but I feel like iteration is really key. My motto for 2017 was ABI: “Always Be Iterating”.
I also try to capture everything I do. I was riding the subway and I saw this amazing scene:
A passenger across me was photographing his phone with his camera, also while wearing snap spectacles. I really love how many cameras are going on in this scene — including my own which is trying to capture it. I feel like artists should document constantly and I feel like alot of what I am doing when I am doing daily sketches involves capturing things. I will code a sketch, then move it around, adjust parameters, and find the right view — I feel like a nature photographer.
I still have one folder (heavily backed up!) that contains everything I do related to sketching. All my screenshots, videos, etc go here:
What I love is that since it contains everything I do sketching related, I can always find my way back. The sketches themselves are like breadcrumbs, making a trail that I can always walk down again.
In addition to using ffmpeg on a daily basis to compress the videos I make, I’ve also started to use kyno to manage media since I have a pretty large collection of clips and screenshots. I tried some of Adobe’s products for this but found kyno much faster and easier to operate. Also, since instagram has a really poor web interface for looking at older work where you have to continually scroll to load content, I’ve started to use the weirdly named !nstaBro app, which, despite what the name sounds like is actually pretty helpful for quickly navigating a ton of posts — it makes a great desktop browser for instagram. Both tools have helped me a ton this year, especially when I am asked to provide high res clips for work or sending links on to client.
Diary
One thing that’s become clear to me is how much daily sketching is also like diary entries, and that your moods get baked into your artwork. If I scrub through the year, I can see my overall mood and environment change through the work I do.
For example, around new years last year, I was deeply concerned about the election results and what the new year would bring. I did a series of animations around the peanuts cartoon, since I felt like were were in some kind of weird reality.
After new years, it felt like we were protesting every weekend, with the women’s march and when my partner and I went to JFK to protest after the travel ban. I did a series of sketches that focused on repulsion — what does it feel like to be in a crowd, pushing and being pushed.
Finally, this year I lost a friend, collaborator and former student Firm Read. When he was in the hospital, I made a sketch for him based on sine waves and modeled after John Whitney’s book cover, since I remembered how much he enjoyed exploring sine waves in my algorithmic animation class at Parsons. It meant so much to me to make a sketch and see him like it. Besides the hospital visits it was one of the last interactions we had.
Video Augmentation
Towards the end of last year, I found out, via prosthetic knowledge, about an algorithm developed at Carnegie Mellon called rtPose (now openPose) that uses machine learning to analyze images / videos and identify skeletons, similar to how the kinect could give a 3d skeleton (in this case, it’s 2d):
It requires pretty optimized hardware to run in real time, but even with the super slow 1 fps I got on my laptop was able to feed it video clips and, after some time, get skeletons to work with offline. When I have a project that would pay for it (or an nvidia sponsorship!) I’d love to try to do this realtime.
I think this is really an interesting area for artists to explore: using machine learning research to get meaningful data about older media or large data sets. I’ve been loosely following some of the ML related artistic endeavors, like GAN, pix2pix, many of which produce striking, blurry, hallucination like images. I find myself gravitating towards using ML for getting augmentation data rather than synthesizing images. (as a side note, I did try to use pix2pix to make a blob out of cat images this year)
Later in the year, I was commission by Hermès to make animations based on daily life in Shanghai and used rtPose as well as some openCV based tracking.
Curves
Several times throughout the last year I meditated on curves — endless lines that bend around the screen.
The previous year, in 2016, I spent time sketching with dubins curves, which have really lovely graphical properties. For these curves, I did something simpler which was create an endless line where every time you step forward, you turn a little. If you turn a tiny amount for every forward step, you make a giant arc, but if your turn is big relative to the stop, it’s a tinier, tight curve. What I am doing to control the curve is picking random curve amounts, adding some smoothing between them, and saying, if you’ve got to the edge of the screen keep curving enough to come back towards the center. I think about it like a drunken curve walk.
Often times I start with a motion or a geometry and then explore different combinations and push the look in different directions.
Sometimes the shading I applied really surprised me. Here I drew the curve as a 2d mesh, but it looks and feels 3d.
Another curve that I love and spent a lot of time thinking about combines half circles (half circle arcs) with straight vertical lines between them — I really love the graphical nature of this curve. With creative coding, we have all these primitives, sin waves, noise, voronoi, particle systems but we can make forms based on other primitives like circles, cubes, etc. I think part of my sketching is to try to identify other kinds of primitives which feel different to me and invite me to explore in new ways.
I found that if you rotate these curves in 3d around their x axis, you can make really lovely revolved forms.
There’s no lighting but these objects feel like they are lit or emitting light. I fucking love these things.
Multiple blobs / Extruded geometry
In my first year of sketching, I spent a lot of time working with blobs, which are essentially particles that repel away from each other connected by a loop of springs. I’ve found that in the past it’s hard to make them blobs touch each other since they can easily get caught and tangled if their edges overlap. I experimented with blob / blob interaction and found some hacks that, although not super efficient, help keep blobs happy as they touch.
This opened up some design ideas — and I spent some time jamming with multiple blobs
Another thing I found interesting, once I start extruding these forms, is to move from blobs, which are kind of expensive to compute, to just using circles or other primitive geometry — animating them against each other and extruding them out over time. I found I could get really organic forms that way.
Brushes / Drawing
At some point during the year, I was feeling like I needed to find a new look to explore. I was watching a lot of instagram videos of calligraphy, shot in a kind of POV perspective and feeling really inspired watching people brush ink to make letter forms. To start experimenting I made images — basically high resolution images of ink — and explored using those as a primitive. It gave the visuals a lovely textural quality. I explored cycling them to animate and creating 3d forms that would be drawn by flat (2d) brushes.
With those instagram videos that show calligraphy, I love how there was always a hand in there. It seems that more burrowed we get into our screens, the more we want to see physical things — ink drops, calligraphy, skateboarding videos. I start sourcing and creating clips of hands drawing a line, and connected aligned these clips to connect the lines. It produced really weird and lovely forms, dozens of synchronized hands making spirals, etc.
Metaballs
Metaballs is an old school computer graphics algorithm where you define shapes via an implicit functions and use a technique like marching cubes to trace the edges of those functions on some sort of grid. I got interested in metaballs to create blob like shapes. I also explored what happens when you slice these function at multiple thresholds. A lot of my explorations were just trying to figure out what was possible with these functions that I didn’t have a ton of experience with.
After experimenting for a bit, I was trying to find a way to find another approach to metaballs which could also make “layered” organic shapes. I stumbled on a solution of doing layers of traditional noise (simplex) — here, you can see an image (computed via a shader) with a bunch of small sub images that represent 2d slices of 3d noise.
It’s easier to visualize when you see them layered on top of each other:
You can change the noise function and where you sample it, to create different looks. I’ve found using noise * noise (ie, like amplitude modulation, with sine waves) can be interesting. I used this technique to create layered compositions with very organic shapes. My studio YesYesNo used this for a project with Grey, creating generative logos based on brain wave sensors to celebrate their 100th anniversary.
Webcam / Face related
One change from 2016 to 2017 is that I spent more time this year working with video and webcam as input. In 2016 I used a video clip of someone moving as a kind of test image for sketches, in 2017 the test image often became video feed of myself.
To be honest, I was a little weary of using webcam footage. I once saw a video with a musician (can’t remember who at the moment) who was making a beat with samples and they said, “if you throw voice sample in there it will always make a mediocre beat sound good.” For that reason, I kind of stayed away from webcam video my first year (besides doing contour based work) but this year had fun exploring it more and more. I was also worried that my artwork would be seen more as a “filter.”
I experimented with breaking up the image, using this segmentation algorithm and coloring / growing the pieces.
As well as lots of experiments with grids and texture manipulation
I’ve long been inspired by Francis Thompson’s NY NY movie, shot over a decade and released in 1957 which depicts a day in the life of NYC.
He used a series of modified lenses to mirror and distort the visuals (much like shaders can be used to distort images now). In that spirit, I experimented with pushing my face in different ways:
I also experimented one weekend with the quadtree algorithm — which is a spatial subdivision algorithm. You start with a a square, split it horizontally, then take the two rectangles formed and split them vertically. You continue recursively dividing the space and alternating the divisions. In these examples, I keep dividing only if the colors in the two splits are different enough, so larger blocks of color stay as one region, and where there needs to be more detail, like my eyes, there’s more division.
I even tried making may face out of GIFs for a client project that didn’t happen. I really want to explore this further.
Another thing I experimented with was splitting video into pieces (using brightness or segmentation)
Or even making layers:
I used this technique for a project I did with adidas. This was my first experience with a sponsored animation where I also posted to instagram as I made it — they gave me an image, and I created a 15 second loop on the image. Most everyone was really positive, but my favorite comment was from a musician who I really admire who said, “fuck shoes.” :) It was a good reminder to keep focused on the art. While I do worry sometimes about mixing commercial postings with my sketches, I really feel lucky to have companies paying me to make more of my weird animations. Usually, when you work with advertising agencies they ask you rip someone else off, at least here they are asking me to rip myself off.
I also did a series of face related masks. This year I was hired by both google and facebook to develop face related masks (similar to snapchat type masks) based on my sketch work and so I spent a lot of time thinking about faces and how you can do some weird stuff with them.
Tubes
Sometimes really simple techniques lead to rich areas to explore. One technique I love to demo and use (I did some of this in 2016 also) is drawing tons of stacked circles along a line to make a more dimensional form. At some point during the year, I started to explore this more. I had seen this animation by Zeitguised and this image by Sawdust Studio, and I wanted to jam with these forms again.
Along the way, these tubes got really magical and weird. For example, this ambiguous shape came out of that. To me this animation is one of the highlights of the year — a totally unplanned, accidental discovery that came out of iteration and play.
Augmented Reality
I have been experimenting for a long time with AR (here, for example, is an AR project called Hidden Worlds, which I made almost 16 years ago w/ Golan Levin) so I was pretty excited to see arkit and arcore come to iOS and android devices. I spent a fair amount of time this summer and fall sketching with my partner Molmol in AR and exploring what it means to have the 3d position of your device in space.
After arkit was delivered at WWDC, I saw a bunch of similar looking demos, putting 3d models on a table top, etc. If you saw the 3d content on it’s own you wouldn’t think twice about it. We started with the premise that there should be no outside content but rather that by using the device itself you can create content in 3d.
We did a bunch of camera related experiments such as leaving photos in the space where you took them and slit scans where you paint in space using the pixels of the camera.
We also had a few of our sketches go viral, such as this one where you record sound in 3d space and replay it by moving through it.
Recently, we’ve been exploring face tracking and AR — using 2d data from face trackers like dlib and combining it with 3d data from arkit / arcore.
I’ll try to write some more about AR in a longer piece but I feel like one of the magical parts of AR is that it produces ambiguous images that your brain has to work a little harder to understand.
Final thoughts
Not sure what to say but I am going to keep sketching in 2018. I really love this process. I can’t really say where I want to go, but there are some techniques I’d like to try, such as adding a 3d renderer like octane to get scenes with shadows and lighting and maybe more work with sound. One of the most interesting moments this year was this animation I made that a musician Breakmaster Cyclinder added sound to, it feels like a new world in a way.
At any rate, it’s a little hard to predict where the sketching will go :) That’s what makes it interesting to do.
Last year, I exhibited the sketches at DLD conference, Carnegie Mellon University and Microwave Festival Hong Kong. I feel excited about showing these sketches in other contexts, so maybe I’ll try to do more of that in 2018? Also, I signed with a commercial rep, much presents and I’m excited to see what we can make together.
If you are considering doing something like this daily sketching, my advice is hell yeah, go for it! I was speaking to Beeple, who has done daily sketches for over 10 years now, and his advice was basically to keep it really simple — something you could do in 5 minutes, since there will be days when you only have five minutes. I think this is good advice. Keep it simple, stick with it, fit it to your life.
I thought I’d end with a collection of some of my favorite textures I made this year. There’s no real organization but these images pop out to me as some of my favorites. So much of what I do in the sketches is about motion, so it feels kind of weird to have a write up that just has a ton of images, but I feel like these are really good snapshots of the energy in the sketches in 2017. Curious to see what 2018 will bring.
Finally, big thanks and love to my family, Molmol and River, for being patient with me, all the times I said, “just uploading instagram, one second! ” and for art directing my life. And thanks to you!
This story is published in Noteworthy, where 10,000+ readers come every day to learn about the people & ideas shaping the products we love.
Follow our publication to see more product & design stories featured by the Journal team.
