Daily Sketches in 2016
(note: this article has a fair number of images and animated gifs. Just an FYI if you are on mobile or slower connection.)
In 2016 I started an experiment of doing daily sketches in the form of short animations posted on instagram. In these sketches I tried out different visual ideas involving geometry, animation, gesture and graphic form. I had no idea what to expect but it felt like a good new years resolution and nice way to experiment with some ideas I had been thinking about. I was inspired by folks like Reza Ali who were doing daily sketches at the time and thought, I should do this.
My step-daughter was having trouble falling asleep by herself at the time and so I would hang out in her room, read some books and then as she tried to sleep, I’d code a sketch and in the morning show it to her. At the beginning of the year she was really positive, “that’s hypnotizing me!” but after a while she started to offer more criticism, “you should try color” or “that looks a little too crazy to me” She pushed me in different directions. I was happy to show her that art making is daily work and small discoveries.
I thought I would take some time and write down things that I discovered and played with throughout the year. I’ll start with my process of producing the videos, my approach and talk about the areas I explored.
Process
One of the things I wanted was to make sure that posting these sketches was as simple and painless as possible. Most of my sketches are coded in openFrameworks (using Xcode) and a few use paper.js. My workflow is really simple:
- use quicktime screen recorder to grab a square recording, usually larger than the 640x640 target resolution.
- trim the quicktime recording to a good length (usually less then 60 seconds) and save out the file.
- use ffmpeg to covert the file. I use the following command line:
ffmpeg -i INPUT.mov -crf 10 -vf “scale=640:640, setpts=1.0*PTS” -c:a copy -tune grain OUTPUT.mov
note, I change the input and output files names. I also adjust the CRF value to try to keep the bitrate around 4000–5000 kbits. Also the 1.0 in setpts= line is for speed change, if I want to make the movie faster I adjust this.
- I use airdrop to copy the video over to my phone.
Finally I have one folder on my hard drive called “everyday”. It’s about 100 GB now, and basically any screenshot or video of the sketching goes there. It’s not very well organized but it’s really useful — if I sort it chronologically, I can see my progression really clearly and even retrace my steps if need be.
As a side note, the timeline of many of my movies look pretty cool :)
Approach
My general approach is to settle on an idea for anywhere from a week to a month and to push that idea in different directions. I usually try to pose a problem “how would I attach things to the contour of a body as it moves?”, “how would I make this blob turn into a letter”, and the sketches are my attempt at solution. Sometimes the sketches show me working through the idea, sometimes they push what can happen with the solution.
The sketches are the opposite of work to me, so I try to be totally un-work like. I don’t use github, I don’t keep code clean, I just make and record without thinking very carefully about anything. I sketch up until the point I think it’s interesting, record it, post it and clock out. It’s the opposite of how I approach commercial work. When I sketch I want to work as messy and mindlessly as possible — I don’t plan, I just see where the wind blows.
I try to work for about 30 minutes on a sketch. It doesn’t always work that way, but in general I try to keep the time involvement as minimal as possible — I have other work to do, etc and I want this to be for fun more than anything else. Sometimes if I have 15 minutes free here or there, I might open up the last sketch again and try to make a variation of push things in a different direction. With shaders, small changes can have a huge effect, so I often times would change even just a few variables and have a new sketch. Lately I’ve taken to using instagram stories and doing screen recording as I work as a way of sharing those different changes.
Folks ask me if I open source this work. At the moment, I haven’t, mostly because these sketches feel like they are in early states and I don’t yet know what they will become. However, when people ask technical questions, I do my best to answer them and explain exactly what I am doing and I also make myself available once a week for open office hours. I am still honestly struggling with this — I am a strong believer in open source but I don’t know the best way to integrate this into sketching without it changing the nature of what I do or me feeling un-afraid to share ideas I don’t fully understand yet. It’s an open question and as I head into 2017 I may revisit this.
Once I post a sketch, I find it’s useful to monitor how it was received. I am curious about the like to view ratio, the comments people leave and I find it helps gauge how other people see my work. It’s interesting to see when I have an idea that I really love that doesn’t connect with people or when there’s something people pick up on that I don’t think as highly of. A lot of times, as an artist, I feel like we’re struggling to find our frequencies and what resonates with the frequencies of the world. This act of sketching is a kind of tuning of these frequencies, trying things that are more harmonious, trying things which are more discordant and generally getting a feel for how others see your work.
These sketches are like diary entries. When I think back to a sketch, I sometimes remember the specific day, where I was, what the temperature was like, what I was thinking about at the time. I get nostalgic just poking through them. They mark my moods and capture all kinds of things in them — it’s really surprising.
Finally, I had a set of simple rules:
- avoid over-used algorithms like delauny triangulation or reaction diffusion in favor of more experimental things.
- avoid random (and to a lesser sense, noise) as much as possible in favor of using mathmatical functions like sin() and tan()
- find sources of inspiration and try to re-code them (for example, I saw this and made this which later led to this)
- don’t take requests or feedback too seriously, just try to develop my own personal intuition. My response to requests was always, “dj doesn’t take requests”
- don’t be afraid to repeat myself or tweak a sketch and repost it
- post the first thing I make that is good enough to share
- surprise myself
Reflection
My initial sketches were all around reflection — what it looks like when light bounces off of surfaces, reflects and refracts, and the kinds of forms that emerge.
Last December, I had made an interactive project called reflection studies where I invited people to play with light interactively using laser cut shapes. I was inspired by Micah Elizabeth Scott’s Zen Photon Garden and I was wondering what an interactive version of a light simulation would look like.
My first instagram sketch had a new years greeting for people. I used the rotating light to line up with walls to reflect off to make out words. To create the text, I use a hershey font and calculated where to put walls to make the lines of the text. I really like how the message resolves:
One of the nice things about starting with reflection for the sketching is that I had a very limited number of choices to make for each sketch. I had to decide where the light was coming from and where the walls for the light to bounce off go. That’s it.
Another nice challenge is that line drawing animations compress terribly, so I kind of cut my teeth with the some of the worst possible material and later as I moved to other topics I had a much better sense of how to manage compression.
Finally, when I started the year instagram had a 15 second limit on videos. I tend to use sin(time) in my animations and the nice thing about that is you can do one or two full cycles and a lot of my early animations are just simple loops.
Later I looked at refraction — what happens if the light bends as it hits a wall — and even added colored light. I moved from a hand crafted solution for fast light simulation (which used a raster image / look up tables, etc) to a more mathmatically precise method involving bounding view hierarchies (a technique used for ray-tracing)
I took these sketches and worked them into an update on the interactive version of this project, which I took to Panorama Music Festival and Signal Festival in Prague. I really enjoy the interactive version of this project, because to me it feels like people are basically playing with my sketchbook — the current version of the software has 12 different modes which each come from this daily sketching.
One thing I experimented with that I found really promising is light interacting with the human body. I’d like to do live full body experiments with this
One of the things I like about the light reflection is that it got me excited about very simple geometry — things like squares and triangles felt really different when they are part of light simulation then if you just draw them with solid pixels. For example, here are a set of horizontal and vertical lines, but when we bounce light off of them, there’s a lot of kinectic energy that’s revealed.
For example, here’s a simple sketch that has a grid of triangles — I like how weird and organic forms emerge from this pattern and repetition.
Blobs
At some point after a period of sketching light I got tired of the lack of color. I moved to blobs. A blob is a cell, a closed form, that moves in an organic and lifelike way.
I remember many years ago seeing processing sketches from Ryan Alexander and being amazed at how these forms he made in processing were moving, and through trial and error developed my own approach which is fairly straightforward:
- connect particles with springs (and connect the last particle to the first particle, making a loop)
- give the particles a repulsion force from each other that is proportional to distance (stronger force when closer) and has a maximum radius for interaction (i.e., if particles are beyond this distance, they don’t have an impact on each other)
Because these blobs are based on physics, they have natural and pleasing motion. For colors I just chose a background color because of the name “peachpuff” and experimented with things that felt right.
I did a lot of experiments with shading — these blobs aren’t 3d but rather, triangulated meshes, and I color the vertices based on the distance to the closest edge and the angle of the closest edge. This gives it 3d feel, but without being 3d. This idea of making things look 3d but not really 3d comes back again and again through my sketches.
I tried thing like putting images inside blobs and distorting the image based on how the blob moves:
There were some interesting discoveries, such as this sketch which connected two blobs to each other at the closest point (i.e., the points on blob A search for the closest points on blob B). I made this sketch almost on a whim, since something I was attempting that was hard wasn’t working, and people really seemed to really like it.
When I saw that people liked, I was wondering, what is special about this? I felt like it might be the use of negative space, and so I started to explore that. Usually, most computer graphics are accretive (i.e. layering) and it seemed interesting to try to make visuals where the shapes you draw reveal worlds below. These sketches use Chris Baker’s ofxClipper heavily.
Letters
I tried making the blobs turn into letters, and even exploring what it would be like to push them with my hands. To turn a blob into letters, it involves adding angle forces, so particles feel agular turning forces (if you think about each set of 3 particles as a hinge in the middle, where the middle particles is the pivot and the two particles on the outside have forces that open or close the hinge to change the angle to match the letter).
Offsets
I looked into offsetting these blobs. I got inspired by the visual language around the 1968 Olympics in mexico and tried to figure out how to take polygons and offset them so that they grow larger and larger.
Jose Luis Ortiz, Jan Stornfeld
First I worked on expanding the polygon based on normals — but it messed up really quickly and led to much self intersection. There’s a mathmatical way of doing this (minowski sums) but I couldn’t find a way to do many offsets fast. When I mentioned this problem to Golan Levin, he connected me with Bryce Summers, whose ofxScribbleSegmenter helped solve for self intersection of those polygon offsets and I was able to do offsets pretty fast:
These offsets proved really interesting — Sometimes I would stumble into looks that felt kind of 1960’s design
Later I tried things like straight skeleton and distance images to get similar results — the straight skeleton is implemented in CGAL and I also used Stalgo. With the distance image (example usage here), I used opencv and thresholded at different levels to get inner or outer contours.
At some point, I tried to make the blob 3d. My first attempt was to replicate the same physics in 3d. I took a primitive shape called an icoSphere (a sphere made out of triangles) and made each vertex of the icoSphere a particle, connected these particles by springs essentially recreating the icosphere’s mesh, and used the same repulsion forces as the original 2d blobs . This created some interesting and weird shapes.
I tried things like slicing the 3d form into layers or having energy travel along nearby vertices.
Another approach to 3d was try to implement a kind of pseudo 3d — basically inflating the 2d shape based on it’s distance from the edge. I was inspired by a project called Teddy, an experimental interface for making 2d drawings 3d. it produced interesting looks as well. My initial experiments looked more like baked cookies than real 3d :)
later I found a project called openTeddy and with that code, was able to integrate the ideas from the original Teddy paper into my sketches. The result were things that looked really curious and fascinating — they were much more controllable forms as well since the basis of the form was 2d.
Since they come from 2d, I could also use things like the contour of a body or hand and even make it 3d — it’s great for making a michelin man:
Circles
I made the blob another way — instead of connecting particles, I tried connecting just circles (i.e., particles with a huge radius) and using these circles with larger radii, tracing around the circles to make the form.
This lead to all kinds of weird and organic shapes that moved the same as the previous blobs but look really different.
I found you could get interesting looks if you extended the shapes out, in a kind of pseudo 3d. I didn’t use 3d, I just drew in 2d and used line line intersection and just drew lines and stopped them if they intersect the another line. This way of rending is kind of ridiculous, but it produces a very unique look.
In particular, this one really looked like a sea creature / Ernst Haeckel illustration:
Later I tried the same thing but using real 3d forms:
Another thing I experimented with was placing adjacent circles across a horizontal line, and tracing a them (i.e., following the top of one circle and connecting it to the bottom of the next circle) to make an alternating shape that felt like a kind of graphical wave. As you stack them, this produces 3d-like forms but it’s just layers of waves that come from circles.
Circle Fitting
One of the interesting properties of circles is that you only need three points to define a circle. If the points are co-linear, the circle has an infinite radius. I found a small snippet of code to solve for the position and radius of the circle based on these three points and I did a variety of sketches based on that.
I used this approach for other sketches — I like how you you can see / find circles in anything.
Later in the year, I discovered Dubin’s curve, a technique for path planning which can connect two points with a start and end angle via a set of circles — it’s used for example to steer elements that can only turn a certain amount.
For years I have been obsessed with Juerg Lehni’s Hektor robot and the visual language around it but I didn’t know how to make smooth motion planning between segments. Dubin’s curves gave me some avenues to explore.
Body
As I was working on the blobby shapes, I kept thinking about the body as a shape and how I could use it in my sketches. I started first with hand shapes, but then moved into bodies. I found two movies from this page Theo Watson put together with “test” opencv movies. These two videos became my “Lenna” — I used these contours for a ton of different sketches. One nice thing about always using the same video was that I got to know it really well and understand the differences in my visuals better since the input was always the same.
I tried different things on this shape, like polygon offsets:
One of the problems I wanted to solve is attaching elements to the contour across multiple frames. When you use openCv to turn a blob into a polyline, it can return a variable number of points every frame and the first point changes from frame to frame. This makes it hard to attach anything to the contour because it shifts so much from frame to frame. My solution was to resample the contour so it always has the same number of points every frame, and then to compare the current frames contour with the previous contour to figure where to put the zero position (start) of the polyline. One way to visualize this is two rubber bands with a single dark mark on each one, I rotate one until the mark matches with the other ones mark.
I was inspired by things like Nick Cave’s sound suits, and Oskar Schlemmer’s costumes, and I started designing a series of costumes — elements attached to and moving around the body.
Sometimes my sketches with the body came from really simple mechanism — For example, I was inpired by these stacked circles from Keetra Dean Dixon and JK Keller.
Inspired by this, I took the contour and looked at where horizontal lines would intersect it (using line line intersection across all the lines in the contour). Any line moving aross the contour will intersect in some multiple of 2 times, 0 if it’s below or above the body, 2 times on the shoulders for example, 4 times on the legs in the left two pictures, etc. I took those intersections and turned each set of 2 into circles, which I rotated in 3d slightly. The result is a body which looked like a stack of circles:
It looked interesting when you rotate around it — I love it when the body breaks apart and reforms:
Another thing I really fell in love with is turning the body into 3d forms by simple extruding or rotating the points like a lathe:
I also explored 3d point sets. You can find BVH motion capture data online, and it’s pretty easy to load this into openFrameworks. I found a walk cycle and on the anniversary of my father’s death it seemed really appropriate to use this solitary walking figure as a way to explore my feelings — I was unsure how to process them. I was feeling really alone those days and this solitary set of 16 points walking really resonated with me.
Distance
As I was experimenting with the using the distance images from openCV to make polygon offsets, I discovered a whole field of work around signed distance fields which got me intrigued with using shaders for computing distances. I could pass in a set of points, and the shader would compute for every pixel the distance to all the points and color that pixel based on inverse proportionality. As I experimented with things like coefficients or raising the distance to a power, I started to see shapes go in and out of focus
I was fascinated in particular with the kind of milky light you could get — here I took lines from the body contour and passed them to the shader
If you take the same distance formula but you use sin or tan with it as input in the shader, you can get some interesting results. Most of these are shader explorations and there’s something I find deeply meditative about programming shaders vs traditional coding. It’s more improvisational and it feels quite close to making math visible. I’ve been really inspired by Patricio Gonzalez Vivo and Jen Lowe’s Book of Shaders and I found it really helpful as a reference for these explorations
Collaborations
Probably one of the most exciting aspects of doing the sketching has been the collaborations that have emerged. I thought I’d discuss a few of them.
I was approached by the New York Times this summer to use a variation on one of the sketches I made which stretched typography. In this sketch I examining letter forms and found good “slice points” where I could cut the letter and extend the slice I’ve made, distorting (but not breaking) the letter form.
We use this to illustrate a story about the influences on the maltese language.
In the fall, South Bank Center and the publishers of Margaret Atwood’s new novel Hag-Seed reached out and asked if I’d be interested in making interactive animations based on the book and I jumped at the opportunity. The book involves a solitary figure who comes from the theatre world and for me it seemed really fitting to make animations that involve the body and typography.
One of the highlights of the year was watching Margaret Atwood dance with it at the South Bank Center, seeing her own words wrap around her silhouette.
In addition, I spent this fall as an “artist in residence” as the school for poetic computation, a school in NYC I help run, and while there, I collaborated with two artists I really admire.
The first was with David Mcleod, a phenomenal artist who primarily uses 3d tools to make hypnotic animation and forms. We experimented with taking the circular forms I had been experimenting with and I built a tool that exports them from openframeworks to alembic (using the amazing add-on from Satoru Higa, ofxAlembic) so he can open them in c4d.
Then David worked his magic on these forms and the result are really stunning.
Shantell Martin is an incredible artist who I’ve collaborated with before. For this experiment, Shantell made a set of drawings and I tried to figure out a good way to animate from one drawing to another. We used Kyle McDonald’s ofxAssignment to figure out of the best match of line segments from one drawing to another:
and then used dubins curves to make the movement as seamless as possible. I had always wondered how to do an animation like this (i.e., how to have the lines of one drawing smoothly move to the lines of the next) and it was lovely to be able to see it happen.
Towards the end of the year, I had the chance to work with Daito Manabe / Rhizomatiks, for a project that was sponsored by Volvo — we used their life paint, a retroreflective spray paint, and made costumes that he and his partner wore on stage and I wrote software to track them — finding the lines in the costume and writing software that animated those, responding to the music.
We also took similar software and tried it with a group of dancers from Elevenplay. One of my favorite sketches was this one which fits circles between the dancers as they move.
Finally, I’ve been working with collaborating with my wife and partner Molmol to create costumes based on her sketches. We take the contour of a body from the kinect and find ways of attaching and manipulating hand drawn shapes which are attached. One of my favorite sketches we made together as a family.
Conclusion
After all this, I guess I want to say there is no conclusion. I’m quite glad I did this in 2016, thankful for the feedback and support I got, and I am excited to sketch in 2017. If you aren’t already sketching, come join me!
