Painting in Clojure

Learning Clojure by building a digital Jackson Pollock. This article and the source code backing it can be found on GitHub. Below is an example of what we will be building, running the code found in this page.

Jackson Pollock

He was an abstract artist who lived through the first half of the 20th century and is most famous for his drip paintings. This style of painting involves him using sticks brushes, sticks and cans to apply paint to the canvas with the motion of his gestures causing the artworks to come alive. You can get a good idea of how this comes together from this youtube video.

Setting the scene

We want to define some facts about the space that our digital Pollock will work in. These facts will not change over the execution of our model and fit Clojure's preference for immutability perfectly. For those who have not come across the idea of mutability before it is simply whether something can be changed in place. In most languages if you set the label some_number to equal 5, further on you can increment the value of some_number to 6 or even 7. In Clojure if you tried to increment some_number you would get a new value rather than changing some_number.

Clojure will let us define facts using one of the following value types:

A number. This could be 5 an integer, 3/2 a ratio/fraction or 3.14 a floating point number;
A string, represented as a sequence of characters, for example "Hello world!";
A keyword, which are very similar to strings in appearance except they are preceded by a colon e.g. :an-identifier. As alluded to in the example they are usually used for identifiers or labels and do not allow spaces.
A list (...), this is a way of grouping values into a collection with an explicit order. You may notice all of the code written takes for form of lists. By default if you have written (...) Clojure will assume the first item is a function and the rest are arguments to be passed in. In order for the list not to be executed you should prefix it with a ';
A vector [...], which is a lot like a list except that they are optimised for appending to the end of the sequence rather than to the front;
A set #{...}. If you are not particularly bothered by the order of the values stored in your collection then you can use a set;
Lastly there are maps {...}, these store pairs of values where the first is a key and the second is a value.

If you would like to learn more about the basic types in Clojure, I suggest you read this great blog post by Aphyr.

The most important fact about the space is its size. We will use metres to measure the size only converting to pixels when we need to draw to the screen. We are going to define size as a vector containing its width, height and depth.

(def space [8   ;; width
            5   ;; height
            6]) ;; depth

We need to know the gravity of the space so it can influence the flow of paint as it leaves the brush. This will be defined as a vector that represents acceleration influenced by gravity.

(def gravity [0 -9.8 0])

Lastly, we need to know the normal of the surface of the canvas that the paint will impact with. This will be used to dictate how paint acts when it spatters from the impact with the canvas.

(def canvas-normal [0 1 0])

Starting points and projection

Our digital Pollock is going to start a stroke of the brush by picking a random point in space. This point will then be projected to find where it impacts with the canvas.

In order to generate a random point inside of the space we need to define a function that each time it is called will emit a vector containing the position of the point. Function values can be created by calling (fn [...] ...) with the first vector being the arguments the function should receive and any follow items in the list are the body of the function and executed with the arguments bound. Rather than calling (def name (fn ...)) Clojure has provided a shortcut function (defn name [...] ...). An example of a defined function is (defn say-hello [name] (str "Hello " name)), this creates a function called say-hello that when called (say-hello "James") it will return the string "Hello James".

We are going to cover a common functional idiom when dealing with lists to change the dimensions of the space above into a random point inside that space. To do this we want to iterate over each dimension of the size of space, generate a random number between 0 and the magnitude of each dimension and then return the resultant list of random numbers as a list. To generate a random number in Clojure we can use the (rand) function, which will return a random number between 0 (inclusive) and 1 (exclusive). The rand function can take an optional parameter `(rand 100), this will define the bounds of the number generated to 0 and 100.

The function map (map [fn] [sequence]) will iterate through the sequence executing the function with the current value of the sequence as its first parameter, the values returned from the function will be stored in a list the same length as the sequence and returned by the function.

We can now define a random point inside of space as follows

(defn starting-point [] (map rand space))

Now that we can generate a random point in space we want to project this to the canvas. We are going to use Newtonian equations of motion, we know the position, velocity and acceleration of the point and we want to know what the position and velocity are when y is 0. In order to work out final positions we need to know the total time the point spent falling, we can do this using the y position as we know that the final position should be 0.

To work out the time it takes for the point to reach the canvas we will solve the following equation for t:

\(r\) = final displacement,
\(r_{0}\) = initial displacement,
\(v_{0}\) = initial velocity,
\(a\) = acceleration,
\(t\) = time.

\(r = r_{0} + v_{0} * t + \frac{at^2}{2}\)

This rearranges to:

\(at^2 + 2v_{0}t + 2r_{0} - 2r = 0\)

We can solve this using the Quadratic Equation, but this will yield us two results. In general we can say that we are interested in the result with the maximum value.

In the next block of code you can see an example of call out to Java(Script). Clojure doesn't have an in-built square root function, so we are calling out to the Java(Script) version. A function named in the form foo/bar means it will call the function bar in the namespace foo. You might be wondering, what is a namespace?.

All good languages need a way to bundle up code that is related, so that it can be reused and accessed only when needed. Clojure's take on this is to provide namespaces. Every Clojure source file will declare its namespace at the top of the file so that other files can reference it, extract values and use functions. Given that Clojure is a hosted language its namespace will related to packages in Java and Google Closure Library namespaces in Javascript.

When hosted on Java all of java.util.* is automatically imported and on JavaScript assorted core and Google Closure Library modules are imported. Both of these languages provide us with a Math namespace which contains a sqrt function.

If you want to learn more about Clojure -> Java(Script) interop then have a read of this article.

(defn time-to-canvas [position velocity acceleration]
  (let [a acceleration
        b (* 2 velocity)
        c (* 2 position)
        discriminant (- (* b b) (* 4 a c))
        minus-b (- 0 b)
        add-sqrt (/ (+ minus-b (Math/sqrt discriminant)) (* 2 a))
        minus-sqrt (/ (- minus-b (Math/sqrt discriminant)) (* 2 a))]
    (max add-sqrt minus-sqrt)))

We can now calculate the time to impact but we want the final position and velocity. For position we can use the same function that we rearranged above to derive the time.

(defn position-at [time initial-position initial-velocity acceleration]
  (+ initial-position
     (* initial-velocity time)
     (/ (* acceleration time time) 2)))

For velocity we can use another equation of motion:

\(v = at + v_{0}\)

(defn velocity-at [time initial-velocity acceleration]
  (+ (* acceleration time) initial-velocity))

These functions we just implemented can be joined up so that, given an initial position and velocity we can return the final position and velocity. This function doesn't explicitly ask for the acceleration acting on the paint, it assumes only gravity is acting using the constant defined earlier on.

(defn project-point [position velocity]
  (let [[i j k]            position
        [vi vj vk]         velocity
        [ai aj ak]         gravity
        
        time               (time-to-canvas j vj aj)
        
        projected-position [(position-at time i vi ai)
                            0  ;; we don't need to calculate as it
                               ;; should be 0, on the canvas
                            (position-at time k vk ak)]
        
        projected-velocity [(velocity-at time vi ai)
                            (velocity-at time vj aj)
                            (velocity-at time vk ak)]]
    [projected-position
     projected-velocity]))

Paint splatter

An important aspect of Pollocks painting is the splatter of the paint hitting the canvas and what this adds to the images. We are going to add a simple splatter model based of the velocity at impact we calculated in the last part.

Not all paint that hits the canvas will splatter, so we need to work out the impact force of the paint and use this as a cutoff for whether the paint should splatter.

We will work out the impact force of the paint by taking the velocity at impact and calculating the force required to reduce that velocity to 0 over a set impact distance.

(def impact-distance 0.05)

We can now use the work-energy principle (https://en.wikipedia.org/wiki/Work_(physics)#Work_and_energy) to calculate the impact force. On one side of the equation we will have the forces at play and the other the energy:

\(F_{i}\) = impact force,
d = impact distance,
m = mass,
g = gravity,
v = velocity at impact.

\(-F_{i}d + mgd = 0 - \frac{1}{2}mv^2\)

This equation can be rearranged to:

\(F_{i} = mg + \frac{mv^2}{2d}\)

For simplicity of code we are just going to consider the y axis as this is the most important when it comes to working out the impact force of the paint into the canvas. The above equation can therefore be expressed as:

(defn impact-force [mass velocity]
  (let [y-gravity (second gravity)
        y-velocity (second velocity)]
    (+ (* mass y-gravity) (/ (* mass y-velocity y-velocity)
                             (* 2 impact-distance)))))

Based of this function to calculate the impact force we can define a predicate that will tell us whether paint should splatter based off its mass and velocity. It is idiomatic in Clojure to end predicates with a ?. We are going to add some randomness to this function so that we don't necessarily just get a uniform line of points. Also defined is a minimum force for us to consider whether some paint could splatter.

(def min-impact-force-for-splatter 30)

(defn does-impact-splatter? [mass velocity]
  (and (> (impact-force mass velocity) min-impact-force-for-splatter)
       (> (rand) 0.8)))

If an impact splatters then we will need to bounce its velocity vector as this is the direction it will leave its current position.

The equation to bounce a vector, \(V\), off a plane with normal, \(N\), is:

\(N\) is the normal vector of the plane
\(V\) = the incoming vector
\(B\) is the outgoing, bounced, vector

\(B = V - (2 * (V.N) * N)\)

You can find out a bit more about the derivation on this Wolfram page.

We are missing a few of the required vector operations used in this equation so we should define some more functions before trying to implement it. The first is the vector dot product, this is defined as the sum of the multiples of each dimension. Otherwise we need subtraction of two vectors and a function to multiply a vector by a constant.

(defn dot-product [vector1 vector2]
  (reduce + (map * vector1 vector2)))

(defn vector-subtraction [vector1 vector2]
  (map - vector1 vector2))

This function will introduce a shorthand for defining functions that is very useful in combination with functions like map and reduce. Rather than writing (fn [args...] body) you can use #(body) and if you want access to the arguments use %n where n is the position of the argument. If you are only expecting one argument then you can use just % on its own.

(defn vector-multiply-by-constant [vector constant]
  (map #(* % constant) vector))

Using the above functions we can now implement the vector bouncing equation. I have pulled \((2 * (V.N) * N)\) out into a variable called extreme for clarity.

(defn bounce-vector [vector normal]
  (let [vector-dot-normal (dot-product vector normal)
        extreme (vector-multiply-by-constant normal (* 2 vector-dot-normal))]
    (vector-subtraction vector extreme)))

When an impact splatters it will only take a fraction of the velocity, otherwise know as being an inelastic rather than elastic collision. We can define a constant that will be used to reduce the total velocity of the bounced vector to reflect this elasticity.

(def splatter-dampening-constant 0.7)

(defn splatter-vector [velocity]
  (let [bounced-vector (bounce-vector velocity canvas-normal)]
    (vector-multiply-by-constant bounced-vector
                                 splatter-dampening-constant)))

Paths vs Points

All of the gestures Pollock makes are fluid paths, even if the velocity along the path might be rather erratic. We now need to work out how to generate a path of points that we can then use the code we have written above to project and splatter.

A Bezier curve is a commonly used curve for generating a smooth curve that can be scaled indefinitely allowing us to have as many points along our path as we care to calculate.

Bezier curves are defined by an list of control points, so we need to be able to generate a potential unbounded list of random control points that should give use limitless different paths to paint with.

In order to generate a list of control points we will need to be able to:

get a random number between two points for distance and steps,
get a random unit vector for the initial direction of the generation,
add vectors together to move between our control points.

(defn random-between [lower-bound upper-bound]
  (+ lower-bound (rand (- upper-bound lower-bound))))

Below is an algorithm that will give well distributed random unit vectors. It was ported from code found in GameDev the forums.

(defn random-unit-vector []
  (let [asimuth (* (rand) 2 Math/PI)
        k (- (rand 2) 1)
        a (Math/sqrt (- 1 (* k k)))
        i (* (Math/cos asimuth) a)
        j (* (Math/sin asimuth) a)]
    [i j k]))

(defn vector-add [vector1 vector2]
  (map + vector1 vector2))

Now that we have a random direction in which to move we need to generate an unbounded path that will move in that direction, but randomise the position of each point within provided bounds.

Firstly, we can define a function that will generate a random vector inside of lower and upper bounds that can be combined with the non-randomised position to provide a randomised path.

(defn random-vector-between [lower upper]
  [(random-between lower upper)
   (random-between lower upper)
   (random-between lower upper)])

In order to provide an unbounded path we can use a lazy sequence. This function returns a value that is somewhat akin to list that never ends. Every time you try to look at the next value in the list it will generate one just in time for you to see no end.

In this function the first value returned should always be the initial starting position, each following value should be a step along the path. You can see this below, it returns the position argument cons'd with another iteration of random-path with the position randomised.

(defn random-path [position step-vector bounds]
  (cons position
        (lazy-seq (random-path (vector-add (vector-add position step-vector)
                                           (random-vector-between (- 0 bounds) bounds))
                               step-vector bounds))))

We can now use this random-path lazy sequence to generate a list of control points given an initial starting point and some bounding variables. The distance, step and variation allow us to request long winding paths or short flicks.

(defn control-points [position min-distance max-distance min-steps max-steps variation]
  (let [direction       (random-unit-vector)
        distance        (random-between min-distance max-distance)
        steps           (random-between min-steps max-steps)
        step-vector     (vector-multiply-by-constant direction (/ distance steps))
        random-positions (take steps (random-path position step-vector variation))
        end-position    (vector-add position
                                    (vector-multiply-by-constant step-vector steps))]
    (conj (vec random-positions) end-position)))

In order to turn this list of control points into a list of points that represent a path we need an algorithm. The most commonly used is a recursive algorithm proved by De Casteljau. There is a great video on YouTube explaining this algorithm that I recommend you watch.

At the core of the algorithm is an equation that will return a point along a line weighted by a variable, \(t\) which dictates how close it is to each end of the line:

\(P = (1 - t)P_{0} + tP_{1}\)

For example, if a line runs from \(P_{0}\) to \(P_{1}\) and \(t\) is 0 then the outputted point with be equal to \(P_{0}\) and if it is 1 then \(P_{1}\).

De Casteljau's algorithm recursively creates a new set of points by using the above equation for a fixed \(t\) against all the lines created by the control points. It does this until there is just a single point, this is a point on the bezier curve. It \(t\) from 0 to 1 and for each step gets a point along the curve.

(defn recur-relation [t a b]
  (+ (* t b) (* a (- 1 t))))

(defn for-component [t component-vals]
  (if (= (count component-vals) 1)
    (first component-vals)
    (for-component t
      (map #(recur-relation t %1 %2) component-vals (rest component-vals)))))

(defn for-t [t components]
  (map #(for-component t %) components))

(defn de-casteljau [control-points step-amount]
  (let [x-vals (map first control-points)
        y-vals (map second control-points)
        z-vals (map #(nth % 2) control-points)
        points (map #(for-t % [x-vals y-vals z-vals]) (range 0 1 step-amount))]
    points))

This can generate paths that go below the canvas, we should set these to 0 as it is the equivalent of painting on the canvas

(defn ensure-above-canvas [path]
  (map (fn [[i j k]] [i (if (< j 0) 0 j) k]) path))

Motion, going through the paces

All the points along the generated path should have an associated velocity. To start with we can generate a linear velocity along the path, given a randomised total time to traverse the path and the total length of the path.

In order to calculate the length of the paths, we will want to do something similar to a map but with pairs of values. Using this we can take two points, calculate the distance between them and then sum all the distances

(defn map-2 [f coll]
  (when-let [s (seq coll)]
    (let [s1 (first s)
          s2 (second s)]
      (if (not (nil? s2))
        (cons (f (first s) (second s)) (map-2 f (rest s)))))))

in order to find the distance between two points we need subtract the two vectors, square and sum the resultant dimensions and then take the root. (https://en.wikipedia.org/wiki/Euclidean_distance)

(defn vector-multiply [vector1 vector2]
  (map * vector1 vector2))

(defn distance-between-points [point1 point2]
  (let [difference-vector (vector-subtraction point1 point2)
        summed-vector (reduce + (vector-multiply difference-vector difference-vector))]
    (Math/sqrt summed-vector)))

(defn path-length [path]
  (reduce + (map-2 distance-between-points path)))


(defn vector-divide-by-const [vector const]
  (map #(/ % const) vector))

(defn velocity-between [point1 point2 total-time total-distance]
  (let [difference-vector (vector-subtraction point1 point2)
        time-between (* total-time (/ (distance-between-points point1 point2)
                                      total-distance))]
    (vector-divide-by-const difference-vector time-between)))

This calculation will leave off the last points velocity, we can just set it to 0

(defn path-velocities [path total-time]
  (let [total-distance   (path-length path)
        number-of-points (count path)]
    (conj (vec (map-2 #(velocity-between %1 %2
                                         total-time
                                         total-distance)
                      path))
          [0 0 0])))

As well as the velocity at each point along the path, we also need how much paint there is falling. Again to keep life simple we are going to model this as a linear flow along the path with there always being no paint left.

(defn path-masses [path initial-mass]
  (let [number-of-points (count path)
        step (- 0 (/ initial-mass number-of-points))]
    (take number-of-points (range initial-mass 0 step))))

Putting it all together

I've pulled a bunch of colours that Pollock used in his seminal work "Number 8" so that each flick of paint can be rendered in a random colour out of this palette

(def canvas-colour [142 141 93])

(def paint-colours
  [[232 51 1]
   [248 179 10]
   [247 239 189]
   [29 16 8]])

(defn pick-a-colour []
  (nth paint-colours (rand-int (count paint-colours))))

Now we need to assemble all of the above functions into something the resembles Jackson Pollock applying paint to a canvas. We start with a point, project a path, work out masses and velocities, project and then splatter. This is all then packaged up with a colour for drawing onto our canvas.

(defn fling-paint []
  (let [position       (starting-point)
        total-time     (random-between 1 5)
        path           (ensure-above-canvas (de-casteljau (control-points position 0.1 2 3 15 0.4) 0.01))
        velocities     (path-velocities path total-time)
        masses         (path-masses path (random-between 0.1 1))
        projected-path (map #(project-point %1 %2) path velocities)
        splatter       (map (fn [[position velocity] mass]
                              (if (does-impact-splatter? mass velocity)
                                [position (splatter-vector velocity) (* mass splatter-dampening-constant)]
                                nil))
                            projected-path masses)
        projected-splatter (map (fn [[position velocity mass :as point]]
                                  (if (nil? point)
                                    nil
                                    (conj (vec (project-point position velocity)) mass)))
                                splatter)]
    {:colour (pick-a-colour)
     :air-path path
     :canvas-path (map #(conj %1 %2) projected-path masses)
     :splatter (filter #(not-any? nil? %) (partition-by nil? projected-splatter))}))

Rendering the canvas

We need to know the available size for the outputted image to fit in. To work this out we are going to have to interface with JavaScript directly. Luckily ClojureScript makes this very easy using the js namespace.

(def image-width (.-clientWidth (.querySelector js/document "#pollock")))

Now we have the width of the image we can use the dimensions of the space to work out the pixel size of the image and how to convert between metres and pixels.

(def pixels-in-a-metre
  (let [[width _ _] space]
    (/ image-width width)))

(defn metres-to-pixels [metres]
  (Math/floor (* metres pixels-in-a-metre)))

We can now use this function to work out the size the sketch should be and how to convert a position in metres over to a position to be drawn in the image.

(def sketch-size
  (let [[width _ height] space]
    [(metres-to-pixels width)
     (metres-to-pixels height)]))

(defn position-to-pixel [[i j k]]
  [(metres-to-pixels i)
   (metres-to-pixels k)])

Now the dimensions of the image our calculated we can use Quil to define the sketch that we will draw into. We also need to define a function that will initialise the image into the state we want it. This function will be run when the sketch is defined.

(defn setup-image []
  (apply q/background canvas-colour)
  (q/fill 0))

(q/defsketch pollock
  :setup setup-image
  :host "pollock"     ;; the id of the <canvas> element
  :size sketch-size)

To draw the trails of paint across the canvas we need draw a path following the defined positions, which takes into account the amount of paint at each position and uses this to set with width of the path. In order to do this cleanly in Quil we need to consider the path as pairs of positions that we shall draw paths between using the initial paint amount as the stroke-weight. This allows for a smooth decrease in the width of the path.

(defn mass-to-weight [mass]
  (* 50 mass))

(defn draw-path [path]
  (doall
   (map-2 (fn [[position1 _ mass] [position2 _ _]]
            (q/stroke-weight (mass-to-weight mass))
            (apply q/line (concat (position-to-pixel position1) (position-to-pixel position2))))
          path)))

For splatter we are just going to draw a point that has a stroke-weight proportional to the amount of paint.

(defn draw-splats [path]
  (doall (map (fn [[position _ mass]]
                (q/stroke-weight (mass-to-weight mass))
                (apply q/point (position-to-pixel position)))
              path)))

Now that we can render the result of flinging some paint around we need a function that will fling the paint and render the result.

(defn fling-and-render [& any]
  (q/with-sketch (q/get-sketch-by-id "pollock")
    (let [{:keys [colour canvas-path splatter]} (fling-paint)]
      (q/stroke (apply q/color colour))
      (draw-path canvas-path)
      (doall (map draw-splats splatter)))))

Lastly, we shall attach to the buttons and cause our image to come to life.

(.addEventListener (.querySelector js/document "#add")
                   "click"
                   fling-and-render)

(def interval-ref (atom nil))
(def fill-count (atom 0))
(.addEventListener (.querySelector js/document "#fill")
                   "click"
                   (fn [e]
                     (reset! interval-ref
                             (js/setInterval (fn []
                                               (if (> @fill-count 500)
                                                 (js/clearInterval @interval-ref)
                                                 (do (fling-and-render) (swap! fill-count inc)))) 100))))