(techno music) - Thanks, John.
Actually, John we met in 2010.
When I was in University of Sydney that was 2013. So it's been a while.
Before I begin my talk, I do want to give a shout out to the web director's What Do You Know programme.
So if you haven't done a presentation before, or this is your first time presenting and you want to do a quick four or five minutes talk, that's actually a really good venue to send your talks to as well as other meet-ups. So the reason I'm standing here today is because I was lucky enough to get into a What Do You Know talk and eventually kind of morphed our five minute lightning talk to a 20 minute format and submit it to the call for paper, if I'm right.
So definitely, I really recommend it.
The speaker workshop is really great as well, so I wanted to vouch for that first.
So today I'm going to be telling you about programming generative arts.
My name is Xavier Ho, I work for Data 61.
My specialisation is data visualisation.
A lot of my work is kind of like not this.
But in a way, when I take work I try to think of things visually.
When you are doing engineering, software engineering perhaps, you might be thinking about Agile, you might be thinking about functional requirements, what are the acceptance tests that we have to make, to make sure the product is what it says. Through doing all of that, you know, those things like agile processes and requirements and things like that they don't really talk about how do we communicate to the client that this work is going to be helping them in some way. That's kind of the shift, okay.
We're kind of doing this work to say we're developing these softwares and these interfaces so that by the time they get to use it, they see what it is. And through the process of interacting with the websites, they understand that this actually does what they need it to do.
My process has always been very visual.
I come from kind of a weird hybrid between software engineering and film, animation background. The reason I got into programming actually was because I noticed in a bunch of film softwares back in the day, like Maya and and other things, they supported Python as a scripting language. And when you are doing film, it's kind of like I'm keying the 100th green screen today in the same film. And you have to do it, right? If you have to insert a CG character, surveying post, some kind of things, you have no choice. But to re-do the different shots of the same scene over and over again, so it will look consistent.
So I got tired of that and repetition is something that software is really good at doing. We're really good at making sure we specify a path, a process, a workflow.
And then you say that given my input such as the place of the green screens, and then these are the outputs what I expect, which is a consistently matched, narratively driven shot of something.
I started to pick up programming as a means to kind of save my time and not be bored down by doing the same thing over and over again.
I eventually decided to take a software engineering degree and I still from time to time go back and do hobby stuff like this.
The stuff I'm going to be telling you today, while it's not entirely practical, hopefully will make your websites a little bit more interesting, and take-away the tools that I'm going to teach you. Something I emphasise in pretty much all the talks I give, is that programming is a creative thing.
People often think of coming to science as like a very logical thing.
People think of STEM as like a very rigid, you can derive one thing to the other.
Everything is deterministic in some way.
But we're solving problems.
We're trying to make sure that what ever the problem is, we're coming up with brilliant, different, unique solutions that's going to solve that problem.
And the problem's solution will always come from the problem itself.
You might have to take something that's quite left field, out of the box, into the solution space so that you're going to have a solution, right.
So I hope that this thing will stay with you. And then I think that people often think about when it comes to programming and learning about generative art or other things like live coding, musics, creative coding, all these kind of crazy, funky things. Generative design, architecture, if you're into that kind of thing.
They look at it and go, "Wow that's kind of crazy. How did you even come up with that?" That pattern is really interesting though.
What the hell.
And it's just like programming.
When you first jump into Javascript land or when you first jump into CSS land you're like "what the hell is all these modules, import, export.
Why do we have to download a million dependencies before I can even start? It's exactly the same thing in generative art, it's exactly the same with learning a different discipline. There is a lot of existing knowledge and tools that people have come up with, documented, written about it on media and blog posts, in tutorials and books that hopefully I'll touch on a little bit today. Really the goal is to take that knowledge and tools and then go into this uncertain land where we said the possibilities that we can really take into and create something that we might want to do.
I want to kind of demystify generative art. It's not really something that people experiment with in the basement for like 10 years and suddenly they have this amazing thing: it's trial and error. So a little bit of digression.
This bottle came from a Siggraph paper which was a computer graphics conference in 2014, in which they came up with a way to generatively manufacture glasses that would give you this caustic effect.
And if you don't know what caustics are it's just that little light that shines in the back. If you have glass of water, you put it down, there's a spot light behind it.
If there's a wall or a table on the other side of the glass, there's going to be a focal point, at some point, in which light will converge into a brighter point.
But often those glasses are kind of just smooth and they're straight.
So the caustics don't really look that interesting. Well these researchers have come up with a way that you can print and manufacture, whatever the shape it is that you want given various brightness of light fields.
So you can imagine them as pixels.
But not in the discrete pixels but like kind of blurry kind of form.
And when I see stuff like this one of the things I like to do is pick it apart. I want to think "How did it do this?" It's kind of when like you're a film director, you're trying to figure out what are the cinematic tools that I have used here.
When you're a writer, when you're writing a novel what are the different mechanics that they have put into this writing so that you can follow the story along? Well, luckily given that Siggraph is a conference that publishes papers.
So one of the things that it came up with is the Verint diagram, in which they say given the normal uniform distribution of these lights, and given the target distribution, in this case Einstein's face, they can rearrange the dots in the Verint form so that there's more lights going into those brighter spots and there's less lights going to the black spots. Therefore, you get a pattern at the end of the day. And it's a very generalised, probably overly sympathised way to talk about this algorithm. But really they just made sure that the glass is carved in some way so that you will get this concentration.
And that's kind of the mindset I go into when I look at other people's work. I try to pick it apart mentally.
How does it work? So when I look at different works, I also relate to what I already know.
So you may have recently noticed that Michael Bullstock started a company called Observables, in which you can do kind of tributary path of notebook, but in Javascript style and along with D3 and other things to have a life, a documentation and perhaps description of algorithms that you want to show. So in this case, he's done something that's called a Vernway-Sleepling algorithm in which case you just kind of do the dots, and when the dots are further apart it appears to be whiter.
And when I look at it, I'm like it's really similar, right? But instead of redistributing the Verint diagrams I'm just distributing the density of points such that when there is a darker area there are more points around.
So you can also think of things as related tools that you can kind of start to step into the realm of the algorithms that you might want to explore.
So today I want to bring you just two different tools that I find personally very interesting.
I play with them quite a bit and I will summarise them at the end and hopefully answer some questions.
So the first thing I want to talk about is the technique that came up by Anders Hoff who is the head of Inconvergent and he coined it "Shepherding Random Numbers." Second thing I will be talking about is the very ancient, from the '60s thing called L-Systems, but kind of a bit of a twist so that it's generative but it's going to mutate, it's going to change.
And finally there's going to be a summary.
So, let's get right into it.
When I think of drawing something, let's say we have a random number.
So, I've got a circle, well, I'm going to draw a circle over and over again, and the circle has a fixed radius and position. But instead of drawing them in the same spot at the same time, I'm going to apply a random off-set to the X and Y of the circle, and in this case, I'm going to be using pseudo-code for pretty much the rest of the slides.
Naturally when you apply a random value to the X and Y, you're going to get a slight offset. So when you draw them over and over again you're going to get a slightly noisy circle. That looks kind of interesting but it really doesn't mean anything.
But this is where we're going to start, we're just going to use random functions that mapped out random to build on our random numbers. So what you can do right away is you can say "Well why apply the randomness uniformly when you're applying random numbers?" Right, you have control.
So we can say we want more vertical shifts instead of horizontal shift, which is going to magnify that random value in the Y, say by 10, and it might reduce the X value of offsets, say divided by 5 or times 0.2.
And you can draw these circles in any kind of arrangement you like and you kind of start to see a pattern emerge. It's kind of like, have you seen those movie openings where like the film opens and the city kind of blurs and it goes into focus? It's still kind of random.
There's really not that kind of sense of what we call a universality, which in nature you see there is a pattern in flowers and leaves.
There's none of that here because these are just pure random values.
But you can start to play with those random values. So the technique called Shepherding Random Numbers, all you do is you say "I'm going to generate a list of random numbers in which I'm apply to the circles all at once. But instead of just adding the random values themselves, I'm also going to keep a running sum.
So what that means is, from the left hand side I'm going to apply my random values.
For every circle that I've seen, I'm going to add the random values which you might remember is between negative 0.5, or 0.5 and one and zero depending on how you do it.
In this case I'm making sure that they're going to cover both the positive and negative spectrum. So when you add in the average, it's going to be zero. So we keep a running sum from the left to the right side, you are going to carry that randomness from the beginning to the end.
So eventually a pattern will start to emerge. I haven't really dictated any algorithms.
I haven't said that this has to follow some kind of thing. But if I just add the histories of some random values, they will kind of seem to follow each other from left to right.
And this might not make sense until you kind of scale it up.
You just start to draw way more circles.
And you really start to see some kind of really interesting pattern to emerge. So this might look like some kind of grassland and if I let it play for a bit probably replay, there it goes, now it's too long. Okay.
So what you can also do, let's just forget about the vertical or horizontal.
Let's just go back to the uniform for the moment. So if we, say, multiply by 10 or whatever number of your choice, you can now get a suddenly different effect. And all I've done here is I've literally changed one scale in my function. So now you suddenly get what looks like and ocean wave or a cloud or however you want to look like, and really the beauty of generative art is, I haven't written that much code, I literally just wrote a thing out for a circle, and that just said for every frame I'm going to draw a circle with a bit of offset, I'm going to draw them in an array many times, and I'm going to run a running sum of random numbers and shift them over and over again.
This is kind of how I play with things.
I just change a number and see what happens. That trial and error process as you will see will become a theme of this talk.
So again, what you can do is so why restrict yourself? So think about the possibilities.
I really like that theme.
I can have the velocity, I can have the running sum, which in this case I'm calling accumulating as in my pseudo-code, but I can also just add another random value, right? So I now have two running things of random values. One of them is being accumulated the other one is just a random value between some numbers. So in this case because I've changed the ratio a little bit, so that they are going to not move too much in the first random value and I'm going to emphasise the second random value, I get a different result again.
So this might look like some kind of edge of the carpet, it could be some kind of fleas.
It is quite interesting.
What's really cool when you start to play this in a bigger scale, and again if you're writing Javascript this might be a loop in a loop perhaps.
And instead of drawing circles, I'm drawing lines between the points.
And so I haven't really started this process to say I'm going to make things that look exactly like this.
I just kind of started with the pattern and I say I'm just going to start to play with these random algorithms and see what I can get out of it.
And I might just like add a line, maybe remove the line, or just play with the scales a bit.
So really the idea for Shepherding Random Numbers here is that you have a starting point, which can be anything. It can be a grid, it can be a circle, it can be a line. It can be a vertical grid or a slanted grid. You then have a list of random points in which you accumulate and you add it back to the offset. So in my London talk I've called it like "From Chaos to Order".
It's kind of like really taking no real world data here. I'm just saying these are random points.
But because I'm adding them together, I'm keeping a history of running sums and that kind of became an interesting pattern. So turning onto the next thing.
L-systems, if you haven't seen it before, is a thing that was kind of invented in the mid or late 60s by a professor named Lindenmayer, and he and his colleague were trying to model the nature of plants.
So he wanted to find out there's these patterns in which tree branches go up, or the leaves go out, or the flowers bloom in some kind of way.
Other mathematical descriptions back in the day, before the invention of the internet, it's all about mathematical models, right? What kind of models can we write to simulate the ideas of nature.
So he came with this thing called L-Systems, and that's a whole talk in itself, in which you can describe the beginning and the sequential order and apply a rule to it. And I'm going to add a little bit of mutation so we can start to play with that side of generative as well.
So there's two parts to L-Systems when it comes to be applying for geometrical generation.
The first thing is what we have is a list of rules and a starting point.
In this case, my starting point is a "B" and I have just two rules: whenever I see a "B", I'm going to turn it into an "A". Whenever I have an "A", I'm going to change it to an "AB".
So you can start from the beginning and say I have "B" or I see a "B", I'm going to apply the rule, it becomes an "A" it becomes "AB".
Well the first A is that I know AB.
The second B is going to be an A.
And you can just recursively or reiteratively or however you do it, apply this pretty much into infinity.
You can do this forever, as much as you RAM or your paper allows.
And this really gives you a sense of pseudo-random in each generation you will get a different list, an orders of things, while at the same time you can go back and forth. So there's a sense of order as well.
Although when you apply it, it's going to be different every time.
The other thing you may have, if you've ever been doing education in primary schools, it's really popular in scratch to graphics, so the idea is that I just want to have a turtle war, some kind of starting point. I want to draw a line and I'm constrained in this case to a grid, so 90 degrees.
And I'm going to draw forward, so I have a sense of direction, I might be facing this way. And whenever I draw forward, I'm going to draw by one unit.
And all a turtle can do is drawing one unit. Or the turtle can turn left or right, by some fixed number of angles.
When you apply these turtle graphics drawing rule to the different states from the L-System, instead of A and B, I'm going to have F minus and plus. And you can define F minus and pluses to be whatever rule, you can replace them to be whatever the rule.
The point is F is the command that tells the turtle to draw forward, minus is to turn right, and plus is to turn left. So you can string these sequence of commands, if can think of it that way, in whatever thing you want. And that would effectively describe the process to generate a graphic.
So let's start with a simple thing.
If you want to draw a rectangle, I can just do F, turn right, F, turn right and that gets me a rectangle to start.
What I can do is with L-System I can start to mutate, or not mutate, I can start to substitute by applying a rule. So with this thing, I'm going to say maybe my rule is I'm going to leave the minus and pluses alone, but I'm going to replace for every F I see, to be this, so: FF-F+F-F-FF.
For whatever reason, right, you can come up with a random string with anything there. At once it's got F's in there, it's going to get replaced. If it has minus and pluses in there, I know what to do with it.
So effectively, if what that would do after the first iteration of applying this rule you would get this massive string down here, because every single F that you see there is now replaced from here, is now replaced with this thing. So you get this massive string over here.
And what that looks like from the rectangle is to this. And I didn't you know like invent this thing, I didn't randomly punch in characters until something interesting like this showed up. There's a whole pool of algorithms and rules and L-Systems that you can look at on the internet and books.
So this is one of the entry examples at a university. So you can do this over and over again.
You can do this over and over again.
Like I said you can just kind of do it for a long time. I haven't changed my code.
My rule is literally three lines or if you included braces, four lines, I suppose. And you can do a lot of things with this.
So L-Systems are so flexible.
People have worked out how to describe a bunch of geometrical formations with L-Systems. So chances are if you see an fractals or if you se any, this is called a space filling curve, or the Hilbert curve in this case. You can describe it in L-Systems.
If you've ever seen any geometrical tiling that's really popular in the M.C. Escher days and other people, you can pretty much describe it with L-Systems.
So this is like this really generic thing that I just started playing around with, right. Like I haven't really written any code that I wanted to do something here.
So what can I do with it? Well, I want to start to mutate this thing. And what I mean by mutation is the very, very simple string substitution that you do in your evolution optimization. So if you've ever done some basic machine learning, I should say basic, if you ever tried to learn machine learning in the evolution optimization form, you would know that one of the things you can do to optimise is to say I'm going to randomly send my workers to do something and then I'm going to say 50% of you are ranked from the best to the worst.
I'm going to cull off the worst, I'm going to take the best, mutate them so I'm going to change a little bit in the configuration.
Now double that to back to a new pool of workers, new staff, rank them from the best to worst, cut off the worst and keep the best, and you repeat that process.
So you are just trying to change it a little bit. You know, flip a gene, see what happens.
So what you can do actually with L-System is, you know that rule with F F dash F, whatever that is? I can apply every iteration a different substitution, so that each time I try this, and this is live, I can go back and go forth, I will get a complete different thing.
Sometimes they end because they just end in some cycles. Sometimes it's a bit chaotic.
And when you do generative art or any kind of experimental thing, I really don't know what I'm expecting here, I'm just kind of flipping back and forth, and I can do this when I'm in the shower, if I have a clicker with me it's kind of cool, I don't know, it's very distracting.
I can do this for days.
(audience laughter) The point is, I have a foundation of a tool that I can experiment with and if I like this pattern, say I really like this pattern for some reason.
That reminds me of the work from Hunter Abbotts actually. I might have already a tool that captures the mutated L-System that I have, save it and do something else with it.
So let me describe what I'm doing here.
I have the rule that you've seen before, I have the F that kind of replaces it with this thing. And at every single stage, I'm going to randomly pick one of those and then replace it with one of the characters that I already know. In this case it's F minus and plus.
So for every iteration I will have replaced this with my L-System so this string would have grown longer, but for the sake of illustration I'm just going to replace them one by one in the linear fashion.
So you can see that I'm just randomly selecting a character and then replacing them with something that I already know.
So that the system will still be able to run without running into strange things. So another thing I like to do is like play with this and then apply random things to it.
So again, that's just mutating one of the things in the dragon curve and I can just apply a random thing when it happens and I run it through a couple times.
I can do it again.
So I might go back.
Oh that's pretty cool.
(laughter) Okay, see if I can get a good one.
No, that's okay.
So, that actually looks like some kind of like, you know like chemistry or biology, you'd extract the DNA anyway, yeah.
So to summarise, you can do a lot with L-Systems. It's just a straight-forward string that replaces. I guess if you want to call it that way.
And you can start to change a little bit of things and see what we can do with it.
It's very simple to write if you know how to write full loops and if you know how to do canvas so you can just draw, get a context and draw a line, move to line two. You can do a lot with these things.
So to quickly summarise, I really want you to embrace uncertainty and embrace the possibilities. Trial and error is really a normal process. It doesn't matter if you're trying to create art, which is a very abstract, fluffy thing, or if you're just trying to do production stuff. There's a lot of things that you are trying that may not work, but that's okay because part of the process of learning is about embracing uncertainty.
And finally, I hope I stressed this point enough, but again programming is creative and so logics can compliment creativity.
And one thing I also want to mention is that creativity is a skill and skills can be learned. So don't feel that you're not creative or whatever. You can learn those skills and apply it to your daily work.
So that's my talk and thank you very much.
(audience applause) (techno music)
