Create fractal music and change them into intriguing music patterns with a Raspberry Pi.
Fractals seem to have gone out of favour when it comes to computers, which is a pity because there are plenty of exciting things to explore with them, especially in the field of music.
Most people think of a fractal as a complex curve and there a few pleasant-looking standard examples. The basic property of a fractal is that it is self-similar; that is, you see the same sort of pattern if looking at a very small magnified portion of the curve as you do when you look at a zoomed out portion. They are both similar, but not of course identical. Music has a similar structure, with patterns of notes repeating but developing throughout the composition. This is a rich, and largely untapped, source of tunes and inspiration.
Fractal music generation
There are many ways to generate fractals, but here we will be looking at one method, the Lindenmayer system, or L-system for short. This is a recursive algorithm inspired by biological system; it works by successive applications of substitution rules to a string of symbols to generate another, normally longer, string of symbols. This output string is fed into the input again and a new string is generated. This process is repeated any number of times and produces a fractal, or self-similar, sequence. The rules and the initial string, called the axiom, determine the outcome.
Let’s see how this works in practice by looking at a very simple example shown in Figure 1. This has just three symbols – A, B, and C – and each symbol has a rule for substitution. So when we encounter an A in the input stream, we replace it with the symbols BA in the output stream. When B is encountered, we replace it with a C. When C is found, we replace it with an AB. These rules are shown on the left of the diagram.
If we start with the simple axiom of C after the first application of the rules, we get the string AB. Then run it through the rules again and the first symbol A is replaced by BA and the second symbol B is replaced by C. This applying of rules to an input string to produce an output string is known as a level of recursion; after four levels, our symbol string is ABCBABAC. The rules can be arbitrarily changed to produce different outcomes and the string can involve as many single-character symbols as you like.
Rules about rules
While the rules can be arbitrary, in order to be successful they need to follow some rules themselves. First of all, each symbol used must have a rule associated with it, and that symbol must occur in at least one of the results of another rule. If this is not observed then some symbols will be isolated and never appear in the output stream. If all the symbol rules map only to another symbol, then the output stream always remains the same length; sometimes you might want this, but normally you will want a sequence to grow. Once an output sequence – or successive sequences – has been produced then you need another set of rules, called production rules, to interpret it into, in our case MIDI notes. Let’s look at a simple example.
Simple example
The code in Simple.py generates a sequence of symbols and then plays them; this then repeats to a given level of recursion. The rules are expressed as a list of tuples; each tuple has two parts, the input condition and the output condition. To make things easy to interpret for us, we have added the string "->" to the end of the input string so that the tuple ("A->","AB") means replace the symbol A with the symbols AB. This just makes it easy for us to spot what a rule is doing and to change it. The code first opens the MIDI port and prints out the rules and axiom. Then these rules are applied six times and the result of each recursion is added to a list called composition.
Finally, the composition is passed to a sonification function called sonify. The rules for turning the symbols into notes here are very simple: each symbol, A to G, is turned into a MIDI note number representing the notes A to G, defined by the notes list, and played for a time defined by the variable noteDuration. This plays each level of recursion with a short gap between each. Quitting the program with CTRL+C will cause the program to turn all the MIDI notes off before quitting so you don’t get any hanging notes.
The code uses MIDI voice 19, the church organ, but you can change this to anything. If you want to alter this on the fly then you can load up the MIDI voice test program from The MagPi #63 to run at the same time. Just navigate to the folder containing it, using a Terminal window and type:
nohup python3 voiceTest.py &
A multichannel version
The previous example just played a single instrument for a single line. In this next example, the last three levels of recursion are played at the same time on different instruments. As each level is of a different length, the note on time is adjusted so that the playing time as a whole is the same for each track. This means that the smaller levels of recursion have longer notes than the higher ones. A lot of the code is the same as the first example; so, instead of repeating the whole listing, we have just printed the changes you have to make to Simple.py in NewFunctionsfor_Simple.py.
These changes are basically a new sonify function along with two additional functions, notFinished and playNext. The instruments and volume levels are set at the start of the sonify function, and we have used the ‘rain’ instrument for the long notes because it has something interesting going on in the background for held notes. Short notes, we think, are best when the sound itself is short, like a bell.
Adding graphics
To add some graphics requires a much longer program and our normal Pygame framework. We have written one that will produce the sound of the last example, only play it back by building up the composition by adding one track at a time. The screen output is shown in Figure 2 and the code can be found in our GitHub repository. It might not look like a classic fractal, but that is because of the very simple mapping of the sonification: one symbol represents one note. To get a bit more flexibility, we need to add a different sort of rules: that of interpreting the fractal string.
Interpretation rules
Interpretation rules are somewhat different to the rules we used before to produce a symbol string. They are not a sequence of substitutions but a set of things to do for when sonifying each symbol. For example, suppose we add some symbols in the production rules to alter the length of a note. When this symbol occurs, the note duration changes but no note is generated for that symbol; that means we can’t use the trick of altering the note length based on the length of the sequence. It’s easy enough to do this, however, and it adds a bit of variety to the composition. The extra code to add this feature is in ExtraCodeforNoteDuration_Functions.py and it shows what you have to change to the Simple.py code: it is just a replacement for the sonify function and a new axiom string and rules list. Here there are three lengths of note defined by the symbols q, h, and c, and these change the duration variable for subsequent notes.
Adding a state machine
These interpretation rules are still direct substitutions of notes and length of notes. To get another level of complexity you have to get these symbols to interact with a state machine, and use the latter to define various parameters of the music. When this system is used for producing fractal drawings, that state machine is a turtle graphics drawing package. Symbols represent turtle commands like move forward, turn left or right a specific angle, or move without drawing, to name but four. It is the cumulative result of these sorts of commands that determines what is drawn at any one time. In order to get separate branches, there are two other types of operation represented by symbols: the [ which places the turtle state on a stack, and the ] which restores the turtle state from a stack. You can have a look at such a system if you install a graphics program called Inkscape. When you run it, go to the Extensions menu, select Render, then the L-system option. You will get a window that allows you to set rules and turn angles; you can set more than one rule by separating them with a semicolon. Figure 3 shows a list of rules for fractals to set you off exploring.
Bush
Axiom: ++F
Rules: F=FF-[-F+F+F]+[+F-F-F]
Dragon Curve
Axiom: FX
Rules: X=X+YF;
Y=FX-Y;
Angle: 90
Koch Island.
Axiom: -F--F--F
Rules: F=F+F--F+F;
Angle: 60
Other fractal
Axiom: W
Rules: W=+++X--F--ZFX+;
X=---W++F++YFW-;
Y=+ZFX--F--Z+++;
Z=-YFW++F++Y---;
Angle: 30
Penrose P3
Axiom: [N]++[N]++[N]++[N]++[N]
Rules: M=OA++pA----NA[-OA----MA]++;
N=+OA--PA[---MA--NA]+;
O=-MA++NA[+++OA++PA]-;
P=--OA++++MA[+PA++++NA]--NA;
Angle: 36
In the same way, you can implement a music turtle that determines the frequency, duration, and any effects you care to specify. So the range of notes is much wider than you can get from a one-to-one mapping of symbol to note. This musical turtle can be restrained to a certain range of parameters by wrapping round the values as they exceed their limits. The code for this is shown in Classic_Fractals.py and although it looks similar to the other listings, it does have many slight changes. For a start, the production rules have been changed to reflect the Inkscape system: where there is no rule for a symbol, that symbol is just copied to the output string. Also, the production rules match: any symbol A to F plays a note and updates the pitch, whereas any symbol G to L just updates the pitch. Note the initKey function; this generates a lookup table in any major key determined by the starting note. The rules in the listing are for a bush whose graphical representation is shown in Figure 4.
Results
Well, what does all this sound like? The uncharitable might say it sounds like a maniac practising scales, but there is a lot more to it than that. We liked the simpler systems best, as we felt there was a tune trying to break out and occasionally succeeding; you could definitely hear the self-similarity coming through. Small changes in rules produced small changes in melodies, which is good for control, and we liked the multitimbral approach of having more than one track playing at the same time.Taking it further
Like no other project, this is one you just have to tinker with. You can have a lot of fun making up rules and listening to the results. This just requires typing them in at the start of the program. There are lots of variations you can make to the production rules, like including a probability factor to some. For example, you can have two rules for one symbol, and attach a probability that one rule will be used over another simply by generating a number from one to ten, and if the number is above some value then use rule one, otherwise use rule two. The production rules for the state machine can be changed to include note duration or even note timbre. For serious music it is probably best to pick out the good bits in a fractal sequence and incorporate that into your own music.import time, copy
import rtmidi
midiout = rtmidi.MidiOut()
noteDuration = 0.3
axiom = "++F" # Bush
rules = [("F->","FF-[-F+F+F]+[+F-F-F]")]
newAxiom = axiom
def main():
global newAxiom
init() # open MIDI port
offMIDI()
initKey()
print("Rules :-")
print(rules)
print("Axiom :-")
print(axiom)
composition = [newAxiom]
for r in range(0,4): # change for deeper levels
newAxiom = applyRules(newAxiom)
composition.append(newAxiom)
sonify(composition)
def applyRulesOrginal(start):
expand = ""
for i in range(0,len(start)):
rule = start[i:i+1] +"->"
for j in range(0,len(rules)):
if rule == rules[j][0] :
expand += rules[j][1]
return expand
def applyRules(start):
expand = ""
for i in range(0,len(start)):
symbol = start[i:i+1]
rule = symbol +"->"
found = False
for j in range(0,len(rules)):
if rule == rules[j][0] :
expand += rules[j][1]
found = True
if not found :
expand += symbol
return expand
def sonify(data): # turn data into sound
initMIDI(0,65) # set volume
noteIncrement = 1
notePlay = len(notes) / 2
midiout.send_message([0xC0 | 0,19]) # voice 19 Church organ
lastNote = 1
for j in range(0,len(data)):
duration = noteDuration # start with same note length
notePlay = len(notes) / 2 # and same start note
noteIncrement = 1 # and same note increment
stack = [] # clear stack
print("")
if j==0:
print("Axiom ",j,data[j])
else:
print("Recursion ",j,data[j])
for i in range(0,len(data[j])):
symbol = ord(data[j][i:i+1])
if symbol >= ord('A') and symbol <= ord('F') : # play current note
#print(" playing",notePlay)
note = notes[int(notePlay)]
#print("note", note, "note increment",noteIncrement )
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
midiout.send_message([0x90 | 0,note,68]) # next note on
lastNote = note
if symbol >= ord('A') and symbol <= ord('L') : # move note
notePlay += noteIncrement
if notePlay < 0: # wrap round playing note
notePlay = len(notes)-1
elif notePlay >= len(notes):
notePlay = 0
time.sleep(duration)
if symbol == ord('+'):
noteIncrement += 1
if noteIncrement > 6:
noteIncrement = 1
if symbol == ord('-'):
noteIncrement -= 1
if noteIncrement < -6:
noteIncrement = -1
if symbol == ord('|'): # turn back
noteIncrement = -noteIncrement
if symbol == ord('['): # push state on stack
stack.append((duration,notePlay,noteIncrement))
#print("pushed",duration,notePlay,noteIncrement,"Stack depth",len(stack))
if symbol == ord(']'): # pull state from stack
if len(stack) != 0 :
recovered = stack.pop(int(len(stack)-1))
duration = recovered[0]
notePlay = recovered[1]
noteIncrement = recovered[2]
#print("recovered",duration,notePlay,noteIncrement,"Stack depth",len(stack))
else:
print("stack empty")
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
time.sleep(2.0)
def initKey():
global startNote,endNote,notes
key = [2,1,2,2,1,2] # defines scale type - a Major scale
notes =[] # look up list note number to MIDI note
startNote = 24 # defines the key (this is C )
endNote = 84
i = startNote
j = 0
while i< endNote:
notes.append(i)
i += key[j]
j +=1
if j >= 6:
j = 0
#print(notes)
def init():
available_ports = midiout.get_ports()
print("MIDI ports available:-")
for i in range(0,len(available_ports)):
print(i,available_ports[i])
if available_ports:
midiout.open_port(1)
else:
midiout.open_virtual_port("My virtual output")
def initMIDI(ch,vol):
midiout.send_message([0xB0 | ch,0x07,vol]) # set to volume
midiout.send_message([0xB0 | ch,0x00,0x00]) # set default bank
def offMIDI():
for ch in range(0,16):
midiout.send_message([0xB0 | ch,0x78,0]) # notes off
# Main program logic:
if __name__ == '__main__':
try:
main()
except:
offMIDI()
import time, random, copy
import rtmidi
midiout = rtmidi.MidiOut()
notes = [57,59,60,62,64,65,67]
noteDuration = 0.3
axiom = "AD"
rules = [("A->","AB"),("B->","BC"),("C->","ED"),("D->","AF"),
("E->","FG"),("F->","B"),("G->","D") ]
newAxiom = axiom
def main():
global newAxiom
init() # open MIDI port
offMIDI()
print("Rules :-")
print(rules)
print("Axiom :-")
print(axiom)
composition = [newAxiom]
for r in range(0,6):
newAxiom = applyRules(newAxiom)
composition.append(newAxiom)
sonify(composition)
def applyRules(start):
expand = ""
for i in range(0,len(start)):
rule = start[i:i+1] +"->"
#print("we are looking for rule",rule)
for j in range(0,len(rules)):
if rule == rules[j][0] :
#print("found rule", rules[j][0],"translates to",rules[j][1])
expand += rules[j][1]
return expand
def sonify(data): # turn data into sound
initMIDI(0,65) # set volume
midiout.send_message([0xC0 | 0,19]) # voice 19 Church organ
lastNote = 1
for j in range(0,len(data)):
if j==0:
print("Axiom ",j,data[j])
else:
print("Recursion ",j,data[j])
for i in range(0,len(data[j])):
note = notes[ord(data[j][i:i+1]) - ord('A')] # get note given by letter
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
midiout.send_message([0x90 | 0,note,68]) # next note on
lastNote = note
time.sleep(noteDuration)
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
time.sleep(2.0)
def init():
available_ports = midiout.get_ports()
print("MIDI ports available:-")
for i in range(0,len(available_ports)):
print(i,available_ports[i])
if available_ports:
midiout.open_port(1)
else:
midiout.open_virtual_port("My virtual output")
def initMIDI(ch,vol):
midiout.send_message([0xB0 | ch,0x07,vol]) # set to volume
midiout.send_message([0xB0 | ch,0x00,0x00]) # set default bank
def offMIDI():
for ch in range(0,16):
midiout.send_message([0xB0 | ch,0x78,0]) # notes off
# Main program logic:
if __name__ == '__main__':
try:
main()
except:
offMIDI()
def sonify(data):
melodyLines = 3 # change for more or less lines
# for more melody lines add more elements to the next two lists
instruments = [112, 0, 96] # instruments for each line
volume = [50, 60, 65] # volume for ech line
lastNote = []
index = []
startTime = []
interval = []
lineLength = []
for i in range(0,melodyLines):
initMIDI(i,volume[i]) # setu up MIDI channel
midiout.send_message([0xC0 | i,instruments[i]]) # set voice
startTime.append(time.time()) # set up lists
index.append(0)
lastNote.append(0)
interval.append(noteDuration * len(data[len(data)-1])/len(data[len(data)-1-i]))
lineLength.append(len(data[len(data)-1-i]))
print() ; print("Playing")
for i in range(0,melodyLines):
print("line",i,"voice",instruments[i],"length",lineLength[i],
"notes of duration",interval[i],"seconds")
while notFinished(melodyLines,lineLength,index) :
for i in range(0,melodyLines):
if time.time() - startTime[i] > interval[i]:
lastNote[i] = playNext(i,index[i],lastNote[i],data,len(data)-1)
index[i] += 1
startTime[i] = time.time()
time.sleep(noteDuration)
for i in range(0,melodyLines):
midiout.send_message([0x80 | i,lastNote[i],68]) # last note off
def notFinished(playingLines,length, point):
notDone = True
for i in range(0,playingLines):
if point[i] >= length[i] :
notDone = False
return notDone
def playNext(midiChannel, i , lastNote, data, line):
note = notes[ord(data[line][i:i+1]) - ord('A')] # get note given by letter
midiout.send_message([0x80 | midiChannel,lastNote,68]) # last note off
midiout.send_message([0x90 | midiChannel,note,68]) # next note on
return note
axiom = "qAhD"
rules = [("A->","ABc"),("B->","BCh"),("C->","EDq"),("D->","AFc"),
("E->","FGh"),("F->","Bq"),("G->","Dc"),("q->","hA"),("h->","qF"),("c->","hF") ]
def sonify(data): # turn data into sound
initMIDI(0,65) # set volume
midiout.send_message([0xC0 | 0,19]) # voice 19 Church organ
lastNote = 1
for j in range(0,len(data)):
duration = noteDuration # start with same note length
if j==0:
print("Axiom ",j,data[j])
else:
print("Recursion ",j,data[j])
for i in range(0,len(data[j])):
symbol = ord(data[j][i:i+1])
if symbol >= ord('A') and symbol <= ord('G') : # it is a note
note = notes[symbol - ord('A')] # get note given by letter
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
midiout.send_message([0x90 | 0,note,68]) # next note on
lastNote = note
time.sleep(duration)
else : # it is a note duration
if symbol == ord('h'):
duration = noteDuration * 2
if symbol == ord('c'):
duration = noteDuration
if symbol == ord('q'):
duration = noteDuration / 2
midiout.send_message([0x80 | 0,lastNote,68]) # last note off
time.sleep(2.0)
