Author: easthackney

Passionate about Music, piano, electronics, programming,...

Each Pianist Has His/Her Style of Playing

Believe it or not, ‘playing style’ is extremely important to Sideman. Your playing style is what Sideman analyzes in order to generate accompaniment. Imagine you are listening to your favorite pianist (mine happens to be Bill Evans and Oscar Peterson). What exactly do you hear? Here are some things I try to pick out:

  • What Key Signature is the performance in? Does the performance modulate between different Key Signatures?
  • What are the chord changes? In a specific key I tend to hear I, II, II, IV, V, VI and VII chords.
  • Is the performer ‘vamping‘ between a small set of chords? For example II min7 <-> VSus11.
  • Is the performer simply ‘comping‘ through a set of chords, as would be the case if he were ‘backing up’ a vocalist?
  • Is the performer ‘soloing’ over a set of chords? If so, is he playing ‘modal scales’.
  • How ‘aggressive’ is the solo? Notes per bar and volume are factors in this.
  • Is the performer playing a Bass pattern?
  • Is the performer playing at a specific ‘tempo‘, or is he simply ‘improvising’ with no apparent tempo.
  • What type of ‘chord voicings‘ are being used: ‘crushed’ block chords, ‘open-voice’ chords spanning several octaves, etc.
  • Are ‘rootless chords‘ being played, where the chord root is either implied or meant to be played by a bass player.
  • Does the performance ‘swing’? Does the performance have a ‘hip-hop’ feel, or perhaps a ‘Latin feel’?
  • What volume is the pianist playing at? Are chords played ‘staccato’?

These are the type of musical nuances that Sideman quantifies in order to guide the accompaniment.

For example, if you simply play in an ‘impromptu’ fashion, with no strict tempo or key signature, Sideman can at most infer only a general sense of what chords are in play. Examining a set of notes being played, Sideman might determine an implied ‘chord root’ (perhaps the lowest note being played), whether a major third or minor third is present, whether a ‘tritone’ is present, etc. From just this information, Sideman might generate a string accompaniment that follows one of the notes being played. A set of chords might contain a common note that is consistent throughout those chords, which provides an excellent note to be played as a Sting note.

On the other hand, the performance might be played at a strict tempo and time signature (think 8-beat, 16-beat, 6/8 beat, swing beat, etc). In this case, Sideman has no difficulty determining the tempo, time signature, and song style.

If the performance is done with a wide range of volumes, Sideman can track these volume changes and adjust the accompaniment volume accordingly. If the user pauses his playing, Sideman can pause the accompaniment and resume as soon as the pianist resumes playing. When the performance ends, it might consist of a sustained ‘ending chord’. Sideman can detect this and generate some ‘ending accompaniment’ that might be little more than an arpeggio of the ending chord, played by violins. You can see how important your playing style is – it is the ‘input’ to sideman’s algorithms that generate accompaniment.

Adding Sideman to your ‘Rig’

The first requirement is that you have a digital piano with a 5-pin MIDI OUT connector. The next requirement is that you have a MIDI Sound Module. If you already have these items, you most likely connected your keyboard to the sound module using a standard MIDI cable. The AUDIO OUT of the sound module goes to some form of ‘sound system’ (PA system, Bluetooth speaker with AUX IN, or even simply headphones). If this is the type of setup you use for practicing or live performing, you are ready to go. However, if you currently lack a ‘MIDI Piano’, there are keyboards such as the MidiPLUS AK490 MIDI Keyboard Controller for under $100. Sound modules vary in price from under $100 to $1,499 for the Roland Integra (my sound module of choice), or the Yamaha Motif-Rack Xs for under $1,000. Keep in mind that regardless of whether you use Sideman or not, this setup of Keyboard, sound module and sound system is a standard setup.

The only change we need to make to your “Keyboard + Sound Module + Sound system” setup is to disconnect your keyboard from the sound module , then connect the MIDI OUT of your Keyboard to the MIDI IN of Sideman, and the MIDI OUT of Sideman to the MIDI IN of your sound module. That is all there is to it.

When Sideman is powered ON, the MIDI OUT of your keyboard is ‘passed through’ by Sideman to your sound module. In fact, at this point you will be unaware that Sideman is even present. That is, whatever you play on your keyboard will sound through your sound system just as it did before Sideman was added to the setup.

Your MIDI sound module, like all MIDI devices, has 16 ‘MIDI Channels’. What you play on the piano will be processed on only one of the 16 MIDI channels, leaving 15 MIDI channels available for Sideman to use. Sideman has a default configuration for voice assignments on the 15 MIDI channels it uses: Bass, Guitar, Drums, Strings & Pads, Chromatic instruments, etc. You are free to use the default voices that Sideman comes with, or you can modify the configuration files to your liking. As an example, you can select the piano voice that sounds when you play (most sound modules have close to 100 piano voices!).

Keep in mind that Sideman is, at its heart, simply a ‘personal computer’, albeit the popular Raspberry PI. Being such, you are free to connect a monitor, mouse and keyboard and use the Raspberry PI as your personal computer. What makes it special is that it can run the Sideman application. When powered up, the Raspberry PI launches the Sideman application and you are ready to go.

In upcoming posts we will detail how Sideman will analyze your piano playing to generate up to 15 MIDI channels of sophisticated accompaniment. Topics will include:

  • How Sideman monitors for when you are ‘playing at tempo’. There are various ways you can ‘play at tempo’, including chords, melody, or playing percussion notes. Playing ‘at tempo’ enables Sideman to automatically begin playing at the same tempo that you are playing. “How does it do that???!!!”.
  • How Sideman monitors your ‘playing style’. Nuances in style include things such as playing a melody that ‘swings’, playing certain types of chord progressions (think Latin, Bossa, Samba, Shuffle, Swing, …), playing staccato or legato, playing softly or loudly, playing modal scales, etc.
  • How sideman monitors your playing volume and adjusts the accompaniment volume to match.
  • How Sideman responds to keyboard inactivity, for example if you pause your playing to take a sip of wine. Or someone comes up to you and wants to ask you how incredible your performance sounds. You pause playing, and Sideman pauses right with you, ready to resume when you are.
  • How Sideman detects when you are ‘vamping’ on a set of chords, or when you modulate to a new Key Signature.
  • How Sideman monitors you chord voicings. Are you playing simple triads? Are you playing primarily major chords (common with standards and ballads). Are you playing extended voicings (min9, min11, sus, altered dominant). Are you playing Latin II-V voicings?. Are you playing ‘funk’ chords?
  • How Sideman responds when you play Left Hand bass patterns along with Right Hand chords.

Stay tuned…

The idea is that Sideman behaves similar to how a Bass player, guitarist, drummer, etc. ‘listens’ to the pianist and provides accompaniment that enhances the pianist.

Stay tuned….

Some History On Sideman

The genesis for Sideman was the “Midi Specification version 1.0”, published in 1983, almost 40 years ago. It’s introduction stated that “MIDI enables synthesizers, sequencers, home computers, rhythm machines, etc. to be interconnected through a standard interface”. At that time I was a young engineer working at a company called Linkabit, designing embedded systems. But my passion, besides electronics and programming, was music – specifically piano and drums.

The systems we designed at Linkabit were microprocessor-based and programmed in assembly language. UARTS were part of the design to enable communication between I/O devices such as terminals and keyboards. Little did I know that UARTS were a fundamental part of MIDI.

MIDI Specification 1.0 included a schematic for a ‘midi interface’. The interface used no more than two 5-pin DIN connectors, an optoisolator, some resistors, a diode, and some buffers. The interface connects to the transmitting device’s UART and the receiving device’s UART. Needless to say I quickly bread-boarded this circuit and eagerly connected it to two of our micro-controller boards to see if I could transmit some bytes between the boards. After adjusting the value of the Optisolator’s output pull-up resistor to get a full 5 volt logic swing, the setup worked! At that same time I was using a Yamaha FB-01 sound module connected to a Yamaha DX7 digital piano, and it was MIDI that enabled these two devices to communicate. After studying the “Midi Data Format” for midi messages sent between midi devices, I focused on the “Midi Note ON” and “Midi Note OFF” messages (and how trivial they were: they included the Midi Channel, the note value, and the notes volume). Armed with this knowledge I set out to send some ‘Note ON’ and ‘Note OFF’ messages from my microcontroller to the Yamaha FB-01, and it WORKED! Of course the next task was to ‘play’ some chords – and that worked TOO! My imagination was now on fire. As time progressed, I began writing code to play chord progressions and even play drum beats on a Drum Machine. Years later this code was generating Bass patterns and Drum patterns as accompaniment to my piano playing.

At some point I realized that with MIDI, so much more could be accomplished. For example, by connected the microcontroller’s UART to the DX7 Midi output, my program could ‘see’ and ‘listen to’ what I was playing on the piano!

Next came some code to decode the chords I was playing. As I played my favorite jazz chords (think ‘Dominant 7, raised 9’) the program would decode and semi-instantaneously display “F Dom #9” on the monitor as diagnostic information. I thought: If code could quickly decode the chord played by a pianist in real-time, the generation of accompaniment (think Bass, Strings, Guitar, etc) was definitely possible.

Then came ‘Key Signature decoding’, ‘Tempo and Time Signature decoding’, and even piano ‘piano style’ detection.

Having a high-speed hardware timer with software interrupts enabled the program to generate accurate timing for generating accompaniment to my piano playing. After creating Bass and Drum accompaniment I named this project “Sideman” – because in those days bassists and drummers were called “sideman”.

Initially Sideman used a large and expensive single-board computer. Today Sideman uses the ubiquitous Raspberry Pi! The Pi has a processing bandwidth unheard of just a decade ago, and for the cost of cheap dinner out.At some point in Sideman’s development I became determined that the guiding philosophy was that the pianist would always be in complete and total control of his performance – Sideman would strictly be a “listener”. At no point would the pianist be interrupted or distracted by the accompaniment generated by Sideman.

Sideman generates accompaniment using software algorithms that “listen” to what the pianist is playing – and to respond accordingly. Such musical nuances as ‘tempo, ‘swing’, ‘Key modulation’, ‘chord voicings’, ‘modal scales’, ‘chord and solo note timing’, etc. would be analyzed and quantified in real-time and used to guide the accompaniment. And just as importantly there would be no knobs, buttons, switches, menus, displays, etc. to take the pianist’s attention away from his performance. On a personal note, since my father suffered from macular degeneration, I have focused a lot of attention on making Sideman accessible to the visually impaired. A year or so ago I posted some videos showing how Sideman could be used by the visually impaired. ‘Starting’, ‘pausing’ and ‘stopping’ sideman was controlled strictly by the performance. For example, volume of the accompaniment auto-tracked the performance, and variations in tempo and style of the accompaniment also tracked the performance.

My vision looking forward is to make pianists aware of Sideman’s features, and how simple and inexpensive it is to incorporate Sideman into their performance. By using videos, the pianist can learn how Sideman automatically provides incredible accompaniment to his performance. As a side-note, Midi Sound Modules can be obtained at prices as low as $65 (MidiPlus Midi Enging). Howeer for the more serious pianist I would recommend sound modules such as the Yamaha Motif-Rack XS or Roland’s new Integra. These high end sound modules are absolutely amazing – and Sideman will take advantage of their voices to make your performance OUTSTANDING!

So please watch for upcoming posts that detail how Sideman does what it does. Thanks for listening, Eastman.