Sonic Instruments
New technologies, inventions are often borrowing analogies from already understandable systems when hitting the audience of the wider society. The analogy of musical instruments can be understood easily in cultural terms. While these instruments are globally present, they also reflect the needs and specialities of local communities. Their structure is simple: making sounds with physical energy. This goal has a truly interesting, deep ergonomic tradition rooted in the past few centuries. Software based interfaces are fresh, ephemer tools that are lacking these type of long term cultural traditions. Therefore, comparative research of traditional instruments and digital tools are raising really interesting questions.
Musical instruments naturally have intuitive dimension spaces embedded in them, shaped by their ergonomic aspects, cultural functionalities and resonating bodies. In the case of software based interfaces, these dimensions are not originating a prior from the physical parameters of the object so they have to be built in explicitly into these systems. This means, that the lack of a physical body induces a new type of inner coherence in software based visual instruments, which drives to unknown territories both designers and players.
Music and sound related visual interfaces are special area within interaction design. They need to give immediate, meaningful feedback upon any type of manipulation. Latency, lags and unintended temporal interruptions can confuse the user, these events can make the usage of a tool hard and uncomfortable. Since the designer needs to embed the cognitive dimensions directly in the system, she needs to define a clear, concise visual language that is not standing in the way of the user's expressivity. Good general starts for these interaction patterns are design guidelines of Apple and Android. Apart of these general principles, the designer of a sonic interface should keep in mind and plan ahead the amount of presence of the following dimensions before starting any implementation:
- Expressive Constraints
- Autonomy
- Music Theory
- Explorability
- Required Foreknowledge
- Improvisation
- Generality
- Creative Simulation
hint: it's useful to take a pencil and draw a diagram of the included dimensions above. also, it's a very compelling task to analyze existing games, instruments and interfaces based upon the dimensions above
image source: NIME 2010 conference paper by T. Magnusson
- HelloPd
- PureDataBasics
- ProcessingBasics
- SimpleUIElements
- SoundAnalysis
- GenerativeMusic
- VisualMusic
- GrainField
Thor Magnusson: Of Epistemic Tools, The Fenomenology of Musical Instruments, Scoring with Code, Sergi Jordá: New Musical Interfaces, Peter Brinkmann: Making Musical Apps, Agoston Nagy: Visual Sound Instruments (thesis introduction)
BangBook (pdf), Programming Electronic Music in Pd
Books by Daniel Shiffman