How does a piano string compare to an ideal physicist’s string? Are there equations that describe the sound a recorder produces? Can the quality of an instrument be quantified? In this episode, we talk to one of the editors of the JASA Special Issue on Modeling of Musical Instruments, Nicholas Giardano, about the wide variety of research efforts regarding analytical and computational techniques to model musical instruments, and how these techniques can help both instrument makers and musicians.
Read the Special Issue on Modeling of Musical Instruments.
Read more from The Journal of the Acoustical Society of America (JASA).
Learn more about Acoustical Society of America Publications.
Music Credit: Min 2019 by minwbu from Pixabay. https://pixabay.com/?utm_source=link-attribution&utm_medium=referral&utm_campaign=music&utm_content=1022
Kat Setzer (KS)
Welcome to Across Acoustics, the official podcast of the Acoustical Society of America’s Publications office. On this podcast, we will highlight research from our four publications, The Journal of the Acoustical Society of America, also known as JASA, JASA Express Letters, Proceedings of Meetings on Acoustics, also known as POMA, and Acoustics Today. I'm your host, Kat Setzer, Editorial Associate for the ASA.
Today, we're highlighting another one of JASA's special issues, the Special Issue on Modeling of Musical Instruments. Joining me is one of the guest editors of the special issue, Nicholas Giordano of Auburn University. Thank you for taking the time to speak with me today. How are you, Nick?
Nicholas Giordano (NG)
I'm well, and thanks for having me. I really appreciate this.
Thanks. So first, just tell us a bit about yourself and your research.
Well, I grew up or I was trained as an experimental condensed matter physicist. But of course, I was an academic. So I was also engaged quite a lot in teaching. And at one point I got involved in developing and then teaching a course in computational physics, and modeling is a big part of that. And I was, and I came upon examples, good examples, for teaching computer physics that involve musical instruments. Specifically, I think we started with piano strings. And we started asking what, how, in what ways is the piano string different than an ideal physicist’s string? And then we went from there, and I got my I and my students got kind of hooked on problems to do with musical instruments. And my research gradually grew to include work of that nature, and now I'm getting close, I'll confess, I'm getting close to retirement. And most of my time now is spent thinking about musical instruments and about modeling. And that's how we came to, to propose and then execute this, or have this special issue.
Awesome. So that actually segues into our first question really well: What does it mean to model a musical instrument? What can you learn about an instrument from modeling?
Well, that's a good question. I get asked that quite often. I would say that a math-- I'm gonna come to this as with a physicist’s background, and of course, physicists, we think of physicists as being either experimental physicists or a theoretical physicist, but what does it mean to be a theoretical physicist? There are very few theories in physics that are exact. So usually a theorist will develop a model of the system that he or she wants to study and then try to solve it in some way. So, in that sense, a model in the in the physics sense, and also I think in the acoustic sense, is a mathematical description of an instrument that you want to gain insight into it. So you might want to say okay, what aspects of an instrument determined the frequencies that it produces? Or what aspects that determine the directionality of the sound coming out of the instrument? And so, you develop, so, the model would be developed-- it would be, it would be a mathematical description, usually based on the laws of physics or acoustics, same, basically the same thing, and then use those mathematical, that mathematical description to understand or to make predictions based on the model and that those predictions are in agreement with what you know from experiments or from playing the instrument. Then you say okay, my model is describing the system well, and then I can use the model to understand okay, exactly what aspects of the model determine the frequency or what aspects are important for determining the directionality and which ones are less important? So, the goal of the modeling is to use a mathematical description to develop insights into the instrument, okay.
When I explained what I mean by modeling to when I ask that question, I almost always come back to a very famous quote by from Einstein (well, I guess, almost all Einstein’s quotes are very famous), but this one's very interesting. He, you know, he was he was known for developing simple, simple models to think about complex phenomenon. And his comment was a model should be as simple as possible, but no simpler. So that is the goal of the modeler is to, when you put your model together, when you assemble it or you when you get, when you abstract it, is to make, is to include the area or the aspects of the, in this case of the instrument, that are important for the for the behavior you're trying to gain insight to. And if you don't, so that sort of, not the trick, but that's—a lot of what's what it what it means to be a good modeler or to develop a good model of a particular instrument or particular phenomenon.
Okay, so how does research involving modeling of musical instruments compare to experimental or theoretical research?
Well, as I said, theoretical research rarely, rarely comes to grips with an exact physical or exact mathematical description of the instrument. So, you're usually making approximations about all the elastic behavior or other kinds of approximations. It's rare to have an exact descript mathematical description of an instrument. So that's, but that's similar to the way most theories are. And then of course, solving the model, or developing predictions from that model, often involve approximations. You'd, you, again, in physics, you can rarely solve a physics problem, you can rarely solve Schrodinger’s equation exactly right; you develop approximate techniques and so on. And the same thing is true often in acoustics. Now, the experiments come in, because they are the, in some sense, the ultimate arbiters of truth, right? So ultimately, you have to be prepared to compare the predictions you make from your model, the insights, with what you can, what you can see, what's measured experimentally, and that's the ultimate test. So in that sense, they work together in much the same way that they work together in what I would consider to be a normal kind of physics research endeavor.
Okay, that makes sense. What are some methods of modeling musical instruments? Or how have these methods developed over time?
Well, I think modeling of instruments goes back a long way. I mean, there's some very famous names that we can look to or point to as having thought about and written papers about and developed models of musical instruments. Helmholtz, his name comes up a lot. Raman, very famous Indian physicist. comes up a lot. They both, they both did a lot of important work on the violin and other instruments. So that work goes back more than 100 years. Now, I would say that starting in the, in the 60s and—well, back in the beginning, I'll say it was popular, when— well, let me back up. A lot of the work in developing a model and then making predictions based on the model, involve computer simulations or computer solutions of this mathematical description of the method, of the physics equations. Well, before computers were widely available, before they were very powerful, it was very common to use what are called “lump models.” In fact, lump models are used a lot even now, because they can be very insightful.
So I guess as an example, let's think about the piano. As in most, or many instruments, you can, you can think of the piano as sort of separate pieces that connect together to form the whole instrument. So you think about the strings, and the hammers hit the strings, those connect to the soundboard. And the sound board then vibrates in a way that pushes the air around that develop, that creates sound that we hear. And so we have these different pieces, these different pieces of the instrument. And so we could start by developing a model that describes the strings, or maybe describes the way the hammers hit the strings. And then add to that the soundboard. How do the strings exert a force on the soundboard and cause it to vibrate? And then finally, how does the soundboard vibrate and cause the air to move and and produce sound, which leaves the instrument in a directional away? So these lump models, this lumped approach, is very useful. And in some cases, it's very accurate, especially in like in the piano, because the strings and the hammers do their thing, exert force on the soundboard. The soundboard exerts relatively little force back on the strings. Now there's some return, because we know about the sustain pedal and so on, but rough—it’s not a terrible approximation to think about this thing is sort of moving the excitation, from the strings to the soundboard and then from the soundboard to the air. Then of course, you have to worry what, can I, do I have a good description of say, how are the strings coupled to the soundboard to strings? The strings, the force putting the strings in the soundboard can be treated in more and more sophisticated ways. The simplest way to treat it might not capture all the physics, all the acoustics you want. And so again, lump models can be developed and become more and more sophisticated depending upon what you're trying to understand.
Another example of a lump model would be say, in say, a recorder or a wind instrument. You can think about the mouthpiece region, where, which is what you blow into and where things happen that do with with complicated airflow and so on, as being separate from the, what’s called the resonator, so the body of the recorder or the body of the flute, then, so you can think about those separately. The body of the flute determines the main sounding frequencies. But the coupling between the mouthpiece and the body is extremely important. It's extremely complicated. So you might have to make approximations or assumptions about that coupling. As you develop better models that treat the mouthpiece and the resonator in a combined way, maybe you can do better and actually calculate that coupling. So those are examples of lump models and some of the complications, or the parameters one has to deal with, in understanding, really developing a lump model to where it can make predictions. So those are two examples. And there are many other examples of models.
Yeah, those are great examples. Thanks.
So and then, of course, it may be that the lump model can't answer or address the questions you're interested in. And then you have to go to a more sophisticated picture, which we can, which for example, in the wind instruments might involve just treating all the airflow using a Navier-Stokes equation—much more complicated mathematically, but that would be the sort of sometimes be the ultimate use of the fundamental physics equations, the fundamental laws of fluid dynamics for that particular instrument.
Amazing. So actually, what are some common equations from physics that are helpful in modeling musical instruments? In what ways do they need to be modified and why?
Okay, let's, let's go back to the piano. Obviously, strings, everything starts with the hammers and the strings or the strings. And as physicists, we've often, we've all taught the behavior of ideal strings. And so we know what kinds of, what their vibrational modes are, the frequencies they can produce, and so on. And we know that the ideal string produces a harmonic series of frequencies, f2, f3, f and so on. And sure enough, the piano, a piano tone, is also very close to being, having that kind of spectral composition. But if you look very, more carefully, you find that the frequencies aren't quite harmonic, and the higher frequencies are called “partials.” And the reason they aren't harmonic is because real strings have some stiffness. So the bending, the restoring force when you bend a string isn't just due to the tension, but it's also due to the elastic properties of that string. So for a piano, which has steel strings, it could be a millimeter in diameter, that restoring force is significant. It's not, it's not negligible compared to the restoring force due to the tension. So that's, those are, so we can start with the basic ideal string equations, we know second order partial differential equations, and then we can add more to it to include the effect of string stiffness. So that would be, those are common equations.
Now, some equations that aren't so common, at least to physicists, or weren't common to me when I got into this business, where the equations that describe the vibrations of a plate. The plate equations are well known to civil engineers and mechanical engineers, and those are fourth order differential equations. So, they're definitely not harmonic. They have more complex kinds of frequency properties. But those are some of the equations. Again, they all come from Newton's laws, but applying Newton's laws of different physical situations, different physical systems—strings or plates. And the same way for a wind instrument, one has to deal with the equations of fluid dynamics and Navier-Stokes equations, which again, are our partial differential equations. Those are there, they're nonlinear, so they're quite complicated. And they needed to be solved, and almost always by numerically. So those are, those are examples of the kinds of fundamental physics equations that we would encounter in modeling a musical instrument.
Okay. So how can numerical algorithms be used for simulating instruments sound?
Well, there are different approaches one might take. So considering, let's go back to the string. We could, one way to think about it or deal with it would be to think about the normal modes of the string. That's a popular thing for physicists to do. So we would think about the vibration of a string as being a sum of normal mode vibrations, so adding up the vibration due to the fundamental frequency, plus the vibration due to the 2f and the 3f and so on. And that would be kind of, that would be, what you could say is working in the frequency domain. The other way to think about it would be to work in the time domain and take the underlying differential equations. For example, it could be second order partial differential equations, and then write them as a system of finite difference algebraic equations, which are very amenable to solutions like, you know, linear or like linear equations with diagonalizing matrices, and so on. So those are two different ways that you might want to work.
I think, in the last 10 or 20 years, the most, much of the work has gone toward the being in the time domain-- thinking about finite difference approximations for these equations-- because it's much simpler, least for at least for me to think about time dependent properties, the attack part of a tone, things like that, if you're working in the time domain, as opposed to working in the frequency domain. You can do it either way, but I think conceptually, staying starting and staying in the time domain can for a musical tone, which we know is has time dependent properties can be a very natural way to work.
Okay. So how are material properties of instruments incorporated into numerical models?
Well, for like, for, again, go back to the piano string. We know what the diameter of the string is; we know what its composition is. I have particular interest in historical pianos. In the old days, back in Mozart's time, or, or even before that, the strings were made of brass or iron, but over, and over time, and starting up and culminating in about the mid 1800s, the strings became steel. So they had different mechanical properties which you'd have to deal with—they had higher tensions and so on. So things like the elastic properties would be one example. Another example, would be, well, how about the soundboard of a, for a musical instrument, for a piano or some other instrument. Wood is an incredibly complicated material. Now, fortunately, the wood for a piano soundboard or for a violin plate, are usually cut from the original piece of wood in a way which makes the grain structure have a simple a simple orientation relative to the flat sides of the wood. But in general, a piece of wood has 27 different elastic constants. So that's a nightmare. And that plus, we haven't even talked about the fact that no two pieces of wood, no two trees are identical, even if they're the same species, and so on. So those can be complications. Now, fortunately, at least in the case of wood, we don't think that the differences between two different samples of spruce that might be used for a soundboard, will cause enormous differences between the properties of a piano. I mean, two different Steinway pianos, you recognize them as pianos, and you might recognize them as Steinways. Now partly the Steinway people know how to choose their wood. The same was true for, you know, elite violin makers, too. But those are the kinds of complications was what that one was deal with, in terms of the properties of the materials themselves. So it's, I think that those complications are pretty much now under control, we understand those properties, and we know how to account for them.
So a lot of this discussion is talking about modeling musical instruments in isolation. But, of course, humans are involved playing instruments and creating sound. How do you account for musician-instrument interaction in models?
Well, that that's a very good question. That was one I think, if you if you want to ask what was one of the themes that emerged from the special issue, it was about the musician-musical instrument interaction, okay. And there's been more and more work done in that area. And that work can be thought of in several different ways. As one example, say for a wind instrument, it's now become very clear, and there have been more and more good experimental studies of the behavior, or how does the player like for say, a flute, how does the players of vocal track or vocal cavity influenced the sound is produced, especially for things like say a trumpet or a recorder or things like that? So we're beginning to develop ways to model the interaction of the player. And I should also say, the experimentalists are being more and more clever, or getting, have been very clever in developing ways to measure what goes on in the mouth of the player. So that's one. So including the player in the model, or in the mathematic description is becoming more and more possible, and we've been getting a good insights into that.
Now, another way in which the interaction is interesting, is in the experimental side. So I said we want to, we want to do an experiment, we want to play an instrument, measure and maybe record what the tone looks like, do different kinds of analysis, or the experimentalist may do other things, they may do their optical techniques for looking at the vortex motion of sound as it moves through an instrument or all kinds of things like that. So, but they all involve having somebody play the instrument. Now, experiments might go on for a long time, and it's very hard to find a trumpet player who's able to blow the trumpet for six hours at a time, in exactly the same way, so you can do experiments that are reproducible, and you can really study the systematics of what's happening. So again, the experimentalist are very clever; they developed, in many cases, different kinds of machines that play the instrument, sort of autonomously. So there are, I think, the first, perhaps the first experiments like this, where bowing machines for violins; I've seen now violin bowing machines, in which we, you know, you can, you've probably seen robot arms that can move and do all kinds of things, and they have all kinds of degrees of freedom. And I've seen cases where a robot arm is bowing a violin, okay, so that's very impressive, okay. Another example would be artificial mouths, where you blow a wind instrument, or you might blow a woodwind instrument. And again, but you can, you can blow them uniformly, reproducibly, and do controlled and detailed quantitative experiments.
You can also imagine, tweaking the way this artificial, artificial player interacts with the instrument: let's change the vocal track property; let's change the way the bowing is set up. And then we can really begin to ask the kinds of questions that that might be helpful for a player in terms of what am I doing? Or how can I change the way I approach my playing to get a different kind of sound or different kinds of tone, or just to change the way, the kinds of tones I'm able to produce? So that's a direction that I think we're going to see more and more of some more, some very good, good work. I have a friend who's done some nice work with this robot bowing arm. And I was kidding him, I was so impressed, I said, “Well, you need to buy two of those robot arms: one to hold the violin bow, and another to hold the rosin as you apply the rosin to the bow.” He thought that was funny, but he said, “That's going to be a little expensive.” But the experiments are getting, are really very impressive and becoming more and more, more and more sophisticated. So I think we're going to see a lot of a lot of really interesting work done in that direction in the coming years.
So now, are these experiments how you test and validate models? Or, I guess, just sort of a general question, how do you test or validate models?
It's by experiments like the ones I just described, with these artificial bowing machines. Of course, the old-fashioned way of just getting a player to play it, still works quite well. Some work that we've been doing, in my own group, with my collaborators, is doing simulations to say, Okay, if I change them the geometry of the mouthpiece of this instrument, how would that change the tone? Now, of course, we make predictions, we think we understand what's happening, but then we have to go out and validate it. So then we would actually make instruments with these new designs to check our predictions. Maybe we're right, maybe we missed something. So those are the kinds of directions and kinds of work, I think we're going to see more of.
That's really fun. So what are some obstacles that come up when you try to obtain high-quality measurements for physical model validation?
One of the challenges is, again, developing artificial players that are realistic stand ins for a real player. So I mentioned this robot arm bowing the violin? Well, I never played a violin, but I know that it's easy to make sounds that don't sound like a violin when you bow it. And so, and to become a proficient violinist takes a lot of time and a lot of practice. I think it's going to be, it's going to take a while for the people who are programming these robot arms to really develop accurate copies of what a real player does to play a violin in a professional way. Joshua Bell did not learn to play a violin overnight. And I think it'll be interesting to see how that works. The same thing is true with an artificial mouth to play a flute or a trumpet. There is a lot of nuance to playing an instrument. And I think that will, that's a direction which that we need to go, and that will be very, I can give a lot of new insights, but it will take a lot of a lot of time a lot of effort.
I think this is also a case where having scientist or acousticians who are proficient at playing these instruments will also be really important. I am a very amateur pianist. But when I was doing work on pianos, the insights I gained from sitting around on my piano, with my piano and noodling around, really helped me understand what we were doing when we were modeling. I think, and I've right now I'm doing some, some work with wind instruments. And we have some collaborators who have, who are proficient players. I think that kind of, their experience, their insights are really important in helping to understand how to put a good model together and what to look for. So I think that's another thing which is, which I think I've pretty much I think we, I found that in most of my interactions with acousticians working in this area, is they often are drawn to this because they like the, they love the instrument, they love theirinstruments. And that helps them become better researchers in doing this, either doing experiments or doing modeling or whatever kind of work they're doing. So that's one thing that's made, I think that's helped push this field along in a very nice way.
That makes a lot of sense—having that familiarity with playing the instrument and using the instrument to understand how to create these models. Yeah. And how to how to simulate a talented player rather than a, you know, random person who's never touched a violin before exactly.
Well, but and you find more and more papers, where you compare the beginner with a professional. And that also can you yield nice insights into what's happening. So
Yeah. So anyway.
So how can modeling results inform instrument makers?
Well, there are a couple of examples I could that are worth noting. In the case of—I’ll go back to the piano, the models of the piano we have now are really getting to be quite good, and they produce sounds that are very impressive. And now some people are using those models, to explore design modifications of the piano. Now, we all believe that the Steinway design is a pretty darn good one, okay? And it will-- probably won't be easy to improve on it. But you couldn't imagine, how do we know that that design of the soundboard is the best one we can do? And it, as you may know a soundboard has ribs on the bottom, and the number of ribs is not fixed, that means different makers do different things, the positions and again number and so on, and their dimensions and their thickness and so on. So you can imagine, or people are now doing developing models of the soundboard where you can hopefully take a software tool, go to a maker and say, “Here, explore different soundboard designs. And we can tell how they will affect the tone of the of the piano you would make with them.” And once they optimize things for the kind of tone they want, they could then maybe then they would then go and make a piano like that. But they could explore many, many designs for the soundboard without having to make each one into a full piano. So I think we're getting close to where, to that kind of a software tool that could be a real help for piano makers.
Violins are a tougher problem, but to understand what does it take to make a good violin? And how the same kind of, I don't think the models in that case are as quite as good as they are for the piano in terms of in terms of being able to tell a good violin from a really good violin. But I think that's the kind of thing we might ultimately get to is, how can we—what kinds of design modifications might we think about to make a better violin? I mean, I think that's, that gets complicated.
I mean—let me back up. To answer that question, we have to first agree on what it means to be good for a musical instrument, right? And, now we've become subjective, and we'll have arguments. So at the same time, people are putting a lot of effort into, okay, what can I measure about a violin that tells me this is a Stradivari violin, and this is a, I don't know, a John Smith, a John, whoever violin, okay? And we're still, I think we have a ways to go to be able to tell the difference, just from making physical measurements. Now, if we, if we play them, if someone like a Joshua Bell plays them for us, or we ask him his opinion, after he's played them, we can probably tell a difference. But it's a tough—that subjective jump is really tough. And it's a part of a musical instrument, right? And in fact, no, you probably shouldn't hope for a universal agreement, right? Because different people prefer different kinds of pianos. Some people love a Steinway. And they or others may really prefer a Bosendorfer. And you can get an argument about all that. And same thing with violins different, different types. So, but again, if we, I think there is some hope that we'll get to the point where, okay, this kind of soundboard is what makes a Steinway more of a Steinway. And this kind of soundboard is what makes the Bosendorfer more of what I expect from a Bosendorfer. I think that's probably, that's possible in the not too distant future. But it's not I don't know if we're there yet.
So it's essentially quantifying quality, in a way, or qualities.
Yes, yes. And that's, yeah, it's not easy.
No, doesn't sound like it. Um, so then, stemming from that, how can modeling results help those who play instruments?
Well, I think I kind of touched on that before. If we can tell a player, “The way you press the key for this piano makes a difference,” or makes a certain kind of difference, or the way you hold, the way you adjust your vocal track, or the way you blow into the instrument, or, for a flutist, the way you shape the air jet as you're blowing into the embouchure. Those kinds of things can help. And we, and I think the musicians I that I've been privileged to talk to who aren't physicists or aren't professional acousticians, they want to understand how and why what they do makes a difference. So even if they didn't take a lot of physics courses, they want to understand how that physics or how that acoustics, how that plays a role in their interaction with the instrument. So I think that alone is a very satisfying thing to be—for me, it's those kinds of conversations are very satisfying, and usually give me new insights into the instrument that I didn't have before. So this idea of having a research acoustician interact with a professional musician, I think there's a lot of a lot of potential for information transfer in both directions, which I think is just a it's— I guess it's part of science. It's part of, it's part of the arts, I would say.
Yeah. It's what makes it interesting, right?
Yeah. So you've already touched on this a bit, but where do you see the future of modeling of musical instruments heading?
Well, I think we'll always be trying to make the models more accurate, more exact, include more physics. I think one of the big directions we've already touched on would be this interaction between the player and the instrument. I think that will be a place that's got a lot of potential offerings. I think we’ll also sort of at the same time, see more and more sophisticated methods being used experimentally. For example, again, I mentioned I'm working now on wind instruments, but getting better pictures, better measurements of what the airflow looks like in the mouthpiece of a flute or in between the lips of a trumpet player will really help us develop better models of those instruments and others like those. So I think those are directions that I'm very excited about. And they're going to be others too.
It's also I think, something we I didn't we haven't mentioned here, but it was one of the themes in the special issue was a lot of, there's been some, lots of interesting applications of modeling to understand, I would say, cultural instruments, so instruments developed for different cultures that aren't part of the Western musical tradition, but are really very interesting, I mean, the didgeridoo we've all heard of, and so on. But instruments that have come from other kinds of cultures and societies, those are becoming, there's some interesting, very interesting studies of those instruments that are being done using a lot of the same methods that have been successful for, again, Western musical instruments, but getting new insights into the culture and into the musical culture of those societies. That's another interesting thing I think we're going to see more of too. And we had some, we have some interesting papers on that in this special issue.
Very cool. Do you have any closing thoughts?
Well, it's funny, my wife often asked me, “Why do you do this? I mean, we know what a piano sounds like or we know what a flute sounds like.” But to me under understanding in a physicist’s or an acoustician’s kind of way, just makes the instrument more satisfying. Understanding why, how the player is manipulating the tones, and make, and conveying his or her expressive desires, onto the music, understanding how that happens makes—is a very nice feeling for me as a listener, okay. But yeah, so I just, it's just , modeling I think is a lot of, has been a lot of fun for me, and it's been a fun thing to do in terms of my research.
Yeah, it definitely sounds fun. It sounds like it's a very—having that extra layer layer of understanding is very satisfying, in a way, so you can appreciate the music more. Well, thank you for taking the time to speak with us today. I've learned a lot about the fascinating endeavor of modeling musical instruments, and I am sure our listeners will be very happy to learn more, too.
Well, thanks very much for having me. It's been my pleasure.
Thank you for tuning into Across Acoustics. If you'd like to hear more interviews from our authors about their research, click subscribe and find us on your preferred podcast platform.