When he died in 1827 aged 56, Ludwig van Beethoven left his 10th symphony unfinished. Only a few handwritten notes briefly detailing his plans for the piece have survived, with most just being incomplete ideas or fragments of themes or melodies.
Now, a multidisciplinary team of computer scientists at Rutgers University-based start-up Playform AI have trained an artificial intelligence to mimic the great composers style and used it to write a complete symphony based on these initial sketches.
We spoke to the lead researcher on the project, Professor Ahmed Elgammal, to find out more.
Beethoven left sketches in different forms, mainly musical sketches, but also some written notes with some ideas in as well. Previously, in 1988 [English musicologist] Barry Cooper used the majority of these sketches, about 250 bars of music, that were meant for a first movement [in his attempt to complete the symphony].
But what was left behind is really very little. So basically, like three bars of music here and four bars of music there and some rough sketches, which sound like basically the starting points of the main themes in the movements that he [Beethoven] wanted to write.
When you look at Beethoven and other classical composers, thats usually the case. I mean, usually they work with a main theme and develop it into a sequence of a couple of minutes and then another theme comes. Thats the traditional way of composing, and thats exactly what the AI needed to learn how Beethoven and other classical composers start with a theme and develop it. Like in the Fifth Symphony da da da dah. And then take that and evolve a whole movement around it.
Read more about artificial intelligence and music:
The way AI generates music in general is very similar to the way your email, for example, tries to predict the next word for you. So, when you write an email, you find it jumps into suggesting what you might want to write next.
Its the same concept, basically the AI has to learn from a lot of musical data. It asks what would be the next note given what you just wrote? And if you can predict the next note, then you can predict the next note and the next note and so on. Thats the main concept.
But what we soon realise is that if you start picking up the suggestions from the phone for next word and start writing just based on the AIs suggestions, it doesnt really hold for a long time. And thats what happens with music. If you just give it a starting point and leave it to predict, yes, it can predict a couple of notes. But then after that, it becomes nonsense more or less, and is no longer faithful to the main theme.
So that was the main challenge. How can we let the AI stick to the main theme and develop it? So this is where the role of the human expert working with the AI comes in. So we had to work with human experts to annotate and label a lot of music for us to tell the AI what the theme was and where the development of the theme was in a lot of pieces of music. So basically, the AI learnt as a student. That made a big difference because then the AI could really keep sticking to the theme.
Also, the AI had to compose the music in a specific musical form. So if you are composing for a scherzo movement or a trio part of the movement or a fugue etc, each of these musical forms have certain specific structure. The AI also had to learn how to write a fugue, how to write a trio, how to write a fugue, and how to write a scherzo.
It was very challenging because Beethoven only wrote nine symphonies. Thats a very small dataset compared to the scale of what the AI needed to do. So, the way we approached this was to first imagine ourselves like a young Beethoven learning about music. What he would have listened to?
So, we trained our first version of the AI as if it was somebody living in the 18th Century listening to baroque music like Bach, as well as Hayden and Mozart. And so that was the first version of the AI, which basically would be the kind of music anyone living in that era would study to compose. And then we took that and trained it specifically on Beethoven on old Beethoven sonatas, concertos, string quartets and the symphonies as well, so not only symphonies.
We first trained the AI to generate the composition as two lines of music, not as a full symphony, which is a typical way of a composer works by just composing first and then orchestrating. So then, we had another AI that would take that composition and learn how to orchestrate it. I believe this is very similar to the way humans learn you cannot really master fourth-level college without going through the first and second and third levels first. Its always incremental.
The symphony was premiered by The Beethoven Orchestra Bonn on 9 October 2021 Deutsche Telekom
The way we harmonise music is very similar to how we use AI to translate languages. Like when you use Google Translate or another AI to translate a sentence from one language to another. These kind of models used in translation learn a lot of background sentences. So, what is the sentence in German? What is the sentence in English? And from that, they try to learn how to translate them.
So basically, imagine you have these models [for harmonisation]. You put the melody in one side and on the other side you put in how Beethoven would harmonise it so the AI learns how to translate a melody line into harmonised music.
The thing about music is that its very structured and follows a lot of rules. But this is very hard for us to capture and write down. You really have to have a PhD in musicology with a speciality in Beethoven to really understand that. But the machine is able to capture that statistically and mathematically in a very implicit way and be able to use that to give us this harmonisation.
You got it right. That decision is just an extension of the harmonisation. We wanted the machine to translate the composition into multi-track instrumentation, which we also did by training the AI based on how Beethoven and other composers would do so.
Their response is really mixed. There are people who loved this very much, and love the idea of having an AI that understands music and can help you finish your composition or have you explore different musical ideas.
But on the other side of the spectrum, there are people who just reject even the concept of being able to complete a Beethoven symphony using AI. They are afraid of AI taking their jobs and think that it has nothing to do with this kind of thing.
Yeah. I have no doubt about that, we did that in visual art a couple of years ago where we developed an almost autonomous AI artist we had look at, lets say, the last 500 years of western art. The task was basically to generate new artworks that didnt follow any existing style.
If the AI generated an impressionist or a Picasso kind of art or a Renaissance-style artwork, it could realise and so it would have to learn how to create something new.
The challenge with this project was actually the constraints the fact that the AI was not generating music by itself but generating music that is based on Beethovens genius and also following the sketches. This makes it even more difficult. The high bar, of course, of expectation was due to the sketches coming from Beethoven. But when it comes to generating music autonomously I think thats an easier task.
Listen to the symphony below:
Read more about music:
Read more from the original source:
- SAP : Helping to Close the Computer Science Education Disparity - marketscreener.com - December 3rd, 2021
- Women of the Well House supports new campus programs: Indiana University Kokomo - IUK Newsroom - December 3rd, 2021
- Diving Into The Strange World Of Xenobots - Science Friday - December 3rd, 2021
- Silicon Valley beckoned, but he went home to Delaware - MIT News - December 3rd, 2021
- Two UB faculty receive NSF funding to study social media in disaster response - UB Now: News and views for UB faculty and staff - University at... - December 3rd, 2021
- Santa Reminds Us to Be Our Best Year Round - goskagit.com - December 3rd, 2021
- Announcing SWENext Award Recipients from WE21 - All Together - Society of Women Engineers - December 3rd, 2021
- App State researchers working to save honeybees with informatics - Watauga Democrat - December 3rd, 2021
- Hundreds of UC faculty named among the world's Highly Cited Researchers - University of California - December 3rd, 2021
- Kids, teens believe girls aren't interested in computer science, study shows - UW News - November 28th, 2021
- At UF, we are working together to provide Florida teachers and students with computer science skills | Column - Tampa Bay Times - November 28th, 2021
- Pakistani scientist ranked among top 1% computer science researchers - Geo News - November 28th, 2021
- Computer Scientist Gets Grant To Strengthen Software Protection - University of Texas at Dallas - November 28th, 2021
- Ahead of their time - The People's Defender - November 28th, 2021
- Computer Conservation - Harvard School of Engineering and Applied Sciences - November 28th, 2021
- Singapore's first free, teacher-less computer science programme to be launched by 2022 - The Straits Times - November 28th, 2021
- A chance to focus on an academic passion at Oxford - Harvard Gazette - November 28th, 2021
- Northern College hosts gaming tournament and hopes to attract future students with a new computer program - CTV News Northern Ontario - November 28th, 2021
- Student cyber competition team is fifth of 120 competing in DOE contest - UAH News - November 28th, 2021
- New algorithm could save lives: Predicts COVID-related intensive care unit resource use University of Copenhagen - EurekAlert - November 28th, 2021
- COVID-19: Vitamin D may be able to 'switch off' lung inflammation - Medical News Today - November 28th, 2021
- In MIT visit, Dropbox CEO Drew Houston '05 explores the accelerated shift to distributed work - MIT News - November 28th, 2021
- This SanDisk NVMe SSD deal is too good to pass up even if you don't need a new SSD - TechRadar - November 28th, 2021
- Where are all the hacktivists? - BCS - November 28th, 2021
- Save 97% off a course bundle that covers all things computer science - BleepingComputer - November 17th, 2021
- Cameron University's CU in Computing presentation to focus on Computer Science and Information Technology degree programs - Duncan Banner - November 17th, 2021
- Bias in Algorithms | The Inference Project - Yale News - November 17th, 2021
- Infosys Commits to 3-Year Investment in Thurgood Marshall College Fund - Yahoo Finance - November 17th, 2021
- The tech behind popular cryptocurrencies, explained - Popular Science - November 17th, 2021
- When will a robot write a novel? Harvard computer scientist shares his thoughts - Harvard Gazette - November 17th, 2021
- At SC21, Plenary Wrestles With the Ethics of Mainstreamed HPC - HPCwire - November 17th, 2021
- Governor Abbott Encourages Texas High Schoolers To Participate In CyberStart America - Office of the Texas Governor - November 17th, 2021
- USF physicists selected as finalists for the 'Nobel Prize of supercomputing' - University of South Florida - November 17th, 2021
- 4 universities nurturing tomorrow's women STEM experts - Study International News - November 17th, 2021
- SAS and Sphero Address Coding Needs of Students with Visual Impairments - T.H.E. Journal - November 17th, 2021
- Portland State University experiments with Attend Anywhere model - OPB News - November 17th, 2021
- The long term effects of sexism on women in STEM - Utahstatesman - November 13th, 2021
- Cameron University's CU in Computing presentation to focus on Computer Science and Information Technology degree programs - Chickasha Express Star - November 13th, 2021
- Certificate in computer science: What you need to know - ZDNet - November 13th, 2021
- Calling outstanding young researchers in mathematics and computer science! - EurekAlert - November 13th, 2021
- Governor Hutchinson's Weekly Address | Arkansas Takes the World Stage with New Technologies - Governor Asa Hutchinson - November 13th, 2021
- New CMU Robotics director says diversity is key to the institute's future - 90.5 WESA - November 13th, 2021
- Simulating Galaxy Formation in Mesmerizing Detail for Clues to the Universe - SciTechDaily - November 13th, 2021
- Collaboration on Data and Computational Sciences Announces 2021-2022 Projects to Advance Cancer Breakthroughs - HPCwire - November 13th, 2021
- Algorithms aren't fair. Robin Burke wants to change that - CU Boulder Today - November 13th, 2021
- Research Associate in Artificial Intelligence / Robotics, Multi-Agent Path Finding job with ROYAL HOLLOWAY, UNIVERSITY OF LONDON | 271630 - Times... - November 13th, 2021
- Report: Southeast New Mexico's rural school districts struggle in computer science - Carlsbad Current Argus - November 10th, 2021
- Washington People: Chenyang Lu - The Source - Washington University in St. Louis - Washington University in St. Louis Newsroom - November 10th, 2021
- GA Dept of Ed Awards Third Round of Grants to Build Computer Science Teacher Capacity - All On Georgia - November 10th, 2021
- Facebook Isn't Shutting Down Its Facial Recognition System After All - News @ Northeastern - News@Northeastern - November 10th, 2021
- Tableau to add new business science tools to analytics suite - TechTarget - November 10th, 2021
- Top 10 Best Women Programmers of All Time - Analytics Insight - November 10th, 2021
- This 26-year-old doubled his income by switching to computer scienceand using a key negotiating trick - CNBC - November 8th, 2021
- Partnership advances K-12 classrooms computer science classes to among the top in the nation - Beckley Register-Herald - November 8th, 2021
- WVU Today | WVU, W. Va. Dept. of Education partnership advances K-12 classrooms computer science classes to among the top in the nation - WVU Today - November 8th, 2021
- More Than Half of High Schools Now Offer Computer Science, But Inequities Persist - Education Week - November 8th, 2021
- West Virginia among top in the country for access to computer science education - My Buckhannon - November 8th, 2021
- $15 million to the University of Utah and $25 million to Utah Valley University will expand computer science programs - Salt Lake Tribune - November 8th, 2021
- Assistant Professor of Computer Science job with University of North Carolina at Greensboro | 401826 - The Chronicle of Higher Education - November 8th, 2021
- Free Tucson course teaches basics of computing to attract new workers - Arizona Daily Star - November 8th, 2021
- 'Ever Brighter' campaign a highlight at Homecoming - University of Miami - November 8th, 2021
- 3 Questions: Blending computing with other disciplines at MIT - MIT News - November 8th, 2021
- Data Analytics: How to prepare for the next big job of the future - Study International News - November 8th, 2021
- 2U Announces Partnership with the National University of Singapore, One of Asia's Top-Ranked Global Universities, to Expand Online Professional... - November 8th, 2021
- UMass Amherst College of Information and Computer Sciences to Unveil Major Private Gift and Significant New State Investment on Wednesday - UMass News... - October 26th, 2021
- $25M gift to help fund computer science, information building | The University Record - The University Record - October 26th, 2021
- Assistant Professor, Teaching in Computer Science job with DURHAM UNIVERSITY | 269406 - Times Higher Education (THE) - October 26th, 2021
- XJTLU now in ESI Top 1% in the world in 3 fields - WSAZ-TV - October 26th, 2021
- Mapping the mind's eye | ASU News - ASU Now - October 26th, 2021
- Hunger Drove Me into it; Computer Science Graduate into Fixing Nails for the Past 16 Years Says in Video Nigeria news - Legit.ng - October 26th, 2021
- Thriving, Where Business and Technology Meet | Maryland Smith - Robert H. Smith School of Business - October 26th, 2021
- New Vice Provost for Digital Education on innovating for educational equity - Stanford Report - Stanford University News - October 26th, 2021
- UA researchers set to take part in energy study - Arkansas Online - October 26th, 2021
- Bachelor's in Computer Science - October 14th, 2021
- Computer Science Courses, Subjects, Colleges, Syllabus ... - October 14th, 2021
- Bringing Computer Science to the Ocean with BluePlanetAI - News@Northeastern - October 14th, 2021
- Project aims to expand pool of computer science teachers and computer scientists - The Den - October 14th, 2021
- Research helps Blugold discover how computer science, health care intersect - University of Wisconsin System - October 14th, 2021
- Researchers receive grant to predict the mechanics of living cells - EurekAlert - October 14th, 2021
- University of Queensland Professor David Abramson Wins High Performance Computing Award - HPCwire - October 14th, 2021