KAIST's AI Pianist Trades Solos With Jazz Musicians at Korea's Jarasum Jazz Festival

Artificial intelligence systems built at the Korea Advanced Institute of Science and Technology (KAIST) shared the stage with human musicians at South Korea's 23rd Jarasum Jazz Festival, the university announced on Friday, Oct. 16. According to The Korea Times, which reported the announcement, 12 jazz artists performed 26 pieces with the AI across 12 performances at the festival in Gapyeong County, Gyeonggi Province, which ran Oct. 9 to 11.

The team, led by Professor Nam Ju-han of KAIST's Graduate School of Culture Technology, presented two systems. The first, an AI pianist named Scott, compares a musician's live playing with the sheet music in real time to keep track of where the piece is. The Herald Business reported that this lets Scott pick up a performance according to the actual flow of the music, even when a player slows down or shifts the timing, rather than following a preset schedule. In some sets, Scott generated new solo passages that reflected the melody and rhythm of the musician who had just played. The Herald Business said Scott played on nine pieces.

Scott's output was sent to a Yamaha Disklavier, an acoustic reproducing piano, which played the music with real key and pedal movements. On stage, an augmented reality character representing the pianist was overlaid on live camera footage, driven by technology that simulates a pianist's hand movements, The Korea Times reported.

The second system, an AI accompanist called DeepPocket, listens to a human performer and generates bass and drum parts in real time, adjusting to the notes, rhythm and tempo it hears. According to The Herald Business, it played eight pieces alongside a jazz pianist without any prepared backing tracks. The Korea Times reported that it improvised with pianists and guitarists without prior preparation.

The program ran under the name Jazz AI Lounge. A preview published by SpecialTimes Global described it as a joint project of the festival, KAIST's Music and Audio Computing Lab (MACLab) and Studio Meta K, supported by the Korea Creative Content Agency. Maeil Business Newspaper reported that pianists Heo Dae-wook and Kwak Jung-min, as well as vocalist Moon Mi-hyang with bassist Kim Dae-ho, performed with the AI pianist system.

The project builds on earlier work by the same team, which in 2023 used AI to adjust the timing of piano accompaniment during a performance by soprano Sumi Jo, according to both reports. The Herald Business said the team sees possible uses as a practice partner for musicians playing alone and in personalized music education.

"The performances demonstrated that AI can listen to human musicians and create music alongside them by responding to the flow of their performances, marking a significant advance from simply playing predefined music," Nam said, as quoted in English by The Korea Times. He added that the team will continue research toward AI as "a musical partner that can exchange creative inspiration with human musicians, rather than simply serving as a tool."

The accounts above are based on KAIST's announcement and Korean press coverage. JazzLib has not reviewed recordings of the sets.