Software Defined Media and LiVRation: Building Interactive Object-Based Viewing Spaces

[2014 – 2019]

A programmable media platform enabling immersive and interactive live experiences through networked control of spatial audiovisual environments.

As internet-native audiovisual services continue to expand, new expectations are emerging for immersive experiences that go beyond traditional one-way broadcasting. At Tsukada Lab, we have been leading the Software Defined Media (SDM) consortium since 2014, focusing on object-based content delivery and real-time, software-defined control of viewing environments.

The SDM architecture virtualizes AV equipment—such as microphones, speakers, and lighting—as IoT-controllable resources. By exposing them via APIs, developers can build applications that dynamically alter sound images or visual perspectives based on user location or preferences. This approach turns conventional live venues into programmable platforms, where user-specific experiences can be generated. The architecture has been evaluated and demonstrated in multiple hackathons and exhibitions.

Based on this foundation, we developed “LiVRation,” an interactive remote live-streaming platform. LiVRation captures live music performances using 360-degree video and 3D audio, allowing viewers with head-mounted displays (HMDs) to explore the event space from any point of view. Through integration with event metadata, users can freely switch perspectives and focus on specific sound or visual sources. The system was awarded First Prize at the Billboard Live Music Hackasong 2017, hosted by Billboard Japan.

This project aims to make media experiences programmable by merging networking and spatial computing. The platform holds potential for diverse applications including smart buildings, remote education, virtual tourism, and next-generation entertainment experiences that blur the boundary between physical and digital realities.

Publication

• インタラクティブな遠隔ライブVR配信プラットフォーム
  粕谷貴司,, 塚田学,, 菰原裕,, 高坂茂樹,, 水野拓宏,, 野村譲誉,, 雄太, 上田, 浩, 江崎. 情報処理学会論文誌デジタルコンテンツ（DCON）, vol. 7, no. 2, pp. 1-14, 2019, ISBN: 2187-8897 [Journal Article]BibTeX
• LiVRation: Remote VR live platform with interactive 3D audio-visual service
  Kasuya, Takashi, Tsukada, Manabu, Komohara, Yu, Takasaka, Shigeki, Mizuno, Takuhiro, Nomura, Yoshitaka, Ueda, Yuta, Esaki, Hiroshi. IEEE Games Entertainment & Media Conference (IEEE GEM) 2019, pp. 1-7, Yale University, New Haven, CT, U.S., 2019 [Proceedings Article]BibTeX
• Software Defined Media: 視聴空間サービスのソフトウェア制御
  塚田学,, 小川景子,, 池田雅弘,, 曽根卓朗,, 丹羽健太,, 齊藤翔一郎,, 粕谷貴司,, 砂原秀樹,, 江崎浩,. 日本ソフトウェア科学会学会誌『コンピュータソフトウェア』「ネットワーク技術」特集, vol. 34, no. 3, pp. 37-58, 2017, ISBN: 0289-6540 [Journal Article]BibTeX
• Software Defined Media: Virtualization of Audio-Visual Services
  Tsukada, Manabu, Ogawa, Keiko, Ikeda, Masahiro, Sone, Takuro, Niwa, Kenta, Saito, Shoichiro, Kasuya, Takashi, Sunahara, Hideki, Esaki, Hiroshi. IEEE International Conference on Communications (ICC2017), pp. 1-7, Paris, France, 2017, ISBN: 1938-1883 [Proceedings Article]BibTeX