Kim Sung-Bin

I am a Ph.D. student at POSTECH (Pohang University of Science and Technology) in the Department of Electrical Engineering, advised by Prof. Tae-Hyun Oh at the Algorithmic Machine Intelligence (AMI) Lab. My research interests lie in audio-visual learning, multi-modal learning, cross-modal generation, and multi-modal large language models.

Experiences

Adobe Research San Francisco, CA, USA

Jun. 2026 – Sep. 2026

Research Intern · Sound Design AI (SODA) Group

Video-to-audio generation and linearizing audio codecs.

Speech, Audio, and Language Technologies Lab, UT Austin Austin, TX, USA

Sep. 2024 – Mar. 2025

Visiting Researcher

Automated video dubbing, advised by Prof. David Harwath.

ROK
Army

Vanguard Unit, Republic of Korea Army Cheorwon, South Korea

Sep. 2016 – Jun. 2018

Sergeant (Mandatory Military Service)

Publications

Papers are sorted by recency. 20 shown · ^* denotes equal contribution.

Conference · Under Review2026

Retrofitting Linearity into Pretrained Audio Codecs

Kim Sung-Bin et al.

Under review

Conference2026

Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs

Kim Sung-Bin, Lee JungMok, J. Jung, Oh Hyun-Bin, H. Ryu, David Harwath, Tae-Hyun Oh

NeurIPS 2026

Conference2026

FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling

Kim Sung-Bin, Joohyun Chang, David Harwath, Tae-Hyun Oh

NeurIPS 2026

arXiv project page

Conference2026

Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources

Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida, Kim Sung-Bin, Toshimitsu Uesaka, Takashi Shibuya, Kyeongyoon Lee, Tae-Hyun Oh, Yuki Mitsufuji

EMNLP 2026

arXiv

Conference2026

Physics-Aware Deepfake Detection via Distance–Speech Consistency

Kyeong-Rae Kim, Kim Sung-Bin, Oh Hyun-Bin, Tae-Hyun Oh

Interspeech 2026

code

Conference2026

Efficient Hyper-Parameter Search for LoRA via Language-aided Bayesian Optimization

Baek Seong-Eun, Lee JungMok, Kim Sung-Bin, Tae-Hyun Oh

ICML 2026

arXiv

Conference2026

SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter

Lee Jung-Mok, Kim Sung-Bin, Joohyun Chang, Lee Hyun, Tae-Hyun Oh

ACL 2026 (Oral)

paper code

Conference2025

AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation

Jeongsoo Choi, Ji-Hoon Kim, Kim Sung-Bin, Tae-Hyun Oh, Joon Son Chung

ACM Multimedia 2025

arXiv project page code

Conference2025

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models

Kim Sung-Bin, Jeongsoo Choi, Puyuan Peng, Joon Son Chung, Tae-Hyun Oh, David Harwath

ICCV 2025

arXiv project page

Conference2025

Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics

Chae-Yeon Lee, Hyun-Bin Oh, EunGi Han, Kim Sung-Bin, Suekyeong Nam, Tae-Hyun Oh

CVPR 2025 (Highlight, 3.7%)

arXiv project page

Conference2025

AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models

Kim Sung-Bin^*, Hyun-Bin Oh^*, JungMok Lee, Arda Senocak, Joon Son Chung, Tae-Hyun Oh

ICLR 2025

arXiv code

Conference2025

SoundBrush: Sound as a Brush for Visual Scene Editing

Kim Sung-Bin, Jun-Seong Kim, Junseok Ko, Yewon Kim, Tae-Hyun Oh

AAAI 2025

arXiv project page

Conference2024

MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset

Kim Sung-Bin^*, Chae-Yeon Lee^*, Gihun Son^*, Hyun-Bin Oh, Janghoon Ju, Suekyeong Nam, Tae-Hyun Oh

Interspeech 2024

arXiv project page dataset

Conference2024

Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert

EunGi Han^*, Hyun-Bin Oh^*, Kim Sung-Bin, Corentin Nivelet Etcheberry, Suekyeong Nam, Janghoon Ju, Tae-Hyun Oh

Interspeech 2024

project page

Conference2024

SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

Hyun Lee^*, Kim Sung-Bin^*, Seungju Han, Youngjae Yu, Tae-Hyun Oh

NAACL 2024 Findings

Also presented at the AV4D and MMFM workshops at ICCV 2023.

arXiv dataset

Conference2024

LaughTalk: Expressive 3D Talking Head Generation with Laughter

Kim Sung-Bin, Hyun Lee, Da hye Hong, Suekyeong Nam, Janghoon Ju, Tae-Hyun Oh

WACV 2024

arXiv project page

Journal2026

Revisiting Learning-based Video Motion Magnification for Real-time Processing

Hyunwoo Ha^*, Hyun-Bin Oh^*, Jun-Seong Kim, Byung-Ki Kwon, Kim Sung-Bin, Linh-Tam Tran, Ji-Yun Kim, Sung-Ho Bae, Tae-Hyun Oh

Transactions on Machine Learning Research (TMLR), 2026

arXiv project page

Journal2024

A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization

Youwang Kim, Hyun Lee^*, Kim Sung-Bin^*, Suekyeong Nam, Janghoon Ju, Tae-Hyun Oh

Transactions on Machine Learning Research (TMLR), 2024

project page paper

Journal2024

The Devil is in the Details: Simple and Effective Optical Flow Synthetic Data Generation

Byung-Ki Kwon, Kim Sung-Bin, Tae-Hyun Oh

The Visual Computer, 2024

arXiv

Workshop2023

Spatio-Temporally Consistent Face Mesh Reconstruction on Videos

Youwang Kim, Hyun Lee^*, Kim Sung-Bin^*, Suekyeong Nam, Janghoon Ju, Tae-Hyun Oh

CVPR Workshop on 3DMV, 2023

poster

Conference2023

Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment

Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Andrew Owens, Tae-Hyun Oh

CVPR 2023

Covered by Yonhap News, SBS, and local media; also presented at AI4CC (CVPR 2023), AV4D (ICCV 2023), and the Sound to Sight workshop (CVPR 2023, invited talk).

arXiv project page paper

Conference2023

Prefix Tuning for Automated Audio Captioning

Minkyu Kim^*, Kim Sung-Bin^*, Tae-Hyun Oh

ICASSP 2023 (Oral)

Covered by Yonhap News and SBS.

arXiv project page paper

Journal2021

Lightweight Speaker Recognition in Poincaré Spaces

Lightweight Speaker Recognition with Discriminative Mapping in Poincaré Spaces

Jieun Lee^*, Kim Sung-Bin^*, Seokhyeong Kang, Tae-Hyun Oh

IEEE Signal Processing Letters, 2021

paper

Domestic Conference2020

Real-time Face Registration and Classification System using Fuzzy ARTMAP

Kim Sung-Bin, Wong Hyong Lee

ICROS 2020 (Best Student Paper Award)

paper

Journal · Under Review2024

IJCV

Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment

Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Tae-Hyun Oh

IJCV, under review

arXiv code

Education

POSTECH Pohang, South Korea

Mar. 2023 – Present

Ph.D. in Electrical Engineering

Advisor: Tae-Hyun Oh · AMI Lab

POSTECH Pohang, South Korea

Mar. 2021 – Feb. 2023

M.S. in Electrical Engineering

Advisor: Tae-Hyun Oh · AMI Lab

Thesis: Learning to Generate Visual Scene and Text Description from Sound

Handong Global University Pohang, South Korea

Mar. 2015 – Aug. 2020

B.S. in Electrical Engineering

Summa Cum Laude

Honors and Awards

Outstanding Reviewer CVPR 2026 · Jun. 2026
SBS Cultural Foundation Scholarship Full-ride scholarship for Ph.D. studies (up to $75,000 over four years) · Aug. 2023 – Present
Summa Cum Laude Handong Global University · Aug. 2020
Best Student Paper Award ICROS 2020 (Institute of Control, Robotics and Systems Annual Conference) · Jul. 2020
National Science & Technology Scholarship Full-ride scholarship during B.S. studies (total $24,000) · Mar. 2015 – Aug. 2020

Academic Service

Journal Reviewer

IJCV
IEEE/ACM TASLP
TMLR

Conference Reviewer

CVPR
ICCV
ECCV
ICML
ICLR
NeurIPS
SIGGRAPH
BMVC
ACM Multimedia

Teaching Experience

POSTECH

Jan. 2021 – Dec. 2021

Teaching Assistant · Visual Intelligence

NAVER & UPSTAGE BoostCamp

Jan. 2021 – May 2021

Mentor · Computer Vision

Patents

Image Generating Apparatus U.S. patent application · P2023111-03-US
Motion Amplification Devices and Method of Using Thereof Republic of Korea · 10-2021-0184021