News and Awards

• Our paper “Deep learning with diffusion MRI as in vivo microscope reveals sex-related differences in human white matter microstructure” was published in Scientific Reports (paper link).
• Our paper “A neural speech decoding framework leveraging deep learning and speech synthesis” was published in Nature Machine Intelligence (paper link). The related press release can be found here.
• We have a new press report about our project “Object-Centric, View-Adaptive and Progressive Coding and Streaming of Point Cloud Video” (jointly with Prof. Yong Liu and Prof. Luke DuBois).

Book Chapters

Published in , 1900

Y Wang, J Ostermann, YQ Zhang, Video Processing and Communications, Upper Saddle River, Prentice Hall, 2002.

Journals and Conference Papers (Since 2010)

Published in , 1900

You can find these papers and more details on the Google Scholar page.

Other Presentations

Published in , 1900

• Presentation in a short course on Deep Learning for Computer Vision in the Deep Learning School in 2022 Autumn (link)
  • Lecture outline
  • Session 1
  • Session 2
  • Session 3
• Keynote presentation at Picture Coding Symposium 2018, “FoV Adaptive 360 degree video streaming”. (slides)
• Keynote presentation at ACM Multimedia Systems (MMSys) 2020, “FoV Adaptive 360 degree video streaming”. (slides)
• Keynote talk titled “Compression for Scene Perception and Understanding: Deep-Learning Approaches” in the 2022 Picture Coding Symposium, December 7-9, 2022.
• Keynote presentation on Learnt compression for visual analytics on the edge at the IEEE Workshop on Coding for Machines, in conjunction with ICME 2023.

PhD Dissertations and MS Theses

Published in , 1900

• Nikola Janjušević, “Noise Adaptive Dictionary Learning based Deep Neural Networks for Imaging Inverse Problems”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Aug. 2024.
• Xingyu Pan, “APPLICATION OF FEATURE COMPRESSION MODEL ON MOBILE DEVICES”, Tandon School of Engineering, New York University, Master Thesis, Sep. 2024.
• Zhongzheng (Jacky) Yuan, “Visual Analytics Through Edge Servers: Learned Feature Compression and Adaptive Video Coding”, Tandon School of Engineering, New York University, Ph.D. Dissertation, May 2024.
• Parisima Abdali, “MR Contrast Synthesis Using Deep Learning”, Tandon School of Engineering, New York University, Master Thesis, May 2024.
• Chenqian Le, “Neural Decoding and Understanding via Deep Learning”, Tandon School of Engineering, New York University, Master Thesis, May 2024.
• Zhiqi Chen, “Deep Learning for Glaucoma Diagnosis and Monitoring and for Video Processing”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Dec. 2023.
• Junbo Chen, “Machine Learning Application to Study Human Brain: The Investigation of Brain Microstructure and Speech Decoding based on Cortical Neural Activity”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Dec. 2023.
• Prerna Luthra, “Automated Respiratory Pattern Analysis for Dynamic MRI of the Lung of Post COVID-19 patients at 0.55 T”, Tandon School of Engineering, New York University, Master Thesis, Dec. 2023.
• Amirhossein Khalilian-Gourtani, “Model-based Signal Processing: Application to Brain Connectivity Analysis and Natural Image and Video Processing”, Tandon School of Engineering, New York University, Ph.D. Dissertation, September 2022.
• Zhipeng Fan, “On Improving the Efficiency and the Accuracy of Human Pose Estimators”, Tandon School of Engineering, New York University, Ph.D. Dissertation, June 2022.
• Ades-Aron B, “Noise and Artifact Reduction in MRI: Impact on Accuracy, Reproducibility and Clinical Translation”, Tandon School of Engineering, New York University, Ph.D. Dissertation, May 2022.
• Yixiang Mao, “Coding and Streaming System Design for Interactive 360° Video Applications and Scalable Octree-based Point Cloud Coding”, Tandon School of Engineering, New York University, Ph.D. Dissertation, May 2022.
• Ran Wang, “Decoding Speech from Human Cortex and Interpreting Cortical Networks”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Jan. 2022.
• Ziming Qiu, “Deep Learning Application to 3D Vision: Volumetric Medical Image Analysis and Camera Pose Estimation”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Jan. 2022.
• Jacky Yuan, “Block-based Image Coding with Autoencoder and Border Information”, Tandon School of Engineering, New York University, MS Thesis, May 2020.
• Jeffrey Mao, “IWASTE: Medical Waste Video Classification”, Tandon School of Engineering, New York University, MS Thesis, May 2020.
• Alp Aygar, “Pareto-Efficient Training of Multi-Task Deep Networks”, Tandon School of Engineering, New York University, MS Thesis, September 2019.
• Zhaoxi Chen, “Reconstructing Speech Stimuli from Human Auditory Cortex Activity Using GAN based Approach”, Tandon School of Engineering, New York University, MS Thesis, May 2019.
• Chenge Li, “Deep Learning for Object Detection and Tracking and for Field of View Prediction in 360-degree Videos”, Tandon School of Engineering, New York University, Ph.D. Dissertation, Jan. 2019.
• Yuan Wang, “Biophysically interpretable recurrent neural network for functional magnetic resonance imaging analysis”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2018.
• An-Ti Chiang, “Kinect-Based In-Home Exercise System for Lymphatic Health and Lymphedema intervention”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2018.
• Shervin Minaee, “Image Segmentation Using Subspace Representation and Sparse Decomposition”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2018.
• Fanyi Duanmu, “Fast Screen Content Coding and Two Tier 360 Degree Video Streaming”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2018.
• Yilin Song, “Machine learning for neural activity analysis and for object tracking in video”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2018.
• Eymen Kurdoglu, “Application Layer System Design for Real-time Video Communications”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2017.
• Yuanyi Xue, “Beyond HEVC: Exploring New Frameworks for Video Coding”, NYU Tandon School of Engineering, Ph.D. Dissertation, 2016.
• Jen-Wei Kuo, “Automatic High-Frequency Ultrasound Image Segmentation And Shape Analysis”, NYU Tandon School of Engineering, Ph.D. Dissertation, May 2016.
• Chenge Li, “A Comparison Of Machine Learning And Feature Selection Methods For Predicting Breast Cancer Related Lymphedema”, NYU Tandon School of Engineering, Master Thesis, May 2015.
• Xuan Zhao, “Automatic image segmentation and Treatment planning for radiotherapy”, Polytechnic School Of Engineering, New York University, Ph.D. Dissertation, May 2014.
• Beril Erkin, “Age estimation and gender classification from detected faces in video”, Polytechnic School of Engineering, New York University, MS thesis. New York, May 2014.

Technical Reports

Published in , 1900

• Yixiang Mao, Yao Wang, “Interactive 360-degree Video Coding Experiments (Supplementary Documents ACM MM2020)”, Last updated: Aug 2020.
• Fanyi Duanmu, “Screen Content Compression: A Brainstorming Report”, Last updated: Aug 2019.
• Yilin Song, Chenge Li, Yao Wang, “Pixel-wise object tracking”, Initial version: Nov. 2017, Last updated: July 2018.
• Yilin Song, Yao Wang, and Johnathan Viventi, “Adversarial autoencoder analysis on human μECoG dataset”, Dec. 2017.
• Yilin Song, Yao Wang, and Jonathan Viventi, “Multi Resolution LSTM For Long Term Prediction In Neural Activity Video”, Initial version: May 2017, Last updated: July 2018.
• Yilin Song, Jonathan Viventi, and Yao Wang, “Diversity encouraged learning of unsupervised LSTM ensemble for neural activity video prediction”, Initial version: Nov. 2016, Last updated: July 2018.
• Chenge Li, Gregory Dobler, Xin Feng, Yao Wang, “TrackNet: Simultaneous Object Detection and Tracking and Its Application in Traffic Video Analysis”.
• Yilin Song, Jonathan Viventi, Yao Wang, “Unsupervised Learning of Spike Pattern for Seizure Detection and Wavefront Estimation of High Resolution Micro Electrocorticographic (μECoG) Data”.
• Zhili Guo, Yao Wang, “Assessing the Visual Effect of Non-periodic Temporal Variation of Quantization Stepsize in Compressed Video”, Submitted to IEEE International Conference on Image Processing (ICIP) 2015. Accepted.
• Zhili Guo, Yao Wang, Elza Erkip, and Shivendra Panwar, “Wireless Video Multicast with Cooperative and Incremental Transmission of Parity Packets”, Submitted to IEEE Transactions on Multimedia. Accepted.
• Yen-Fu Ou, Yuanyi Xue, and Yao Wang, “Q-STAR: a perceptual video quality model considering impact of spatial, temporal and amplitude resolutions”, accepted, IEEE Transaction on Image Processing.
• Yuanyi Xue, Beril Erkin, and Yao Wang, “A novel no-reference video quality metric for evaluating temporal jerkiness due to frame freezing”, submitted to IEEE Transaction on Image Processing.
• Yvonne W. Lui, Yuanyi Xue, Damon Kenul, Yulin Ge, Robert I. Grossman, and Yao Wang, “Classification algorithms using multiple MRI features in mild traumatic brain injury”, submitted to Neurology, 2014.
• Yuanyi Xue and Yao Wang, “Video coding using a self-adaptive redundant dictionary consisting of spatial and temporal prediction candidates”, submitted to IEEE International Conference on Multimedia and Expo (ICME) 2014, Chengdu, China, Jul. 2014.
• Yuanyi Xue, Yilin Song, Yen-Fu Ou, and Yao Wang, “Video adaptation considering the impact of temporal variation on quantization stepsize and frame rate on perceptual quality”, Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, Arizona, Jan. 2013.
• Yuanyi Xue and Yao Wang, “Perceptual quality comparison between single-layer and scalable videos at the same spatial, temporal and amplitude resolutions”, Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, Arizona, Jan. 2013.
• Yongxia Zhou, Yao Wang, Damon Kenul, Yuanyi Xue, Yulin Ge, Joseph Reaume, Robert I. Grossman, and Yvonne W. Lui, “Detection of mild traumatic brain injury utilizing multifeature analysis of MRI”, ISMRM 2013, Salt Lake City, Utah, Apr. 2013.
• Zhan Ma, Hao Hu, Meng Xu, and Yao Wang, “Rate model for compressed video considering impacts of spatial, temporal and amplitude resolutions and its applications to video coding and adaptation”, March 2012.
• Yen-Fu Ou, Yuanyi Xue, and Yao Wang, “Q-STAR: A perceptual video quality model for mobile platforms considering impact of spatial, temporal and amplitude resolutions”, submitted to IEEE Journal on Selected Areas of Communications, Aug. 2011.
• Yuanyi Xue and Yao Wang, “Perceptual quality comparison between single-layer and scalable videos at the same spatial, temporal and amplitude resolutions”, submitted to IEEE ICIP, Jan. 2012.
• Hao Hu, Xiaoqing Zhu, Yao Wang, Rong Pan, Jiang Zhu, and Flavio Bonomi, “Proxy-Based Multi-Stream Scalable Video Adaptation over Wireless Networks Using Subjective Quality and Rate Models”, submitted to IEEE trans. on Multimedia, Nov. 2011.
• Xuan Zhao, Dewen Kong, Jenghwa Chang, Edward K. Wong, Gabor Jozsef, Silvia C. Formenti, Yao Wang, “Automatic beam placement for breast radiotherapy using a Support Vector Machine based algorithm”, accepted by Med. Phys., Mar. 2012.
• Yen-Fu. Ou, W. Lin, H. Zeng, and Y. Wang, “Perceptual Quality of Video with Frame Rate and Quantization Variation : a subjective study and analytical modeling”, Submitted to Circuits and Systems for Video Technology (CSVT), IEEE Transactions on, 2012.
• Yen-Fu Ou, H. Zeng, Y. Wang, “Perceptual Quality of Video with Quantization Variation: a subjective study and analytical modeling”, Submitted to IEEE International Conference on Image Processing (ICIP), 2012.

Critical Region Prediction

Project Summary

Current neurosurgical methods for mapping the language cortex, such as Electrical Stimulation Mapping (ESM), face significant limitations including invasiveness, time consumption, and patient cooperation challenges. Electrocorticography (ECoG) has emerged as a potential improvement by offering enhanced spatial and temporal resolution. However, its application in language mapping has been constrained by traditional analysis methods that focus narrowly on signal strength at individual electrodes. In this study, we propose a novel approach by employing transformer-based machine learning models to analyze ECoG data comprehensively. Our findings suggest that transformer architectures can advance the precision and effectiveness of language mapping techniques, potentially overcoming some of the limitations of current methods.

Neural Speech Decoding

Project Summary

Decoding human speech from neural signals is essential for brain–computer interface (BCI) technologies that aim to restore speech in populations with neurological defcits. However, it remains a highly challenging task, compounded by the scarce availability of neural signals with corresponding speech, data complexity and high dimensionality. Here we present a novel deep learning-based neural speech decoding framework that includes an ECoG decoder that translates electrocorticographic (ECoG) signals from the cortex into interpretable speech parameters and a novel diferentiable speech synthesizer that maps speech parameters to spectrograms.

Title

Project Summary

Title

Project Summary

Talk 1 on Relevant Topic in Your Field

Published: March 01, 2012

This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!

Tutorial 1 on Relevant Topic in Your Field

Published: March 01, 2013

More information here

Talk 2 on Relevant Topic in Your Field

Published: February 01, 2014

More information here

Conference Proceeding talk 3 on Relevant Topic in Your Field

Published: March 01, 2014

This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.

, , 1900

ECE-GY 6123 – Image and Video Processing (Fall 2024)

Image and Video Processing Project Guidelines

, , 1900

Course Project Requirements:

, , 1900

ECE-GY 6123 – Image and Video Processing (Spring 2022)

, , 1900

ECE-GY 6123 – Image and Video Processing (Spring 2023)