Our group’s research focuses on human-centric computer vision, including human identification and activity recognition, AI Robotics and 3D Modelling, and the related machine learning algorithm research.

 

 

Human Identification and Activity Recognition

 

We have a long research term for researching human identification, including person re-identification and face recognition. Since 2013, we also have extensive research on activity recognition and assessment.

 

[Person Re-identification]

 

The person re-identification is to match the same person’s images captured at different space and different time across non-overlapping camera views in a visual surveillance system. We have worked on this problem since 2008. Our research covered from metric learning (relative distance comparison early proposed in 2009) to deep neural network solution. We have in-depth research on the challenging problems in person re-id, including our early investigation of occluded person re-id, RGB-Infrared person re-id, low-resolution person re-id, clothing-change person re-id, fine-grained person re-id, open-world person re-id. Recently, we revisited person re-id again and present the consistent person re-id problem existing in almost previous person re-id solution, where a consistent tracking is ignored when training conventional re-id models.

图示 描述已自动生成

图示 描述已自动生成

图片包含 图示 描述已自动生成

图示 描述已自动生成

图形用户界面, 应用程序 描述已自动生成

图示 描述已自动生成

 

- Representative Publications:

1. Yi-Xing Peng, Yuanxun Li, Wei-Shi Zheng. Revisiting Person Re-identification by Camera Selection. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), to appear, 2024.

2. Jingke Meng, Wei-Shi Zheng, Jian-Huang Lai, Liang Wang. Deep Graph Metric Learning for Weakly Supervised Person Re-Identification. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 44, no. 10, pp. 6074-6093, 2022.

3. Wei-Shi Zheng, Jincheng Hong, Jiening Jiao, Ancong Wu, Xiatian Zhu, Shaogang Gong, Jiayin Qin, and Jian-Huang Lai. Joint Bilateral-resolution Identity Modeling for Cross-resolution Person Re-identification. International Journal of Computer Vision (IJCV), vol. 130, pp. 136-156, 2022.

4. Qize Yang, Ancong Wu, Wei-Shi Zheng. Person Re-identification by Contour Sketch under Moderate Clothing Change. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 6, pp. 2029-2046, 2021.

5. Jiaxing Chen, Xinyang Jiang, Fudong Wang, Jun Zhang, Feng Zheng, Xing Sun, Wei-Shi Zheng. Learning 3D Shape Feature for Texture-insensitive Person Re-identification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

6. Hong-Xing Yu, Ancong Wu, Wei-Shi Zheng. Unsupervised Person Re-identification by Deep Asymmetric Metric Embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 42, no. 4, pp. 956-973, 2020.

7. Jiahang Yin, Ancong Wu, and Wei-Shi Zheng. Fine-grained Person Re-identification. International Journal of Computer Vision (IJCV), vol. 128, pp. 1654–1672, 2020.

8. Ancong Wu, Wei-Shi Zheng, Shaogang Gong, Jianhuang Lai. RGB-IR Person Re-Identification by Cross-Modality Similarity Preservation. International Journal of Computer Vision (IJCV), vol. 128, pp. 1765–1785, 2020.

(Conference Version: Ancong Wu, Wei-Shi Zheng, Hongxing Yu, Shaogang Gong, Jianhuang Lai. RGB-Infrared Cross-Modality Person Re-Identification. IEEE Conf. on Computer Vision (ICCV), 2017.)

9. Ying-Cong Chen, Xiatian Zhu, Wei-Shi Zheng, and Jianhuang Lai. Person Re-Identification by Camera Correlation Aware Feature Augmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 40, no. 2, pp. 392-408, 2018.

10. Xiatian Zhu, Botong Wu, Dongcheng Huang, Wei-Shi Zheng. Fast Open-World Person Re-Identification. IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2286-2300, 2018.

11. Xiang Li, Ancong Wu, Wei-Shi Zheng. Adversarial Open-World Person Re-Identification. In European Conference on Computer Vision(ECCV), 2018.

12. Ancong Wu, Wei-Shi Zheng, Jianhuang Lai. Robust Depth-based Person Re-identification. IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2588-2603, 2017.

13. Wei-Shi Zheng, Shaogang Gong, and Tao Xiang. Towards Open-World Person Re-Identification by One-Shot Group-based Verification. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 38, no. 3, pp. 591-606, 2016.

14. Wei-Shi Zheng, Xiang Li, Tao Xiang, Shengcai Liao, JianHuang Lai, Shaogang Gong. Partial Person Re-identification. IEEE Conf. on Computer Vision (ICCV), 2015.

15. Wei-Shi Zheng, S. Gong, and T. Xiang. Person Re-identification by Probabilistic Relative Distance Comparison. IEEE Conf. on Computer Vision and Pattern Recognition, 2011.

16. Wei-Shi Zheng, S. Gong, and T. Xiang. Associating Groups of People. BMVC 2009.

 

 

[Action & Activity Recognition]

 

In our early stage, we were interested in the Human-Object-Interaction (HOI) recognition, either between human and object or between human and human. In the meanwhile, we also have ever investigated the action anticipation problem. During this period, we have quite a lot of algorithms on multi-modal collaboration modelling.

 

Recently, our interests on activity understanding go towards action assessment, which aims to tell whether an action performs in principle. This includes the first perspective and the third perspective modelling for activity. We have proposed learning adaptive assessment function rather than manually designed for each action, as well as learning to quantify the interaction for more detailed assessment of interactive actions. Currently, we are focusing on first perspective modelling, so that we can do the assessment when a human is working with wearable device (such as vision pro etc.)

 

图示 描述已自动生成

图片包含 图形用户界面 描述已自动生成

图形用户界面, 图示 描述已自动生成

 

- Representative Publications:

1. Haoxin Li, Wei-Shi Zheng, Jianguo Zhang, Haifeng Hu, Jiwen Lu, Jian-Huang Lai. Egocentric Action Recognition by Automatic Relation Modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 1, pp. 489-507, 2023.

2. Jibin Gao, Jia-Hui Pan, Wei-Shi Zheng. Automatic Modelling for Interactive Action Assessment. International Journal of Computer Vision (IJCV), 2023.

(Related Conference Version: Jibin Gao,  Wei-Shi Zheng, Jia-Hui Pan, Chengying Gao, Yaowei Wang, Wei Zeng, Jianhuang Lai. An Asymmetric Modeling for Action Assessment. In European Conference on Computer Vision (ECCV), 2020.)

3. Kun-Yu Lin, Jia-Run Du, Yipeng Gao, Jiaming Zhou, Wei-Shi Zheng. Diversifying Spatial-Temporal Perception for Video Domain Generalization. Advances in Neural Information Processing Systems (NeurIPS), 2023.

4. Jiahui Pan, Jibin Gao, Wei-Shi Zheng. Adaptive Action Assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 44, no. 12, pp. 8779-8795, 2022.

(Conference Version: Jia-Hui Pan, Jibin Gao, Wei-Shi Zheng. Action Assessment by Joint Relation Graphs.  In IEEE Conference on Computer Vision (ICCV), 2019.)

5. Angchi Xu, Ling-An Zeng, Wei-Shi Zheng. Likert Scoring with Grade Decoupling for Long-term Action Assessment. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 

6. Peizhen Zhang, Yongyi Tang, Jian-Fang Hu, and Wei-Shi Zheng. Fast Collective Activity Recognition Under Weak Supervision. IEEE Transactions on Image Processing, vol. 29, no. 1, pp. 29-43, 2020.

7. Fa-Ting Hong, Xuanteng Huang, Wei-Hong Li, and Wei-Shi Zheng. MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection. In European Conference on Computer Vision (ECCV), 2020.

8. Jian-Fang Hu, Wei-Shi Zheng, Lianyang Ma, Gang Wang, Jianhuang Lai, and Jianguo Zhang. Early Action Prediction by Soft Regression. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 43, no. 11, pp. 2568-2583, 2019.

9. Shaofan Lai, Wei-Shi Zheng, Jian-Fang Hu, and Jianguo Zhang. Global-Local Temporal Saliency Action Prediction. IEEE Transactions on Image Processing, vol. 27, no. 5, pp: 2272-2285, 2018.

10. Jian-Fang Hu, Wei-Shi Zheng, Jianhuang Lai, Jianguo Zhang. Jointly Learning Heterogeneous Features for RGB-D Activity Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 39, no. 11, pp. 2186-2200, 2017.

(Conference Version: Jian-Fang Hu, Wei-Shi Zheng, Jian-Huang Lai, and Jianguo Zhang. Jointly Learning Heterogeneous Features for RGB-D Activity Recognition. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2015.)

11. Xiaobin Chang,  Wei-Shi Zheng, and Jianguo Zhang. Learning Person-Person Interaction in Collective Activity Recognition. IEEE Transactions on Image Processing, vol. 24, no. 6, pp. 1905-1918, 2015.

 

 

AI Robotics and 3D modelling

 

This is our new efforts on learning activity when interacting with robotics in the future. For this purpose, we recently start researching AI robotics from the computer vision perspective. We are focusing on learning to grasp. We conceive that grasping is fundamental function for a robotic, which is still a big challenge. The grasping we learning including two-finger grip, three-finger grip, and five-finger grip(namely Dexterous Grasping). In the meanwhile, we also learn human grasp generation.

图示 描述已自动生成

桌子上放着一些玩具 中度可信度描述已自动生成

图片包含 文本 描述已自动生成

图示 描述已自动生成

图示 描述已自动生成

卡通人物 中度可信度描述已自动生成

 

- Representative Publications:

1. Zhixuan Liu, Zibo Chen, Wei-Shi Zheng. Simulating Complete Points Representations for Single-view 6-DoF Grasp Detection. IEEE Robotics and Automation Letters (RA-L), 2024.

2. Yu-Kun Qiu, Guo-Hao Xu, Wei-Shi Zheng. Inner-Outer Aware Reconstruction Model for Monocular 3D Scene Reconstruction. Advances in Neural Information Processing Systems (NeurIPS), 2023.

3. Zibo Chen, Zhixuan Liu, Shangjin Xie, Wei-Shi Zheng. Grasp Region Exploration for 7-DoF Robotic Grasping in Cluttered Scenes. International Conference on Intelligent Robots and Systems (IROS), 2023.

4. Zhixuan Liu, Zibo Chen, Shangjin Xie, Wei-Shi Zheng. TransGrasp: A Multi-Scale Hierarchical Point Transformer for 7-DoF Grasp Detection. ICRA 2022: 1533-1539

 

 

Related Machine Learning Algorithms

 

In our work, our group works on 1) online classifier; 2) fast search; 3) large-scale clustering. Recently, we are interested in continual learning and weakly supervised learning.

 

- Related Publications:

1. Xiao-Ming Wu, Dian Zheng, Zuhao Liu, Wei-Shi Zheng. Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training. IEEE/CVF International Conference on Computer Vision (ICCV), 2023.   

2. Yu-Ming Tang, Yi-Xing Peng, Wei-Shi Zheng. When Prompt-based Incremental Learning Does Not Meet Strong Pretraining. IEEE/CVF International Conference on Computer Vision (ICCV), 2023.

3. Yu-Ming Tang, Yi-Xing Peng, Wei-Shi Zheng. Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Data. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

4. Tanli Zuo, Yukun Qiu,  Wei-Shi Zheng. Neighbor Combinatorial Attention for Critical Structure Mining. International Joint Conference on Artificial Intelligence (IJCAI), 2020.

5. Ganzhao Yuan, Li Shen, Wei-Shi Zheng. A Decomposition Algorithm for the Sparse Generalized Eigenvalue Problem. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

6. Shuhan Tan, Jiening Jiao,  Wei-Shi Zheng. Weakly Supervised Open-set Domain Adaptation by Dual-domain Collaboration. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

7. Long-Kai Huang, Qiang Yang, Wei-Shi Zheng. Online Hashing. IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2309-2322, 2018.

(Conference Version: Longkai Huang, Qiang Yang,  Wei-Shi Zheng. Online Hashing. International Joint Conference on Artificial Intelligence (IJCAI), 2013.)

8. Chenghao Zhang and Wei-Shi Zheng. Semi-supervised Multi-view Discrete Hashing for Fast Image Search. IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2604-2617, 2017.

10. Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. Quantized Correlation Hashing for Fast Cross-modal Search. International Joint Conference on Artificial Intelligence (IJCAI), 2015.

11. Qiang Yang, Longkai Huang, Wei-Shi Zheng, Yingbiao Ling. Smart Hashing Update for Fast Response. International Joint Conference on Artificial Intelligence (IJCAI), 2013.

12. Wei-Shi Zheng, Jian-Huang Lai, and Pong C. Yuen. Penalized Pre-image Learning in Kernel Principal Component Analysis. IEEE Trans. on Neural Networks, vol. 21, no. 4, pp. 551-570, 2010.

 

 

Computational photography

 

Our group also works on computational photography. Sometimes, I am also interested in several research topics, such as AIGC etc.

 

- Representative Publications:

1. Qing Zhang, Hao Jiang, Yongwei Nie, and Wei-Shi Zheng*. Pyramid Texture Filtering. ACM Transactions on Graphics (ACM TOG), SIGGRAPH 2023.

2. Qing Zhang, Jin Zhou, Lei Zhu, Wei Sun, Chunxia Xiao, and Wei-Shi Zheng. Unsupervised Intrinsic Image Decomposition Using Internal Self-similarity Cues. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 44, no. 12, pp. 9669-9686, 2022.