CoCoSys: Co-Design of Cognitive Systems

Background

A figure illustrating the overview of the different themes in CoCoSys.

CoCoSys aims to develop the next generation of human-AI systems by advancing algorithms, specialized hardware, and collective intelligence. These innovations will be applied to collaborative robots, digital assistants, and mixed-reality systems, which require responsiveness, energy efficiency, and trustworthiness. The project seeks to improve current AI systems, moving towards a future where digital humans reliably work alongside real humans.
To achieve this, CoCoSys will focus on four key areas:

  1. Algorithms: Creating explainable AI that goes beyond perception to include reasoning and decision-making.
  2. Algorithm-Hardware Co-Design: Ensuring algorithms and hardware are optimized for each other.
  3. Technology-Driven Hardware: Designing efficient and high-performance cognitive hardware.
  4. Collaborative Intelligence: Enhancing how AI systems work together and interact with humans.

Why CARE

The CoCoSys project exemplifies the union of advanced technology with a strong commitment to society, aligning with the core mission of CARE. This project is socially responsible by ensuring that AI systems are designed to work in harmony with humans, enhancing collaboration without replacing human roles, thus fostering a more inclusive future. Trustworthiness is built into CoCoSys through the development of transparent and explainable AI algorithms, ensuring that AI decisions are understandable and can be trusted by users. Sustainability is at the forefront of CoCoSys, achieved by creating energy-efficient hardware and AI systems designed for long-term use, reducing the environmental impact.

Our Focus

Theme II: Algorithm-Hardware Co-design

Full-stack Optimization and Software Frameworks for Cognitive Systems: This task is vital for ensuring that AI algorithms and hardware work together seamlessly. By optimizing the entire system, from hardware to software, this task enhances performance, reduces energy use, and ensures the system handles complex tasks efficiently. We are co-optimizing the entire system, considering all components, using advanced modeling and simulation to fine-tune performance. This work is key to making the AI systems in CoCoSys both powerful and efficient.

Acknowledgment

Defense Advanced Research Projects Agency

Semiconductor Research Corporation

Project Team

PI: Yingyan (Celine) Lin

Ph.D. Student: Yonggan Zhang

Ph.D. Student: Luke Zhang

Ph.D. Student: Chaojian Li

Internal (GT) Collaborators

Prof. Arijit Raychowdhury

Prof. Tushar Krishna

Prof. Larry P. Heck

External Collaborators

Related Publications

  1. Sixu Li, Yang Zhao, Chaojian Li, Bowei Guo, Jingqun Zhang, Wenbo Zhu, Zhifan Ye, Cheng Wan and Yingyan (Celine) Lin, “Fusion3D: Integrated Acceleration for Instant 3D Reconstruction and Real-Time Rendering”, The 57th IEEE/ACM International Symposium on Microarchitecture (MICRO 2024)
  2. Zhongzhi Yu, Zheng Wang, Yonggan Fu, Huihong Shi, Khalid Shaikh, Yingyan (Celine) Lin, “Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration”, the Forty-first International Conference on Machine Learning (ICML 2024)
  3. Yongan Zhang, Zhongzhi Yu, Yonggan Fu, Cheng Wan, Yingyan (Celine) Lin, “MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation”, LAD’24: International Workshop on LLM-Aided Design
  4. Yujie Zhao, Yang (Katie) Zhao, Cheng Wan, Yingyan (Celine) Lin, “3D-Carbon: An Analytical Carbon Modeling Tool for 3D and 2.5D Integrated Circuits”, Design Automation Conference 2023 (DAC 2024).
  5. Zhongzhi Yu, Zheng Wang, Yuhan Li, Ruijie Gao, Xiaoya Zhou, Sreenidhi Reddy Bommu, Yang (Katie) Zhao, Yingyan (Celine) Lin, “EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning & Voting”, Design Automation Conference 2023 (DAC 2024).
  6. Yongan Zhang, Yonggan Fu, Zhongzhi Yu, Kevin Zhao, Cheng Wan, Chaojian Li, Yingyan (Celine) Lin, “INVITED: Data4AIGChip: An Automated Data Generation and Validation Flow for LLM-assisted Hardware Design”, Design Automation Conference 2023 (DAC 2024).
  7. Haoran You, Huihong Shi, Yipin Guo, Yingyan (Celine) Lin, “ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer”, Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)
  8. Shunyao Zhang, Yonggan Fu, Shang Wu, Jyotikrishna Dass, Haoran You, Yingyan (Celine) Lin, “NetDistiller: Empowering Tiny Deep Learning via In-Situ Distillation”, IEEE Micro 2023.
  9. Yonggan Fu, Yongan Zhang, Zhongzhi Yu, Sixu Li, Zhifan Ye, Chaojian Li, Cheng Wan, Yingyan (Celine) Lin, “GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models”, The IEEE/ACM International Conference on Computer-Aided Design 2023 (ICCAD 2023).
  10. Yonggan Fu, Ye Yuan, Souvik Kundu, Shang Wu, Shunyao Zhang, Yingyan (Celine) Lin, “NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations”, The Fortieth International Conference on Machine Learning (ICML 2023).
  11. Zhongzhi Yu, Yang Zhang, Kaizhi Qian, Cheng Wan, Yonggan Fu, Yongan Zhang, Yingyan (Celine) Lin, “Master-ASR: Achieving Multilingual Scalability and Low-Resource Adaptation in ASR with Modular Learning”, The Fortieth International Conference on Machine Learning (ICML 2023).
  12. Yang (Katie) Zhao, Shang Wu, Jingqun Zhang, Sixu Li, Chaojian Li, Yingyan (Celine) Lin, “Instant-NeRF: Instant On-Device Neural Radiance Field Training via Algorithm-Accelerator Co-Designed Near-Memory Processing”, Design Automation Conference 2023 (DAC 2023).
  13. Sixu Li, Chaojian Li, Wenbo Zhu, Boyang Yu, Yang Zhao, Cheng Wan, Haoran You, Huihong Shi, Yingyan Lin, “Instant-3D: Instant Neural Radiance Field Training Towards On-Device AR/VR 3D Reconstruction”, The 50th IEEE/ACM International Symposium on Computer Architecture 2023 (ISCA 2023)
  14. Yonggan Fu, Zhifan Ye, Jiayi Yuan, Shunyao Zhang, Sixu Li, Haoran You, Yingyan Lin, “Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design”, The 50th IEEE/ACM International Symposium on Computer Architecture 2023 (ISCA 2023)
  15. Yonggan Fu, Yuecheng Li, Chenghui Li, Jason Saragih, Peizhao Zhang, Xiaoliang Dai, Yingyan Lin, “Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence”, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023)
  16. Haoran You, Yunyang Xiong, Xiaoliang Dai, Bichen Wu, Peizhao Zhang, Haoqi Fan, Peter Vajda, Yingyan Lin, “Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention During Vision Transformer Inference”, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023)
  17. Zhongzhi Yu, Shang Wu, Yonggan Fu, Shunyao Zhang, Yingyan (Celine) Lin, “Hint-Aug: Drawing Hints from Foundation Vision Transformers towards Boosted Few-shot Parameter-Efficient Tuning”, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023).

Related News

We presented our posters and demos at the CoCoSys annual review!