Research

We develop a new LfD model for the collaborative robot to learn from human demonstrations and assist its human partner in shared working situations. The human could program the robot using natural language instructions according to his/her personal working preferences. The robot can leverage its learned knowledge to actively assist the human in the collaborative assembly task.

Human comfort is significant in human-robot collaboration since it can influence the task efficiency and quality. In this project, we propose a computational Human Comfort Model (HuCoM) to model and quantify human comfort during human-robot collaborative manufacturing. Based on the defined primitive comfort rewards and combined comfort rewards, the HuCoM is developed with the incorporation of the static comfort model and the dynamic comfort model. ​With the human EEG signals, the robot actions can be customized by the human comfort levels in real-time.

Artificial intelligence is significantly improving the efficiency and productivity of the manufacturing sector. In this work, we develop a deep Q-network-based robot learning paradigm for human-robot collaborative manufacturing tasks. The proposed approach enables robots to learn task customization and execution by observing and analyzing human operations through a continuous feedback loop, thereby fostering adaptive, efficient, and intelligent human-robot collaboration in smart manufacturing environments.

In multi-human multi-robot collaboration, task allocation and scheduling present many underexplored challenges. We propose a nature-inspired, objective function-constrained task scheduling optimization framework for multi-human multi-robot collaborative remanufacturing. Based on the Dingo Optimization Algorithm, multiple objective functions are established to investigate how different optimization objectives influence task allocation, human participation, and the interpretation of the remanufacturing process in multi-human multi-robot collaborative settings.

The Multimodal Collaborative Robot System (MCROS) is developed to advance a broad portfolio of research initiatives spanning four colleges and 12 academic units (departments, institutes, and centers) at MSU. The MCROS will enable new research opportunities and significantly expand existing capabilities across five strategic focus areas: (1) intelligent systems and advanced computing, (2) ecology and chemistry sciences, (3) smart urban agriculture and food science, (4) human-factors engineering and social science, and (5) interactive learning.

Artificial intelligence has advanced rapidly, with large language models (LLMs) evolving into multimodal agents capable of sophisticated reasoning, planning, and interactive problem solving. We implemented multiple state-of-the-art LLMs, including variants of OpenAI ChatGPT, Google Gemini, and Anthropic Claude, on humanoid robot platforms to investigate human-AI interaction and collaborative task execution in complex, rule-based environments. These embodied AI systems support natural language communication, adaptive task learning, and collaborative decision-making, enabling more natural and effective partnerships among humans, AI, and intelligent systems.

We design and develop a controllable and customized human-AI interactive platform, AI sandbox, to advance cross-disciplinary research and education with large language models (LLMs). The AI sandbox features its integration and embodiment with generative AI, mainly including a multi-function graphical user interface (GUI), a reliable and security-oriented infrastructure, and a multi-LLM-enabled engine system. The AI sandbox is able to provide users with a reliable and scalable environment for developing, testing, and deploying AI applications.

Adopting a multi-robot team in a collaborative workspace can significantly boost the throughput and efficiency of manufacturing tasks. In this project, a merge sort-based task planning approach is proposed for multi-robot collaboration in shared tasks. We develop a multi-robot simulation platform with three (can be scaled to three more) six degrees of freedom (DOF) collaborative robots sharing a common workspace. The merge sort algorithm is used for the multi-robot team to plan collaborative assembly tasks. 

On automotive assembly lines, many associates are employed to clean the dust inside the vehicle body by holding industrial vacuum cleaners. In this project, to reduce the cleaning time and facilitate ergonomics, a Collaborative Robotic Vacuum (CoRV) system is developed for humans in automotive body cleaning tasks. A human-CAD co-guided robot motion planning approach based on the cost function is proposed for the CoRV to automatically generate optimized cleaning waypoints and trajectories with collision avoidance. 

Robotics and artificial intelligence (AI) have increasingly expanded smart and precision technological solutions for the agricultural industry, especially in time-consuming and costly farming tasks. We developed a robot-assisted crop maturity recognition and harvest system to accurately detect the stages of crop ripeness. The proposed approach integrates collaborative robotics, computer vision, image processing, and AI. A transfer learning-based model is created for the robot to recognize the crop in its maturity stages and locate the crop during real-time detection.

Soil monitoring and management are essential yet labor-intensive practices that underpin sustainable agricultural production. In this work, we developed an autonomous robot soil sampling system capable of monitoring key soil properties and providing data-driven insights to support precision and sustainable soil management. The system is integrated with a mobile collaborative robot and incorporates a resistive soil moisture sensor and an RS-485 NPK (Nitrogen, Phosphorus, Potassium) sensor, controlled by dual microcontrollers for automated sensing, data acquisition, and soil analysis.

We develop a human manipulation understanding approach using a wearable sensory system, which has a natural and simple configuration and can be easily used by humans. This approach could make the robot recognize human hand-over intentions and enable the human to naturally, flexibly, and conveniently control the hand-over process. In addition, the approach could recognize the attribute classes of the objects in the human’s hands using wearable sensing and enable the robot to actively make decisions to ensure that graspable objects are handed over from human to robot.​

We develop a task cost evaluation model using online optimization in human-robot collaboration. First, we extract the task model by graphical representations and design the collaboration cost functions which incorporate time consumption and human efforts. After that, the robot action planning algorithms are developed to online plan robot actions by evaluating and optimizing the collaborative assembly cost. In addition, appropriate robot actions can be planned by the proposed algorithms to ensure the accomplishment of assembly process when the human happens to conduct wrong actions during the collaboration.​

We propose a FAP method for autonomous vehicles via the computer vision. By taking advantage of the input images from a rear camera on the vehicle, a convolutional neural network (CNN) based on the Caffe deep learning framework is trained to automatically output the steering and velocity commands for vehicle control.​

In the high-energy large-scale scientific facility system, the controllers and devices are operated on three electric potential platforms: ground, 350 kV, and 400 kV. We develop a high-speed redundancy optical fiber ring network with high availability for monitor level and control level. The specialized intranet can complete the network reconfiguration within a few milliseconds when the communication failure occurs, and has no effect on the progress of logic control and data terminal.​