Virtual Reality platform for Cartesian manipulation of digital twin in a UR3 robot
Article Sidebar
Main Article Content
Introduction: the development of trajectory programming and control platforms for collaborative robots has significantly advanced with the integration of emerging technologies such as virtual reality.
Objetive: the primary objective of this study is to develop and implement a virtual reality platform that enables the programming and control of trajectories for a UR3 robot. Additionally, the study aims to improve system accessibility and usability by allowing users to define Cartesian trajectories through visual interaction and by integrating a digital twin of the robotic arm.
Methodology: the system was developed using Unity 3D, enabling the creation of an interactive visual interface for users. A digital twin of the UR3 robot was integrated, which synchronizes with the Meta Quest 2 VR headset to provide an immersive experience. Users can define linear trajectories by placing control points, with the ability to easily add or delete points within the virtual environment.
Results: the implementation of the platform allowed users to effectively define and control trajectories within a virtual reality environment. It was observed that users were able to interact with the system intuitively, creating trajectories without prior robotic programming knowledge. Moreover, the use of the digital twin provided an accurate real-time visual representation of the robot’s behavior..
Conclusions: this study demonstrates that the integration of virtual reality with collaborative robot trajectory control enhances user accessibility and interaction with the system. This approach not only facilitates learning and trajectory programming but also lays the foundation for future improvements in the graphical interface and control functionalities, enabling system customization according to specific user needs.
- Virtual Reality
- System Trajectory Programming
- Robotics
- Digital Twin
Ministerio de las tecnologías de la información y la telecomunicación - Aspectos básicos de la industria 4.0. [Online].; 2019 [cited 2023 03 23]. Available from: https://colombiatic.mintic.gov.co/679/w3-article-124767.html.
Rejikumar G, Raja V, Arunprasad P, Persis J, Sreeraj K. Industry 4.0: key findings and analysis from the literature arena. Benchmarking: An International Journal. 2019. https://www.emerald.com/insight/content/doi/10.1108/bij-09-2018-0281/full/pdf
Moran R, Teragni M, Zabala G. Physical Bits: A Live Programming Environment for Educational Robotics. In Lepuschitz W, Merdan M, Koppensteiner G, Balogh R, Obdržálek D, editors. Robotics in Education; 2021; Cham: Springer International Publishing. p. 291-303. https://doi.org/10.1007/978-3-030-67411-3_26 DOI: https://doi.org/10.1007/978-3-030-67411-3_26
Mital A, Pennathur A. Advanced technologies and humans in manufacturing workplaces: an interdependent relationship. International Journal of Industrial Ergonomics. 2004; 33: p. 295-313. https://doi.org/10.1016/j.ergon.2003.10.002 DOI: https://doi.org/10.1016/j.ergon.2003.10.002
ESIC. Robótica Colaborativa: Qué es y Qué papel tienen las personas. [Online]. [cited 2023 03 04]. Available from: https://www.esic.edu/rethink/tecnologia/robotica-colaborativa-papel-tienen-las-personas.
Kohrt C, Stamp R, Pipe A, Kiely J, Schiedermeier G. An online robot trajectory planning and programming support system for industrial use. Robotics and Computer-Integrated Manufacturing. 2013. https://doi.org/10.1016/j.rcim.2012.07.010 DOI: https://doi.org/10.1016/j.rcim.2012.07.010
Brito T, Queiroz J, Piardi L, Fernandes L, Lima J, Leitão P. A Machine Learning Approach for Collaborative Robot Smart Manufacturing Inspection for Quality Control Systems. Procedia Manufacturing. 2020. https://doi.org/10.1016/j.promfg.2020.10.003 DOI: https://doi.org/10.1016/j.promfg.2020.10.003
Zhu Z, Hu H. Robot Learning from Demonstration in Robotic Assembly: A Survey. Robotics. 2018. https://doi.org/10.3390/robotics7020017 DOI: https://doi.org/10.3390/robotics7020017
Cohavi O, Levy-Tzedek S. Young and old users prefer immersive virtual reality over a social robot for short-term cognitive training. International Journal of Human-Computer Studies. 2022. https://doi.org/10.1016/j.ijhcs.2022.102775 DOI: https://doi.org/10.1016/j.ijhcs.2022.102775
Litvinova Y, Rilke RM, Guenther C. Me, myself, and I: Image concerns and honesty in immersive VR. Computers in Human Behavior. 2023 December; 149: p. 107950. https://doi.org/10.1016/j.chb.2023.107950 DOI: https://doi.org/10.1016/j.chb.2023.107950
Holubek R, Ružarovský R, Velíšek K, Janíček M. Novel trend in case study of industrial robot programming and production system design using the virtual reality. IOP Conference Series: Materials Science and Engineering. 2021. https://doi.org/10.1088/1757-899X/1009/1/012023 DOI: https://doi.org/10.1088/1757-899X/1009/1/012023
Park BJ, Hunt SJ, Martin C, Nadolski GJ, Wood BJ, Gade TP. Augmented and Mixed Reality: Technologies for Enhancing the Future of IR. Journal of Vascular and Interventional Radiology. 2020; 31: p. 1074-1082. https://doi.org/10.1016/j.jvir.2019.09.020 DOI: https://doi.org/10.1016/j.jvir.2019.09.020
Bolano G, Roennau A, Dillmann R, Groz A. Virtual Reality for Offline Programming of Robotic Applications with Online Teaching Methods. In 17th International Conference on Ubiquitous Robots (UR); 2020. https://doi.org/10.1109/UR49135.2020.9144806 DOI: https://doi.org/10.1109/UR49135.2020.9144806
Costa G, Petry M, Moreira A. Augmented Reality for Human-Robot Collaboration and Cooperation in Industrial Applications: A Systematic Literature Review. Sensors. 2022. https://doi.org/10.3390/s22072725 DOI: https://doi.org/10.3390/s22072725
Fang H, Ong S, Nee A. Robot Path and End-Effector Orientation Planning Using Augmented Reality. Procedia CIRP. 2012. https://doi.org/10.1016/j.procir.2012.07.034 DOI: https://doi.org/10.1016/j.procir.2012.07.034
Michalos G, Karagiannis P, Makris S, Tokçalar Ö, Chryssolouris G. Augmented Reality (AR) Applications for Supporting Human-robot Interactive Cooperation. Procedia CIRP. 2016. https://doi.org/10.1016/j.procir.2015.12.005 DOI: https://doi.org/10.1016/j.procir.2015.12.005
Ostanin M, Klimchik A. Interactive Robot Programing Using Mixed Reality. IFAC-PapersOnLine. 2018. https://doi.org/10.1016/j.ifacol.2018.11.517 DOI: https://doi.org/10.1016/j.ifacol.2018.11.517
Rosen E, Whitney D, Phillips E, Chien G, Tompkin J, Konidaris G, et al. Communicating Robot Arm Motion Intent Through Mixed Reality Head-Mounted Displays. In Robotics Research.: Springer International Publishing; 2019. p. 301-316. https://doi.org/10.1007/978-3-030-28619-4_26 DOI: https://doi.org/10.1007/978-3-030-28619-4_26
Zhang T, Mccarthy Z, Jow O, Lee D, Chen X, Goldberg K, et al. Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. In ; 2018. https://doi.org/10.1109/ICRA.2018.8461249 DOI: https://doi.org/10.1109/ICRA.2018.8461249
Baklouti E, Jallouli M, Amouri L, Amor B. Remote control of mobile robot through 3D virtual reality environment. In International Conference on Individual and Collective Behaviors in Robotics; 2013. https://doi.org/10.1109/ICBR.2013.6729276 DOI: https://doi.org/10.1109/ICBR.2013.6729276
Kot T, Novák P. Application of virtual reality in teleoperation of the military mobile robotic system TAROS. International Journal of Advanced Robotic Systems. 2018 January; 15. https://doi.org/10.1177/1729881417751545 DOI: https://doi.org/10.1177/1729881417751545
Ružarovský R, Holubek R, Delgado Sobrino DR, Velíšek K. A case study of robotic simulations using virtual commissioning supported by the use of virtual reality. MATEC Web of Conferences. 2019. https://doi.org/10.1051/matecconf/201929902006 DOI: https://doi.org/10.1051/matecconf/201929902006
Murhij Y, Serebrenny V. An application to simulate and control industrial robot in virtual reality environment integrated with IR stereo camera sensor. IFAC-PapersOnLine. 2019. https://doi.org/10.1016/j.ifacol.2019.12.473 DOI: https://doi.org/10.1016/j.ifacol.2019.12.473
Garg G, Kuts V, Anbarjafari G. Digital Twin for FANUC Robots: Industrial Robot Programming and Simulation Using Virtual Reality. Sustainability. 2021. https://doi.org/10.3390/su131810336 DOI: https://doi.org/10.3390/su131810336
Crespo R, García R, Quiroz S. Virtual Reality Application for Simulation and Off-line Programming of the Mitsubishi Movemaster RV-M1 Robot Integrated with the Oculus Rift to Improve Students Training. Procedia Computer Science. 2015. https://doi.org/10.1016/j.procs.2015.12.226 DOI: https://doi.org/10.1016/j.procs.2015.12.226
Crespo R, García R, Quiroz S. Virtual Reality Simulator For Robotics Learning. In International Conference on Interactive Collaborative and Blended Learning (ICBL); 2015. https://doi.org/10.1109/ICBL.2015.7387635 DOI: https://doi.org/10.1109/ICBL.2015.7387635
Meta. Get started with Meta Quest 2. [Online].; 2020. Available from: https://www.meta.com/quest/products/quest-2/tech-specs/.
Downloads
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors grant the journal and Universidad del Valle the economic rights over accepted manuscripts, but may make any reuse they deem appropriate for professional, educational, academic or scientific reasons, in accordance with the terms of the license granted by the journal to all its articles.
Articles will be published under the Creative Commons 4.0 BY-NC-SA licence (Attribution-NonCommercial-ShareAlike).