Team Members

Ihyun Leo Park - Electrical Chad Engineering

Brandon Espinoza Balbuena - MAE

Ashenling Yang's Braised Chicken Rice - MAE(Anagram for LAME, L's just a bonus)

Project Overview

Our project used tesseract and EAST text detection with openCV to detect text, then which the car will go and pick up the specified object using a model trained with the YOLO algorithm. For example, if we show the car the word apple, it will go and pick up an apple. We used openCV and imutils library to control the camera, pyVESC to control the vesc and the motor, and finally the python servo motor library to control the servo motor to pick up the actual item. We used a brushless DC motor, VESC, Jetson Nano, 1080p webcam, and a servo motor.

Main Components

Physical Setup

Car

Progress

Final Product

Schematic

Implementation

Software and Hardware List

The following are needed to complete this project.

Jetson Nano
64GB MicroSD card
Camera
Tesseract
pyTorch
seaborn
imutils
argparse
OpenCV
Probably a lot more I'm forgetting

Source code

The source code of the project can be found here. Good luck running it :): https://github.com/SikGek/ECE_148_Final

Steps Overview

Jetson Nano setup
Clone git repo: https://github.com/SikGek/ECE_148_Fina
Download all the libraries that it requires you to do so, you are probably missing at least 3 to 5
Show the car a text from the list of detectable objects
Let the car spin around until it finds the shown item and watch it as it goes and pick it up.

Results

Our team was able to successfully have it retrieve 3 different types of items, although 2 is shown for saving time. The throttle is slow to compensate for the delay in camera feed due to poor processing power of the Jetson Nano.

Tesseract is painfully slow due to the model being trained for all the characters in the alphabet, while we reduced the YOLO model to the smallest size for best possible performance.

Gantt chart

Demonstrations

Entire Project at Work on an Apple and a Banana

The Camera feed to Showcase that it only detects one object

Challenges

Challenge	Solution
Inconsistent object detection	Higher resolution camera improved results
Little time to develop in ROS	Used python libraries instead
Jetson doesn't have enough power to power the servo motor	Use i2c power switch
Poor and slow object detection	Training our specific model

Potential Improvement

Use ROS for VESC, camera and servo control
Obtain a better camera
Change the camera angle so that it can detect a basket after picking up the object and place it inside of it
Train a new model with more detectable objects
Implement speech detection for input instead of text detection for convenience

Weekly Presentations

Final Project Proposal
Progress Week 8
Progress Week 9
Final Presentation

Reference

We would like to give special thanks to:

Professor Jack Silberman
TA Dominic Nightingale
TA Ivan Ferrier
ECE Makerspace
All my fantastic classmates

Disclaimer

I kinda copied team 7's format, hope that's ok. Their project was very sick, go press a like and subscribe to them. Thank you. ^^7

2022SpringTeam3

Contents