robotvision-onboarding

Note: These tasks aren't intended to take too long. Do as good of a job as you think you are capable of with your own brain, a search engine, and the time you have available.

Task 1: Download a Dataset and Visualize It

The first task is pretty open-ended, but with a clear goal at the end.

1. Go to the nuScenes website, and make an account. Download the "Mini" dataset (it should have a size of 3.88 GB). You are welcome to do this on your own computer, but it would be better if you did this in Colab or some other remote computing setup. Note it will take a while to download and unzip this dataset.

2. Write a script to visualize this data in an interesting way. The nuScenes devkit has code on interfacing with the dataset and accessing data, as well as some tutorials.

• Consider the different sensors and annotations available to you. How could you present this information to make it greater than the sum of its parts?

3. To ensure you didn't vibe code this solution, please include a link to a video in which you step through the code and demonstrate your visualization. The video shouldn't be longer than 4 minutes.

Task 2: Short Response Question

Now that you have explored a real robotics dataset, I want to better understand your creativity. Without Googling/asking an LLM explain how you would design a system that tries to take sensor data as an input and has to steer the car as the output. Some things to consider:

• What should the output of the system be? (There are many right answers)
• What inputs are most/least useful, and why?
• Take a step back and forget about neural networks and ML. What would your answer be without these? (These tend to be the best solutions.)

Please note that it's really obvious when you are consulting an outside source. I care about your own thought process, not you repeating back information I could also search for.

Deliverable

Compile the following information together for your deliverable:

• Link to your video for step 1
• Link to you code for step 1 (please fork this repository and work from there.)
• Your answer to the short response question for step 2 (no more than a paragraph or two, with a diagram)

If you have any questions, please DM @mgagv on Discord. Start your message with "ML@P".