TLDR; shifted from DataGAN to building end2end self-driving from cameras using Deep Neural Nets. Here’s a memo outlining project specifics + my goals. Here’s also a tweet of where I’m at with steering with an example of it being ran on.
Goals. 5 letters. the biggest thing you end up spending your time on.
whenever i've done goal setting, i’ve always felt as if they’re something that I need to be committed to; something that you have to do in order to feel results and “success.” i remember talking about this with Michael (mentor @tks.world) and the main takeaway I had from that meeting was,
You are never supposed to be commited to your goals. Goals are made based on your situation + the outcomes you want. The first time you set goals, you probably won’t get 50% of them done. But the more and more you practice goal setting, the better you become at creating tangible + productive goals.
my whole November was planned around DataGAN and expanding this out. i met with Seung [who’s an incredible person] and understood that synthetic data is not a problem. instead, it makes more sense to use GANs as a method for 3D simulation and be able to train Deep Neural Nets in it.
and so what did I do? start building this out? what did I realize? i’m relying on external people to help me with the issues that I have (compute). this is where stagnancy and burnout start to come up. i made the assumption that I would get compute and that I’d find people who would give me compute.
Understanding that rewards = efforts was crucial to my development in November. Sometimes I feel like I deserve a reward or this opportunity but the reality is you have to earn it.
so, what am I doing right now? building a self-driving car.
sri, where are you gonna get compute to train this? from my computer.
understanding end2end learning in self-driving cars
this was my main focus after my pivot from DataGAN. i want to now start focusing on understanding how self-driving truly looks like, from an AI-first approach [and with cameras].
end2end learning has been one of the most fascinating ways of training a self-driving car that I’ve come across. especially cameras. i’ve always found it super interesting how you can go straight to control from camera inputs.
came across Wayve and had the chance to meet Haibo + her amazing team (thanks so much Satvik!). really LOVE the work they’re currently doing; super interesting and fascinating how they're able to extract such semantics from single cameras. would recommend reading their paper on Learned Urban Driving via Imitation Learning.
here’s what my timeline is looking like right now:
connecting to the OBD port. this is the main priority right now. i want to get the Arduino parts required to collect data + apply the transformation to get my data in degrees (in the case of steering).
create Neural Nets for steering + brake + acceleration + gear settings and get training up and running.
stitch them up and then be able to run this in real-time as I drive.
here’s a demo of what my steering model is looking like right now:
I’ve also created a memo/writeup better explaining my goals with this project + my intentions with this project. Link to the memo is here.
I’ve even open-sourced my notes so far. Not fully done with the book so far (halfway done) but it’s written in paragraph form - makes it easier to digest + understand
going through calculus right now. Want to focus on developing theoretical + technical knowledge right now.
planning to go through The Deep Learning Book soon and be able to fully digest it. I’ve gone through a part of this book before but that was mainly skimming through. Now I want to go deep and develop theoretical intuition behind concepts.
asks right now
would really, really appreciate your thoughts on the approach i’m focusing on (simultaneously building while developing theoretical knowledge). how do you create + build valuable projects? am i even focusing on the right things?
the other thing on my mind right now is how you really create novel ideas in self-driving. how do you break down problems and scale up from there? i’ve had a lot of ideas in the past little bit; what does the process of validating them look like?
hey, thanks so much for reading this newsletter! Would love to hear your thoughts and updates on what you’re working on, i’m only one email away.