Upgrading machine learning hardware!

I started this project by training on a GTX 970M, yeah a mobile graphic card. It was slow as hell and training took days for mediocre results. But eventually I got down to a model that gave an usable accuracy and I decided to go all out with a GTX 1080Ti (I've been using AMD on desktop for years). This was right after crypto crash in 2018 so the used market was flooded with cheap cards so I managed to get a used 1080Ti for cheap. The models that I use today only required overnight to train, so I was able to rapidly test different models and managed to nail down something that was accurate and performed well in about 2 weeks. I've continued to use the 1080Ti to this day for development and public testing.

I've been eyeballing the RTX 3000 series since their launch last year. The added tensor cores since the 2000 series meant huge bump in training and inference performance compared to 1000 series. But due scalping, pandemic, and yet another crypto boom the prices for these cards has gone absolutely insane. I've been on the EVGA queue since December 2020 and they have yet to fulfill orders from November! The crypto prices fluctuates a lot but as long as Bitcoin mains above $30K the prices for these cards are never going to drop to MSRP level, and it will be years until chip fabs can have new factories up and running.

So after some moderate research I found out that I had the best chance of getting one from the Zotac store. The site was very bot unfriendly due to captchas, short login timeouts, and poor performance, so it was the best site for human to get one.... as long as I was quick. They drop about 400 cards per day on a somewhat predictable schedule. So after trying for a week I managed to nab a RTX 3070! It is still more expensive than MSRP but I can sell my GTX 1080Ti to recoup majority of the cost. Yeah, that's how crazy graphic card prices are these days.

I'm expecting big jump in performance in both training and inference, and will be my first foray into using tensor cores. So I'm going to take a slight detour from my feature blacklog to upgrade TensorFlow and CUDA SDK. I will have to rebuild TensorFlow from scratch and repackage into the current development software. I may even want to retrain the model so this entire process may take a few weeks (it took me quite awhile just to get current tensorflow to build on my current system). So I will report back once I have it up and running!.

Stay tuned!