With any project I take on, the biggest problem I encounter is often to do with what API versons I use. Sometimes it seems I’d be better off writing my own algorithms, but that is not at all practical and defeats the intent of everything I’ve studied. Most recently, my issues were with Jax and Keras versions. With the help of threads from others who have experienced similar issues, I resolved both problems. AI was useless in each case.
How can I find good environments beforehand?
Why do CS grads have such a tough time when (in my experience) AI is often useless for these types of things?
I am actually a CS grad(AI) and the main issue is lack of experience. Universities only teach you so much and many modules on AI are not in regular CS courses or are optional. So with the lack of experience provided at an academic level most graduates do not know certain vital topics in AI. Stuff like Keras was only taught in my subject and not any other CS subject in the University, that being one of my last modules at Bachelors level. So if the graduates are not trained in these models they will not be able to use them well as that requires self teaching which relies on documentation or in most cases, the CS student just using AI which is useless at giving reliable results. Especially as version conflict isn’t covered widely with tools CS students use.
I used to have a bunch of issues like this. The problem lies in the APIs themselves. Tensorflow is very prone to failures to maintain backwards compatibility (at least in my experience). I’ve never used JAX–a functional language isn’t the greatest abstraction in machine learning, where you need a persistent state of weights to gradually modify.
My main experience is in pytorch, which has been pretty stable over the years. The hardest I’ve ever had to struggle with it is getting CUDA support for an abandoned AI SBC. Pytorch is pretty big on backwards compatibility. Code that was written 7-8 years ago still works, which is important when I want to experiment with older papers whose code is available publicly.
Finding a ‘good’ environment is not something I can help with that much, since it really depends on what you are doing, but one thing you might try is looking into the version requirements for a modern github repo that uses tensorflow and jax. Whatever versions they are using must work together nicely, or otherwise the project would break.
There isn’t any good way to consistently stay out of dependency hell besides creating a new virtual environment for every single project you make or install. The best suggestion I could make is to get some polished linux distro (like ubuntu, everyone seems to like that) and use the package manager whenever possible.