This is exactly why I’m starting a consulting practice focused in this area. Similarly, I’m thinking of starting an AI executive coaching service focused on helping individuals empower themselves with AI tools. So much potential ahead!
Last year I had multiple gigs in data governance and data privacy for clinical trials. This year most of the interest is in building AI platforms. I’m also seeing interest in folks wanting to know how generative AI affects their individual role across pretty much every industry.
> A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023.
ChatGPT
The statement is an exaggeration and not entirely true. While advancements in AI technology have made certain tasks more accessible and streamlined, it is not accurate to claim that a wide range of AI tasks that previously required years of research and a dedicated team can now be accomplished easily with just API documentation and a spare afternoon.
AI development still requires a solid understanding of the underlying principles, algorithms, and methodologies. Complex AI tasks, such as training deep learning models or solving intricate problems, still require significant expertise, time, and computational resources. While APIs and pre-trained models can simplify certain aspects of AI development, they do not eliminate the need for expertise and careful implementation.
It's important to recognize that AI development is a rapidly evolving field, and advancements in technology continue to make certain tasks more accessible. However, it is still a complex and specialized area that often requires dedicated time, research, and expertise to achieve meaningful and reliable results.
I think both routes are possible. Some products take immense knowledge, time, and brilliance to build. Then there are those apps like that one popular app from Russia that generated like 20 reimagined photos from scratch for $4.99 a pop and was making like a million bucks a day. As someone formally trained in AI research and engineering in school and at a couple big tech companies, it made me sick how easy that would’ve been to build. I’m seeing apps pulling in insane money that my Junior AI consultants could build in a few weeks. Understanding the market is harder than AI engineering much of the time. If I could predict the markets, I’d be pumping out million dollar a day AI apps using API documentation every afternoon.
This is great and thanks for all of the links and references.
FWIW - I agree that the under the hood stuff really isn't necessary for AI engineering in the future because it's not even really necessary now. Yes learning from first principles matters, but most DS and MLE for 10+ years would set up their models with sci-kit learn and understood that they had to optimize for key business metrics, latency and performance, and minimum viable precision / recall metrics rather than understanding the fundamentals of linear algebra and tensorflow to make the VERY best and accurate models from scratch.
And I personally love treating AI LLMs like a black box because it allows this AI engineer role to train a skillset that combines business + product + implementation that I think is almost always more valuable.
Great article! And timely appearance of the AI Engineer Summit! I organize even more niche event online - Prompt Engineering Conference - just two days after the Summit. Free registration and CFP are open!
I thought I'd have to follow links here and there to get the gist of the article like most tech specific articles are but this one was extremely well written !!!
Hey swyx, thanks for putting together a well-articulated "description" of an evolving job - AI Engineer. Out of curiosity, I have been doing these things since ChatGPT came out and once in a while scratching my head wondering: "What kind of job is this?"
Hi @swyx! Extremely informative article - I'm currently working as a Marketing Automation Manager, but more and more tasks I get, the closer they are related to automating something with AI (i.e. by fine-tuning LLM)
As my job is being "influenced" by AI, I am looking to get really deep into AI Eng. Don't know a lot about programming, but previously I've done CS50 course by Harvard.
What path would you recommend for a beginner to start with AI Engineering?
You might need to learn new skill, but no new role is needed. A good engineer is what a company need. There is no need to spend too much time to think about this at all, a good developer will already learn new skills and be relevant.
This is exactly why I’m starting a consulting practice focused in this area. Similarly, I’m thinking of starting an AI executive coaching service focused on helping individuals empower themselves with AI tools. So much potential ahead!
would love to hear what problems you think people really want to hire for most!
Last year I had multiple gigs in data governance and data privacy for clinical trials. This year most of the interest is in building AI platforms. I’m also seeing interest in folks wanting to know how generative AI affects their individual role across pretty much every industry.
"AI executive coaching service" 👈 UGoBro! Huge need for this.
great followup from @jjacky here: https://jjacky.substack.com/p/ai-will-not-replace-you-but-you-will
> A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023.
ChatGPT
The statement is an exaggeration and not entirely true. While advancements in AI technology have made certain tasks more accessible and streamlined, it is not accurate to claim that a wide range of AI tasks that previously required years of research and a dedicated team can now be accomplished easily with just API documentation and a spare afternoon.
AI development still requires a solid understanding of the underlying principles, algorithms, and methodologies. Complex AI tasks, such as training deep learning models or solving intricate problems, still require significant expertise, time, and computational resources. While APIs and pre-trained models can simplify certain aspects of AI development, they do not eliminate the need for expertise and careful implementation.
It's important to recognize that AI development is a rapidly evolving field, and advancements in technology continue to make certain tasks more accessible. However, it is still a complex and specialized area that often requires dedicated time, research, and expertise to achieve meaningful and reliable results.
I think both routes are possible. Some products take immense knowledge, time, and brilliance to build. Then there are those apps like that one popular app from Russia that generated like 20 reimagined photos from scratch for $4.99 a pop and was making like a million bucks a day. As someone formally trained in AI research and engineering in school and at a couple big tech companies, it made me sick how easy that would’ve been to build. I’m seeing apps pulling in insane money that my Junior AI consultants could build in a few weeks. Understanding the market is harder than AI engineering much of the time. If I could predict the markets, I’d be pumping out million dollar a day AI apps using API documentation every afternoon.
That's ChatGPT in 2021, mind you.
This is great and thanks for all of the links and references.
FWIW - I agree that the under the hood stuff really isn't necessary for AI engineering in the future because it's not even really necessary now. Yes learning from first principles matters, but most DS and MLE for 10+ years would set up their models with sci-kit learn and understood that they had to optimize for key business metrics, latency and performance, and minimum viable precision / recall metrics rather than understanding the fundamentals of linear algebra and tensorflow to make the VERY best and accurate models from scratch.
And I personally love treating AI LLMs like a black box because it allows this AI engineer role to train a skillset that combines business + product + implementation that I think is almost always more valuable.
Great article! And timely appearance of the AI Engineer Summit! I organize even more niche event online - Prompt Engineering Conference - just two days after the Summit. Free registration and CFP are open!
https://promptengineering.rocks/
@swyx, let's set up a partnership!
This was surprisingly succinct
I thought I'd have to follow links here and there to get the gist of the article like most tech specific articles are but this one was extremely well written !!!
Hey swyx, thanks for putting together a well-articulated "description" of an evolving job - AI Engineer. Out of curiosity, I have been doing these things since ChatGPT came out and once in a while scratching my head wondering: "What kind of job is this?"
I write AI Made Simple. We have very synergistic audiences. Do you want to collab?
Hi @swyx! Extremely informative article - I'm currently working as a Marketing Automation Manager, but more and more tasks I get, the closer they are related to automating something with AI (i.e. by fine-tuning LLM)
As my job is being "influenced" by AI, I am looking to get really deep into AI Eng. Don't know a lot about programming, but previously I've done CS50 course by Harvard.
What path would you recommend for a beginner to start with AI Engineering?
Aren't you focusing too much on LLM and natural language processing? In the end this is just a subset (as you highlighted in the news trends graph)
Grateful to have read this. What are your thoughts on certifications like the Microsoft Azure AI Engineer as a starting point?
You might need to learn new skill, but no new role is needed. A good engineer is what a company need. There is no need to spend too much time to think about this at all, a good developer will already learn new skills and be relevant.