Data Jobs & AI?

Many of the current “AI” tools are half baked toys that don’t add value, the current exception to this are coding tools like Cursor and Claude.

But, things are progressing very quickly. Jansen Huang (CEO Nvidia), here explains they’re increasing the processing power by approx. 10x EVERY YEAR.
https://youtu.be/m1wfJOqDUv4?si=_vgOxOO2dl2mfTRL&t=1160

In a few years, AI will be thousands of times better than it currently is. With processing power, the underlying algorithms & software improving exponentially.

Here is my key point.

Companies that use the big/popular fully integrated data platforms from Oracle, Microsoft, AWS, Snowflake etc, these are the ones already laying off data workers. WHY?

Because these providers are already starting to bake AI into their platforms. This video of Larry Ellison (CEO Oracle) talking about what they’ve just rolled out.
https://www.youtube.com/live/4eCFmbX5rAQ?si=mm0zB5sFX7Fh1BJk&t=1815

Notice he is asking questions of his data directly from the database (no Analytics needed). This has just been rolled out at Oracle, and all companies will be able to do this with their own data. (It’s worth watching his whole talk).

Companies that have old/poorly designed data platforms. Mostly these are smaller companies but really they could be any size. This is where Data jobs will last for decades to come.

Recently I saw a mid-size company (100 employees) that had 8 different data sources.

• Data sources that were CSV downloads from various ticketing websites.

• APIs that only had a “reporting end point” that consisted of a few aggregated metrics. I.e., it was useless.

• APIs that were rate limited, slow and were down most of the time.

• A CRM system that you’ve never heard of and is just awful.

There are thousands of companies operating like this.

There are many smaller but popular business tools that are simply awful for getting data out, and reporting. This is where Data Analysts and Engineers will have jobs for a very long time.

Currently there is no AI tool that can take multiple messed up data sources and turn it into insights.

In Summary:

Data jobs will be around for a very long time especially for companies using poorly integrated, old legacy platforms. Mainly, Medium to Small size businesses.

If you’re an Analyst working for a multinational company that has the latest integrated tech, you’ll be made redundant eventually.

Advice:

If you’re a Data Analyst, you should add Data Engineering skills to your repertoire. Python, Cloud computing (Azure, AWS, Google, Snowflake), pipeline orchestration.

If you’re a Data Engineer, learn the AI offerings from the big providers. Learn to deploy AI solutions.

Also, there is truth to this meme, a lot of what people call Data Science is built on Data Engineering.

Zach the ex Netflix & Meta Data Engineer, is saying "AI will eventually take every job".
https://www.youtube.com/live/RcEzdnyPh9M?si=G_GLbCIAy7aqHvvK&t=1425

With great change, comes great opportunity. Become an expert in integrating AI into the data handling process and you’ll be in demand.

Cheers Shano.