Posted on

Cover letters, chatGPT, and MS Word

This won’t be a super organized post, but I wanted to put this out into the ether. As I’ve been on the job hunt I’ve experimented with using chatGPT to help me craft cover letters (example here ).

I thought that a demonstration of using a relatively new technology would demonstrate creative thinking and maybe help me get my application to the top of the pile.

However, it seems like the chatGPT output is missing something, because I seem to be having better luck by crafting cover letters myself. I haven’t found it fruitful to collect data on this (frankly my time is better spent writing cover letters and sending applications). It might be something I revisit down the road.

However, I want to automate everything I can, and to that end I’ve created a word template that uses fill-in fields to allow me to automatically insert information into the template. This has the benefit of saving a bunch of time but still enabling the letters to be individualized to each job and company. You can read more about how to do this in MS Word here.

Posted on

GPT, English SDK, LLMs and the future

It’s official, Databricks has released their English SDK for Apache Spark.In short, this tool looks like it makes writing Spark code as simple as writing plain English instructions.

I haven’t tested it yet, but I have been using ChatGPT for some time to help me write cover letters for job applications. I’m assuming that the fine folks at Databricks have released a product that will make a lot of data engineering tasks way too easy, and make it easier for folks without a coding background to create ETLs, or even whole data infrastructures, without the help of people experienced in writing code.

THIS IS A GOOD THING, and it highlights an important thing to be mindful of , whether you’re a data engineer, data scientist, or marketing executive.

Memorizing the use of a complex tool (like Spark or SQL) does not make you a good engineer. Knowing how to use a complex tool to get results that are asked for from your boss or a stakeholder doesn’t make you a good engineer.

Understanding relationships, whether mathematical, cause/effect, logical, or interpersonal, make you a good engineer. 

What do I mean by this? Let’s use this new Spark tool as an example. When it comes to data engineering, the important part isn’t understanding how to write the correct Spark. It’s understanding how the data you’re manipulating is being collected, how it will be used, what it represents in the real world, and how it can be efficiently stored, retrieved, transformed, and related to other data in your infrastructure.

This isn’t exactly a hot take. I’m sure something similar is being repeated on a thousand different blogs right now. But I wanted to weigh in because I also see a lot of doom-posting about the end of data engineering as a profession. On the contrary, tools like this (and Copilot, and ChatGPT, etc.) are just going to enable people to apply more computational power to their projects. Technical folks like myself will be necessary to help non-technical folks avoid mistakes, refine and optimize processes, and fine-tune plug-and-play LLMs for specific business uses. To me, this is the democratization of complex computation; folks with real expertise in analytics will still be necessary to act as subject matter experts. But we’ll have to spend less time memorizing syntax, and more time understanding what we’re trying to build.

Posted on

audentes fortuna adiuvat

This blog is going to catalog my professional life. Here’s where I’m at.

I took a severance from my last job in May. I decided to take some time to jumpstart a project I’ve been dreaming about: an analysis of the U.S. Bureau of Labor Statistics (BLS) Consumer Price Index (CPI). This index is the best-known and understood index of inflation in the U.S. Inflation is topical, and I find that there’s a lot of misinformation and misunderstanding out there about what it is and how it’s measured.

I’m going to make further posts shortly that will catalog what I’ve done so far on CPI. I’ll also make entries related to my job hunt as I begin that more earnestly.

Why did I decide to leave my last role? I’ve learned that all relationships, business or otherwise, are co-created. I’ve also learned that the only way to grow is to focus on what I can control. It’s tempting to either blame myself and feel defeated and unworthy, or to blame others, when a crossroads like leaving an employer is reached.

Instead of doing either of those, I’ve tried to be mindful of two things:

What do I want from a prospective employer? This will help me evaluate jobs before I take the plunge, and hopefully lead to a better relationship with my next employer. In short, I want an employer that will respect my expertise and has problems that need my skills. I don’t want to maintain ETLs; I want to design and build them. I don’t want to fix broken dashboards, I want to create north-star metrics. I don’t want to work at a giant organization, I want to work with a small group of passionate people. This is incomplete but you get the idea.

What can I learn from my own mistakes? This is the most important point. I know for sure that I could have handled collaboration with stakeholders in a more productive way. To be frank, I think I became a little too arrogant about my skills and role, and failed to be a good business partner. People talk a lot about impostor syndrome, but a talented professional can fall off their horse on the other side too; it can be really tempting to see non-technical people as adversaries who don’t understand, when the truth is, they’re likely experts within their own domain and should be treated as such. As I go on in my career, I’m glad I had a chance to self-reflect and temper my arrogance into healthy self-esteem and pride.

I’ll leave with the translation of this post’s title (and my site’s tagline): “Fortune favors the bold.” It’s always been among my favorite sayings, and it’s older than England for a reason. It’s my personal core value. I hope it serves as an introduction to my general attitude as a person.