Peter Nordmark in Development

Rant: Database fundamentals is a lost art

Year by year I have noticed a constant decline in database skills amongst developers. 20 years ago, database design and implementation could consume 60% of the total development time spent in a software engineering project. Why? because effective data storage and retrieval is a huge and fundamental part of data-driven applications.

This seems like news nowadays. Many developers use the database as a simple bucket. Trust me, it's not a bucket. You can treat it as such, but you won't end up with a good product.

Object-relational mapping

I think it all started with the rise of ORM frameworks. Data access on the client-side is tedious and time-consuming and the initial motivation was to speed up the process. In the end, it moved developers away from the fundamentals. SQL is now generated and skills in how to optimize database querying were lost. What we got was generated generalized sub-optimal generated SQL. In those days I often found myself spending time on "cheating" the framework a lot to do what I want. In short, abusing the high-level ORM QL to do what I want on the SQL level.

Another thing that happened was that knowledge of database integrity constraints was lost, leading to more cases of data corruption. Indexing know-how was also lost. I'm pretty certain most developers today can't even explain how a b-tree works.


When NoSQL databases started to appear, many ran to them without a real plan with the main motivation to quickly solve performance issues, making database access easier and lessen the burden on the developers. NoSQL databases have its uses but it is not a replacement for relational databases. As they became more widely used the knowledge of ACID(Atomic, Consistency, Integrity, Durability) also began to fade.


The most recent major factor playing a part in the problem is DevOps. Why not let the developers just do it all? Who needs specialized roles like DBA and Ops experts. We'll just run it all in the cloud with auto-scaling virtualization software!


To sum up, ORM, NoSQL, and DevOps are all progress. They are tools but it does not replace the need for a fundamental understanding of what you are doing. The ever-growing demand for quick fixes is the essence of the problem.

I will leave you with this amazingly well thought out database design by the engineering team behind Instagram. There are links in the post to their approach to scaling etc. It's just a great example of a cool design with the fundamentals always present and written in 2012 but it still holds.

That's my CTO rant of the day 😂 I hope it sparked some thought. Feel free to disagree in the comments.

About the top image: Yeah that's my ragdoll Idefix. The confused look is due to me putting him in a bucket 😊

Leslie Clarke

Leslie Clarke

Nothing beats a rock-solid design and understanding of how something works from the inside out. So if you have a leaky bucket rather than the structured approach Peter. How does this reflect on maintaining a product and problem-solving when things go wrong?

Peter Nordmark

@Leslie Clarke things going wrong? thats called version 2
Leslie Clarke

Leslie Clarke

@Peter Björklund Aha.

Peter Nordmark

@Leslie Clarke when the code is totally fubar at version 12 and returns indetermenistic results you can rebrand it as AI 😂
Do you want to read more like this? Hit subscribe. It’s FREE!

Never miss out on Developer Blog: Haaartland!

Community negotiated deals. Exclusive events. Posts. Polls and more. Free to members.