It is not a surprise that designing, building and maintaining huge distributed systems is no easy feat. But for me, what is surprising is the amount of knowledge of the problems that we are solving and technology tools that fall in place to enable the successful implementation of such systems.
The data system bible
Recently, I come across the book Designing data-intensive applications as I was searching on the internet for good resources to learn about designing data pipelines that supports every data need of an organisation.
Barely into the first chapter, I was treated to a proper discussion of system design considerations when it comes to the software industry’s favourite term: scalability, as well as real life examples of distributed systems to seal the discussion.
Glancing ahead at the table of contents, the author touches on these topics:
- data modelling
- storage and retrieval
- distributed systems(e.g. CAP theorem)
- data architecture(batch & stream processing)
With this, I boldly proclaim that the book Designing data-intensive applications should in every software engineer’s must-read list, because the purpose of the author, Martin Kleppmann, in writing this book is to discuss fundamental ideas in distributed systems, dissect modern systems and show how thing work underneath the hood, in order for the reader to be able to make informed decisions regarding the choice of technology to solve their own problems.
I strongly believe that any tech professional(software engineer, architect, technical product manager, product engineer) who has some stake in the design, implementation and maintenance of back end and data system will reap the benefits that this book brings and become a self sufficient professional equipped with the necessary knowledge and skills to choose the right tool for the job.
For the seasoned architect, tech leads, senior engineers, this book will serve as a refresher as to systems had evolved over the years due to rapid rate at which data changes.
For the entry level folks like me, this book serves as an “aha” eye opener that introduces the technical considerations when it comes to “using the right tool for the job” as well as showing real life examples of such systems in action.
I will be following this book closely in the upcoming weeks according to the next step in my software engineer learning roadmap.