“An investment in knowledge always pays the best interest.” – Benjamin Franklin
Many of the best software developers have T-Shaped Skills: Deep expertise in programming and software development, and broad knowledge of diverse areas including testing, DevOps, UX design, team organization, customer interaction, and their domain areas. While there is unfortunately no substitute for experience, reading is probably the next best thing. Over the past 10 years I’ve read a lot in an effort to deepen and broaden my knowledge as a software developer. Along the way I’ve been organizing books and concepts into the reading list I share below. I have been trying to design a core curriculum for “modern” software development by asking myself:
- What core concepts are required to be a world class software developer?
- What is the best book for introducing and teaching each concept?
The result is a list of 16 essential software development concepts presented by 16 excellent books…
Methodology & Other Notes (skip to reading list)
Reading a book is a significant time investment, so I don’t recommend a book lightly. In order to design an effective and efficient reading list, I have tried to structure it around a concept list satisfying these criteria:
- Complete: Don’t omit necessary concepts.
- Essential: Don’t include optional concepts. Concepts should be relevant to most developers during their career.
- Non-redundant: Limit overlap and repetitive content.
- Universal: Technology & domain independent.
- Applied, not Theoretical: So this list excludes great theoretical books like Gödel, Escher, Bach.
- Software Development, not Programming: The distinction is quite fuzzy, but I think of programming as getting a computer to do what you want and software development as evolving valuable software with a team. Learning programming deserves an entirely different reading list (like this one). Every programming reading list should include Code Complete, SICP, and this article: Teach Yourself Programming in 10 years. (If you are learning programming, you may be interested to peruse my notes about Why & How I Write Java.)
- Software Development, not Computer Science: (The following observations are not a consensus. They are my opinions based on my experience which includes being a computer science teaching assistant for C++, assembly, logic design, algorithms, databases, and artificial intelligence, followed by 7 years of professional software development.) There is limited overlap between a professional software development (SD) curriculum and a traditional computer science (CS) curriculum. A computer science degree often includes learning programming plus courses including automata theory, algorithms, programming language theory, operating systems, networking, cryptography, databases, artificial intelligence, logic design, hardware architecture, etc. A functional understanding of algorithms, data structures, and complexity (big O) is important for programming. A CS database course provides a theoretical foundation for the broader SD challenge of data management in practice. Other than algorithms and databases, I’ve found many CS subjects to be optional (but occasionally helpful) for many kinds of software development. In some areas of SD a background in UX design or domain knowledge might actually be more helpful than CS.
- Beyond listing the 16 core books for each concept, this page also include notes, links to further worthwhile reading, and related quotes. Some of the notes may not be useful until after reading the book for context.
- I also link to other reading lists at the bottom of this page.
- I prefer books with human narrative over a lofty academic writing style. I find that staying awake significantly improves my reading comprehension.
- The book links are to Amazon where you can read reviews and view the table of contents by clicking on the book image. I assume many are available as eBooks. Some are also available online as PDFs.
- By its nature this is an evolving list. I would love your feedback about the concept list, the book selections, my concept notes, and any worthwhile books or quotes that I’ve missed. Thanks!
Outline: A Software Developer’s Reading List
Software Design Concepts
3. Code Design: Clean Code
4. Component Design (Object-Oriented): Object Design
5. Domain Design: Domain Driven Design
6. System Design: Software Systems Architecture
7. System Design (Availability): Release It!
8. System Design (Security): Foundations of Security
9. Legacy Code: Working Effectively With Legacy Code
10. User Experience Design: Designing Interfaces
Software Delivery Concepts
11. Lean Systems: Leading Lean Software Development
12. Agile Team Organization: Essential Scrum
13. Requirements Design: User Stories Applied
14. Testing Products & Services: Agile Testing
15. Continuous Delivery: Continuous Delivery
16. Customer Development: Lean Startup
I believe these 16 books are a great start for any professional software developer (after learning programming). It is certainly a much more direct path than I took. By no means are these the only excellent books or the only books you should read. This list is simply the most effective and efficient overview I know how to recommend. The “Further Reading” sections contain many important and classic books.
The reading list begins with 2 basic software evolution skills: Test-Driven Development and Refactoring. Next are 4 different levels of software design granularity: Code Design, Component Design, Domain Design, and System Design. We then expand on System Design by digging into 2 commonly important system qualities: Availability and Security. We round out Software Design with 2 more perspectives: Legacy Code and User Experience Design. Next are the 6 Software Delivery concepts present how to organize tools & teams to successfully deliver valuable software to customers and users.
I have tried to place the books in a natural sequence, but I would read them in whatever order makes sense for you and your work. Many of these books are worth reading cover to cover, but some have sections you can safely skip or defer. (Perhaps keep in mind that if you have time to read 1000 pages, reading 200 pages of 5 books is often worth more than reading 500 pages of 2 books.)
New concepts based on feedback
I think it’s important to keep this list to a manageable size, however based on feedback, I will probably be adding 4 concepts:
- Professional Craft: Software development is a craft. This has implications for how it is learned and what constitutes professional conduct. The classic book here would be The Pragmatic Programmer.
- Component Design (Functional): I want to split Component Design into Component Design (Object-Oriented) and Component Design (Functional). It looks like Functional Thinking may be the right functional book when it is released (earlier articles and presentation by the author).
- Data Management: This is a core skill that is missing from the 16 concepts. I’m not sure what book to use. Subjects include relational vs. key-value, normalization vs. performance, transactions, big data, sharding, consistency, distribution, UUIDs, backup+recovery, scaling, schema evolution, etc. One option is a computer science textbook like Database Systems, but I worry about this choice because there is a sizable gap between theory and practice in this area.
- Managing Humans: Peopleware is the classic book (3rd edition in 2013). I stole the concept name from the book Managing Humans, which will be in the further reading section (along with the associated blog Rands in Repose). I would also include a link to this article.
Software Creation Concepts (#1-2)
1. Test-Driven Development: Writing automated tests to discover requirements, guide software design, enable refactoring, and prevent regressions.
Concept notes (this concept has more notes than the others):
- Testing software early is important for many reasons:
- The cost of an undetected defect rises dramatically over time. Once automated tests are in place we find many bugs shortly after they are introduced, so we easily know what changes caused the problem.
- Writing tests helps us understand (and often discover) the detailed requirements.
- Tests enable us to improve the design of software via refactoring. The fear of changing untested software is worth experiencing: it fosters intrinsic motivation for testing.
- Testing improves modularity by forcing us to invert dependencies (statically depend on interfaces, not implementations).
- Tests provide executable requirements documentation, enabling us to write less textual documentation (trust code, not comments).
- Tests force us to support a 2nd client: production is the 1st, and the test is the 2nd. A 2nd example provides valuable design feedback.
- Testing guides us to write testable code. Untested code is often untestable. You end up with coupled Legacy Code.
- When practicing Test-Driven Development you write tests before implementing the functionality:
- Before adding a new capability or fixing a bug, write a failing test.
- Make the test pass in the easiest conceivable way. Don’t plan much; design as little as possible; just get the test passing. If the implementation is embarrassing (but passing), you’re doing well at this point.
- Under the cozy protection of a passing test, refactor the implementation until it is “reasonably clean“.
- There are many types of software testing including unit testing, integration testing, acceptance testing, system testing, regression testing, smoke testing, non-functional testing, stress testing, performance testing, usability testing, etc. Unit testing is the foundation. Here is a comparison of unit, integration, functional, and acceptance.
- Most pieces of software (i.e. units, components, systems) use and depend on other pieces software. These other pieces of software are the dependencies. An important part of testing software is deciding what implementation to choose for dependencies during testing. For example, if you were persisting Account objects you would use (depend on) a database. Let’s say you use a MySQL database implementation in production. If you also use MySQL in your unit tests, you will often find that it is too slow and inconvenient. Instead you will want to use an in-memory database when running unit tests. This in-memory database is an example of a Test Double, or more specifically a Fake Object. With vocabulary derived from Mocks aren’t Stubs and the xUnit Test Patterns (table), here are several options for dependency implementations:
- Real Dependency: Using the same implementation in tests as in production. (MySQL in the example.)
- Test Double: Using a different implementation in tests:
- Fake Object: A different but fully functional implementation (in-memory database in the example)
- Dummy Object: An object that is passed around, but is never used by the code we’re testing (i.e. providing null, knowing it doesn’t matter)
- Stub Object: An object that provides canned (hardcoded) answers to method calls.
- Mock Object: From wikipedia, a mock is a simulated object that mimics the behavior of real objects in controlled ways. This is an extremely flexible Test Double, but the tradeoff is your test ends up knowing implementation details of the code you’re testing (coupling the two). For this reason I use fakes instead of mocks whenever possible. My favorite mocking library for Java is Mockito. (When deciding between mocks and fakes, perhaps also consider that mocks nail down behavior where fakes may only nail down end state. If you loop through rows in a database instead of doing one large query, the final state might be correct, but the looping implementation may be a behavioral bug causing inefficiency. In this case the test may want to verify the implementation details, despite the reduced modularity between test and code-under-test.)
- In order to use a Test Double implementation for a dependency, that dependency must be inverted. Instead of statically importing an implementation (i.e. class or instance), the implementation should be provided at runtime, often as a constructor parameter. In a language with interfaces you would statically import the interface. By inverting the dependency, the test is able to provide a Test Double (i.e. in-memory database) while production provides the Real Dependency (i.e. MySQL database). It think it is fair to say that the unavoidable price of testability is dependency inversion. The family of mechanisms used to provide dependencies at runtime is sometimes called Dependency Injection.
- Test Driven Development: By Example (This isn’t really further reading as much as it is a more introductory book demonstrating the TDD experience in more detail.)
- xUnit Test Patterns: Refactoring Test Code
- Practical Unit Testing with jUnit and Mockito
- Article: Unit Tests
- Article: Mocks aren’t Stubs
- Google Tech Talk: Don’t Look For Things
- Stack Overflow: List of TDD anti-patterns
- Martin Fowler’s Tags: Testing
- Conversations between DHH, Fowler and Beck about whether TDD is “dead”
- “Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.” – Michael Feathers (Legacy Code)
- “Successful software always gets changed.” – Fred Brooks (Turing Award #34, Mythical Man Month, No Silver Bullet)
- “It is my firm belief that all successful languages are grown and not merely designed from first principles.” – Bjarne Stroustrup (Grace Murray Hopper Award, C++)
- “Never in the field of software development have so many owed so much to so few lines of code (JUnit).” – Martin Fowler (Refactoring, Book Series, Agile Manifesto)
2. Refactoring: Evolving the design of software.
- Refactoring means changing the design of existing software. This reading list covers 4 different levels of software design granularity: Code Design, Component Design, Domain Design, and System Design. Refactoring at each level presents new challenges. Here is where to look for related discussions at each granularity level:
- Code Design: Clean Code discusses design, Refactoring discusses refactoring
- Component Design: Object Design discusses design, Refactoring to Patterns discusses refactoring
- Domain Design: Domain Driven Design discusses both design and refactoring
- System Design: Software Systems Architecture discusses design, and Michael Nygard’s talk Architecture Without an End State discusses refactoring
- Time Allocation Heuristics:
- Refactoring to Patterns
- xUnit Test Patterns: Refactoring Test Code
- Refactoring Databases: Evolutionary Database Design
- Martin Fowler’s Tags: Refactoring, Evolutionary Design
- “The best writing is rewriting.” – E. B. White (The Elements of Style)
- “No great thing is created suddenly.” – Epictetus
- “Before software can be reusable it first has to be usable.” – Ralph Johnson (GoF)
- “He who rejects change is the architect of decay.” – Harold Wilson
- “I made this letter longer than usual because I lack the time to make it shorter.” – Blaise Pascal
- “We tried to add the feature to our application but found that there was no place for it. So we first made a place, and then added the feature there.” – Ward Cunningham (Invented Wikis, Technical Debt, Agile Manifesto)
- “Creativity is allowing yourself to make mistakes. Design is knowing which ones to keep.” – Scott Adams (Dilbert)
- “It’s about whittling. It’s about taking something and whittling and whittling and getting it sharp and perfect. Then you’ve got something.” – James Victore
- “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” – Antoine de Saint-Exupéry
Software Design Concepts (#3-10)
3. Code Design: Writing intention revealing code.
- Professional code is written for humans to read. It is a communication medium. (The code itself, not just comments. Comment the why; the code should make what and how evident.) The person reading a developer’s code may be their teammate, or it may be themselves months or years from now. Coding is like writing or speaking: we learn what minimum level of clarity is necessary to communicate successfully. We meet this baseline of clarity even when we’re in a hurry (i.e. email, voicemail, rapid prototype). When the message is important we put extra effort into clarity (i.e. essay, speech, reusable library). The reality is that if someone is incoherent then nobody will want to listen to them (or work with them).
- I feel that programming becomes software development when you collaborate with a team on a shared codebase. For a team to avoid accidental complexity, established standards should often take precedence over personal preferences, with all team members helping to guide what those standards should be. Pair programming is a great way to discuss, debate, and share such standards.
- Code Complete
- Effective Java
- Implementation Patterns
- 97 Things Every Programmer Should Know
- The Pragmatic Programmer
- “Simplicity is the ultimate sophistication.” – Leonardo da Vinci (Biography, Notebooks, Medici Documentary)
- “Programs must be written for people to read, and only incidentally for machines to execute.” – Abelson & Sussman (SICP)
- “Good visual layout shows the logical structure of a program.” – Steve McConnell (Code Complete, Rapid Development, Software Estimation)
- “It’s the little details that are vital. Little things make big things happen.” – John Wooden
- “The average teacher explains complexity; the gifted teacher reveals simplicity. ” – Robert Brault
4. Component Design (Object-Oriented): Managing interfaces to manage complexity.
(Update: When I do a full revision of this reading list, I think I’m going to switch my top recommendation in this category to Practical Object-Oriented Design in Ruby.)
- There are many great Component Design books, and it’s definitely worth reading several in this area. To get started I recommend Object Design because I think roles, responsibilities, and collaborations are at the heart of the matter, and because it reads like a human narrative. (One aspect of this book I’m unsure of is CRC cards because I’ve never tried them.)
- Here is a mental model I’ve developed over time for how a bunch of component design principles (heuristics) relate to each other.
- As a programming subject, the dichotomy between Functional Programming and Object-Oriented Programming isn’t addressed in this software development focused reading list. That being said, mutation and side-effects are complexity creating monsters. I believe that regardless of the implementation language, the best software designers apply functional principles when possible. I’ve never used a functional language in production, but I’ve benefited enough from these functional principles to consider them beyond doubt:
- Practical Object-Oriented Design in Ruby (potential alternative 1st book). I haven’t read this book yet (it was suggested in the comments), but it has the most 5 star Amazon reviews I’ve ever seen. I’m definitely looking forward to reading it.
- Object Thinking (potential alternative 1st book)
- Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development (potential alternative 1st book)
- Object-Oriented Software Construction (potential alternative 1st book)
- Head First Design Patterns or Design Patterns: Elements of Reusable Object-Oriented Software (Gang of Four, also known as GoF, is a very influential design patterns book from 1995. The Wikipedia has pages for all of its 23 design patterns. Some of these patterns are used all the time, some are used infrequently, some are specifically responses to Java’s limitations, and there is even singleton which is often an anti-pattern. The GoF patterns I use most are factory method, composite, adapter, iterator, observer, and template method.)
- Agile Software Development: Principles, Practices, Patterns (SOLID Principles)
- Beautiful Code
- Object-Oriented Design Heuristics
- Presentation: How to Design a Good API & Why it Matters
- Wikipedia: Unix Design Philosophy
- Martin Fowler’s Tags: Encapsulation, Object Collaboration Design, API Design, MVC
- “Fools ignore complexity; pragmatists suffer it; experts avoid it; geniuses remove it.” – Alan Perlis (Turing Award #1, ALGOL)
- “Computer Science is the first engineering discipline in which the complexity of the objects created is limited solely by the skill of the creator, and not by the strength of raw materials.” – Brian Reid (Grace Murray Hopper Award)
- “The competent programmer is fully aware of the strictly limited size of his own skull.” – Edsger Dijkstra (Turing Award #7)
- “Controlling complexity is the essence of computer programming.” – Brian Kernighan (K&R)
- “So much complexity in software comes from trying to make one thing do two things.” – Ryan Singer (37 Signals)
- “The cost of adding a feature isn’t just the time it takes to code it. The cost also includes the addition of an obstacle to future expansion… The trick is to pick the features that don’t fight each other.” – John Carmack (Id Software)
- “I think most programmers spend the first 5 years of their career mastering complexity, and the rest of their lives learning simplicity.” – Buzz Andersen
- “Nothing is particularly hard if you divide it into small jobs. ” – Henry Ford (Biography, Autobiography)
5. Domain Design: Molding software to fit its domain.
- This unique book introduces the importance of discovering a Ubiquitous Language for discussing the domain with customers/users/experts, as well for naming the nouns and verbs in the software representation of the Domain Model. It also places the Domain Model at the center of our system, with the persistence (Repositories) and distribution (Application API) at the edges.
- If you are used to web application frameworks like Rails or Django, it may seem unusual to have a persistence independent Domain Model. These frameworks can lead you towards using ORM objects as your Domain Model, which couples your persistence details and your Domain Model (at least they led me down this path). Essentially you trade modularity for getting started quickly, which may or may not be profitable based on how large and long lived your system is. Bob Martin gave a great presentation about what an alternative architecture looks like: centered around the Domain Model, with persistence details hidden behind Repository interfaces, and with delivery details (like HTTP) decoupled from the Domain as well. I prefer this alternative architecture, but modularity can increase the number of layers, which can have downsides such as increased learning time and worse performance due to layer translation (but if disk or network is your bottleneck, often this won’t matter).
- Implementing Domain Driven Design
- Suggestions on Stack Overflow
- Martin Fowler’s Tags: Domain Driven Design
- “Human language appears to be a unique phenomenon, without significant analogue in the animal world.” – Noam Chomsky
- “I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.” – Fred Brooks (Turing Award #34, Mythical Man Month, No Silver Bullet)
- “When you actually sit down to write some code, you learn things that you didn’t get from thinking about them in modeling terms… there is a feedback process there that you can only really get at from executing some things and seeing what works. ” – Martin Fowler (Refactoring, Book Series, Agile Manifesto)
- “Everything should be made as simple as possible, but not simpler.” – Albert Einstein (Nobel Prize, Biography)
- “There’s a point when you’re done simplifying. Otherwise, things get really complicated.” – Frank Chimero
6. System Design: Holistic views of a system’s structure and qualities.
- Books about System Design & Software Architecture generally describe how to think about, work with, and document systems. Then they hone in on the most commonly important system quality attributes (non-functional requirements). There are dozens of quality attributes and their importance varies from system to system.
- Software Systems Architecture has chapters dedicated to Security, Performance & Scalability, Availability & Resilience, and Evolution.
- Software Architecture in Practice is a standard textbook (and a potential alternative System Design book), with chapters dedicated to Availability, Interoperability, Modifiability, Performance, Security, Testability, and Usability.
- ISO 9126 is an international standard for the evaluation of software quality (more description here). Its quality model contains these characteristics and sub-characteristics:
- Functionality: Suitability, Accuracy, Interoperability, Security
- Reliability: Maturity, Fault tolerance, Recoverability
- Usability: Understandability, Learnability, Operability, Attractiveness
- Efficiency: Time Behavior, Resource Utilization
- Maintainability: Analyzability, Changeability, Stability, Testability
- Portability: Adaptability, Installability, Co-Existence, Replaceability
- Software Systems Architecture points to further reading at the end of each chapter.
- Software Architecture in Practice (potential alternative 1st book)
- Software Architecture for Developers
- 97 Things Every Software Architect Should Know
- Patterns of Enterprise Application Architecture
- Service Design Patterns
- Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions (spoiler: asynchronous message queues are wonderful)
- Beautiful Architecture
- The Architecture of Open Source Applications: Volume I, Volume II
- Articles: AWS Whitepapers
- Article: AWS Tips I Wish I’d Known Before I Started
- Photo: Latency Comparison
- Martin Fowler’s Tags: Application Architecture, Enterprise Architecture, Application Integration, Expositional Architectures
- Maxim Chernyak’s article the CMS Trap is really good. I wonder if some of these lessons may only be learnable by over-engineering a system yourself and experiencing the resulting pain over a period of months or years. You learn to prefer simple dumb short-term in-code solutions over fancy long-term predictions crystalized into a persisted model. (Except when doing that is the wrong decision ;)
- “Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves.” – Alan Kay (Turing Award #38, TED Talk, OOP, Smalltalk)
- “All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.” – Grady Booch (UML)
- “Software design must resolve a set of mutually contradictory requirements.” – Kent Beck (TDD, XP, jUnit, Book Series, Agile Manifesto)
- “The cheapest, fastest, and most reliable components are those that aren’t there.” – Gordon Bell
- “All computers wait at the same speed.” – Thomas E. Bell
- “A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work.” – John Gall (The Systems Bible)
- “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius – and a lot of courage – to move in the opposite direction.” – Albert Einstein (Nobel Prize, Biography)
7. System Design (Availability): Designing for environmental variation and failure conditions.
As a system attribute, Availability is discussed as part of the System Design concept. However Availability is important enough to justify its own book on the reading list, especially when a book as excellent and practical as Release It! exists. There are many system quality attributes that contribute to Availability. They are interrelated and it can be difficult to keep the terminology straight (at least it is for me). Here is my attempt to provide definitions and relationships in a way that is consistent with Release It!:
- Stability: Ability to continue handling requests in the face of component failures or a workload that exceeds capacity. You may not handle all requests, and handling quality may be degraded.
- Request: An abstract unit of work.
- Workload: A set or stream of requests.
- Mixed Workload: A combination of different types of requests.
- Redundancy: No single point of failure for capabilities or data.
- Scalability: Ability to add capacity to a system.
- Capacity: For a workload, the max throughput with acceptable performance.
- Throughput: Number of requests served in a given time window.
- Performance: How long a single request takes.
- Availability: Handling all requests acceptably. Availability often depends on Stability and Scalability (among other things).
- Web Operations
- Scalable Internet Architectures
- Blog: High Scalability
- Article: Availability, Reliability, Scalability
- Article: Performance, Scalability, Stability
- “As a rule, software systems do not work well until they have been used, and have failed repeatedly, in real applications.” – David Parnas (Information Hiding)
- “The unavoidable price of reliability is simplicity.” – Sir Tony Hoare (Turing Award #15, ALGOL W, Null Reference Mistake)
8. System Design (Security): Protecting data & resources.
As a system attribute, Security is discussed as part of the System Design concept. I was undecided about including an additional book focused on just Security. Security is universally important, but many Security books have too much depth to provide excellent ROI for many software developers. However when I discovered Foundations of Security it made the decision easier. This excellent book has a practical breadth and depth, making it worthwhile for most software developers.
- Software Security: Building Security In
- 24 Deadly Sins of Software Security: Programming Flaws and How to Fix Them
- The Web Application Hacker’s Handbook: Discovering and Exploiting Security Flaws
- Security Engineering: A Guide to Building Dependable Distributed Systems
- The Art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities
- AWS Whitepapers: AWS Security Best Practices, Auditing Security Checklist
- A few examples of users getting their accounts hacked: Paypal & GoDaddy, Apple & Amazon, Twitter Employees. (LPT: Use two-factor authentication for important accounts, especially email accounts because other services’ password reset procedures use email addresses.)
- “If you give a hacker a new toy, the first thing he’ll do is take it apart to figure out how it works.” – Jamie Zawinski (Netscape)
- “The key to social engineering is influencing a person to do something that allows the hacker to gain access to information or your network.” – Kevin Mitnick (Ghost in the Wires)
- “Only amateurs attack machines; professionals target people.” – Bruce Schneier (Liars and Outliers, Schneier on Security)
- “The hacker mindset doesn’t actually see what happens on the other side, to the victim.” – Kevin Mitnick (Ghost in the Wires)
9. Legacy Code: What happens when a codebase is untested, large, and coupled? How can you fix it?
I was initially undecided about whether Legacy Code was relevant to most software developers. Some of us are lucky enough to not work with a legacy codebase. I decided to include this excellent book because it provides a view into the implications of technical debt over time. Every software developer benefits from knowing the implications of design decisions and being able to communicate them. This book also provides a toolkit for digging yourself out of trouble. Even if you don’t work with a legacy codebase, there are usually dark corners that need to be managed carefully. For example, if you decide to build on a prototype instead of throwing it away, you may be starting with some amount of Legacy Code.
- “To me, legacy code is simply code without tests. I’ve gotten some grief for this definition… I have no problem defining legacy code as code without tests. It is a good working definition, and it points to a solution.” – Michael Feathers (Legacy Code)
- “The legacy code was so tightly coupled that if you put a chunk of coal between the classes you would get a diamond.” – Jeremy Miller
- All software is legacy (essay)
10. User Experience Design: How do users experience our products & services?
- User Experience (UX) is how someone responds to a product or service, including their behaviors, attitudes, and emotions. For users, the UX is the software. A central part of a user’s experience is the User Interface (UI): the collection of mechanisms a person uses to interact with a product or service (i.e. pages & widgets). A good place to start thinking about UX & UI is this essay. A variety of skills and perspectives go into crafting an amazing UX:
- Interaction Design (IxD): Designing an interactive UI to match & satisfy user needs. (This is the focus of the recommended book: Designing Interfaces.)
- Graphic Design: Communication, stylizing, and problem-solving through the use of type, space, and image.
- Information Architecture: Organizing software and knowledge to support findability and usability.
- Technical Writing: Writing to support the user, both inside the product as messaging and outside the product as documentation. (MailChimp’s writing style guide shows what is possible in this area.)
- Usability Testing: Evaluating the usability of a product by observing users interacting with it. (See related further reading below.)
- Human Computer Interaction (HCI): A broad field encompassing studying, planning, and designing interactions between people and computers.
- Software Development: As software developers, when we evolve a product’s software we affect the UX in many ways (directly and indirectly).
- To be somewhat self-sufficient I think it useful for a software developer to be able to put together “decent” user interfaces. Otherwise you can’t create fully working applications or prototypes without seeking help. Designing Interfaces is a good starting point because it has lot of concrete patterns and examples to get you started quickly.
- Building on a UI framework like Bootstrap makes it much easier to create decent web interfaces. You benefit from a mature style guide, with reduced novelty as the tradeoff. (Style guides are described in Lean UX.)
- Designing the UI shouldn’t wait until the last minute. Although a layered architecture can decouple the UI code from Domain code via Application API, in practice the Application’s capabilities must evolve to support an evolving UI. If you follow the agile practice of delivering complete features early & often, then you get feedback about the UI’s needs early & often. (Netflix is an extreme example of UI needs driving the Application API, where a proliferation of UI clients forced them to go beyond a one-size-fits-all API.)
- Depending on what you work on, an Interaction Design book focusing more on Mobile, Tablet, or TV paradigms may be more directly applicable to your area. (Either way, there is considerable overlap in principles and patterns.)
UX Design is an entire profession, so reading a single patterns book only scratches the surface. There are many excellent and non-redundant books:
- Interaction Design
- Usability Feedback
- Visualizing Information
- General Design
- Feeling vs. Functionality
- I’m throwing this in to suggest that there is more to user feelings than engineering and design. The difference between Apple and other consumer electronics companies cannot be explained by product quality alone. Consumers’ feelings towards the brand matters a lot. Apple has communicated a promise that their customers believe in, and their products (generally) deliver on that promise. I enjoyed this presentation about brands by Sasha Strauss: Part 1, Part 2.
- “Form follows function – that has been misunderstood. Form and function should be one, joined in a spiritual union.” – Frank Lloyd Wright (Organic Architecture)
- “We have always thought about design as being so much more than just the way something looks. It’s the whole thing: the way something works on so many different levels.” – Jony Ive (Biography, iMac, iPod, iPhone, iPad)
- “The ability to simplify means to eliminate the unnecessary so that the necessary may speak.” – Hans Hofmann
- “Business owners need to realize that their design is a reflection of their business even if it is not intentional. If you don’t care about your design then your design is telling people that you don’t care about your business.” – Marco Suarez
- “It is far better to adapt the technology to the user than to force the user to adapt to the technology.” – Larry Marine
- “Testing one user early in the project is better than testing fifty near the end.” – Steve Krug (Don’t Make Me Think, Rocket Surgery Made Easy)
- “The value of a prototype is in the education it gives you, not in the code itself.” – Alan Cooper (About Face, The Inmates Are Running the Asylum)
- “Design should never say, ‘Look at me.’ It should always say, ‘Look at this.'” – David Craib
- “Don’t design for everyone. It’s impossible. All you end up doing is designing something that makes everyone unhappy.” – Leisa Reichelt
- “I’ve learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel.” – Maya Angelou
- “A good design is driven by needs and defined by constraints.” – Astik Pant
- “Design is like a mom, nobody notices when she’s around, but everybody misses her when she’s not.” – Santiago Borray
- “The advance of technology is based on making it fit in so that you don’t really even notice it, so it’s part of everyday life.” – Bill Gates
- “When I design, I don’t consider the technical or commercial parameters so much as the desire for a dream that humans have attempted to project onto an object.” – Philippe Starck
- “If we want users to like our software we should design it to behave like a likeable person: respectful, generous and helpful.” – Alan Cooper (About Face, The Inmates Are Running the Asylum)
- “Very often design is the most immediate way of defining what products become in people’s minds.” – Jony Ive (Biography, iMac, iPod, iPhone, iPad)
- “Real artists ship.” – Steve Jobs (Biography, On Life, On Design, On Innovation, Commencement Speech, Amazing 1980 Reflections, 1983 Gameshow with Bill Gates, 1983 Presenting Famous 1984 Superbowl Ad, 1984 Unveiling Macintosh, 1990 interview, 1997 Return, 2001 iPod Introduction, 2007 Interview with Bill Gates)
Software Delivery Concepts (#11-16)
11. Lean Systems: Organizing an efficient & adaptive human system.
An argument could be made that Lean Systems and Agile Team Organization are too similar to warrant separate treatment on this reading list. However, I include Lean Systems separately because it provides the big picture motivation for just-in time adaptive workflows like Agile, including the historical manufacturing context. Lean Systems sets the stage, and Agile Team Organization outlines a way to establish a Lean System for software product delivery.
- Implementing Lean Software Development: From Concept to Cash
- Lean Software Development: An Agile Toolkit
- The Poppendiecks’ reading list
- Article: Agile vs. Lean
- “The most dangerous kind of waste is the waste we do not recognize.” – Shigeo Shingo (Toyota Production System)
- “A bad system will beat a good person every time.” – W. Edwards Deming (PDCA, National Medal of Technology, Principles)
- “There are many experts on how things have been done up to now. If you think something could use a little improvement, you are the expert.” – Robert Brault
- “Different isn’t always better, but better is always different.” – Unknown
- “A relentless barrage of “why’s” is the best way to prepare your mind to pierce the clouded veil of thinking caused by the status quo. Use it often.” – Shigeo Shingo (Toyota Production System)
- “The best approach is to dig out and eliminate problems where they are assumed not to exist.” – Shigeo Shingo (Toyota Production System)
- “A stitch in time saves nine.” – Benjamin Franklin (Biography, Autobiography)
- “I’m too busy mopping the floor to turn off the faucet.” – Unknown
- “If you do what you always did, you will get what you always got.” – Albert Einstein (Nobel Prize, Biography)
- “Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” – Melvin Conway (Conway’s Law)
12. Agile Team Organization: Organizing incremental & iterative incremental product development.
- Agile is a set of methodologies for responding to these and other realities:
- Like writing an essay, software development is an creative learning process.
- The most important input to this learning is customer feedback, so you need to deliver working software to customers early & often.
- Future You is a genius compared to Current You, so defer decisions when possible. Any plans made today will be made with the least information we will ever have.
- Handoffs don’t work well.
- Too much work in progress causes problems.
- I think the Agile Development and Lean Systems communities correctly describe many of the challenges of evolving valuable software with a team. I also think their proposed solutions are generally in the right direction. Not everyone agrees. I wouldn’t say everyone must practice Agile & Lean, but I would say that every professional should know the problems they address and the solutions they propose. You can pick and choose solutions based upon preferences and context. (For example, I am a bit skeptical of Velocity because I am regularly baffled by how difficult it is to predict what issues will arise during development.)
- Scrum is the most popular flavor of Agile, but you’ll naturally want to choose an alternative book if you’re going to practice a different flavor (i.e. Extreme Programming).
- I find Scrum to be a list of sensible and low ceremony practices:
- Sprints: Build software incrementally and iteratively. Deliver working features so you can get customer feedback early and often.
- User Stories: Write down ideas to work on and prioritize them. Defer figuring out the details (when this is responsible) because you’ll know more later.
- Scrum Meetings: Regularly talk as a team about what is going on.
- Sprint Reviews: Regularly show people what you are building to get feedback.
- Sprint Retrospectives: Regularly discuss how you can work better as a team.
- Product Owner: Have someone who has the knowledge, authority, and availability to prioritize features.
- Scrum Master: Have someone focused on process improvement and coordination. This person should ask questions and employ coaching. They should not give orders.
- The Agile Manifesto
- Scrum Shortcuts without Cutting Corners: Agile Tactics, Tools, & Tips
- Succeeding with Agile
- The Art of Agile Development
- Agile Estimating and Planning
- 97 Things Every Project Manager Should Know
- Waltzing With Bears: Managing Risk on Software Projects
- Organizational Patterns of Agile Software Development
- Extreme Programming Explained: Embrace Change
- Beautiful Teams: Inspiring and Cautionary Tales from Veteran Team Leaders
- The Art of Community: Building the New Age of Participation
- The Mythical Man-Month: Essays on Software Engineering
- Agile and Iterative Development: A Manager’s Guide
- Rapid Development: Taming Wild Software Schedules
- Software Estimation: Demystifying the Black Art
- Article: Flaccid Scrum (is bad)
- Slides: Netflix’s Culture: Freedom & Responsibility
- Martin Fowler’s Tags: Agile, Agile Adoption, Agile History, Scrum, XP, Team Organization
- “Everyone has a plan until they get punched in the face.” – Mike Tyson (Agile Knockouts)
- “It is not the strongest of the species that survives, nor the most intelligent. It is the one that is the most adaptable to change.” – Charles Darwin (Biography, Autobiography, Evolution)
- “Well done is better than well said.” – Benjamin Franklin (Biography, Autobiography)
- “Question: How do you eat an elephant? Answer: A bite at a time.” – Unknown
- “Over and over, people try to design systems that make tomorrow’s work easy. But when tomorrow comes it turns out they didn’t quite understand tomorrow’s work, and they actually made it harder.” – Ward Cunningham (Invented Wikis, Technical Debt, Agile Manifesto)
- “If you find yourself in a hole, stop digging.” – Will Rogers
- “You can’t catch one hog when you’re chasing two.” – Moe Schaffer
- “Don’t spend a dollar’s worth of time on a ten cent decision.” – Peter Turla
- “Focused action beats brilliance” – Mark Sanborn
- “I shall try to correct errors when shown to be errors, and I shall adopt new views so fast as they appear to be true views.” – Abraham Lincoln (Biography)
- “You don’t get harmony if everyone sings the same note.” – Steve Honey
- “Start with a sketch, but make it quick. There’s real work to be done.” – Stefan Hartwig
- “You can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future.” – Steve Jobs (Biography, On Life, On Design, On Innovation, Commencement Speech, Amazing 1980 Reflections, 1983 Gameshow with Bill Gates, 1983 Presenting Famous 1984 Superbowl Ad, 1984 Unveiling Macintosh, 1990 interview, 1997 Return, 2001 iPod Introduction, 2007 Interview with Bill Gates)
13. Requirements Design: Cross-functional teams collaboratively designing software.
- Although the terms requirements and specification are not perfectly defined, here is a common and useful distinction:
- Requirements: From the customer and user perspective, what constraints should a product or service satisfy? What should the system do?
- Specification: From the engineering perspective, how will the requirements be satisfied? How will the system be implemented?
- Questions for product development teams to answer include: How should requirements be represented? At what level of detail? When should they be written? In Agile projects, User Stories are light-weight goals serving as reminders to have a just-in-time design discussion. After a design discussion occurs, should the conclusions be recorded? Agile’s focus on conversations and working software over documentation suggests that you can often jump into writing tests and coding the feature (after breaking the story into smaller implementation tasks). But what if the design discussion involves complexities such as decision trees, workflows, timing, or other business logic? Then medium-weight representations such as Use Cases (Writing Effective Use Cases) and UML Diagrams (UML Distilled) can help support collaboration and mutual understanding. In contrast, heavy-weight requirements documents are more along the lines of Big Design Up Front.
- Regarding User Stories vs. Use Cases, here is Alistair Cockburn’s description of when he employs Use Cases on Agile projects. (User Stories Applied also includes a comparison of User Stories and Use Cases.)
- Ideally requirements are written and prioritized by the customer, or with as much of their input as possible. Writing requirements is a software design activity, so software developers can help guide the process by explaining:
- How to structure stories.
- When a story is too large for an iteration (or too small).
- How to split up stories by feature rather than layer (thus delivering working features for each story).
- How to avoid specifying user interface details, because requirements should only constrain outcomes (not how outcomes are achieved).
- Writing Effective Use Cases
- UML Distilled: A Brief Guide to the Standard Object Modeling Language
- Software Requirements
- Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing
- Specification by Example: How Successful Teams Deliver the Right Software
- Martin Fowler’s Tags: Requirements Analysis, UML
- “If we’d asked the customers what they wanted, they would have said ‘faster horses’.” – Henry Ford (Biography, Autobiography)
- “Computers are good at following instructions, but not at reading your mind.” – Donald Knuth (Turing Award #9, TAOCP)
- “Know thy user, and you are not thy user.” – Arnie Lund
- “I love software, because if you can imagine something, you can build it.” – Ray Ozzie (Lotus Notes, Microsoft CTO)
14. Testing Products & Services: Do our products & services provide what our customers need?
Testing products & services requires a diverse skill set. I believe software developers should strive to do as much of this as possible themselves. Books like Agile Testing and How Google Tests Software point the way. However these books also make it clear that several categories of high value testing and clarification are most effectively pursued by dedicated testing experts. Agile Testing is structured around Brian Marick’s testing quadrants:
- Technology-Facing, Supporting the Team: Unit Tests, Component Tests
- Business-Facing, Supporting the Team: Functional Tests, Examples, Story Tests, Prototypes, Simulations
- Business-Facing, Critiquing the Product: Exploratory Testing, Scenarios, Usability Testing, User Acceptance Testing (UAT), Alpha/Beta
- Technology-Facing, Critiquing the Product: Performance & Load Testing, Security Testing, “ility” Testing
- How Google Tests Software
- How We Test Software at Microsoft
- Beautiful Testing: Leading Professionals Reveal How They improve Software
- Domain-Specific Languages: It may seem like a stretch to say a DSL book is related to testing products & services. However a central tenet of Agile is directly including the customer (and other non-engineers) in product development. One way to do this is to write tests that non-engineers can read, or even write. This is where domain-specific languages shine. We can discuss, think about, and represent tests in the DSL, and then translate that language into executable tests. (This is very similar to how we think about and represent software in high-level programming languages, and execute them by translation into C or machine instructions.) Tools in this area include Fit, FitNesse, and Cucumber.
15. Continuous Delivery: Automated pipeline for integration, testing, & release.
Continuous Delivery is a broad concept that relates to Continuous Integration, Automated Testing, Configuration Management, Automated Deployment, Release Management, DevOps, and much more. Combining these practices into a (fully or mostly) automated delivery pipeline is essential to establishing a lean/agile/just-in-time/adaptive software delivery workflow. The actual release schedule is a business decision, but we should be technically able to release after every sprint (if not every day). Here is a sample set of Delivery/DevOps tools for a Java web application:
- Revision Control: Git and GitHub
- Testing: jUnit and Selenium (headless browser via xvfb)
- Continuous Integration: Jenkins
- Build & Dependency Management: Maven (it’s the most established platform so I just hold my nose and get it over with)
- Artifact Repository: Artifactory or Nexus
- Virtualization: Vagrant for dev/prod parity via virtual machines
- Declarative Provisioning: Puppet or Chef
- Monitoring: There are many options including Amazon CloudWatch, Nagios, Graphite, Logstash, etc.
- Lean Enterprise: Adopting Continuous Delivery & DevOps (I can’t wait to read this.)
- Continuous Integration: Improving Software Quality & Reducing Risk
- Configuration Management Best Practices: Practical Methods that Work in the Real World
- Software Configuration Management Patterns: Effective Teamwork, Practical Integration
- The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps
- Heroku’s 12 Factor App
- Martin Fowler’s Tags: Delivery, Continuous Integration, Build Scripting
- “Release early. Release often.” – Eric Raymond (Cathedral & Bazaar, TAOUP)
- “By far the dominant reason for not releasing sooner was a reluctance to trade the dream of success for the reality of feedback.” – Kent Beck (TDD, XP, jUnit, Book Series, Agile Manifesto)
- “Stop the line, so the line never stops.” – Toyota Production System
- “Amateurs work until they get it right. Professionals work until they can’t get it wrong.” – Unknown
- “Besides black art, there is only automation and mechanization.” – Federico Garcia Lorca
- “Usage is like oxygen for ideas. You can never fully anticipate how an audience is going to react to something you’ve created until it’s out there. That means every moment you’re working on something without it being in the public it’s actually dying, deprived of the oxygen of the real world.” – Matt Mullenweg (WordPress)
16. Customer Development: Creating customer value.
- Lean Startup ties together several of this reading list’s concepts: Lean Systems, Agile Team Organization, Continuous Delivery, and Customer Development.
- Software is valuable if it is used. A software developer’s goal is to create valuable software. The idea of Customer Development is to approach value creation as a science, not as a black art. The strategy is to iteratively apply the scientific method to:
- Problem/Solution Discovery: Discover a problem there is value in solving, and evolve a solution to the problem.
- Market/Product Discovery: Discover a market there is value in addressing, and evolve a product that fits the market.
- They Made America distinguishes between Inventors and Innovators. Inventors create inventions (Tesla, Wozniak). Innovators bring inventions to the masses (Ford, Gates). Some inspiring people succeed at both (Edison, Jobs, Musk).
- Your era matters: 14 of the 75 richest people ever were born in 1830’s USA as a result of industrialization. These individuals were part of a larger group of people who successfully responded to the profound technological changes at that time. Change creates opportunity. Today is also a time of profound technological change. Much of this change is centered around software. If you’ve read this far, you’re probably a software developer like me. I feel extremely fortunate that individuals with our particular skill set, at this particular time in history, are able to have such an unprecedented impact on people’s lives and the world in general.
- Software is changing many areas: phones, TVs, cars, exercise, communication, retail, homes, books, news, healthcare, drugs, education, dating, networking, movies, golf, government, cities, printing, politics, and pretty much everything else. What do you want to improve?
- 4 Steps to the Epiphany goes into more depth about Customer Development specifically. Here is the author’s article about why the Lean Startup changes everything
- Tilt: Shifting Your Strategy from Products to Customer
- They Made America: From the Steam Engine to the Search Engine: Two Centuries of Innovators (Choose the hardcover, it is one of the most beautifully illustrated books I own.)
- Resource: AWS Startups Page
- “I think if you do something and it turns out pretty good, then you should go do something else wonderful, not dwell on it for too long. Just figure out what’s next. ” – Steve Jobs (Biography, On Life, On Design, On Innovation, Commencement Speech, Amazing 1980 Reflections, 1983 Gameshow with Bill Gates, 1983 Presenting Famous 1984 Superbowl Ad, 1984 Unveiling Macintosh, 1990 interview, 1997 Return, 2001 iPod Introduction, 2007 Interview with Bill Gates)
- “In a startup, both the problem and solution are unknown. ” – Eric Ries (Lean Startup)
- “A problem well stated is a problem half-solved.” – Charles Kettering
- “It is not unscientific to make a guess, although many people who are not in science think it is.” – Richard Feynman (Nobel Prize, Surely You’re Joking)
- “Anything that won’t sell, I don’t want to invent. Its sale is proof of utility, and utility is success. ” – Thomas Edison (Biography, The Oatmeal’s Critique)
- “The worthwhile problems are the ones you can really solve or help solve, the ones you can really contribute something to.” – Richard Feynman (Nobel Prize, Surely You’re Joking)
- “The best way to predict the future is to invent it.” – Alan Kay (Turing Award #38, TED Talk, OOP, Smalltalk)
- “The world we have created is a product of our thinking; it cannot be changed without changing our thinking.” – Albert Einstein (Nobel Prize, Biography)
- “Solving any problem is more important than being right.” – Milton Glaser
- “Don’t make something unless it is both necessary and useful; but if it is both necessary and useful, don’t hesitate to make it beautiful.” – Shaker Philosophy
- “I am not discouraged because every wrong attempt discarded is a step forward.” – Thomas Edison (Biography, The Oatmeal’s Critique)
- “We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten. Don’t let yourself be lulled into inaction.” – Bill Gates
- “It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.” – Teddy Roosevelt (Epic Biography)
One last note: Learning a craft requires a balance between studying and doing.
- “An investment in knowledge always pays the best interest.” – Benjamin Franklin (Biography, Autobiography)
- “Reading furnishes the mind only with materials of knowledge; it is thinking that makes what we read ours.” – John Locke
- “One glance at a book and you hear the voice of another person, perhaps someone dead for 1,000 years. To read is to voyage through time.” – Carl Sagan (Pulitzer Prize, Cosmos, SETI)
- “Most papers in computer science describe how their author learned what someone else already knew.” – Peter Landin
- “Mathematicians stand on each others’ shoulders and computer scientists stand on each others’ toes.” – Richard Hamming (Turing Award #3)
- “Forgive him, for he believes that the customs of his tribe are the laws of nature!” – George Bernard Shaw
- “If people knew how hard I worked to get my mastery, it wouldn’t seem so wonderful at all.” – Michelangelo Buonarroti (Biography, Medici Documentary)
- “I have been impressed with the urgency of doing. Knowing is not enough; we must apply.” – Leonardo da Vinci (Biography, Notebooks, Medici Documentary)
- “I hear and I forget. I see and I remember. I do and I understand.” – Confucius
- “All our knowledge begins with the senses, proceeds then to the understanding, and ends with reason.” – Immanuel Kant
- “Be self aware, rather than a repetitious robot.” – Bruce Lee
Other Software Development Reading Lists
Be sure to note the dates on some of these.
- The Jolt Awards
- Jeff Atwood’s
- Joel Spolsky’s and his management list
- Mary & Tom Poppendieck’s
- Jurgen Appelo’s
- Tan Ban Leong’s
- Ben Watson’s
- InfoQ Editor’s
- UC Irvine Institute for Software Research’s
- Intel’s (hardware leaning)
- Addison-Wesley Signature Series: Kent Beck, Mike Cohn, Martin Fowler
- Great programming reading list
- StackOverflow list of freely available programming books
- StackOverflow list of influential programming books
- Quora list
- List of books and book lists from sidebar of /r/learnprogramming
- Ryan Singer’s