How to handle a programming problem

A couple students whom I mentor through a software development course recently asked me for a generic advice on a way to approach a relatively large software development - something that would help them know where and how to start producing a solution. The question definitely provoked several pretty deep discussions. Since I do expect other students to seek advice of a similar kind I decided to devote a bit more time to the framing my advice.

While contemplating and discussing the topic I came up with a number of points that depict the approach that I use to build software. Sometimes I would follow this process to the point, in other cases I just adhere to the general line of thinking that it suggests - it definitely depends on the kind and size of a particular project. Thus, even though I present the approach as a sequence of steps, it is most reasonable to treat it as a description of a way of thinking about a programming problem.

Note that it so happened that I faced the question soon after going back to the Code Complete book to check a couple chapters. Moreover, I was reading How to Solve It by George Polya roughly at the same time. Because both of these have a lot to do with the topic at hand, the post definitely builds on the ideas found there. I absolutely recommend both of these books in case you didn't give these a try yet.

So, you find yourself with the task of producing a piece of software that would solve a particular issue. What should you do?

1. First and foremost, state your problem in writing - what are you trying to solve?
2. What requirements should the solution satisfy? Jot every one of these down as well.
3. Imagine the final solution, state in writing what the thing that you are about to build will look like. Focus on the general shape of the final result here. See how it will solve the problem and meet the requirements.
4. Think over the data model. Decide what kinds of data you will be handling, how to structure it better, where it comes from and where it should go. Imagine how the information is split into tables and collections, spot the relations between them. Consider which bits you need to store and how. Do this even if you don't have any kind of database - maybe building only a frontend application - data is something that you work with in any software and it definitely plays a great role in shaping the solution. Write all of this down.
5. Decompose your future system or program based on what you have understood so far - think over how you can split the problem and the solution into pieces. Depending on the scale of the task you may be thinking on the level of services, modules or classes and functions here. The larger the problem - the larger should be the size of units considered on this design stage.
6. In this process follow the breadth-first approach - e.g. don't go into designing functions before you produce an overview on the level of modules. List the large-scale units first, state their purpose clearly and only once this is done permit yourself to descend to the next level of detail. Notice when your start jumping between different levels of design - like thinking of modules, then functions, then modules again. If this happens stop and pull yourself back to the highest level that still needs clarifying.
7. Don't overdesign and keep moving. Most of the technical design will happen while you're coding. Your key goal at the moment is to draw yourself a clear image of the whole solution, its components and relations that govern them and to see how you will meet every bit of requirements. Design can consume infinite time, so don't let yourself get lost in it - details will come.
8. After facing your components think over your past experience - each component poses a smaller problem to solve and many of these smaller problems you may have solved in the past. If so, check which bits of previous projects you can use now - this may mean anything from pulling in a dozen existing files to just reviewing how you approached a particular issue a couple months ago.
9. Throughout all these stages use pencil, draw figures, sketch stuff, write things down - it helps you think. Once you have started producing a diagram, your mind will follow. Draw how the system or its particular component is decomposed, outline relations between the pieces, sketch the data flow, chart user actions sequence.
10. Devise a plan. Decide in which order you will build your components. Write that down (in any form or medium) - that will help you see and maintain progress, especially with longer projects
11. Pick the first component on the list. Jot down the steps to build it. Think how to test it, list the functions to implement, decide on the order, add checkboxes. If you don't know how to pick the starting point, pick any single one of them. At this stage it’s most important just to start the job, so start anywhere. Even if you choose a wrong initial task, you will soon be able to see and correct that.
12. Go and start writing the code. The first lines may go hard, but with all the preparatory work you should be able to break through them and the rest will follow easily.
13. Don't try to write perfect code from scratch - start dirty. Starting is a challenge by itself, so simplify the task and relax the requirements. It is much easier to produce a dirty solution and clean it once you see it working. Just don't forget to always clean the mess.
14. Write tests as you go to stay confident in the state of your code. Confidence does matter a lot here and the lack of it may slow you down significantly.
15. Iterate. Pick small tasks, complete them, polish the solution, celebrate, step back to take a look at your design and plan, repeat.
16. When you face challenges and are not sure where to go from there, take a pencil and think. State the problem that you're trying to solve, write out the required characteristics of the solution, draw, see it and then implement it.
17. Yes, look for what others have to say about your problem - stackoverflow, github and google are your friends. Don’t just grab any code that you find on the web though - take time to understand what it does, why it works and what kind of new issues it may throw at you.
18. When all else fails, go away for some time - take a walk, wash the dishes, smoke a pipe, use your hammock time. Let your brain wander away. Most times you will come back with new ideas. In the worst case you will get some rest and have more energy to continue attacking the problem.

And finally, if there are no more tasks to do, everything is polished and you feel happy about the result, celebrate the success, reward yourself, think over the lessons to learn from this project and go look for the next issue to solve. If you can program, there is an infinite number of problems waiting for you!

Fixing the Wrong Problem, Once Again

The last week I spent two or three hours looking for the cause of a bug really close to the place where it was actually located – but not close enough. From the exact moment when I opened the bug description in Visual Studio everything pointed me in the right direction – towards a couple lines in our code base, fixing which would have resolved the issue. However, when reproducing it according to the steps outlined in the issue tracker and exploring the related behavior of the system I have caught a glimpse of something peculiar, that could have been the reason of my issue, and it made me waste a lot of time on trying to fix the wrong problem.

A particularly important detail about this new problem that I discovered was that its incorrectness didn’t look like something trivial and minor. It seemed to me that I was looking at a very severe bug spanning a number of components of our system. At some point, I was almost sure that I have found a really big problem, much broader than what the reporter could have thought. This feeling of stumbling upon something significant that might have far-fetching consequences quickly made me forget my initial objective.

Moreover, I was excited in a very negative way by the idea that I see something that other devs on the team have overlooked, and the desire to show everyone how cool I am pushed me even further away from what I should have been fixing. The fact that the newly found problem looked much more complex than what followed from the original description also meant that it was more interesting. Hence, my eagerness to fight it instead of taking a broader look and examining all the aspects of the problematic process.

It took me a couple hours of digging and a relatively long break to persuade myself that I should try and go through all the steps that cause data corruption and the related error. Initially, I have stopped my attempts to find the exact source of the issue once I had spotted something strange and unexpected. Should I just follow the description of the issue, carefully crafted by the QA engineer, up to the point where the problem is clearly seen, I would have reached the root of the problem much faster – maybe in a matter of minutes.

What happened instead is actually very familiar to me – that’s not the first time I make this mistake. From time to time, when tracing down a bug I face something strange-looking in the middle of the way and start exploring it. I do this despite being fully aware of the idea that any problem must be solved from where it manifests itself down to its actual reason – without taking any shortcuts. I know that one should under no circumstances skip any steps on this way, but I still do this occasionally and it never brings me close to where I would be happy to find myself.

This exact problem is what the strategy of moving slowly and in tiny steps aims to solve. For me the nature of the issue began getting clearer once I pushed aside the keyboard and started writing – with pen and paper. These tools make one work slower, they don’t allow to hack one’s way through the code and skip crucial points. Moreover, they facilitate deeper and calmer thinking, while rapidly stepping over the lines of code in a debugger or executing numerous variations of an SQL query usually descends one into a state where only wild guesses and ‘let-us-try-this’ hacks are possible.

My takeaway is once again that one must not rush to solve any problem that they happen to see – frequently what we initially perceive as incorrect behavior turns out to be desired in the end. Instead, one should work towards fixing what they aimed at first. In many cases problem solving works better if one maintains a queue of the problems – not a stack. Fortunately, even though our minds seem to be using a stack internally, we can always push extra problems out of it – to a bug-tracker or to a good old sheet of paper – so that the original, the right problem can be fully comprehended and solved, giving way to new glorious fixes.

CodeProject