Heading West

These days I am working hard on my diploma project. The defense is scheduled for the 26th of June, so I have a great and nervous month ahead. Obviously, finishing the six year course of study is an enjoyable achievement. Being an ending of one significant period of my life, this event will at the same time mark a beginning of another one and this new beginning worries me because I am still uncertain not only about the things that it will bring, but about the direction I should go from there as well. One particular opportunity to which I am encouraged is continuing my education in the form of pursuing Master's degree in CS in USA or Canada.

 

The idea is great and there are lot's of reasons to try. First of all, I like the process of learning new skills and getting knowledge, so the chances are high that I will enjoy another couple of years at university. At the same time, the fact that I lack trust in Russian system of higher education makes me look for an opportunity to deepen and broaden my own education elsewhere. Moreover, even if my assumption about the quality of university education being higher in America doesn't hold, it would still be great to explore new grounds and se what do things look like there. Another feature that makes a degree obtained abroad more valuable for me than one got in Russia is that, as it seems to me, western MS is valued much higher not only in US, Canada and Europe, but in Russia as well.

 

However, while there are huge benefits offered by Master's program in a good university, I see drawbacks as well. The most important of these is the fact that the path I want to go is the path of a software engineer. While mastering this field indeed requires descent education, what is even more crucial for a developer is actual experience in tackling real-world problems. Since I will hopefully get diploma in a month it may be better for me to get into industry instead of spending another 20 months or so on education. This point is made even stronger by the fact that I am almost 24 now and still have only minor job experience. Put another way, I feel it's time to focus on producing things - not only preparing myself for that. I acknowledge that it is possible to create great stuff even while studying at a university, although the word 'focus' is important here and the availability of numerous MOOCs allows me to not merely trade further education for job but to get both at the same time.

 

One more thing that really frightens me when it comes to studying abroad is money. The education alone costs a great deal and one has to take into account the cost of dwelling and other stuff as well. The price really seems too high for me and I am not sure whether I will be willing to pay it - no matter through loans or from my parents' pocket. On the other side, I do believe that generally these costs are fully justified by the merits one gets from university in the form of skills, new contacts and the degree itself. Although, because I am close to getting a degree it is not that easy to estimate the value of such an investment in comparison to going straight for a job.

 

Even with all the arguments laid out it is still very difficult for me to make any reasonable decision. The idea that should I choose to pursue Master's abroad it is better to do this as soon as possible also doesn't make the decision easier. Fortunately, these days I have to concentrate all my efforts on the diploma and put most other issues aside.

C#: Side Effects and LINQ's Defferred Execution

It is difficult for me to imagine a program that doesn't deal with collections of some type – all our applications do proliferate with Arrays, Lists HashSets, DataTables and dozens of others. When writing code in C# the first tool I consider when faced with a collection is LINQ. Its capabilities combined with the fact that LINQ queries can be applied to nearly everything make it a very powerful tool saving a lot of time for any .NET developer. Still, no power comes for free and every piece of equipment must be used with some caution.

 

One of the paradigms widely used by LINQ is lazy evaluation. Most LINQ methods, although it seems that these just return results immediately, are actually deferred. What it means is that whenever you write something like var strings = myArray.Select(n => n.ToString()); the only thing done at the moment of execution of these lines is creation of a query object. Real work in this case is not approached until someone wants the results. This is a well-known fact about LINQ and generally doesn't do any harm. However, when the processing wrapped into LINQ queries is not pure in the functional sense of the word quite strange and difficult to track down problems may occur. Below I will try to demonstrate such a case. 

C#: LINQ and IEqualityComparer

When writing programs in C# with the help of LINQ I have come across the IEqualityComparer<T> generic interface several times. The name conveys the purpose of the interface clearly, still, surprisingly, my first attempts to apply it were marked by strong confusion - the thing merely didn't work the way I wanted it to. Moreover, after going to Google and StackOverflow I have noticed that I'm not the only one to face difficulties with it. After inspecting multiple SO questions and answers related to the topic as well as some other articles I have both found the solution that fitted my needs for the moment and got some understanding of the way IEqualityComparer is actually used by LINQ operators. Here I will try to explain what I got from there.

 

Tutorials Are Not That Bad

From the time when I started to study programming in earnest I somehow disdained tutorials of any kind. There are dozens of them in the world of programming as well as in any other field involving operations performed by means of a personal computer. These tutorials composed of exercises or tasks, which end up being long lists of "go-there-type-this-click-that"s, sometimes interleaved with descriptions of "note-how-the-thing-changes" kind jump at you whenever you type a query involving a name of some technology or tool in Google. Moreover, almost every book dealing with some technology - a programming language, for instance, - contains such exercises. Maybe, it is even safe to say that the more exercises a book of this kind contains the better it is, so I don't state that all tutorials are useless and hardly deserve any attention and I am not even going to discuss the wide range of their quality. What I say is basically that during several years of study I used to think that an exercise inviting me to reproduce its author's actions is not worth the effort.

 

Actually, most of the tutorials, which I came across while exploring various languages and techniques, were very helpful and provided me with valuable insights, thus leaving me no reasons to lament the time spent. Furthermore, I really appreciate the work of people who produce such things just to make their knowledge and expertise accessible to others. Still, I had a reason or an excuse - the latter sounds closer to reality - to avoid tutorials when pursuing my learning goals. My explanation is as plain as possible: I merely used to believe that trying to apply a thing that is being studied to your own problems or to compose such a problem yourself for the purpose of learning and attempt to deal with it is a much more efficient approach, because it leaves a lot of room to creativity. At the same time I doubted that dumbly following the track of a tutorial can be that helpful and educating. I don't reject the idea that creativity is a rich source of both skills and knowledge, although now I barely think that the low level of creativity inherent to the process of doing most tutorials makes them bad equipment for learning.

 



Indeed, it is easy to spot a flaw in my reasoning about the educational value of tutorials. Above I have stated that following a track of an exercise dumbly wont teach you much - chances are high that this particular claim is correct. What is false is my belief that the only way to do tutorials is the stupid one, because obviously one can thoroughly inspect every step of the tutorial they struggle with and think over each action that they perform. Such an approach, even though it is guided, turns out to be quite intelligent and efficient. Moreover it seems to me that it is very difficult to do tutorials dumbly - even when I begin an exercise merely following the instructions in it, sooner or latter I find myself noting interesting features and how-to's highlighted by the author and imagining how I could use the tricks in my own solutions. So it's hard for me to accept the fact that I used to yell that tutorials are not proper learning tools.

 

It must have been my eagerness to do my own stuff, to invent things that made me reject other's expertise in the form of tutorials, but there are too many obvious benefits offered by such exercises to ignore them. First of all, their authors try hard to make one's first steps in a field as easy and as efficient as possible, so they tend to focus on the key aspects and to draw one's attention to the most important things of the technology or the tool under study. Therefore a tutorial usually provides one with understanding of the basics underlying all other matters that one is going to deal with. This is notably important whenever you explore something quite complex, incorporating dozens of features and cluttered with details, the purpose of which is hardly understandable from the first glance. Such focusing on the basics, that is delivered by tutorials and saves one from being totally overwhelmed by complexity, is made possible solely by the vast experience of their creators. Beside this, there is one more thing that an experienced teacher can give you through pages of a tutorial, and it has to do with creativity. The problem is that creativity has a dark side - the side that leads one to an invention of complex solutions to simple problems, which were, in fact, solved long ago. Unnecessary complexity of this kind is produced in huge quantities by those who study new fields and, although frequently the students learn the right way to achieve their goals later, sometimes these beasts survive in the minds for quite a long time so that later they can creep, for example, into production code. While I strongly appreciate creativity and acknowledge that even inventing a bicycle once more can be educating and worthy, I still think that when one starts to learn something new, it may be better for them to be shielded from such things by the experience of those who know the proper ways.

 

An interesting thing that I have noticed about doing tutorials while exploring SQL Server Analysis Services with the help of one is that the closer I follow the instructions in the exercise, the more it gives my. I mean that when I carefully inspect each statement that the author makes - even when some points are repeated - I end up with much better understanding, than in case I leap over instructions that seem obvious or too dull to me. Definitely, this has to do with the fact that when I skip something I risk missing minor aspects that turn out to be quite important, and this leads to holes in my mental models. These gaps, in turn, tend to produce mistakes later, but may be hard to spot at the same time. On the other hand, when deliberately examining each sentence in an exercise I focus not on achieving a goal itself, but on learning a good way to achieve it. This frees me from pressure - particularly, from the desire to complete a task as fast as possible and to leap to another problem - and allows me to keep a comfortable pace, stopping whenever I need to explore some aspect more thoroughly or think about a question. Surprisingly, learning how to achieve a goal and achieving a goal are not the same activities.

 

This said, my takeaway from the last tutorial that I have done are not only new skills and knowledge, but the obvious idea that one should not ignore other's experience merely for the sake of doing their own things and producing their own results. Experts always have a lot to say to us - newcomers - and the insights provided by their instructions may be much more valuable than the product of one's creative attempts to master something from scratch. Moreover, such intelligent and wise instructions can even spur a lot of creativity in students because they tend to provoke interesting problems and make one look for the solutions. In the end, you always learn something through facing questions and answering them - tutorials just encourage you to ask the right ones.

 

Our Predictions Are Bound To Fail

Some years ago, when I did pay much attention to politics, I have found multiple articles by Valeriy Panyshkin (here is some info about the journalist, alas it's in Russian only), who wrote a lot about politics as well as numerous other problems, which one could see in Russia. Despite the fact, that many of his articles, opening my eyes to the things, which I didn't necessarily want to see, were somehow depressing, I read almost all of them. Moreover, even after I had become less concerned about these things - closing my mind once more - I continued to follow the man's writing. Interestingly, Valeriy gradually switched his attention from politics to a narrower set of issues - he got focused mostly on various problems that children and those who try to help them face in our country. He still highlights many other things too, although, as I see it, now, working with organizations that raise money for healing badly ill children or try to protect orphans' rights, he devotes most of his time, efforts and writing to these particular issues.

 

Recently I have read his article (the link - in Russian too) discussing a set of principles, upon which, according to Valeriy, every charity should base its work. The most surprising thing about the article was the idea that any organization of this kind must aim at ceasing its existence. Although this may seem a bad mission, the idea is quite simple and makes sense when elaborated. Valeriy doesn't claim that every charity should give up helping people and go find a more reasonable niche. What he states is that everyone who works in such a foundation should dream of the day when the problem, that he fights with, is finally solved: a treatment is found, which heals blood cancer in a matter of minutes, orphans receive such a crazy amount of support from government that life is as easy for them as it can be and so on. This way the main purpose of any charity is not only to help people, but to put its best efforts into improving the situation so that one day there will be no need for their help at all. This notion of aiming at becoming obsolete, although looked interesting, seemed to me something local to the field of charities at the first glance. However, it is easy to see that it is inherent (or, at least, should be) to many other kinds of activities and institutions - from medical research centers and police departments to ministries of Justice and parliaments.

 

Because I am a programmer it was particularly interesting for me to notice a trace of this idea in the world of software. First of all, there is a multitude of software development jobs, which pursue a goal of solving a particular problem, and once the solution is brought to life, its consumers, roughly speaking, don't need the awesome guys, who built it. However, this kind of relation between producers and consumers, although widely present in my field, is quite common elsewhere. Maybe its because it looks so usual, that it does not seem that interesting. It stays hardly exciting even if we consider all the benefits that many of us could get in case we did our best to make working with our products and extending them absolutely independent of our own presence. Still, I have finally stumbled upon another idea that looks somehow reminiscent of the principled proposed by Valeriy.

 

There is a system at my university that eases collection and aggregates information about the progress of students in the course of a semester as well as their performance during final exams. As I have undoubtedly mentioned in this blog, my diploma project is going to involve analyzing this data and making predictions based on it. What I want to do is to build a system that is capable of predicting the results of exams for a particular student from their performance during some part of a semester. Such a tool may be useful for faculty staff - for instance, they might want to know who's chances to be expelled are high way before exams. Wait a minute... The people, who teach and do different kinds of administrative work at the university pursue the goal of providing every student with quality education - they barely want anyone of their students to be expelled. So, for what sake would they want my software to say that certain students perform poor enough to fail examination and thus put an end to their education? The answer to this question is not subtle at all, because once you know that someone in your responsibility has serious problems - with study in our case - you gain a chance to help them. Thus, my system is not designed to allow teachers and other faculty staff to concentrate only on those students, who work well, but to draw their attention to those who may fail and to do this before it is too late to change anything. This way I am working on a system that will hopefully predict events that are not supposed to happen thanks to these same predictions. Furthermore, I am quite satisfied with this situation and will work hard to make its failed predictions as precise as possible. Clearly the notion of forecasts, that we make with the purpose of making them fail, is not equivalent to the idea of charities aiming at self-elimination, still I feel there is something similar to the nature of these two things.

 

Not surprisingly, I am not the only one who strives to make precise forecasts to see them fail in the end. The most popular example of applying machine learning techniques, demonstrated in various courses devoted to the subject, is predicting whether a patient is going to have cancer (or has it already.) When such a thing is done by medics outside the lecture-room, their main goal is to save a patient from the disease. The same holds for exploiting data on cardiac arrests or CVAs - healthcare field is rich with examples of this kind, as many other areas are. For instance, analysis of constructions is usually performed to find flaws in them and discover where the thing should be strengthened so that no disaster will happen - not to know where to strike. Aircrafts and vessels are overseen to avoid collisions, which are predicted first. The list is quite long and there is no need to continue it, because my point should be clear already.

 

Actually activities of this kind took place much earlier than the first computer was built. People always tried to foresee things, which they wish didn't happen, and they did this not only because of the desire to know the future, but because they knew that as soon as one is aware of bad things ahead, one can escape or prevent these. So I am not opening something new - I just feel fascinated by the simplest idea that was here since the first days of humanity or even earlier. The invention of computers merely gave us new powerful tools and techniques to foresee things, and it seems astonishing to me, how frequently we try hard to make precise predictions that are bound to fail just because we do succeed.

Software Bucket

I have recently come across a short blog post devoted to having a personal list of software pieces, that one wants to develop - a software bucket. Of course it was interesting for me to think over the possible contents of my own bucket. The first lines of my list are occupied by the things I am currently working on. The topmost is definitely a simple tool for arranging personal tasks. I know that there are plenty of such applications, nevertheless I have decided to create one more - mainly because I want a tool, which is as convenient for me as possible, and hope that someone else might also find it useful.

 

I have originally come to the idea of creating such an application almost a year ago, but it was only in the end of October when I have finally made up a deadline for the first version of the tool. Being fascinated by the new Windows 8 OS I have chosen to deliver my app through the Windows Store. The key advantage of this approach is that it will provide me with both the means of distributing my creation and the feedback channel - in case someone downloads the application. Beside that, Windows Store applications development feels quite suitable for me because I have got reasonable experience with .NET development and, particularly, with Windows Presentation Foundation.

 

Despite having done a significant share of work, I have missed the initial deadline, but feeling ashamed for this I have already set up the new one. I feel determined not break one more promise given to myself. After delivering the first version, on which I am working now, I plan to continue expanding this project and add some more features to help me and  potential users plan activities in even more effective manner. I will struggle to avoid making the app overloaded with details and inconvenient, so that other Windows 8 users will be able to consider it helpful and easy to use.

 

Another thing in the bucket is my current job project - I am developing a little toolset for generating manufacturing-related reports from the data provided by a complex PDM-system. Although the task may seem not very interesting, I feel quite absorbed with it, because it offers me a great chance to practice TDD and refactoring techniques to build and deliver an efficient and reliable piece of software, which is capable of giving valuable help to our customers. Additionally, I believe that the need to continuously align the thing to fresh portions of requirements will build me up towards multiple challenges that I will face in future.

 

On the other side, I was stupid enough not to test drive the application from the first steps, so it developed in a way which is far from perfect. However, I feel surprisingly glad about that, because for me this is an opportunity to feel all the pain caused by my own code and design decisions - now I am refactoring the project and supporting it with automated tests. Lots of interesting details of this piece of software and decisions made while working on it, as well as the desire to clean and develop it further, make it an important part of my software bucket.

 

One more item on the list, which is easy to come by, is my diploma project. This one is going to be a data analysis tool for monitoring students' progress data in my university and making predictions based upon it. I am not sure whether there is a substantial chance for it to be adopted and used by the university staff, nevertheless it provides me with important experience and offers a lot of interesting opportunities. First of all, I would like to explore existing data analysis tools and techniques and to adopt some of these to real data. Besides, while working on the project I have to use an external system, which is aggregating and providing the data consumed by my application. The fact that I have been working with it for a considerable period has taught me to put significant effort into shielding my own creations from the fluctuations in the consumed systems' behavior - positively, that's an important lesson. In this sense this looks a bit like my job project, where my tools retrieve data from an external system. My diploma and job feel even more similar because both have something to do with reporting activities, although the projects focus on different problems. The key aspect of the former is data analysis, while the latter revolves around providing flexible data retrieving and processing tools to bridge the gap between the PDM-system, that our customers use, and the reporting engines. My work on the diploma project, as I see it, will involve facing multiple software development problems, learning new techniques and, hopefully, obtaining valuable results - the project is worth the effort.

 

The rest of my software bucket looks much less clear. Being fascinated by the world of software development and its numerous applications, I cant say where exactly I want to direct my work. Moreover, I love the process of designing, writing and testing programs so much, that it is very easy for me to build things merely for the sake of building things. I feel that's a problem and I have to fight this passion sometimes, because it prevents me from focusing on the issues that are truly important, and not only is it harmful to me, but it can also become a source of pain for my employers and customers. On the other hand, this same passion for software allows me to make out a couple of things, which are attracting me as a young developer.

 

Among these things one can find code analysis tools. I have come across multiple mentions of such software recently and it so happened, that the idea to contribute to some open-source project to taste the idea of open-source myself has come to me almost at the same time. That's why I plan to search for a project in Clojure, which will allow me to practice understanding others' code and design, to make contributions to a system that may feel alien to me and to better understand functional programming and Clojure itself. I don't really limit the search to code analysis projects only - I acknowledge these may be absent - but I feel eager to work on a project of this particular kind, as this may help both in understanding the ideas behind code analysis and writing better programs.

 

All the items listed above are united by the concept of tool - they are designed to assist users with their work and provide them with some solutions to the problems, which they face in some area of their life. At the same time any tool is marked by the lack of autonomy and relies heavily on the signals from its consumer. No hammer drives a nail into wall itself - it always needs a person who will pound it on the nail. Definitely, that's not the only way software can function, and I really wish to participate in building an entity, that is capable of analyzing situation on it's own, making sensible decisions and acting correspondingly. I cant say that I am fully determined to work on, say, self-driving car, but I suppose that the area of autonomous (and autonomously developing) systems - either robotics or less physical applications of AI - may turn out quite suitable for me. Anyway, it won't be easy for me to reject an opportunity in the field.

 

The key aspect of such a 'self-driving' system is the ability to function in a self-directed manner without a flow of instructions from human operator, who serves as a link between the thing and the world around it. The idea that 'reality' of this world may be varied - in the sense that the world of the thing need not be the same as the world where I write these paragraphs - combined with my affection for video games leads to another piece in my software bucket: creating an AI for a video game is a challenging task. While I can hardly imagine earning my living in the game development industry, I could sooner or later participate in some open-source game project. Although the game development field may seem not serious, I still think that it can sometimes serve as a source of inspiration and insight for other, more serious areas. Furthermore, my minor attempts to create some simplistic computer games showed me that this kind of work is highly paying in emotional sense, because it is always amazing to play with your own creations.

 

So these projects and ideas constitute my software bucket today, although due to the reasons mentioned above the growth and expansion of the list are barely constrained. This said, I will have to practice saying 'no' a lot in the near future to avoid getting torn apart by my own curiosity and desire to build things. Still, I hope that willingness to say 'yes' will guide me towards doing what I can do best and therefore must do.

Courage

Recently (well, that is several months ago), I have changed the name of this blog from meaningless 'Not that important notes' to something that seems at worst equally meaningless, but probably less irrelevant. To make sense of it I should explain what I feel courage is.

 

First of all, there is the common meaning of the word 'courage' which deals with being not prone to fear. Because I am very diffident person it is sometimes quite difficult for me to share notes with other people - especially when it comes to the things written in English, since I know that my language skills are far from excellency. This way it turns out that while posting something to the blog - including this particular entry - I am in fact exercising my courage. The quality of being open to others and not afraid of sharing my thoughts and ideas is of great importance. I believe that the blog helps me foster this trait, so the new title apparently makes some sense.

 

 

 

On the other hand, writing itself allows me to better understand different aspects of the things I deal with in any area of my life as well as the ways I am dealing with them, thus directly or indirectly improving my decision making skills. This reminds me of the definition given by a samurai in Hagakure:

 

To rush in to the thick of battle and to be slain in it is easy enough, and the merest churl is equal to the task, but it is true courage to live when it is right to live, and to die only when it is right to die.

 

Although Hagakure is a book for warriors written by a warrior, the quote suggests that courage is not about being able to get into battle without any hesitation or fear. Instead it states that courage is more of an ability to know what to do next. Such a quality is as important for warriors as it is for software developers and for everybody else. Beside this, it is required and at the same time hard to master for me as a young man, because being a man, as opposed to a child, has a lot to do with making my own decisions and dealing with their consequences. Thus, as the blog is used to communicate ideas - and its title is not an exception of any kind - I am communicating an idea: courage is very important both for me and you.

 

One more quote encourages me to value courage so high:

 

Courage is a litmus paper for all other human qualities.

 

I am not sure where does this definition originate from and don't even remember who is its author. However, what I feel sure about is the fact that this blog may serve as a good test or indicator for all my personal traits.