First principles, Roshan. Read Marcus Aurelius. Of each particular thing ask this: what is it in itself? What is its nature? What does it do, this matrix you seek?

A faceless Marcus Aurelius with the voice of Anthony Hopkins and the intensity of Hannibal Lector kept repeating this to me most of this week.

Last Sunday night something interesting happened – I believed I had a solution to something – a problem in quantum computing - a way by which I can take any arbitrary quantum function that takes ‘m’ bits and produces a superposition of ‘n’ bits and construct an equivalent reversible function. The interesting thing that happened was that I realized was wrong – when I started looking at the matrices that my solution generated, I realized it would not work for any function that produced more than one output bit.

After some amount of thinking about it, the problem was narrowed down to looking for a matrix that was involuntary, the first row of which was decided by the function I was trying to model. Though the problem could be stated in a simple way, it was surprisingly hard to solve.

It kept running in my head everyday, all the time, for a major part of this week and it just didn’t make sense. I could look at the problem from many angles but the matrix kept eluding me. What is the nature of this matrix? it is involuntary. But that did not help me solve the problem.

The idea that finally helped me solve it came from a friend, Abhijit Mahabal, one of the smartest people I know in the department. Abhijit got me to look at the involution represented by a matrix as the reflection of a point about an ‘n-1’ dimensional hyper-plane in an ‘n’ dimensional space. I knew that given a function that produces ‘n’ bits, the matrix would be a ‘(2^n)x(2^n)’ square matrix and hence I was looking for the reflection of a point about a ‘(2^n)-1’ dimensional hyper-plane. That was the true nature of the matrix, the fact that it is useful to construct a reversible function was only incidental.

Of course, when Abhijit first explained this to me and did the math in 2-dimensional geometry, it was a little too much for me to get my head around. I refrained from this approach for two days since the math seemed too hard. But after looking at the problem in several different ways, I went back to asking myself about the nature of the matrix – then it was simple. I worked out the math for the multi-dimensional geometry and coded it up and ran it - and it worked at the first go. Ah, the pleasure of first principles.

Yesterday I had a chance to look at my Professor Amr Sabry’s, approach to the problem and I was stumped. I was used to being stumped by the things he did (after knowing him for about a semester) and I was often reeling from the effects of what a few well thought out lines in Haskell can express. However this time I was stumped because I thought this was something I understood and yet he had a very different solution - one that wasn’t even an involution!

For a major part of last night and today it bothered me that I spent all that time trying to solve the wrong problem. It annoyed me a lot – what was the point in putting in all that energy to solve a problem if it was the wrong one?

Thinking about this caused me to realize something important – in research, unlike in the commercial software world where I spent the last several years – it is equally important to understand why the problem exists and to question to nature of that which causes the problem to exist, as it is important to solve the problem itself.

This should have been obvious, but it was not. The world of commercial software or in general the world that practices software engineering in a way that is not research oriented has certain rules of problem solving that are different  from those of research. The more creative and intellectually challenging of those environments let you take your problems and solve them in any way you want. The lamer environments put too many bounds on how you solve your problems. Very rarely, almost never, in commercial software do we get the freedom to ask the following question in a non-trivial way ‘why is this the problem that we need to solve? why is this its nature?’. If you are not in a position to make decisions about the nature of the business, you are not in a position to question the nature of the problem. 

This will be tricky lesson to unlearn when I go back to an environment that doesn’t give me this freedom.