May 21, 2011

Two Techniques in Functional Programming in Mathematica

When I code for customers, I often end up using techniques of functional programming that  many people haven't seen before. This often ends up being problematic, and I've begun to identify the techniques I use that cause the most confusion and explain them when needed. Nevertheless, I use these in my own code because they help in creating well organized, readable code.

There are two techniques in particular that I find the most useful: decorators and closures.

Decorators
I use the term decorator very broadly. For this case, I define a decorator to be a function which takes in a function or a result and instead of using it as input for some other process causes some kind of side effect (it's not a very good definition, but will work here). In Python, decorators are used when defining a function, but I often use the term to apply to higher order functions I wrap around other functions which have some generic usage.  A basic example is a "deprecated" decorator. To test if a function is a decorator, remove it from your code. It shouldn't really affect the core computations of your program. Decorators aren't really aren't integral to a  program. They simply decorate. A decorator should also be modular -- it shouldn't  be built to work with a specific function, but should be able to be used generally across many different functions.

This example of a decorator is used to mark functions as having been deprecated.

deprecated[function_] := 
    Function[args, 
        Module[{}, Print["This function has been deprecated"]; 
        function[args]]
    ]
We can then simply use this when defining functions to give them a deprecation warning.
test = deprecated@
   Function[x, x + 1];
When test is ran now, it will warn that it is deprecated. The decorator could be used on pretty much any function definition. More advanced versions are possible. This version allows us to customize the deprecation message:
deprecated[replacement_?StringQ] :=  
    Function[function, Function[args,
        Module[{},
                            Print["This function has been replaced by "<> replacement];  
              function[args]]   
        ]]


Which can be used like:
test = deprecated["blarg"]@
   Function[x, x + 1];

test[1]
This function has been replaced by blarg
Closures
Closures should be recognizable to anyone who knows functional programming. They provide a nice way to have state in a language without really explicitly mentioning state. Here is a really simple example in Mathematica:
 makeCounter[] := Module[{count = 0}, Function[{}, count++]];
counter = makeCounter[]; 
counter[]
1
counter[]
2
....
Counter is a function which has a state because it references a variable in its parent function makeCounter.  This is cleaner than creating a global variable to hold the global count. The value of counter can only be properly accessed by using counter as we have intended it. I can't give a full treatment of closures and their uses here, but I hope this gives a good idea of what they are.

Combining them
I've used the both closures and decorators together with great synergy in a number of tasks. For example, let's say we want a good way to keeping track of how many times certain functions have been called. We can make a closure which returns a pair: a function to be used as a decorator on functions which are to increment the counter and a function to access the value of the counter. For example:
makeCounterSystem[] := Module[{count = 0}   
    {Function[result, count++; result],Function[{}, count]}];

{counts, totalCount} = makeCounterSystem[];
counts@ Sin[RandomReal[]] ;
totalCount[]
1
This decorator here is different from the previous ones in that it decorates functions not where they are defined, but where they are ran. This can be modified to be a decorator on the definition of the function easily if needed. Whenever we call a function which has counts@ preppended to it, the count will incremented and can then be access by calling totalCount[].

There are numerous uses for this kind of combination. A common use is to keep a log of the results of a function silently:
makeLoggingSystem[] := Module[{log = {}},
  {Function[function, Function[args,  
                Module[{output=function[args]},
              AppendTo[log,output]; output]]],
           Function[{}, log]
     }
     ];


{logger, getLog} = makeLoggingSystem[];
test = logger@
   Function[x, N@Sin[x^2]];
Now whenever we call test, the resulting value is secretly logged and the entire history of the output of the function and any other function decorated by logger is accessible by running getLog[].

Apr 29, 2011

I am Bored

I am Bored

Apr 22, 2011

Everyday Life


This is taken from the a joke thread on stackexchange. I think I pretty much see this situation everyday.
A physicist, an engineer, and a statistician were out game hunting. The engineer spied a bear in the distance, so they got a little closer. "Let me take the first shot!" said the engineer, who missed the bear by three metres to the left. "You're incompetent! Let me try" insisted the physicist, who then proceeded to miss by three metres to the right. "Ooh, we got him!!" said the statistician.

Apr 19, 2011

Mar 26, 2011

Finding a Regression for a Circle

I was recently asked to fit a circle to a bunch of data points much like the one right here. You can see that the points are roughly distributed around in a circle of some kind. The task then is to find the equation of the circle which best fits this data.

The Problem

Fitting an equation to a set of data is not an uncommon problem. There´s a large amount of work done in algorithms for finding the ¨best¨ fit for many different definitions of ¨best¨. The person asking me to solve this problem was under the assumption that the normal method, least squares regression, was the best way of going about solving this problem. Actually, performing least squares regression in this case doesn´t really make any sense since the theory applies to only functions. Circles are not functions and so the theory doesn´t apply.

Despite this fact, there was a desire to stick with something like least square regression. I guess it´s like a brand name that people have come to trust. There was talk of some rather complicated solutions, like breaking up the circle into two branches and performing least squares regression on each half in some way, then averaging the results. Anyway you slice it, you´re coming up with an arbitrary algorithm for finding a fit for the function. So really you might as well come up with a sensible arbitrary algorithm.

The solution for the problem is in fact incredibly easy. If you understand what least squares regression is, then you would never attempt to use it for this kind of problem, or anything really remotely close to it.

I resolved the issue with two very clear ideas that cut the Gordian Knot:

  •  The center of the circle is probably around the center of the points. 
  • The radius of the circle is probably the average distance from the center.
The Simple Solution
This is what the end result is when you plot that circle with the data.  It´s probably pretty close to minimizing the absolute residual of the data against the function. The solution is perfect for what it needed to be used for and took about one line of clear Mathematica to program. Relying on math that you don´t understand often causes you to complicate otherwise simple problems.

Feb 6, 2011

Quantum Zombies

The idea the physics induces different kinds of computing must at this point be taken seriously. If were now looking at quantum entanglement in eyes of birds, then the idea that quantum mechanics might have a significant role in the physics of consciousness is not really that crazy of an idea.

As I believe I've pointed out earlier, besides being completely ridiculous by themselves, Kripke's arguments for qualia seems completely useless in the case of quantum consciousness. I wrote a big paper on this subject called "Quantum Zombies". I won't link to it here, since the paper itself is terrible.

Philosophers love to use the analogy of zombies to discuss consciousness. What you won't see in their discussion is any mention that intelligence might have some kind of quantum nature. Like the theory of computation did before, philosophers have assumed a kind of Newton model of physics when discussing these zombies. If their arguments are revisited knowing that the mechanics of thought might involve quantum mechanics, then their arguments about zombies stop making any kind of sense.

For example, Kripke suggests that it would be allowable that there could exist a perfect copy of a person. This copy would be a zombie copy. In a quantum model of thought, this idea of a copy would be ridiculous.

Feb 3, 2011

Design Versus Implementation

The process of software design is often broken up into software design and software implementation. Like all forms of organization and classification, it's inherently incorrect and misleading. It's also necessary for an intelligent conversation on software.

In many of the software projects I've worked on, there has been a tension between these two parts. Very often the roles are separated out and worked on separately, when in a deep sense they are inseparable. The separation of work into design and implementation, architect and foreman, brings about a great danger. If the architect doesn't understand the material and structures houses are built with and the foreman does not understand the purpose of the architect's design, not only will there be conflict, but a chance for holistic creativity is lost. Innovation often requires mastery of both halves.

In popular thought, business people are often guilty of this dilemma. There are many stories of business people who have an idea for the next great website. They only they need to get a programmer to implement their idea. All the hard work to them seems done. The business people are caught unaware of how software was made in practice and become frustrated. Not aware that software creation must be part of the creative process, their projects fail.

Directing programmers is difficult, because software design isn't problem solving. It is problem avoidance. You can ask a programmer to solve a problem, but very often instead she will find a way to avoid it all together. The path of least resistance is often the best and not the one the designer had in mind.

Insert Taoist quote here about how programmers are like water or something like that...
-Laozi


Many have gone so far as to completely discredit the design part of the process. Many stories about internet startups are about how they stumbled into their niche. All they had were quality team of programmers and software designers. They went where their code took them. To some degree this is true. The book, Do More Faster, suggests that ideas are much more worthless than people believe. This is the over arching theme of its first section. Despite the ridiculous name of the book, I recommend it.

The greatest master of both implementation and design might be Frank Llyod Wright. His writings about architecture stress the importance of knowing the whole process of construction. Wright devoted a large amount of his time to understanding the construction methods and materials of his houses and this reflected in his teaching. His college of architecture was known for making students do menial work like carpentry and even farming on top of standard architectural education. Ayn Rand's Howard Roark was a construction worker for much of The Fountainhead. His understanding of the building materials and construction techniques allowed him to innovate where his competing architect's ideas stagnated.