Let's pay some folks with Elixir

July 16 …

Well, here we are, another day at the coffee shop, and I think we’ll do a bit more of the payroll experiment with Elixir.

It seems to be going pretty well for two morning’s work. We have a little payroll pipeline in embryo, that starts from time cards, retrieves matching employees, and computes a beginning paystub with regular pay in it. I’m not at all sure it’s good Elixir, but it does seem to be working and doesn’t look too awful.

Have you noticed, though, that it’s not going exactly as I imagined it would? I was expecting to have some closures containing references to enclosed values, and I was thinking there’d be some really nice list things going on. What we have isn’t quite as I was envisioning.

This is pretty normal. Partly, of course, I don’t even know this language, so it’s unlikely that I’d make very good guesses about what I’m going to do. However, this always happens, even in languages I’m quite familiar with. The fact is, even with a half-century of experience, the designs I imagine are different from the designs I build when the code is there with me. I think that’s as it should be.

So, in the spirit of calling my shots even if I miss, what I think we’ll do today is start to build a payroll pipeline, using the Elixir pipe operator, |>. I think I understand what’s going to happen. Consider something like this:

func1(x,y) |> func2(z,w)`

The idea is that the result of func1 is “piped” to func2. It’s as if you wrote:

temp = func1(x,y)
func2(temp,z,w)

The result of the function on the left becomes the first argument to the function on the right, and so on. You can build a pipeline as long as you want.

The idea for the payroll is that we’ll build a “paystub”, then pass it along through various calculation functions, each one adding values to the stub, and passing it to the next person.

Note that Elixir variables are immutable. You can’t really add an item to a list or map: you can only create a new one, like the old one, but with the new stuff in it. So what will really be happening is that we’ll be creating a new stub, better than the old stub, each time.

July 17 …

Yes, well. Tozier showed up and we got to talking and pairing, and in all the excitement, I forgot to write. My options are to report on what we did, which means I’ll forget a lot, or to do it over. Either way, you’ll miss out on a lot of hilarious mistakes. I’ll start over and fill in with learnings from yesterday as if I always knew them.

Ahem. So, the idea is that there would be some kind of a list or map, called a paystub.(Stub for short. as it were.) The stub would be a growing thing, with each step in the payroll adding one or more elements to the stub. Something like this:

Initialize -> [id, name]

Regular Pay -> [rate*hours, id, name]

OT1 Pay -> [rate*1.5*ot1_hours, rate*hours, id, name]

OT2 Pay -> [rate*2.0*ot2_hours, rate*1.5*ot1 _hours, rate*hours, id, name]

This just goes on and on, each payroll step adding information to the list and passing the list on to the next step.

Seems simple enough to be worth trying, and it might possibly work. We may find it necessary to do something more complex, but we can find out by doing it.

Move to plain lists of tuples

We talked briefly and experimented in iex a bit adding things to lists and maps and so on. It seemed to us that we should start with the simplest structure, the list, rather than the more complex map or struct. Our list elements would, however, be a keyword list, namely a list of tuples, the first of which is the name, and the second, the associated value. So our initial stub might look like this:

[ {:id, 221}, {:name, "samwise"} ]

This can also be written as

[ id: 221, name: "samwise" ]

because Elixir.

Based on this thinking, we updated our little snippet file thusly:

joe = %{name: "joe", rate: 10}
sue = %{name: "sue", rate: 12}
emps = %{1 => joe, 2 => sue}

pay_stub = fn(card) -> 
  ee = emps[card.id]
  [id: card.id, name: ee.name]
end

regular_pay = fn(pay_stub, card) ->
  ee = emps[card.id]
  [ regular_pay: ee.rate*card.regular_hours] ++ pay_stub
end

ot1_pay = fn(pay_stub, card) ->
  ee = emps[card.id]
  [ ot1_pay: ee.rate*card.ot1_hours] ++ pay_stub
end

The bit at the beginning is defining a small “employee database”. We only have two employees, joe and sue, so the database is just a map from their ids (1 and 2) to their employee record, including their name and pay rate. We use that database in all three of the functions shown, to fetch various information from the employee as we need it.

So we do a little test:

iex(79)> import_file "ee-snip.exs"
#Function<12.90072148/2 in :erl_eval.expr/5>

iex(80)> timecards
[%Timecard{id: 1, ot1_hours: 0, ot2_hours: 0, regular_hours: 40},
 %Timecard{id: 2, ot1_hours: 10, ot2_hours: 5, regular_hours: 40}]

iex(81)> [first, second] = timecards
[%Timecard{id: 1, ot1_hours: 0, ot2_hours: 0, regular_hours: 40},
 %Timecard{id: 2, ot1_hours: 10, ot2_hours: 5, regular_hours: 40}]

iex(82)> first
%Timecard{id: 1, ot1_hours: 0, ot2_hours: 0, regular_hours: 40}

iex(83)> pay_stub.(first)
[id: 1, name: "joe"]

iex(84)> pay_stub.(first) |> regular_pay.(first) |> ot1_pay.(first)
[ot1_pay: 0, regular_pay: 400, id: 1, name: "joe"]

We check our timecards list, match to first and second to get the two cards, run pay_stub on the first card to see that we get the initial stub, then run the whole pipeline, and, voila! we have a suitable paystub for joe.

Joe didn’t work any overtime, but sue did, so we’ll check her as well:

iex(85)> pay_stub.(second) |> regular_pay.(second) |> ot1_pay.(second)
[ot1_pay: 120, regular_pay: 480, id: 2, name: "sue"]

Looks good! (And it’s also looking like time to learn about unit testing in Elixir. But not just now. (Just wait: that’ll turn out to be a mistake.))

Concept proven!

OK, this little exercise shows us that we can define a pipeline functions and pass a list representing a stub along the pipeline, building up all the information we need to produce a paycheck.

Moving to a module

Let’s try it as an Elixir module. I built what follows up in one function at a time steps, but let’s look at the resulting code and then go through it bit by bit. Here’s the whole module file:

defmodule Employee do
  defstruct id: 0, name: "", rate: 0
end

defmodule Timecard do
  defstruct id: 0, regular_hours: 0, ot1_hours: 0, ot2_hours: 0
end

defmodule Pay do
  def timecards do
    [
      %Timecard{id: 1, regular_hours: 40, ot1_hours: 0, ot2_hours: 0},
      %Timecard{id: 2, regular_hours: 40, ot1_hours: 10, ot2_hours: 5}
    ]
  end

  @joe  %{name: "joe", rate: 10}
  @sue  %{name: "sue", rate: 12}
  @emps %{1 => @joe, 2 => @sue}

  def pay_timecard(card) do
    ee = @emps[card.id]

    pay_stub = [id: card.id, name: ee.name]

    regular_pay = fn(pay_stub) ->
      [ regular_pay: ee.rate*card.regular_hours ] ++ pay_stub
    end

    ot1_pay = fn(pay_stub) ->
      [ ot1_pay: ee.rate*card.ot1_hours] ++ pay_stub
    end

    ot2_pay = fn(pay_stub) ->
      [ ot2_pay: ee.rate*card.ot2_hours] ++ pay_stub
    end

# this guy can see card and ee and does not need them

    base_pay = fn(pay_stub) ->
      [ base_pay: pay_stub[:regular_pay] + pay_stub[:ot1_pay] + pay_stub[:ot2_pay]]  ++ pay_stub
    end

    pay_stub 
      |> regular_pay.() 
      |> ot1_pay.()
      |> ot2_pay.()
      |> base_pay.()
  end

end

Here’s what happens when we run it:

iex(86)> Pay.pay_timecard(second)
[base_pay: 660, ot2_pay: 60, ot1_pay: 120, regular_pay: 480, id: 2, name: "sue"]

Sure enough, we pay sue correctly. (I really am wishing I had tests … might just have to code up some semi-automatic ones, though I’m sure there’s some kind of Elixir unit testing thingie as well.) Anyway, let’s see what we’ve got here:

defmodule Employee do
  defstruct id: 0, name: "", rate: 0
end

defmodule Timecard do
  defstruct id: 0, regular_hours: 0, ot1_hours: 0, ot2_hours: 0
end

defmodule Pay do
  def timecards do
    [
      %Timecard{id: 1, regular_hours: 40, ot1_hours: 0, ot2_hours: 0},
      %Timecard{id: 2, regular_hours: 40, ot1_hours: 10, ot2_hours: 5}
    ]
  end

This bit we’ve seen before. It just defines the Employee and Timecard structs and then sets up a timecards convenience function to build some … um … timecards.

Then we have our little employee database. (I should probably call it that.) It’s just there so that we can drive the payroll from timecards: if you want to pay employee 1, put in a timecard for them, and the payroll fetches their rate info and such from the database. In future, we can imagine that this will contain addresses, bank account numbers, and other info we may need to pay folks.

  @joe  %{name: "joe", rate: 10}
  @sue  %{name: "sue", rate: 12}
  @emps %{1 => @joe, 2 => @sue}

That brings us to the main event, the Pay.pay_timecard function. Let’s look first at the end of it:

    pay_stub 
      |> regular_pay.() 
      |> ot1_pay.()
      |> ot2_pay.()
      |> base_pay.()

This is just the pipeline we’ve had in mind right along. It’s a little different from the experimental one above: a little better in some ways, and it raises some questions as well.

One odd thing is the need to dereference the functions with dot, and to include empty parentheses. If you remove either of these, Elixir will object in various ways. At this point, I’m just treating it as learning the syntax, but I suspect there’s some deeper reason why it’s this way.

Now let’s look at just one of those functions. After that we’ll review the whole blob.

    ot2_pay = fn(pay_stub) ->
      [ ot2_pay: ee.rate*card.ot2_hours] ++ pay_stub
    end

This function creates a list containing a tuple: { :ot2_pay, 60 } (or whatever value) and then puts it on the front of the input pay stub, returning the new list as its result. They’re all more or less like this. Let’s look at the whole thing now:

  def pay_timecard(card) do
    ee = @emps[card.id]

    pay_stub = [id: card.id, name: ee.name]

    regular_pay = fn(pay_stub) ->
      [ regular_pay: ee.rate*card.regular_hours ] ++ pay_stub
    end

    ot1_pay = fn(pay_stub) ->
      [ ot1_pay: ee.rate*card.ot1_hours] ++ pay_stub
    end

    ot2_pay = fn(pay_stub) ->
      [ ot2_pay: ee.rate*card.ot2_hours] ++ pay_stub
    end

# this guy can see card and ee and does not need them

    base_pay = fn(pay_stub) ->
      [ base_pay: pay_stub[:regular_pay] + pay_stub[:ot1_pay] + pay_stub[:ot2_pay]]  ++ pay_stub
    end

    pay_stub 
      |> regular_pay.() 
      |> ot1_pay.()
      |> ot2_pay.()
      |> base_pay.()
  end

The order of things in this function strikes me as odd, and we tried various other ways and they do not work. All the statements but the last are assignments, and like most assignments, they need to be done in a logical order. So we need to assign (and therefore declare) variable ee¹ before we use it. Possibly we could define it by assigning it nil and then assigning the actual value later but that wouldn’t impress me much.

We define pay_stub as a literal list rather than as a function returning that list. Same thing either way, we just did it that way. Either way, when we reach the pipeline, it’s the starting list to which we prepend our new name-value pairs. Tuples. Keyword list elements.

Then we define the various pay functions. These have to be assigned before the pipeline runs, since otherwise it can’t call them. Not much to it, really. Assign a few values, define and assign some functions, then run the pipeline.

What’s not to like?

Well, actually, Ron, there are a few things …

Most important, I think, is that the variables ee and pay_stub, and the parameter timecard are visible to all the functions. The regular_pay, ot1 and ot2 functions need the time card, to get the time, and the employee, to get the rate. As the comment points out, however, the next function, base_pay, does not need those values, but it does have access to them. In addition, all these functions have access to the employee “database”, @emps, and they could in principle do something with it as well.

To me, this is not a good situation. It suggests that these things are not properly factored. I’d expect that in a good program, functions would have access to what they need, and nothing else. We might let the database slide, as it is just a stub (not pay stub) for a real database. But the others are more troubling.

Secondly, although nested function definitions like this are legal Elixir, and even kind of looked on favorably according to what I’ve read so far, they make it hard to get the flow of the payment function, whose real point is just the pipeline.

I conclude that we need to look at some refactorings, and to learn a bit more about what good Elixir code is. We’ll do that in upcoming articles. And, of course, we built that timecard list so that we could run multiple people through our payroll, and we did a Map experiment somewhere sketching how that might work. And we have lots more payroll functionality to add.

Good times. For now, this is enough. We’ve got a tiny payroll pipeline running and the concept seems to work well enough to go forward. We’ll stop and commit some files for today.

Keep those cards and letters coming! Thanks!

===