A few days ago I announced a new open source library that provides querying methods for Go collections. By far, it’s my most popular project on GitHub, with around 300 stars. Before talking about it more, go ahead and take a look to get a rough idea.

First of all, Go’s type system sucks, everybody knows that and it is by design. At least, Go language creators are fine with it, apparently it gets their work done in Google infrastructure level. In fact actually this makes the language very easy to learn (you can read the whole thing in 1-2 days and probably be an expert in a month). But this makes the type system more primitive than most languages out there.

Since this is my first project in Go, I took my time to learn more about type system and reflection in Go. Several aspects that make things harder to design and use a library like this in Go are:

  • There are no generics, every variable has only one and fixed type at compile time. You have to explicitly cast the variable if you need.
  • There are no lambdas, this was making things easier in C#, Python etc. But you have explicitly write inline functions to query and manipulate objects.

Given these and many historical discussions in golang-nuts Google Group saying LINQ in Go is probably not a good idea, I spent my holiday season doing this and practicing Go in detail.

Why is it not a real LINQ implementation?

There is no yield/enumeration. It is just a for-loop that goes over your Go array (slice) and filters/projects values from it.

There is no lazy evaluation or deferred execution. Because I have not implemented it. This is clearly an improvement area for the project.

Not all methods are implemented. At this moment methods like SelectMany, ToDictionary, GroupBy, Zip are not implemented, you can contribute if you’d like.

It’s not really “language integrated” since the compiler does have support for statements like from s in students select s.Age. But Microsoft also calls it LINQ even though when it is method syntax.

How can it get better?

In addition to ideas I just talked about, there have been ideas like extending this project to make it an implementation of Reactive Extensions:

So the official account of Reactive Extensions team at Microsoft is hopeful about the idea of GoRx but I am afraid nobody would ever need LINQ or Rx in Go. I’ll talk more on this in a bit.

Syntactically, there would be a big improvement by making the errors returned from passed functions optional and enabling users to omit type assertion for the values used in the passed functions (see examples).

These two can be made with the help of reflection, which will make the library super slow, at the same time, makes the codebase cleaner and legible. This will be possible with implementing type parametric functions described in Andrew Gallant’s blog post and his project.

Before moving the whole library to the reflection, I need to write benchmark scenarios for many operations, varying from Where to OrderBy, which involves calling the specified function many times. Only then I can proceed with reflection implementation and measure the results.

Testing

I have used GoConvey for continously testing the project as I was coding it. I have used Test-Driven Development (TDD) approach in this project, for the first time and it felt really better.

Also having unit tests and a few scenario tests is really great while making small changes. I am sure my old open source projects were full of tons of bugs now. But I praise applying testing practices now.

This project has 100% code coverage, tested using “go cover” tool and builds pushed to GitHub are continously tested on Travis-CI.

Will it get better?

I don’t know for now. It seems like it will not get any further. The idea of making something like “GoRx” (reflective extensions) library sounds like a dream and a worthless effort at the moment.

Go is not a language for that and the creators don’t want it to be a language where people can do anything they want. The language has points it’s good at and it really stands out for certain type of problems.

So I really can’t tell if I will be willing to move it to make a breakthrough out of it (perhaps you will). It will be just incremental updates and fixes from now on.

What people say

The story was on Hacker News and /r/programming. Especially there was some good constructive crisiticm on Reddit. It’s worth reading to get more context about what I just told here.

However project got a significant traction on GitHub and stayed on most popular 2nd repository of the day on GitHub for a day and 3rd the day after that. Thanks everyone for being supportive. I’ll just leave a few more tweets here.