• Tom Stuart

    Refactoring Ruby with Monads (or, Monads: The Good Parts)

    MC: Last year we have a speaker from from Berlin and probably know him. He's a computer scientist and programmer, and he lives in London. He's working in consulting and is a trainer that helps companies improve their approach to software developing. And he's an organizer of a Ruby conference and part of the London Ruby User Group. Tom is responsible for a theoretic part of the conference and he will tell you more about Ruby Monads, so please meet Tom. (Applause) Tom: Hello everyone, thank you for chosing to give me your attention for a finite portion of your life. I'm going to talk about refactoring Ruby with Monads. The ideas in this talk may not be unfamiliar to some of you; but before we get stuck into the details let's warm our brains up with something simple. People always use analogies to explain Monads, and it doesn't help; so, there'll be no burritos or space suits or elephants in this talk. But I do want to talk about some related ideas. A new version of Java is available, you'll be glad to know (Laughing) do you quantity to talk about some related ideas and not analogies but other ideas that will help put your brain in to sort of a receptive state. So I'll start with a rhetorical question. What is a stack? Well, it's a kind of value with certain operations and those operations are called push, pop, top and empty. I'm talking an immutable stack here so push and pop don't state the stack, they just return the new within to use top to look at the top element. Need a class method called empty that creates an empty stack ‑ to qualify for a stack it operations have to it will lee certain rules, the rules just say that the operations do what you intuitively expect them to do, when you push a value on to aistic that becomes a top value, pushing ‑ an empty stack is empty, and a stack with something pushed on to it isn't empty. We can implement those stack operations however we like, here's a class called array stack that implement it is operations in particular way. It stores the stack content in an array of values, push adds the value on to the font of the array and pop removes the first item. This class linked list stack implements the same operations but in a different way, it stores the top element and the pointer to the rest of the stack; so, top and pop are just attributes, and push stores it's value in a new instance pointing at the old one. We can use those stack operations without knowing their implementation. If we make an empty array stack, and push two values on to it and then pop one of them and ask for the top value, we get the result we expect if we do the same with a link list stack it works in exactly the same way. We can define more operations in terms of the old ones, for example, here's a method called size that recursively calculates the size of a stack by calculating the amount of times it has to be popped till it's empty. As long as a stack has working implementations of empty and pop methods we can implement size on top of them. It doesn't matter what stack implementation we use. Size always works the same: an empty array stack has size zero. If we push two values on, it's sizes two, an empty listling stack; and if you push two values on, it sizes too. So what do we mean by stack, is really a specification. An implementation of a stack, provides certain operations that follows certain rules. There are two benefits to that. First, these operations provide a common interface across many possible implementations. And second, they allow us to build shared functionality like that size method that works with any implementation. So, here's another question: What's a collection, at least in Ruby? Well a collection is a kind of value with certain operations, actually just one operation called each. And that operation follows certain rules, actually just one rule, which is the each cause a blocks with value zero or more times in meet sequence, that's it ‑ immediate sequence, we can impresent that operation any way we like. Here's a hard coded collection. Each method literally calls block one, two all the way up to five hereby's a generated collection that calculates inside a loop and call asblock with each one. We can use that each operation without knowing its imhe Minnesotaation, from the outside hard coded collection andierer rated collection both behave like a collection of number from one to five. We can define more collection operation on top of the one we already have, here's ammeter called select that takes a lock block then calls each and cumulate the values that make the block return true. As long as a collection has a working implementation of each we can impresent select on top of it. It works the same for both implementations, of course. In Ruby we have a module newspaper perble, account, map, inject and tons of other stuff that all sets on top of each operation - this one each operation. What does this mean by collection? It's really just a specification; it's an implementation of a collection that provides an operation that follows a particular rule. And again, we get two benefits that each method gives us a channel interface against many possible implementations of collections, and allows us to share functionality like the select method that works with any implementation. So what name do we give these ideas? Are they design patterns? Are they interfaces or APIs or duck types? Those words are all appropriate and think all overlap to an extent. I think the concept of a stack or a collection kind of sits somewhere in the middle of all of them. But in my opinion, the most specific, and therefore the most accurate term, is abstract data type. That literally means a kind of value which has certain operations that follows certain rules. Stacks and collections are abstract concepts, but they're abstract in a good way, the abstractions give us power and we expect programmers to understand them. Nobody talks about stacks in hushed tones, they're simple and useful. Okay, that's enough priming of your brain, let's do some refactoring. First I'd like the have a look at some deed has to deal with Nils, let's imagine we have a project management App with different kinds of models, each project has a person who created it, each person has an address, each address has a country, each country has a capital city, and each city has weather information, which is just going to be a string. Let's say that in our user interface we want to display the weather next to each project for some reason. That involves traversing all of those associations. Here's ammeter that does that, maybe you'd write a rail viewer like this, there's good reasons not to write code like this and there's good reason to use it and people will do it no mat wharewe tell them. If you have city and an address in that country and a person with that address and a project created by that person we can parse that project into whether forand it works fine. But if we page a bad project, one that has an address with no country associated with it, weather for blows up. Nils were invented by a man in 1965 and he calls them his billion dollars mistake which have probably caused a billion dollars of pain and damage, and this is why. Well it may be a mistake but Ruby has Nils, so we're stuck with them. To make the weather for tolerate Nils, we have to explicitly check for them. We need to introduce local variables to hold every intermediate result. And then we have to check each intermediate result before we try to call a method on it. While we are at it, we may add well client the project itself is Nil. The code is drifting right and becoming a pyramid of doom, luckily it works the same if we platen it like that. So, this code works but it's pretty clumsy, and hard to remember to do something like this every time we might possibly have Nils to deal W fortunately Rails have a solution to this problem, Rails the active support gem, monkey patches support nil class with a method called try, which delegates to the public send method if the object is not nil and just returns nil otherwise. When everybody object in the system has got a try method instead of doing nil check ourselves we can let try do it for us. And now we're back to just traine training methods so we can take the local variables out again I'll make that a bit bigger, so at the moment this is as good as it gets, it's bet we are the version unless nil all over the place, can we do any better? Well, monkey patching has it's place, but monkey patching every single object in the system isn't great, is it. This is kind of code smell, let's not do it. Okay, so, try has gone away again now, you can relax. (Laughing) when we want to add a method to an object the good sort of object oriented programming solution is to use decoration, which is where you non‑inI havesively add functionality to one object by wrapping it up inside another object. Let's make a decorator class called "optional" with instances that have a single attribute called value. Instances of this list of nations class wrap up another value, I can make a new one containing value like "hello", then I can take "hello" out again, if the value I put out again is nil, I'll get nil out later. Instead of putting try method on object question put it on the optional class, if the value is nil, it returns nil, otherwise it sends the appropriate method to the underlying object. So now we can call try on the decorator and call it on the underlying object as long as it's not nil, if the value inside the optional is nil, try will just return nil. So instead of calling try on the actual project object and on the actual person object and so on, we can write the method like this. We decorate the project with an optional object and then we call try on that optional object. Then we connector rate the result of that, which may itself be nil. And then we call try on that. And then we connector rate the result of that meth call and call try on that and so on. And at the end we pull the value out and return it. So these unwieldily but at least we oar not among I can patching every object in the system anymore (We oar not monkey patching -) we actually just want to refactor away the nil check, but try also sends the value a message, but what if we wanted to use the value in only other way when it's not nil, try is over specialized, it has too much responsibility. So, instead of hard coding the else clause here, let's allow the caller to supply the block which controls what happens next. Now we can pass a block to this try method and do whatever we want with the underlying value, send a message or use it in a method call or print it out or whatever. And this ability to pass a block to try is actually a little used feature of active sufficient port try method as well. So now, instead of calling try, with a press age name and having to understand that it's going to send that message to the underlying object we call wit a block and in the black we send the message ourself and decorate the result in an optional and we can do anything else with the value like print out a log message or something. That worktion fine when there aren't any nils, but we broken it when nils are involved because we're returning nil when the block doesn't run. Easy to fix, instead of raw nil we'll decorate it with an optional. And then, it works in both cases. But there's a new smell, I don't think that try is great name anymore because we've change I it do something more general or different from the main use case of it's name sack in active support. Let's rename this method to and then. Because it really just says start with the deck core rated value and then do some ashtray thing with it as long as it's not - Arbitrary thing. Because we're training and then calls we can get rid of local variables again. This is verbose, but quite nice we databasing rate the possible nil project in an optional object then safely traverse all the association and pull the possibly nil value out again at the end. So, a few, okay, how is our refactoring going? Well, we might not be monkey patching anything and it's conceptually quite clean but there's a huge final male that nobody wants to write code like this, it might be better than active support try method but in practice it's worse. But we can add some sin tact sugar, here's - sin tack sewer Gary Reals, uses and then to delegate any message to underlying value when ever it's not nil. Now we can replace all of the and then optional new with just normal message send and let method missing take care of the details I'll just reformat that, there we go. So this is actually really good, you can see clearly that we wrapped up the possibly nil projects into an optional and saferly do our chain of methods an then pull the possibly anymore method objects out at the end. To recap, this is the whole thing, an object which stores an value that might be nil and a method called and then which incap late the nil logic, we add method missing and if you're doing this for real you'd want to write a respond to method as well. I'd like to very briefly point out that we only need to do the decorating and undecorating with the compatibility with the rest of the system, if the rest of the system passed in an optional object and expected us to return an optional object we wouldn't need did that bit. And then we wouldn't have to remember to check for nil at all, we would just write the method the way we did in the first place and it would just work, imagine that. All right that refactoring was very detailed, we're going to do two more, but I'll skip that much detail to save time. He's refactor code that has to handle multiple results. Imagine we have a content management App with different kinds of models there are several Blogs, each Blog has many categories, each post has many comments and for sake of simplicity let's say comments are just strings. And let's say that we want the fetch all of the words from all of the comments within certain Blogs for some reason. That involves traversing all of these associations again, this time each association has an ray inside it. Here's a method that does that, time not going to dwell on it, each level we map over collection and traverse the association from each inside it. We split if comment on white space to get constituent word. We have to use flat map instead of normal, flatten once, we want a flattened array of word instead of a nested one. Make a couple of Blogs with couple of categories that contain posts that have comments which contain some words and this accurately represent the usual level of discourse for Blog comments, the word in method can pull all of the words out, we're not duplicating them, we just want the raw words. But this method has got a bit of a pyramid of doom going on plus hard to distinguish between the bits of code doing actual work and bits of code boilerplate dealing with multiple values, we clean that up by introducing a class called many, which like optional will decorate something, but this time it decorates a collection of values, like optional, it's got an and then method that takes a block but then it calls the block for every value in the collection and flattens the results together. So we can replace all of 2 calls to flat map with instances of many and calls to and then, and that allows us to flatten the pyramid. And if we - if I reformat that code a little bit, we get that. So, again, this is pretty clear, but we can add some sintax sugar by defining methods missing - this lets us replace all of the and then many.new calls with just simple message sends. This is very nice we put in the Blog most the many aspects - and take the values out at the end. If the rest of the system could deal win stances of many, we could expect one and return one. To rerecam here's the class we just made. We are going to tackle writing as in concurrent resolution nows code, does anyone know who the most influential Rubiest is. We'll find out by using the get hubPI to find who's made the most commits on most popular Ruby project. When you make a get request you get back some Json, it looks more or less like this, gifts us the URI template for finding out information about any organization, here, look. So, now we know what URL to use to get information about the Ruby organization. When we make a request to that UL we get some JSON back that contains URL to get a list of all the Ruby organization, we fetch the list of repository which includes information about how many watches each one has, and we can see which repository has the most watches, and the URL for that repository rep evennation in the API. When we fetch that repository's information, we get another URL that tells us where to get a list of contributors. Then we load that list, which includes information about how many commits each contributor has made; pick the one with the most commit, a user called Nobu; and fetch information about Nobu from the contributor list. It turns out that Nobuyoshi Nakada has made the most commits on the most popular, thanks Nobuyoshi (Applause)! Okay, well, that was exhausting, so let's write some code to do it for us. Let's assume we have all right got this get Json method, it makes an ATP get request, passes on the response and calls a balk with the data or imagine a single threaded non‑blocking equipment in something like an event machine if you like, to do what we just did, we have to get the URL template from the - then fill in the template with the name of the Ruby organization, then get the organization day tax find the URL for the list of it's repository, then get the list of repository, find the URL with the most watches, get the information for what repository, then find the URL for the list of it's contributor, then get the list of contributors, then find the UL of the contributor with the most commits, then get the information on that user, and then print out their real name and user name. So, this code works, but once again it's starting to drift to the light and very difficult to understand and maintain deeply nested code like this. But we can't flatten it because of the nested call back, each block needs the information from it's containing block to do its job. Very briefly, the solution is to make an eventually class that decorates a block, the idea is that the block computes a value that might take a while fro deuce, the run method runs the block with a balk for it to call when that value becomes available. We don't have time for the details, but here's an and then method that we can use the add extra asin concurrent resolution nows processing that's produced by an eventually, more come collated than the ones earlier achieves exactly the same thing, detail is about how the wire up the call back so they all fit together properly. So that lets us rewrite this code by putting each asin concurrent resolution now call inside a block that we decorate with an event object. We connect all the eventuallies together with and then and then we run them at the end. This code isn't super readable either, but now we can Powell each logical part of it into it's own method. The code that gets all the URL template called get hundred API URL, this returns an eventually which decoratetion a Blog which calls it's call become where a result of calling and fetching the JSON, we can relays the line at the top, with GitHub API UrL, the next bit of code that fetches for Ruby organize? can go into a get Org method and we can call et, and so on for the rest of it I'll gist clean that up so now that we're just creating eventually objects at each step, we don't need to call and then on each one immediately, we can let each object be returned for it's enclosing Blog before we call and then on it which is a contrived way to say we can platen this to get this I'll reformat that to slightly differently this is much nicer than before, each is nicely encapsulated in it's own method and parts are connect in the a clean and obvious way, this pattern might be familiar to you, similar to promises or futures or defeared from Java script and event machine and things like that. So, to recap, there's the whole eventually class that we write. Okay, so, what was the point of all that? You've now seen three decorator classes that all have this and then method. In each case it takes a block and somehow calls it with information from the decorated objects and that decorated object could be nil or it's multiple values or it's a value that arrives asin concurrent resolution nosily. All three of these things, optional, many, and eventually are implementations of Monads, and I think the most useful way to think of a monad is an aquae abtract I did that type like a stack or collection. Like any abstract data type, Monads have got some standard operations in fact they've got two operations, the first one, and then is the operation weave already seen, the other operation is class method called from value, we haven't seen it yet, it's very simple, it just takes a value, calls the con instructornd too make sense of the - constructed to make sense of the monad, how to call the con instructor with a simple value that varies between - the main rule is the and then must call a block zero or more times, that's a much weaker guarantee - it doesn't say whether the block will be called at all or how many times it'll be called or when. Another rule is that the and then method must return an instance of the same monad and that's what makes and then chainble in a way that we've seen, the and then implementation in many and eventually already explicitly build an instance of the same monad but optional just trusts the block to return one, we could enforce that on and then on optional, like rays and results is an option or manage something. The third rule is that the and then and from value should not mess with the value and that just means when you construct a monad with from value you're guaranteed to get the same value out again when you call and then and for optional that rule only applies for nonnil values but you should only call from value with something that's not nil. The big benefit of Monads, is that they give us a come somebody interface which allows us to do one thing, connect together a sequence of operations. The and then method means do the next thing, but the way it does the next thing depends on which monad you use. For optional it only does the next thing if the value isn't nil. For many, it does the next thing many times once for each value. For eventually, it only does the next thing once the value becomes available. And as you might imagine, there are plenty of other Monads with different behaviors. Like the size method on stacks, or the select method on collections, we can define new functionality what sits on top of the common monad interface and use it when we're dealing with any monad. For example, we can write a method called within. Within is like sugar on top of and then, instead of expecting the block to return a monad it expects it to return a single value and automatically decorates it again with that from value method so you don't have to inside the Blog. Because that within method hides the implementation specific business of putting a value back inside the monad, you can use it to write code that works with any monad. So for example, here's a method that takes some monad containing Json representing a GitHub user to pause that Json and then assemble the string containing the real name and log‑in of the user all within whatever the original monad. So if we feed it an optional Json string into the method, we get an optional description as a result. And we now have to get a value of that if we need to. If we pass in an optional containing nil, the ?? doesn't blow up because the of nation monad won't let description from even try to pass Json it just immediately returns nil. If we make a many object containing multiple raw Json strings, description from will return many descriptions. And we can extract the actual strings ourselves if we want to. And finally, if we make an object that will eventually return some Json by asyncronously fetching from GitHub and calling a success call when that JSON arrives, description from will return an eventual description. Around we have to run that eventual description to get that call out - eventually. It's extremely useful to write one method like this and have it work with any monad, it's common - by and then and from value and the operations we can build on top of them. Like stacks and collections, Monads are abtract concepts, but they're abe abtract in a good way, the abstractions give us power, as programmers, we should understand them, we shouldn't talk about Monads in hush tones, they're simple that I understand're useful. By allying the power of Monads wisely, we can unhang nested call backs, and generally make our programs better. If you want to play with the monad implementation from this talk, I've put them on GitHub Tomstuart there shall Monads and I've also packaged them up as a gem for some reason. That's all I wanted to say for some reason, thank you very much (Applause). Sorry we don't have time for questions Florian has something to say. Florian: Okay, so, we will have the outside panel soon. I would recommend taking a break just at the outside, the break will be as long as it takes to set up the technology over there and a bit of time to breathe for the team. Thank you! (Applauds)

    About Tom Stuart

    Tom is a computer scientist and programmer. He has lectured on optimising compilers at the University of Cambridge, co-organises the Ruby Manor conference, and is a member of the London Ruby User Group. His latest book, Understanding Computation, was published by O’Reilly in 2013.

    This talk

    Monads are in danger of becoming a bit of a joke: for every person who raves about them, there’s another person asking what in the world they are, and a third person writing a confusing tutorial about them. With their technical-sounding name and forbidding reputation, monads can seem like a complex, abstract idea that’s only relevant to mathematicians and Haskell programmers. Forget all that! In this pragmatic talk we’ll roll up our sleeves and get stuck into refactoring some awkward Ruby code, using the good parts of monads to tackle the problems we encounter along the way. We’ll see how the straightforward design pattern underlying monads can help us to make our code simpler, clearer and more reusable by uncovering its hidden structure, and we’ll all leave with a shared understanding of what monads actually are and why people won’t shut up about them.