• Christian Wirth

    JRuby’s Truffle Backend

    >> Hello, very glad to welcome Christian Wirth from Linz, he came over from Austria for us, he is working for Oracle Labs and he gets to work full time on the Truffle backend, really awesome. He just mentioned he's not a Rubyist, working on Java, lower level stuff. So, I will ask you to take it away, thank you very much for coming. (Applause) >> Christian: Okay, hi, everybody, can you hear me everybody, in the back, thanks for the introduction, it's true, I'm not a Ruby guy, I hope I can answer all your questions you have for me during this talk, but what I'm actually working on in that respect, I'm working on the truffle backend, so the Truffle language implementation framework which might be a new backend for JRuby. It'll be the last transition to the new backend, I'm not so sure about, that we suggest a new one. >> Charles: This will be in 9 K so people can try it. >> Christian: What I'm working on is truffle itself and Java imMr. Mennation, the guy that should be here instead of me today is Chris Seaton, he's working on the JRuby backend, he's at another conference today, so I have to jump in, I hope I can tell you what you need to know. Both of us are working for Oracle Labs, actually, which means we are a research facility, so I have to show you this slide that whatever we do is jussive of it will never be a product, don't sue Oracle if nothing happens. Of course all of us hope that this will be public some day, and it's already open source, so you can use it at the moment, even without Oracle's consent. So, what is Truffle? Truffle is a language implementation framework it helps you implement your programming language on top of the JVM, on top of modified version of JVM actually which we call Graal, which is a new new, Compiler. The one thing we hope to get with that is very high performance. So what we've seen in JavaScript, we can compete with other engines like the V8 engine of Google ‑‑ without too much work, you don't need the be a Compiler expert, really to write that stuff, because we are just providing an AST interpreter, so back to AST, sorry. And it's still gives us very high performance because in interpreter is compiled to a Compiler. So you don't have to did that stuff. That's done by Graal in Truffle. We support Ruby, so it's not a subset, we're not cheating, or tricks here, we try the cover all that Ruby has there for you. Right now we have better support for some of the features that are in JRuby or that are in Ruby. I will show you that later on in more detail. We are open source, so, of course, Oracle wants to make some money out of that, but all we gave to Ruby, you can use, it's open, it's free, just use it. And we try to bring the research we're doing in to Ruby first because Ruby is now the main implementation we have public, the JavaScript one is not public, but Ruby is the public one and it's already out there. A few disclaimers, of course, it's not ready for production yet. So we can run benchmarks and tests, we pass certain amount of the specs, but it's not finished yet of course. We don't expect to replace anything that exists now, so the existing IR at any time soon, it's still experimental, but we hope to get along with that. We're not as complete as the rest of JRuby, we pass much less of the spec, but, we have already implemented all major features, so it's really rather corner cases and details that we are missing, not that we're missing major features. Very important to mention we run a modified version of JVM, you are run Ruby it just won't be as fast, when you reason it on a normal JVM, you'll have just an AST interpreter, when you run it on this modified version, then you get all the fancy stuff where you can get really really quick, really really good high performance. Again, this modified JVM is also open source. So what's my talk going to be about. First I'll give you an introduction what Truffle is, what Truffle works, what Truffle does for you. I will talk about performance, so how fast are we, where can we get fast, why are we so fast. I'm talking about completeness, so what can we do already, what can we do in the future, where can we be better than current implementations. And especially on that point, I'll give you a few hints where we might do some tricks or might be able to implement stuff that others are not able to. So, how does the Truffle work?I said before we have an AST interpreter, and that's the main magic behind that, it's not very magic, actually. What we have is an AST interpreter, which creates nodes for you, and the things is those nodes are specialized depending on what happens at runtime, the case you see here is, it specializes on the types you actually see in your program. Assume this is an add node with two inputs might be local variables, might be constants, and at runtime they happen to be integer values, then we respecialize the AST to an AST that can only work with integer nodes and that will be really really fast because it doesn't have to do any checks, as you said before, Charles, functions will be very small, and that's what the JVM likes, and what we do in the next step, we compile that code in a method that's called partial evaluation, so all ‑‑ this whole AST is compiled to machine code to make the code, and to some code that is really, really fast, that is we hope as fast as if you would have implemented the same thing in native C. Of course, there's an assumption here, the assumption that those two todays all stay integers and never again change to something else. They could transit to a Double or a String or to some object value. What happens northwestern? Well, we have the JVM, we can, what is called deoptimization, we throw away the compiled code, change the AST again to what we need, you see we replaced integer node to double node and the add operation changed to a double node, and after some time, this new code is compiled again. So it's still the fastest possible code for what happens in your execution, it's not as fast as the pure integer case perhaps, but it's still the best you can have. So that's the system structure we have, on top of the operating system, you have some VM so JVM, you have the Truffle optimizer, which is our Compiler, which compiles to native code, and you have some APIs here that support you, in writing the AST interpreter. And that give a few additional optimization possibilities to you. The top thing here in the red section is your implementation, this is for instance the JRuby AST interpreter, on top of that, you execute your application. So what you have to provide is just guest language implementation and Truffle and Graal and optimizer do the rest for you, that's provided for you, you adopt have to write the full compile, you just have to write the AST interpreter. Okay, so, all that is already in JRuby, and you can use that already. It was an indent implementation originally, so last year we had an intern working on that, and later with the permission of the JRuby core team that got open sourced and got include in the the JRuby. It's still kind of separate, so you don't have to use it, you're not forced to use it. And we're working on still completeness issues. Because, the main thing, what's pointed out here is we try to share as much code as possible with the JRuby code already there, so we don't have to reimplement everything, we want to use the existing code, and that's a bit tricky from time to time because when you just use the code that's already there, then we lose all the advantages we might have with Truffle. So, you can already try it out, it's already out there. So, download the stuff, install it and run your application on top of that. We don't guarantee it will really work. If it doesn't work, if anything breaks, just inform us please, send us your code examples, and we will really try to fix whatever's out there, what's not working on our engine right now. Of of okay, now to performance. We tried tinned pooh benchmarks that are interesting for us, perhaps you have other benchmarks, please send them to us, we're happen if I look at them, we took Chinky Png Ruby. So, graphic files, the second one also works on photographic files and Photoshop images, and they read and modified and do arithmetic operations, so we used those gems and implemented some benchmarks on them, we didn't change any code, we just put a benchmarking harness around it and called functions in there. And it uses many arithmetic operations, looping, computations, objects, objectory EBTed code and some dynamic features of Ruby. How fast are we on that? You see we are 80 percent as fast as MRI with C extensions, so only of those benchmarks are implement in the C, we tried to get as fast or faster than MRI, we're faster than Ruby, this is the old byte interpreter, why are some of the engines so slow the is because those benchmarks are heavy on two things, you need very, very good inlining, and you need a way to avoid to allocate objects. And we are especially good on both of those, if in‑line that's one of the key ideas of Truffle, and siding object allocations, that's done by Graal our Compiler, eats a technique called escape nail sis that's already in the JVM, we have a better version in more cases do that, so we have to allocate fewer objects and that's why we get so much better performance. Okay, now on completeness, when we are that fast we tried to cheat somewhere, we hope we're not, we're really trying to implement everything, we tried to implement all features that are out there, we oar not one hundred percent complete, but all the tricky parts we tried to implement at least with some basic support. So, for instance we support monkey patching, Fixnum operation is called just like a function and as far as we know, please correct us if you find something, but as far as we know, we're not taking any short cuts somewhere, so we're really trying to stick to the Ruby spec and try to execute the same code that JRuby can execute in the same way with the same semantics. If you or we find something, then we'll fix it. We know that the performance won't degrade when we include more stuff because we had ‑‑ we have some history on the same thing for JavaScript. People also complained, yeah, you're 0 percent complete, we're sure the last 20 percent will drop your performance to nowhere ‑‑ 80 percent complete ‑‑ that has not proven to be true, our performance stayed the same, whatever we add in Truffle is really an addition and doesn't degrade your current code. So we know we can reach the goal of keeping up the good performance even if we include more of the specs. So that's a list that Charles made at some point in times, you see check marks everywhere so we have at least basic support for all the stuff. At some points we are better than JRuby, at some point we use the same stuff that JRuby does, for instance the continuations and variables where we have the same limitations having that said we have our own JVM so we have patches for continuations, we have patches for calls and code routines that we try to get into the JVM at some point in time or at least into our Graal JVM. The one check mark you see missing is Ruby on Rails, that's because Rails uses so much of the spec that we cannot just execute it because we need more completeness. But that's more a matter of engineering time spent into getting more complete, it's not that there are features we think that are hard to impresent, it just takes some time to get that done. Okay, completeness, we're using the Ruby spec completeness, we're not sure why Ruby is so low in that matrix, we don't trust numbers, so please don't hit us if JRuby actually passes more of this Ruby spec, if you see in the important take away here, we are around as good as Topaz, so we pass around 66 percent of that test suite. We tried to get to 100 percent or very close to that in the near future, that's the main thing that we're currently working on. Okay, where are we better than JRuby at the moment, at the moment, might not be the next version of JRuby, I know you're also working on some of that stuff, we fully support the ObjectSpace and the set trace function to be turned always on without any performance degradation. So this feature ‑‑ degradation, this feature is turned on all the time, only if you use it it will take some of your computation time. For the set trace function, for instance, you can run it at no cost if you don't use it, and at one point in time, in your execution, you add this trace, it will be compiled and it will be there at, of course, some costs, because the code needs to be executed, but compiled and optimize intoed the rest of your program, that's as good as you can get it for those kind of features. We also can work with binding and Eval a bit better than JRuby can at the moment, we don't do any tricks around that, and we can also walk the stack probably a bit better, we can walk our native stack and the stack of the interpreter and don't need to recreate that from the Java system. Okay, now, in the future, what wilt Truffle help you? Even if you don't care so much about speed, will there be any features that might be interesting to you? Again, a disclaimer, some of that work is in early stages, some of it might not be public yet, we oar trying to get it public, but we still have some problems, so it's not open yet. The one ‑‑ the most important thing, we think is C extensions. So, we can call C coded from Truffle and we do that in a way that we can really optimize C code Andrew by code together. We do that by executing C also on top of Truffle, so we have C interpreter for Truffle, and then we can optimize those calls together. So you can in‑line a C call into Ruby and for our Compiler it will look like just one function. This allows you to, for instance, you can push point from Ruby to C code and modify it back and forth, so, on next slide. I show you this really brings some performance where around 20 times faster than JRuby or three times faster than MRI, on some of those benchmarks. That's an average of the benchmarks, why can that be? Because we can really inline through this border C Andrew by, we can in‑line the C code into Ruby Code. If you view for instance a Ruby constant, usually C code wouldn't know it's a constant, our interpreter knows that and he can optimize on that and remove that code again as we've seen in the previous talk, even though there's language boundary in between. And we have some inline Cache, we can do it much faster than the current implementations. Very similarly is debugging, so, we have a debugging mode that is always enabled or can be always enabled because it doesn't cost any performance to you, it only cost performance once you set break point somewhere, if do you this again, this is compiled, if you use it on your fast path in a heavily executed method, everything is compiled and will run as fast as possible, couldn't run fastener a C Compiler, you can die dynamically attach break points ‑‑ so, we heard the question before on start up time, we is a project that's called substrate VM, and Charles already told us, we're working on compiling the stuff together, so we want to compile JVM, Truffle, Graal and your language implementation, so for instance JRuby to one binary, that will give you a higher ‑‑ a lower memory footprint and much, much faster start up. So we've seen, as we can start up as fast. You can dynamically load classes once you're in. We have some ideas how we can work around that, but that's rally something for the future. So how does it look like. We have some static analysis, what classes do you need, what code do you need, and this ahead of time compilation step, where one binary is created that you can use that's like an exit file, an executable file that can run your Ruby Code. Okay, I told you something about C and integrating with C, but we have many more languages out there, we have the JavaScript that my team is on, we have R, we have SmallTalk, Python and there could be more languages that you or somebody else could help us write with. And as we can do with C, we can also integrate those languages. Which means in the future, when you use Truffle, you can call between any language you like that is supported on top of Truffle. You can even pass objects, for instance, on those languages, so, in an ideal world you could write a Ruby application but let, for instance vector calculation being done in R, a statistical language, so that is pretty fast on vectors, or you can use JavaScript code or C code and use it all together in a very highly optimized fashion. Okay, what we're working on is to still increase the performance, of course, the completeness. We're working on profiling tools, we are working on performance tools nice thing with those tools they work with all of Truffle, it will be the same for all languages we support. Even if you have calls over the boundaries you won't recognize them or you will see them in the tooling. What we also have is some research into parallelism, I think we will hear one talk about con currency, Petr will give that, perhaps we should coordinate a bit what you're dog, we might be really interest in the including stuff like that into Truffle that will help par are rise stuff.  ‑‑ par are legal stuff. Wrap up. Key points to take away. We hope to create a new back end that will give you very high performance of Ruby. That will support some features in a very nice way, in a very transparent way in a very quick way. At no cost, so no overhead if you don't use them, and very little overhead if you use them. We can run C extensions, probably in the future faster than MRI can, and we have many many languages that we can use in your projects and connect them to your current implementations. And all that will come to JRuby and the JVM first. So, if anyone of you wants to get involved, we don't think we have so many low‑hanging fruits unfortunately. Chris Eaton who is working on that, his main task is to try to find out how we can better integrate stuff that is already there into Truffle. We claim that Truffle is very easy to use and learn, still it will be another thing that you is to learn on top of what you need to learn for JRuby. So, whatever you contribute to JRuby will probably help Truffle in the future because we can use that. If you're really serious in supporting Truffle effort, please get in contact with Chris Seaton, he can give you all the help you need, he's very supportive. He can give you an introduction and get you started with some kind of project that you can work on. Of course, another option is you can implement your own language, whatever you're interested in, using the Truffle framework, that allows you in the future to use them also in your Ruby programs, what we hope. So here we want to thank the Ruby community for being supportive to us, without your support we wouldn't have be able to include that in to JRuby. We're very open your feedback, so if there's anything we break or we do, what you dislike, please just tell us, we try to avoid being a disruption to your work. We just try to help you so if you don't care about us, just don't use us, but please give us the feedback, it would be really helpful to know if there's annoy answer, if we break something, if we can do stuff Bert than we currently do. Okay. So thank you, just another list of people at Oracle and many universities we collaborate with who made that possible. But apart of that, I thank you ‑‑ (Laughing) yeah, has to be here a second time. Thanks for your interest in the topic and do you have if I questions now? Three minutes (Applause) >> In the slide about the limitations on how you would optimize Ruby from the list that Charles had come up with, mention of a gill was there, is that a limitation on the existing Truffle interpreter the Graal underneath or where the GIL comes into Truffle and JRuby. >> Yeah, we have the limitation right now basically because it's a Truffle limitation at the moment soothful itself is not thread safe currently. That's what we're working on. We're very interested in getting more paralill Asian in that stuff we have some search in that direction, we support some software transaction on memory for JavaScript where it works nasally, JavaScript doesn't have parallelism in the language. We still need to make Truffle really thread safe, that's the key issue we have to do, and then, we can use much of that stuff also for Ruby. So it's not a general limitation, it's just that we have not done that yet, because it's not the main thing for us to do, for Java script. >> How about interacting with Java ‑‑ >> Is there a Java interpreter in Truffle? >> Yes. >> Unofficial answer yes there is, we have a student project on that. Official answer is, for instance in the JavaScript implementation, we tried to copy ‑‑ the official JavaScript engine in the JDK is doing and you can interact with Java code directly from JavaScript code. In JavaScript code you can create new Java objects, so we allow that in JavaScript I don't think we support that in Ruby, currently, I'm not the expert on that, but you can do that easily, have shown it's possible in JavaScript. Having a JS interpreter in Truffle would ‑‑ your problems, that's not an all too hard thing to do. If anybody's interested in writing that, please speak up and come forward. >> One more question. >> So, just from an outsider's point of view, we have in our situation where you're ‑‑ interpreter for Ruby on Truffle. And also on JRuby head we can enable flag to use the truffle backend, but sort of these two ‑‑ like these two features, give us sort of a Ruby interpreter when using the normal JVM and when using the Truffle back end. It seems a bit competing, so, what is in it for an enduser, like ‑‑ when I can use either of them on Rails, for example, which is actually where most of the Ruby applications are being used. >> So what we're really competing against is the current backend, so the current byte code interpreter or the new one. At one point this time, I guess there will be the main ‑‑ one main one that the user will see usually. If Truffle happens to be at one point in time the main backend you have then the user won't even recognize. For the user won't ‑‑ only thee an experience user or administrator will know I'm using this or that backend. We're among one back end among several you have in JRuby, and, yeah, for that, we think we need our own JVM, again, we try to get our ‑‑ so all the modifications we have into the main line of JVM, but, this is research, so we try to negotiate that with the JVM product group, they have not decided yet, if they will do that, when they will do that. I cannot guarantee this will ever happen. But then, if that happens you can use any Java virtual machine to run our code, so even also JRuby can do the same. >> From the floor. Okay, sorry, just a continuation. Will there be two interpreters at the end? Like the JRuby interpreter and your interpreter written on top of Truffle or we will just use JRuby on top of Truffle at the end? >> In the end, I cannot answer the question, it's up to the community you decide what back end you want to use and what will be your default back end. >> Charles: Will generally switch to one or the other, don't know if there's much value to use more than one at a time, >> There was an idea by Seaton you could Trufflize some code that you knew truffle did if really good at, for the very specialized code that you benchmark and realized that Truffle did a Bert job then you would break out of it just for that small code. >> Okay, I'm afraid we don't have time for any more questions. Thank you very much to Christian for your talk (Applause) we now finally have a fifteen minute break, please be back here at 11:30 ‑‑ coffee and cake is available.

    About Christian Wirth

    Christian has a PhD in computer science. He joined Oracle Labs in 2013. While his main focus is on writing a JavaScript engine, he and his team develop the Truffle framework that is also used in JRuby as optional compiler.

    This talk

    JRuby’s new Truffle backend has been in development at Oracle Labs for over a year and is now open source and part of the JRuby 9000 branch. Truffle is a new approach to implementing JVM languages that achieves performance well beyond what is currently possible in JRuby and other similar projects. We’ll begin with a high level introduction to what the Truffle backend is and how you can use it, which will be accessible to all. We’ll then explain in more detail exactly how it achieves such high performance compared to all other implementations of Ruby. We’ll finish by outlining where we think Truffle fits into the JRuby ecosystem and introduce some exciting new possibilities in areas such as integration with other languages and debugging.