• Austin Seraphin

    RubyMotion and Accessibility

    Hello again, I hope you enjoyed the activities and workshops that we had. Let's continue and I would like to introduce our next speaker, Austin came to us from Philadelphia yeah, he started programming at the age of 7. He has a accessibility freelance company and uses Ruby potion, he enjoys cooking and meditation, most recently he lopped an amazing technique and developed an maiding technique and started to publish. He will talk about accessibility matters and how the iPhone revolutionized the life of the blind. He'll tell more about RubyMotion. So please, meet Austin Seraphin. Austin Seraphin: Thanks very much. We've been dealing with some technical issues, so, all right, so my name is Austin Seraphin, and as you just heard, I've been blind since birth and I started programming when I was 7 on an apple 2 E, that was the first computer with a screen reader, that's how a blind person uses a computer by using a program that reads the screen to them, usually through synthesized speech and also sometimes through Braille. In the '90s, after the apple I used MS DOS, I used windows, unfortunately and then Linux, then in 2010, I got an iPhone and that was the turning point for me, after that I got two Macs and an iPad. The iPhone changed my universe, the blind are used to getting nothing from companies so we felt exemptcal about apple when they introduced the touch screen, we just made fun of it, figured they were trying to get good PR, as it turns out all of apple's products have accessibility built in, there's screen read ergs screen magny if Ication. There's a switch control for people with motor difficulties, there's guided access for people with learning difficulties, and special hearing aids now ... oh, I just lost my place ... okay, sorry. So, that's how we felt then, we didn't know then about Apple's accessibility. A friend of mine bought an iPhone and went nuts about it, especially the touch screen, I couldn't believe it. Really we were just making fun of this, three days ago, come on. So I went to the AT&T store with my mom to check it out, I read the manual the night before, so I knew what to expect, probably the only person that did read the iPhone manual (Laughing) so I began using it immediately, it was great, I picked it up, I checked the weather, the stock, and charts, that sold me that showed me apple's detail to accessibility, for the first time I could read a stock chart. As we were taking it up to the front, my mom asked if I could get a text message on it, and, we are getting all kinds of great notifications here – my mom asked if I could get a text message on it, and the guy at the AT&T store, I don't know, I don't think so. And that's can be a problem, sometimes when you go to these places, often they'll know the accessibility features exist, but they don't know anything else; they might know how to turn it on, but not about it. If someone tells you it's impossible, it's probably not true. So ... – my mom, said there's a good way to find out if I get a text, my mom sent me one, it said hello awe tin, even if guy at the AT&T store, before I get too much further, I wanted to demonstrate some basic Voice‑Over usage so you have some idea what I'm talking about. It supports speech and Braille through the use of Braille display, you can turn it on by going to settings then general then accessibility, Siri can do it as well. Don't turn it on till you know what you you're doing, I've gotten e‑mail, I saw your talk, I tried to turn on Voice‑Over, I couldn't turn it off, couldn't use my phone, make sure you know what you're doing. The apassing thing about Voice‑Over, it has two dimensional and one dimensional paradigm, the screen has rows and columns, in other words, withand hate, speech ton other hand is one dimensional, you can go forward and back, one dimensional line, the task of a screen reader is to take a two dimensional out put and reverend it into a one dimensional out put. So let me just give you a quick demonstration, we can do all these wonderful notifications, I'll be turning on my phone here in a second. But we don't need that yet, so the two dimensional paradigm is, if you just tap around the screen, here's how fast I have it. (Speaking really really fast) I'll turn it down for you all. 50 percent. If I tap around the screen. – important, I well now speak at ... – got a nice retweet there, if I tap my finger over the screen, it speaks what's under my finger, I'm going to turn on my iPhone so you can all see this, if I swipe right with one finger it goes, and so tapping around the screen is the I did two mentional paradigm, it also give you spatial relations (Two dimensional) you can tell where things are on the screen. If a sighted person said tap a button in the upper right hand corner, a blind person would say they have no idea what it means. The one dimensional paradigm is swiping, if I swipe right with one finger, takes me to the next element, if I type left, takes me to the previous element. Let me turn on my air play. And – and we're not seeing it now. Great ... and then, while I'm trying to figure this out, if you then hear an item to activate you double tap and you're on the screen and it will then activate that item. You can also scroll with three fingers. And I'm on you camp script, and I'm on the right network, I'm not seeing the air play. Sorry for these technical difficulties, I don't know guys, I'm in the seeing it. – all right, whatever we're going forward. So the standard cocoa elements are accessible by default, and that's really important because that means that accessibility in a lot of cases is built right N so if you're a developer and using the standard controls and label they feel then you've done 90 percent of the work. And Apple does have guidelines, by the way. So, getting back to my story. After I got my iPhone, I took it home and I was so excited because now finally I could run apps, everyone was so oh, the iPhone could run apps, no one told me which apps, they just told me I could run apps, so I thought cool, all right. So I took it home, and I started thinking, and this is the thing to understand about what make it is iPhone so amazing, is that it makes a whole bunch of aust devices for the blind obsolete have always been devices made just for the blind, pieces of technology, limited park et, so they often cost a lot of money, aren't very well constructed or supported. I was thinking about these different devices, and, for instance, I thought about a color identifier, so I typed color identifier, in the App store, and an App came up, color I D, cost two dollars, I downloaded it, installed it, ran it, my heart is just like this is the awesomist thing, the iPhone is awesome. And it just kept saying, black, black, black, my heart sunk, I was just like, oh harsh reality reasserts itself, the iPhone is not great after all. Before I turned it off in frustration, I thought for a second, wait a minute, it's 2:00 in the morning, I can't sleep, it's pitch black, I have no lights on, because I don't need to, yeah, I forgot, sighted people need light to see color. (Laughing) yeah, yeah, imagine that. So, I'll just quickly turn on the color I D, not going to be able to see it unfortunately, you'll be able to hear it at least: Her's color I D, and (Button, peal purple ish blue, I don't know what it's picking up. Gray ish green, see, you can hear the different colors, pretty cool. A color identifier itself cost 200‑dollars, the App cost two dollars. And an interesting thing, said remember accessibility is built in, he had no idea, never thought of the blind being able to use it. So, that's what's so cool. There's a device called a money reader. In the States we don't differentiate our bills ones, fives, ten, twenties etc., so blind people have to come up with their own system, often folding the bills differently, and people say, oh, that's so clever, that's so great you do that, no, it's not, it's stupid and annoying, and we shouldn't have to do it. So, here, of course in Europe the Euros are all different sizes which is really nice, there's a device for a hundred dollars that will do it and identify the bills or you can get an App for ten dollars or less that will identify currency. There's objective recognition, there's a device for 1,200‑dollars you can buy or get an App for tap tap see for free and pay a reasonable subscription and tell you what a can of soup or box cereal is, there's GPS, you can spend 800‑dollars on a GPS device, I like one called "blind scare" and it costs 30‑dollars, so, blind square is cool because it links into four square's database, as well as – App, you get a crowd sourced, all the things that are around you, which sighted people take for granted you can look around, see what is passing or things, this will tell you as well as standard GPS function 's a scientific calculator, those cost 250 bucks, a five dollar App, a talking alarm clock is 40‑dollars, of course this is built right nigh in. Note takers, there's been devices for years that just do basic note taking functions, notes address books, things like that, and those costs around a thousand dollars, of course all of that stuff is built right into the iPhone and with a much Bert web browser, the web browsers on these note takers are pathetic. Terrible! So, if we we add all of things up, that's 3,600‑dollars for assistive devices and let's see 247, let's say we throw in the beer buddy App, since we near Berlin, can splurge a little to help identify the German beers, let's throw in a blue tooth keyboard, that's 3,600‑dollars verse 300‑dollars, would you spend twelve times more on astives devices than you would on an iPhone and assistive apps. That's the difference that the iPhone makes for the blind. And accessibility really helps everyone, it's not just about helping the blind or a particular group, accessibility helps everyone, great example is Siri, it makes me loft now, and sighted people, oh, it's great my phone will read stuff to me,sira, yeah, that's great, that's how I do everything. Another cool example is audio books, the blind have been reading books on tape in various forms for decades and decades, finally I manage one clever marketer was like, hey, sighted people would like these too, and now audio books are extremely popular. Reading a menu at a restaurant, little things, another thing is having constant communication, unless you're traveling internationally, of course. So, getting back to my story, after the color ID revelation, my first thought was this is awesome, the iPhone really is the greatst thing for the blind, my second thought is, of course, I need to start programming this thing. So, people often ask how I program computers, and I say I program just like you do, I use the visual editor just like you do. Now, I thought, I'm pretty good, you know, I've be programming for a long time, I'm good at picking up languages right. Wrong X code works horribly with Voice‑Over which is apples IDE, if you think X closing is confusing, try closing your eyes and using it it with Voice‑Over. That's confusing, the interface builder as well is totally inaccessible, which is a lot of developers construct their views, I had to learn how the build views programmatically. So I hit this wall, I was feeling really frustrated, I head this kind of brick wall, this is ... that happened until I found out about RubyMotion. Now, RubyMotion is a tool chain for developing IoS and MacS in the in the fall it will have Android support as well. Which is awesome, you can share a lot of your code. I think that's going to be a big thing that they're adding Android as well. It's a commercial product, it costs 200‑dollars, I would spend it on beer if I had to use X code and objective C every day. It balances out. It's fully App store compliant, compiled internetted, these are apple's build tool. Uses Ruby with all of it's advantages, you probably know about I since you're here, the clean syntax, which I think sounds better with speech it's so clean and concise, you know, blocks, iter raptors open classes. Modules, mixes, all the things we love about Ruby. Come to RubyMotion. All of apple's SDKs, and frameworks are available. So no restrictions there, third party libraries all that stuff. You have to know enough of objective C to interpret it it to look up stack over flow answers, RubyMotion has translated docks, I recommend using the dash document browser. Built on top of standard cocoa classes, NS object, is object, instead of string it uses NS string, all that's handled for you, it also handles memory management for you, which think is way cool, it has rake, gem, bunked Earl, all that stuff is available. Unit testing, we love unit testing in Ruby, it brings that to IoS and Mac OS as well. It has an REPL, the read evalprint loop for Ruby Code. You can actually interact with the Redding application, which is really neat. If things get really heavy you can use the RubyMotion debugger, LLDB, it can connect to interspeck processes, you use it on the device, really cool. Great open source gems, great community. For instance a gem called bubble wrap gives a lot more Imatic code for cool things, button up when. (Idiomatic) it has JSON, RSS nice cool liting thing. Motion kit provides styling and auto layout, motion lets you build forms, promotion lets you easily lay out apps. RubyMotion query is a powerful front end, lets you do great things, setting up views, styling them, adding actions and all kinds of cool stuff, core data query, makes core data very easily, I thought I would never master core data and never use it, but CDQ changed that, and motion bundler lets you use most spannard Ruby gems as well. Which is really nice. So that brings me to my own gem, which is motion accessibility. First I wrote some wrappers around apple's accessibility APIs, I started writing simple games, but I realized problems, the IOS simulator is not very accessible. It doesn't work very well with Voice‑Over, so that's why I wrote the motion accessibility console. So, let me just demonstrate that. I hope this will work. We seeing it? Good, good, okay . , no or yes Yes Good. So we have the IO S simulator here, you see the simple App I wrote to demonstrate work flow. I don't expect it to get on App advice or anything any time soon. What we see here is in this tab here there is a label, there's a text field and an update button, you fill in the text field and the update button, so the first command is browser B, and we'll see that it says Bruceing UI window, it says UIV with three sub views, navigation bar and a tab bar, so we're going to Bruce be number one which is the UI view there, is our label and text field and button, so, let's, the next V view gives you the next view you're Bruceing, then to interact with something, you just touch, so I could say, touch, 2, because that's our text field EuruCamp rocks! Okay, you can see that's update there had in the simulator, you can say touch "update" and you'll see that it's updated. So, that's just the basic work flow, just a simple demonstration there. So, the next thing I want to show you is the accessibility inspeck sore. So, we can use accessibility or AL Y is a common abbreviation, it's A is 11 eleven letter, when you to type accessibility millions of times it gets annoying. So there you see there's our UI label. And, I don't have time to go through all these, but I'll go through a few of them, the accessibility label is a very important one, that's what Voice‑Over read, when you hear Voice‑Over reading those different things that's the accessibility label, that's very, very important to remember. Another thing is the accessibility traits, in this case it's static text, that tells Voice‑Over how to interact with that element, there are a bunch of them, static text button image, link, lots of different one, they're all documented. And the other important one to know is accessibility element that tells Voice‑Over whether or not to regard it in the first place. So, that is what you need to know about that. So, now I want to show you, see, you now might be thinking all that sufficient is great, but I'm not going to remember all that, even if I read all the dock mention,ration, so that's why I'm proud to announce motion accessibility 3.0 has automated accessibility testing. So, the most annoying thing if you ask a blind person, the most annoying thing is an unlabeled button in an App. Because you just hear, button, button, button, won't tell you what the button does and it's horrible and annoying, and only takes 30 seconds to fix, now you have to give it an accessibility label let's created an unlabeled button, it's a controlled environment. Button new, button.frame, doesn't matter. We're just going to say button.frame equals CG ... one two Lee four, whatever. Now let's inspect the button. And, you eel notice it says accessibility false it says you must set the accessibility label. You can use the title for state method to do this on a button. So, let's try that, button set title. "hello Potsdam! " for state UI control state normal. Yeah, good, and now let's inspect the button. And it'sible true. So there you go, it tells you exactly what you have to do to make that buttonnible. Very simple. Now, you might wonder – oh, I want to show you about the specs, so the really cool thing, I cede RubyMotion has unit testing built in, so you can use the accessible predicate in your specs, and the advantage to that is you won't break accessibility, the only thing worse than an inaccessible App is an App where they break the accessibility, you're using it maybe for your job even and update the App and it's totally broken and useless. And a good clue that that's happened is the words totally redesigned user interface in the change log when ever you see that blinded people cringe and go oh, no, what have they broken, because sadly that's the truth, they don't think about it and update it with some cool new visual interface so use your test that won't happen. RubyMotion has a little accessibility testing if you're using the functional test and testing your controllers and tapping your buttons, those have to have the accessibility label, you to label your buttons if you're testing them, this takes it one step further. So, let's – I want to show you a little demo here. Okay, so, this is just a button that I just did, and you'll see now this is a spec if you know R spec it says describe the – it's accessible, new, and then thank you for all the tweets by the way, down here it says button should bible. Very simple, the first time I wrote that, I wrote view should bible and code it sent a chill down my spine, it actually works! So, let's try this, as we know the first thing in test driven development is to run your spec and watch it fail, right. And you'll notice it's commented out there, we have to do it to save time, so ... okay. So, we're going to run the spec. Okay. And of course, it failed. Good. And it told us again, it gave us that same message, right in the log, so developer just have to know exactly what to do. So now let's move the comment here save it of run it again. Okay. And Boom. And there you go, zero failure, zero errors. So, that's the idea of accessibility testing in your specs. And by the way, you can do controller accessible. Apps should be accessible, I wouldn't recommend you can get confused going down the higher oar I can, if you say a certain button is inaccessible, you might wonder which one it is, I have ten buttons on the screen F you is a custom view, how it supposed to know? And that's use the accessibility test method for that, custom.accessibility test equals UI button or whatever. That's motion accessibility, so I hope you enjoy that little demo, it's available for free on GitHub and and on Ruby gems, I do give a discount if you use my gem, something to keep many mind. In conclusion, the iPhone allows the blind to do wonderful things. Accessibility helps everyone, it's the right thing to do. RubyMotion increases productivity especially for the sighted and blind. In most cases you can make your App accessibility with little effort and high reward, those extra ten minutous spend labeling those buttons or what have you make all the differences, and usually it's very simple stuff, it can get more complicated, often that's all it. That's true for web sites, if you're using standard elements, labeling them, doing it, you're probably good go, the combination of RubyMotion and motion accessibility makes a great come by fashion, especially for the blind, I – especially for the blind, if companies like App wouldn't have made their products accessibility, I wouldn't be standing here giving this talk. Thanks very much! (Applause) okay, we oar going to try the dome know now that I'm not – we're going to try the demo now that I'm not under pressure. Should we do questions and answers if anyone has any quick questions. I can't see your hand raised so just yell out a question (Laughing) while I'm getting my WiFi set up. Is there any questions? I'm not seeing eurucamp script now. Oh, wait, here it is. Eurucamp script, okay. I'm going to turn this speech back up, actually ... you'll want to hear this, it's going to drive me crazy this slow. Okay. No, no questions. (Laughing) all right. You got it, yep, you're rite . For some reasons our devices keep switching networks, I don't know. Okay. Something just happened ... (Collected, Austin, mirroring, switch, on, it says it's mirroring, okay, collect, do you see it? No ... screen one ... do I have to turn this off? Now we're seeing the screen again, everything is green again, that means nothing to me. Try to put on the mirror again. Thank you, Rachel, mirroring, oh, yeah, it's switched off. Oh, air play ... yeah! Okay, yeah, good! Whooo! (Applause) all right, all right, so let's try to demonstrate a few of these apps. That's cool. All right, what do we want to do first. All right, so we did color ID, but now you can see it. That's pretty cool. Very light green ish blue ... gray ish purple. That's colorI did. Bluish gray, very cool. Okay, that's fun. Now let's show you, how about let's do money reader. And let me just get out some currency here. All right, all right – here's an American 20‑dollar bill, would youn't know it, here it is, (Applause) pretty cool, pretty cool. And I have to fold it back up so I know it's a 20, and check this out, this is a fun little surprise for me as well. – oh air play is back off. Money reader running. ... 50 Euros. And there's 50 Euros. It's not on, again, really. No, the iPhone lost the connection. Oh, air play, it just works (laughing) what what's the problem, what do we have do here? What's the trouble. You're right, air play is off again, it's not air play, it's WiFi, eurucamp script, right. I'm on eurucamp script for both. Now I'm not seeing airplane. Oh, there it is, it's back, okay, all right, all right. Sorry folks for this, don't blame me, I didn't do it, – okay. Are we good ... all right, all right, if anyone sends me messages or tweets, it's going to be displayed on a very large projectorment you heard the fifty Euros, cool, it could identify euros, that's cool . so now let's try – tap, tap C, so this is pretty cool, I hope we can do this, this has internet, right – I guess. So, let's just take a picture of this computer and see what it says. (Picture two in progress) have to wait a second. – nothing I can do about that. So, tap, tap C is a combination of crowdsourcing – ... oh, now air server just popped up again and just lost it, oh my good goddess, I don't know about this air service thing. There you go, Mac with pearl – it's a Mac book air, that's pretty damn good, that's pretty damn good (Applause) and, so, we try to get it for one last – I wanted to show you blind square the GPS App. I don't know. Oh, air play, coming back. This is supposed to be a trial version of this air server. All right, all right. Thank you, I have a new follow, all right, getting all kinds of notification, so this will be kind of fun, because, I'm using, of course, the blind square English speech oh, I have to reopen it again. Going to be using the beta version this is great, I'm getting so many notification. So this is going to be funny, it's going to be about as good of a job pronouncing the German as I would. So we're going to see here. It's not going to have any idea what to make of this, so hope that – all right, all right. It's going to – I don't foe if it's going to be able to give us our location, I don't know if it's going to be able to give us our location, it couldn't earlier, but that's not it's fault. It's trying, it's thinking.Ful (Radius of 200 ... oh, no, this is – okay, well, whatever, I just want to show you – (Searching ... of) see, I'm just swiping back through the controls, tools, button, alert, select action, look around, so, check this out, if I move it around and people will stop tweeting for a second. No place – within 245 feet. – other button ... to the southeast – no places found within radius of – let me turn up the search radius. Yeah, yeah, yeah, search places within a radius of – 0.3 $4 3‑miles, to the north, no places found within radius, I guess the GPS – 0.2 – there you go, N – ... I don't know. No places found, well, that's blind square, it's not get ago good GPS in here, sorry, but blind square is really cool and it will tell you, all right, all right, all right ... be good. And, the last thing I want to show you, suggested I show you an App that you all perhaps may have used yourself, except that we've lost our stupid air play connection. All right, we're going to it to one more time. One more time one more time ... one more time ... air play awe tin's Mac book air, now let's it to before it crashes. All right, and here's Twitterriffic, I thought I was talking about how great it is to have constant communication, so here's your cam's Twitter, Twitter riffic, here's the back button for eurucamp and recent tweets, that's a heading, here's the tweets, the table, I can move my finger down and hear the tweets. (Here's an important one. Anyone carrying an apple TV around at eurucamp, we could use one. You can scroll, with three fingers, so it's really that simple, so that's Twitterriffic. I don't think we should torture this air play thing anymore, so there you go. (Applause) Thanks. All right. Cool. Anything else, any questions anyone have have any questions about accessibility or anything No more time for questions, no more time. All right, well, stay tuned, we'll be doing another speech after this ... or we doing the panel – what's happening after this. We're doing another, and the talk after mine, if it's the one I think it is also relates to accessibility as well, hope you enjoy it, thanks everyone!

    About Austin Seraphin

    Austin became blind at birth. He started programming at age seven on an Apple II/e and since then has used all major operating systems. He currently uses a Mac, Linux, and loves his iPhone. He does freelance accessibility consulting, and uses RubyMotion to program in iOS. He lives in Philadelphia and also enjoys cooking vegetarian food and meditation.

    This talk

    How can a blind person even use an iPhone? Austin Seraphin will explain this, and give personal examples of why accessibility matters and how products like the iPhone have revolutionized life for the blind. He will introduce RubyMotion, a Ruby toolchain for writing iOS apps, and he will demonstrate motion-accessibility, a Ruby gem to make accessibility features easier to implement by sighted programmers and which opens up the world of iOS development to the blind. This talk has received acclaim at #inspect, the RubyMotion developer's conference.