Sharing the Story

(Annie jumping back in here for the latest update!)

With the prototypes taking shape, it was time to give them names and share what they could do. This blog and casual conversations had been the main ways people knew about the project, but next up was to talk about it more and get some feedback for the next stage of development.

First, Luke—known for finding creative names for servers and computers—wanted to christen the prototypes. He threw around a few ideas and landed on character names from the X-Men comics. Two became Shadowcat and Pyro, and James chose Phoenix for the prototype he built. We will introduce you to the fourth, later.

Shadowcat

Pyro

Phoenix

These super prototypes traveled to the Balihoo All Paws meeting to make their public debut. (Balihoo likes to use names too, and their former mascot and logo is a dog, Scout. They also have meeting rooms named for dogs. "All Paws" and "Pawsitive Projects" are their clever canine puns.)

Setting aside his dislike for public speaking, Luke gave a presentation as a way of reporting back to his employers and co-workers on a) how the team had put Balihoo's gracious grant money to use, and b) what the company could do to give input to the Styra Project. Pyro, Phoenix, and Shadowcat got to show their stuff and then stuck around for the day so that people could come by, try them out, and comment in accompanying note pads about the ease of use and any suggestions for what the devices should be able to do. All the comments are being taken into consideration for fine-tuning different keyboard functions.

At the meeting Luke also introduced Phill Thomas, James Hatmaker, Herb Ankrum, and Roy Kimball and thanked them for their contributions as part of the Styra Project.

Then it was back to the lab with kinks to flush out (also, more on that in the next post) and more ideas to fiddle with in planning for the next few months of the project. An Eastern Oregon journalist called Luke, wanting to do a story on the Styra Project for a small newspaper. He consented, always eager to talk about the project at least if he doesn't have to do an on-camera interview. A link to Brianna Walker's story in the Valley Herald follows. Here is a photo of the article.

The Valley Herald
part 1

part 2

Much more continues to generate in Luke's brain and in his lab, often keeping him awake at night and springing ideas on him in the shower. Be ready for an upcoming post.

Putting it All Together

Today's post is all about modularity. I have had a number of questions about how someone using the Styra Project designs will be able to customize their own keyboards. This post will hopefully demonstrate how different button layouts can be achieved with LEGO bricks and a little bit of imagination. All of the plates shown in this post were designed and printed by Phill Thomas.          

                       

Figure 1:  Top and bottom of button plate

The modularity of the Styra keyboards stems from the button plate design. Each button plate has a flat top and a reinforced button hole through which we can snap in an inexpensive joystick button. The bottom of the plate has a LEGO compatible interface around the edges so that it can connect with standard LEGO bricks.

Figure 2:  Assembled plate with button

Each button has a 12" cable terminated in a two pin header that can easily connect any of the custom Styra controller boards discussed in previous posts. The relatively long cable allows for a range of placement options without the need to perform any soldering. It is also possible to use extensions to move a button farther than 12" from the controller.

When working with LEGO bricks, the variety of sizes and shapes allows the creation of things limited only by one's imagination. I decided that this same philosophy would provide the greatest flexibility for the button layouts. After a bit of cajoling on my part, Phill consented to design a wide array of button plates. There is a downside to this approach; the spacing between the buttons cannot be any smaller than two lego units. This means that a keyboard layout for n buttons cannot be as compact as it could be if we custom designed a plate for each layout. The benefit is that layouts can be designed on the fly without modifying the 3D model and printing a new plate for each change.

 Figure 3:  Complete set of button plates

Figure 3 shows a complete "kit" of button plates. The contents of a kit were somewhat arbitrary; I worked through a few different layouts to figure out the quantity of each plate type. The following plates are included:

• 1x1 button plate - 8
• 1x2 button plate - 4
• 1x3 button plate - 4
• 1x4 button plate - 4
• 2x2 button plate - 2
• 2x3 button plate - 2
• 2x4 button plate - 2
• 3x3 button plate - 1
• 3x4 button plate - 1
• 4x4 button plate - 1

Phill had the 3D printer working day and night for a couple weeks printing two complete kits. These have proven extremely useful in our prototypes.

Talking about plates and layouts is one thing. Seeing them in action is another. The following images were taken while I was experimenting with a new button layout:

Figure 4:  Plates with buttons

The first step in designing a new layout is to use your imagination. Think about what might make a good button configuration. Play with the blank button plates to see what looks good and what doesn't. Once the design looks good, select the appropriate plates and snap the buttons in.  

Figure 5:  First layer of LEGO bricks

The first LEGO layer is critical. Sound LEGO engineering practices are recommended; mainly, try to make sure to provide as much connectivity between the plates as you can. Larger bricks (2x4) are really good for this for several reasons but the biggest is that the 3D printed plates don't always snap in tight with the blocks. This is par for the course, but larger blocks seem to stick better where smaller once may fall off.

The second layer is just as important. When applying the second layer of bricks, position them so that the seams between do not link up with the seams from the previous layer. In other words, span the seams.

Figure 6:  Third layer of LEGO bricks with blue accents and red cable paths

The third layer should be applied the same as the second, spanning the seams. With these shallow buttons, if the tabs on the back are bent over (gently), then it is possible to stop after three layers of bricks. It is tight and there is not a lot of room to route wires, but it is doable. For style, I like to add a splash of color to one of the layers. In this case I used blue bricks in the third layer. This layer is also where I routed the button cables and used red bricks as caps.

Figure 8:  Fourth layer of LEGO bricks

Finally, add a fourth layer in the same way as layers two and three. At this point the framework is strong enough to be entirely self-supporting. For my prototyping work I prefer to have a LEGO base to work with, so I built several base plates. They consist of 16x32 LEGO pads glued to inexpensive bamboo cutting boards. Figure 9 shows the completed button layout.

Figure 9:  Completed 16 button controller layout

Just connect a controller and assign macros to each button and this keyboard is ready for action! 

Styra Navigation Keyboard Prototype

Now presenting the Styra Navigation Keyboard prototype. This keyboard is designed to allow someone to browse websites without use of a mouse.

Figure 1:  Styra Navigation Keyboard

This keyboard required several iterations before we had a working prototype. It all began with Phill making a list of the keyboard shortcuts he uses most frequently. Then he added a couple extra functions that he typically uses a mouse for. Armed with that information I built the first draft of this keyboard. Using the keyboard helped to refine the list further. Figure 1 shows the prototype I have been working with. And yes, that is dry-erase marker on the buttons. We still need to figure out the button labeling, but for now it works. The following is a list of the functions this keyboard can perform:

• cursor up
• cursor down
• previous page (back button)
• next field (tab)
• select (enter)
• previous field (shift tab)
• open bookmark manager
• add bookmark
• new browser tab
• close browser tab
• next tab
• previous tab
• reload page
• zoom in
• zoom reset
• cancel (esc)

After using the keyboard, we have identified several more functions that can easily be added to the navigation keyboard:

• username - automatically type out a preconfigured username
• email - automatically type out a preconfigured email address
• password - automatically enter a preconfigured password.  Not secure, I know, but very convenient.

Beyond the basic functionality shown above, a lot of work has gone into the internal software that makes this keyboard function. First is the keyboard library itself. Each button press invokes a macro that consists of two components. The first is the action—or key strokes—that will be sent to the computer. The second is the action type, or how the action will be performed. For instance, sending the action CTRL + ALT + DEL to the computer one key-press at a time doesn't do anything except possibly delete a character. However, pressing them all simultaneously triggers a reboot. The Styra keyboard library supports up to 28 characters in the action field and the following five action types:

• Click - simultaneously send all action characters to the computer in a single event.
• Double-click - simultaneously send all action characters to the computer twice in two separate events.
• Press - simultaneously press and hold all action characters until the button is released.
• Latch - simultaneously press and hold all action characters.  Buttons are released when the button is pressed a second time.
• Sequence - send each action character one at a time to the computer

The second library developed for this prototype is the controller library. This simplifies working with the custom 16 input modules and includes a routine to scan the i2c bus and automatically detect connected controllers (as long as they have unique addresses). This is the first step toward being able to modify the hardware configuration of the keyboard without recompiling the Arduino firmware.  This library handles polling the buttons and managing the button numbering to make it easier to assign configuration macros to a button.

The third library is an interface that defines the macro actions and action types and how the keyboard library can interact with them. The first implementation of this interface uses an eeprom chip to store the macro definitions. I'm really proud of this library because it was my first time using the SPI bus and my first time working with an eeprom. I am using the SPI bus because I had a bunch of these chips laying around from a kit I purchased several years ago that I never built. :)

Figure 2:  Case for the 16 input controller

The Navigation Keyboard Prototype also debuts the first draft of a case Phill is designing for the 16 input controller boards. The camera flash really accents the printing lines, but the case is a perfect fit with a lego compatible bottom and Roy's Styra logo embossed on the top!

Styra Joystick Prototype

The first fully functional prototype to roll off the production line is the Styra Joystick. The concept with this prototype is to provide a mouse alternative for pointer control. After several trial runs, I decided on the following criteria for this prototype:

1. The prototype needed to provide a perfect "click" and "double-click," regardless of how long a button was pressed. 
2. The prototype needed to provide a way to grab, aka click-and-drag, an item.
3. The prototype needed a way to adjust pointer speed on-the-fly, not through an OS control panel setting.
4. This prototype should not require any modification on the host PC to function.  Just plug it in and go. Since the design is based on the Arduino Leonardo, the requirement is easy.

Figure 1:  State machine for processing clicks

I worked through several different approaches for implementing the desired click functionality before I finally decided on a state machine. Figure one shows the three different flows for processing click, double-click, and latch functionality.

Figure 2:  Flex sensor

Figure 3:  Flex sensor in action

Mouse pointer movements for the Arduino are sent to the computer as X,Y position changes relative to the current pointer position. To implement on-the-fly pointer speed control, we read a value from a sensor connected to an analog input pin on the Arduino. The value from the sensor controls the speed of the pointer.

There are quite a few types of sensors to choose from, but the one that shows the most promise is a flex sensor, shown in figures 2 and 3. This sensor changes its value based upon how much it is bent.  Bend it a little, it returns a small value. Bend it a lot, much greater value. (That could be backward, but you get the point.) A sensor like this could be sewn into something attached to a wrist, ankle, foot, finger, etc. Just bending a finger or moving a foot could control the speed of the pointer.

We have two of these sensors to play with. If you have suggestions on how they might be used, leave some feedback. We're open to ideas.

Figure 4:  Styra Joystick Prototype

Figure 4 shows the completed Styra Joystick prototype. This prototype uses an Arduino Leonardo connected to one of the custom 16 input modules, though it only requires 8 inputs. The knob at the upper left corner of the prototype controls the pointer speed. The four buttons perform the following: left-click, double left-click, left-click hold/release, right-click.

The Styra Joystick requires a little getting used to, but I have used it for several hours to browse the Internet and it really isn't too bad.

We're still getting caught up on the blog posts.  More coming soon!

PCB Modules

Playing catch up again...  The PCBs arrived from Germany on August 21st, professional and practically perfect. Somehow the 'T' in Styra didn't show up, but the design for the electronics printed out just as it should. The boards were assembled and tested in short order and prototyping work is well under way. Before we jump into the prototyping work, I wanted to show off the beautiful printed circuit boards from Fritzing!

Figure 1: 32 input shield for Mini-PAC

Figure 2: 16 input module (version 2)

Figure 3: 16 input module (version 3)

Figure 4:  $250 worth of printed circuit boards

I ordered these boards through the Fritzing project from Germany. I paid a little more for shipping than I would have if I had ordered the boards from a fab in the US, but they were able to process the native Fritzing file format AND part of the proceeds goes to fund further development of the open source Fritzing development environment. I only ordered a total of eight boards, but they included an extra board of each design without any additional charge. The boards turned out beautifully.

Figure 5:  Assembled 16 input module (version 2) 

The fully assembled through-hole version of the 16 input PCB module shown in figure 5 was easy to put together and only required about 25 minutes each to assemble.

Figure 6:  Assembled 16 input module (version 3)

The fully assembled surface-mount version shown in figure 6 required nearly 45 minutes to assemble due to the amount of time I spent checking and double checking the solder connections. Both versions work exceptionally well and they both can be used interchangeably (and simultaneously) in keyboard prototypes. The four pin headers in the lower left of the PCBs allow up to 8 input modules to be daisy-chained together, providing up to 128 button inputs for a single keyboard.

The first batch of Styra 16 input modules is ready for integration into prototypes!  :)

Opening Doors

Tonight marks a first for the Styra Project: We ordered a batch of custom printed circuit boards (PCBs). For perfectionists like James and I, it is hard to make the call that something is "good enough." And since this was our first attempt at designing custom PCBs, we wanted to get it right.

For the keyboard controller prototypes, we're investigating two different approaches. The first approach repurposes a device designed for building arcade cabinets. The Mini-PAC is a fully integrated keyboard controller designed for attaching arcade buttons, joysticks and other devices to a computer. This device is compact and inexpensive. It supports a large number of buttons and is potentially a fantastic fit for our project. The downside is that it is closed-source and only produced by a single manufacturer in Europe. The PCB board shown in Figure 1 will plug onto the Mini-PAC as a "shield" and allow the connection of up to 32 buttons using standard two-pin headers.

Figure 1: 32 input shield for Mini-PAC

The second approach we're investigating for the keyboard controller is one that leverages open-source hardware and software, specifically the Arduino Leonardo. There are several advantages to this approach, specifically that it allows for a great deal of flexibility in sourcing components as well as customization. Unfortunately the Leonardo does not have sufficient I/O inputs, and while there are techniques that can be used to add more buttons (charlie plexing / scanning/ bit shifting), I have decided that for the Styra Project it makes sense to create input modules that attach to an I2C bus.

Due to the limited scalability of using software debouncing, I decided to add hardware debouncing to the modules as well. Figure 2 shows the design for a 16 input module with hardware debouncing using entirely through-hole components. This design requires more components on the board, but is easy to solder. The EDE2008 chips used in this design have a debounce interval of 25ms, which is more than enough for the buttons we are using.   

Figure 2: 16 input module (version 2)

Figure 3 shows a second 16 input module but with a couple of surface mount components. This design utilizes the MAX6818 chips, which have a 50ms debounce interval and may feel sluggish to people who push buttons lightning fast. The layout is cleaner but the surface mount components can be a pain to solder.

Figure 3: 16 input module (version 3)

The PCBs will be on their way soon, and we will only have a couple more orders to make to have all our hardware for the project.

Finally, I would like to give a shout out to Roy Kimball for his design of the new Styra Project logo. His design was able to capture what this project is all about: opening doors.  :) We got it just in time to get it onto the PCB boards.

Bug Removal

“If debugging is the process of removing bugs, then programming must be the process of putting them in.” (Edsger Dijkstra)

(This post comes straight from Luke, as only he can describe the details of his work over the last few weeks. Stay tuned for his report of his meticulous work from the last few days.) 

The Debouncing Continues

 The prototype keyboard depends on a single button press triggering a single corresponding action. As any engineer knows, a project like this requires a lot of testing. However, testing in this case means pressing buttons many, many, MANY times. :) So what I needed was a test harness to push buttons for me! I read several posts online about people having similar bounce issues when working with relays. I tested a few on the oscilloscope and found that they bounce similarly to the buttons I am using for this project. The end result was that I was able to use a second Arduino to drive the relays in specific patterns, then monitor the inputs of the prototype keyboard controller connected to a computer to ensure the pattern is received without error. 

Shown above is the test harness circuit. The Arduino Uno is at the top of the image, connected to one of the MCP23017 chips that I have been playing with for The Styra Project. The MCP23017 is connected to a bank of eight transistors that drive the 12volt relays at the bottom of the image.

This picture shows the test harness connected to the button inputs of my prototype keyboard controller. The laptop is capturing the button presses and tracking any errors. When the test harness is running it sounds like a swarm of crickets in the garage. With the lights off it is like a scene from the Star Trek TNG Schisms episode.

Using the test harness I was able to diagnose several intermittent problems with my initial design. The first was that after five to ten minutes of pressing buttons, the prototype controller stops sending button presses. This was traced to an issue with the i2c bus and a lack of pull-up resistors on the signal lines. A second issue was found in my software debouncer that limited how well it would scale as I increased the number of buttons.

The prototype keyboard project is designed around the idea of assembling panels of buttons into modules that can be connected together.  Up until this point, my testing has been with a single panel, but to prove my design, I need to connect multiple modules together.  The above picture shows a total of five i2c devices connected to a single bus. This configuration supports up to 64 buttons, hardware or software debouncing and hardware interrupts. The oscilloscope is tracing the clock signal to make sure I have good values for the pull-up resistors.  I need to do further testing to determine if I can fine a single resistor value that will work for multiple devices or if I will need to build a chart of values to use depending upon the number of modules attached.

One final note, the gray button panel shown above is Phill's latest design from the 3D-printing side of the project. We had some initial trouble successfully printing larger parts, but Phill has spent many hours perfecting the process and we are now able to print designs the full width of the print bed. Phill made this design specifically to help with my testing as I needed the ability to connect (and manage) 16 buttons to a single module.  Phill is currently working on panel layouts for specific navigation and browsing functionality.  Picture coming soon!  :)

3D Printing and Button Debouncing

The 3D printer had more lessons to teach once it was put together and working. After several successes and failures in printing, it decided Luke needed to fix some wires to keep it running smoothly. Back in action, it went to stay with Phill for awhile so he could focus on printing while Luke switched to working on the electronics and programming elements. 

Phill designed and printed a way to mount the spool of filament. 

Luke working with an oscilloscope and Leonardo to debounce buttons.

(Luke's learning about surface mount soldering. This is a tiny chip to put in place!)

If you don't know what debouncing is (and perhaps your first thought was Rabbit wishing to debounce Tigger—hey, we've got a toddler in the house) then here is a handy definition from Whatis.com:

"Bouncing is the tendency of any two metal contacts in an electronic device to generate multiple signals as the contacts close or open; debouncing is any kind of hardware device or software that ensures that only a single signal will be acted upon for a single opening or closing of a contact" (Margaret Rouse TechTarget). 

If you find your TV remote doesn't respond correctly when punching buttons, likely it wasn't debounced. Your computer keys, however, should register only the single signals as you type so you shouldn't want to throw the keyboard the way you might throw your remote. 

Some Assembly Required

With a clean workspace, a sturdy table, and most of the materials ready and waiting, it was time to really get started on the project. Assembling the 3D printer was a painstaking process. Luke and Phillip Thomas spent the majority of one day building it. Though lots more work than just buying a pre-built model, it was a great opportunity to learn the ins and outs of the machine. 

Phill:  "All the parts just fit together. This will be easy!"

Turns out there was more than just matching screws to holes. Calibrating the machine was no easy step. 

10 hours later...

Once test chips could be printed it became "Goldilocks and The Three Bears": This one's too rough, that one's too thick, that one's not square; This one's too hot, this one's too cold. Oh no! this one scratched the bed. Try, try again. Many days of failed prints followed.

Finally, Luke was able to print without scratching the bed.

 But he still hasn't figured out how to remove the prints without stripping the tape off as well.

These are the first three prototype button mounts, designed by Phillip Thomas. The buttons snap easily into the mounts, which interlock tightly with LEGO bricks.

This is a sample 2x6 layout that combines buttons with LEGO support.

With the 3D printer in action and progress being made, the excitement around here is contagious! Special shout-out to Daddy's little helper—when he wasn't throwing LEGO bricks around the garage!