2/2/2016, MPF interface…duh, duh, dun, done!

As I write this, the OPP platform interface to MPF is done.  It went rather well despite simply reverse engineering another platform interface.  A couple new commands were added to the board interface to make talking to MPF a lot simpler.  These include setting all incandescent bulbs as either on or off in a single command, updating an individual solenoid configuration, and updating the color on a single LED.  Easier to add those commands rather than trying to shoehorn the MPF to use the first version of the commands.

MPF works very differently than the OPP pinball framework.  OPP framework gathers all the updates to incandescent boards and configuration changes and sends them down to the card in one single command.  MPF wants to update solenoids one at a time.  This means the OPP framework is a little more efficient on the serial link, but moving to Gen2 the link went from 19.2 kbps to 115 kbps, so that doesn’t seem to be an issue.

I still have a little more work to do.  I need to add support for version 1 cards.  The only people who probably care about this are Cactus Jack, maybe Joe, and myself.  It is mostly for me and updating the SS3 pinball machine.  I’m too lazy to rewire all of the solenoids and inputs, so why not just use the old first generation OPP cards as they are, and simply add one or two cards to support incandescent bulbs.

That would make the upgrade to SS3 that much simpler (and forces me to finish the layout on the low side/high side driver card for bulbs).  Hey…Iteadstudio is currently having a 10% off sale.  That might put a fire under my butt to get it done by the end of the month.

Need to have an easy way to update OPP gen2 cards.  Right now, a user would need to download Cypress’ PSoC Creator, and download new code using that tool.  (That’s a 604MB download).  The bootloader protocol is documented by Cypress.  I found a version of cyflash which is supposed to do what I need, but of course it crashes immediately when trying to run it.  I have also found some C++ windows code that is supposed to implement the protocol.  I’m going to start by basing the updater of cyflash because it is written in Python.  That should be the simplest for people to use across multiple different platforms.

1/18/2016, Love that new code smell

So I started writing the MPF interface for the OPP hardware.  Brian offered to write the interface, but I know a lot more about the OPP hardware so I think that it is just as easy for me to write it and get it correct and optimize it properly.  I didn’t want to get into a situation where Brian spent a bunch of his time, I looked at the code and wasn’t happy with it, and rewrote it myself.  That would waste both of our time.

Installing MPF did not go as smoothly as possible.  MPF assumes that it is the first time that Python, Pygame, and PySerial are installed on a machine.  I had older versions of all three of these programs, so when I tried to start the demo machine in MPF it failed.  “DLL load failed: %1 is not a valid Win32 application.”  I’m was thinking to myself, oh great, I’ve just set myself on a couple day hunt and peck to try and figure out what went wrong.  Luckily it simply required uninstalling PyGame and reinstalling it again over the the new version of Python.  (I blame this issue on upgrading from Python 2.7.6 to 2.7.10).  I then had trouble running TK.  (import TKinter was giving the same issue as above.)  I uninstalled Python 2.7.10 and reinstalled it again, and now I was able to import TKinter.  MPF does not use TKinter, but my pinball framework uses it for running the debugging window.  I was happy to see that I could now run both frameworks successfully.  Last configuration change I had to make was to allow Python connections through my computer’s firewall.  MPF BCP connect sockets using a local loopback connection.  These were actively rejected on my laptop (Win 7 Enterprise Edition), so I had to Control Panel->System & Security->Windows Firewall->Allow Program Through Firewall, and browse to c:\Python27\python.exe.

Woo, hoo.  Up and running with the demo machine executing.  That was actually relatively painless.  (A couple more error checks could be used in the install script, but it does clearly state that the script assumes that no previous version of Python, PySerial and PyGame are installed.)

The thing in MPF that is used to support actual hardware is called a platform.  There isn’t currently any information on writing your own platform file, so you basically look at previous examples of platform files, and reverse engineer what they return to the rest of the framework.  There are already platform files for the FAST and the PROC and P3-ROC platforms.  They support some of the same functions that I support in the OPP hardware.  I based the OPP platform file off the FAST platform file.  I don’t think it would have changed things significantly to base the file on the P3, but I happen to choose the FAST file.  It is going well, but the new platform file isn’t done being written yet.

One thing that I do miss using C, C++, C#, etc is that it is very easy to define an interface.  Python does not really have any concept of what an interface is.  It also has the added benefit that if somebody calls your object, expecting a certain function and it doesn’t exist, it just crashes.  (Yeah, no type checking because that would restrict how things could be used)  Do you get a compiler error, oh no.  That wouldn’t be the Pythonic way to have to know what type of object you are talking to.  Instead it simply gives a back trace dump.  Hopefully all the functions are called frequently enough that I can easily find all the interface errors.

So, first thing I have to say about the MPF is that the documentation is really great, especially for a beginner.  Within a couple of hours, I was easily able to get a simple machine working that would do an attract light show, flip through my attract slides, and start a three ball game.  Most of that is built right into the framework which is really nice.  If you are following the tutorials, everything works really well.  In the tutorials, some of the background processing is explained which makes extending the example and modifying it to your liking easy.

When you get to the end of the tutorials, things become a little rougher around the edges.  I was not willing to dig into the core MPF code, but had some difficulties doing some very simple things.  There are currently no tutorials on playing sounds.  From programming my pinball framework, I knew this functionality should be very simple to get at using Pygame.  My main problem was that I didn’t know how to do it in MPF.  I immediately went to the forums but had difficulty finding information.  It was great that MPF has forums, but they aren’t very useful at this moment because they aren’t searchable.  After a day or two of mucking around, I finally figured out how to get it to work, so I could play a sound at the start of a mode.

I am planning on moving all of the SS3 rules over to MPF and posting the final files.  Hopefully this will give others a working example of running a full featured pinball machine using MPF.  This little exercise should force me to examine the differences between the two pinball platforms.

One major thing that needs to happen is that the MPF forums need to be made searchable.  I remember seeing a post that was related to something that I was putting off implementing, and even after going back and searching for a few hours, I could never find it.

Here are two quick pictures of comparisons between running the OPP framework and MPF.  I will describe more of the differences in a later post.

OPP Framework:

PinballFramework

MPF Framework:

MPFFramwork

One quick thing to note is the jaggedness of the score font.  Both of the frameworks are using the exact same true type font file.  I might have to look into how MPF is generating its text.  This is one area that the OPP framework was optimized for printing text because it is assumed that it is going to be used for scoring.  The MPF framework puts much more emphasis on DMDs, and I think the code to display text is less mature.

When the OPP framework prints text, it is always right justified so it looks old style pinball displays.  I haven’t found an easy way to get this to happen in the MPF framework.  In MPF, the text anchor is always on the top left corner, so I will have to pre-process all scores to add extra spaces to get the text to not move as the score goes up.  MPF also post-processes numbers and adds the commas automatically.  That would be great, but in some cases, I don’t want to add commas to the numbers.

1/4/2016, What the Heck (not Ben) am I doing?

I was looking back at the archives to try and figure out what the goals were from the previous year and if I met them.  Fortunately I can’t find any goals, so that must mean that I met all of them.  At the end of 2014 I came up with a dopey idea of bringing a machine to the Pintastic show.  That happened, but it was a knock down flat out fight.  I was personally not happy about many aspects of it (mostly the software that was running the rules) and just didn’t have enough time to really work out the bugs.  (Yes, I was the guy huddled over the pinball machine the night before the show updating code.)  Some of the updates worked well, while others introduced bugs.  I didn’t have enough time to really play test the code, and some of the long timers that I was using to kick off songs in attract mode didn’t work or didn’t clear properly.  The majorly fatal flaw was that I had a mode where I didn’t clear the kickout hole, so if it went into that mode, the ball would get stuck and never clear.  My bad for not having enough time to do the proper things.

After Pintastic, I was pretty much done with pinball for quite a while.  I needed a vacation.  At some point Joe told me that I should keep going.  After discussing some design ideas with Joe, I decided to start working on the second generation of cards.  Those cards are about twice as dense as the original cards (meaning up to 16 solenoids, 32 inputs, support for Neopixels, etc).  They were also a good deal cheaper to make.  If there wouldn’t have been a significant improvement to the cards, I doubt if I would have pursued making them.  I sent out for a few boards to be made and decided hey, I’ll get to the code when I get to the code.  At some point around October, I decided that I was going to give myself a hard deadline of finishing the firmware on the cards by the end of the year.  That put a whole bunch of pinball hobby time into the end of the year, and finally, today, I finished testing out a card and sending it to somebody else to play with it.

My wife asked me why I was working so hard over the last month or so…what was the point.  Well here is the deal.  There are another couple of guys who have spent the last year and a half of their lives working on another aspect of making pinball more accessible to the masses.  Out of respect for them, and all of the countless hours they have spent working on their project, they deserve me spending the month or so of my time to support them.  It was a foolish deadline to tell them it would be done by the end of the year, but when you give somebody your word, you should try and keep it.

Yesterday morning I was crimping wires and my wife again asked me what I was doing.  I responded it was so when they get the card, they can immediately start playing with it.  I wanted their barrier to trying the stuff out to be as small as possible.  Here’s a quick picture of the demo board that they got:

GE

Gen2 Test Board

So the card is a little bit of a Frankenstein board.  Instead of using the normal communication channel, I used the USB to serial adapter that comes with the card.  (I didn’t want them to have to get a USB to serial port adapter or a RS232 to 5V converter and the board contains all that stuff anyway)   After initially using the board for a while, I wasn’t happy with the USB connection which is simply a card edge connector that is plugged into a USB port.  Grabbed an old USB connector, soldered the wires onto the edge connector, and added some epoxy for stress relief.  Now it never loses connection.  I also didn’t buy any parts to make these cards.  I just used left over parts and modified things that I had laying around because I never expected to send these cards to anybody.  This card is made up of the original cards (I built two of each) that I built for testing the hardware and nothing else.  Haven’t done a Mouser order at all, so the solenoid Molex connector is actually a cut down 24 pin header, not the real 6 pin header.  The 8 pin headers, are two 4 pin headers soldered side by side, etc.  Totally Frankenstein but it should be plenty for the integration they need to do.

So that project is done.  I have some more features that I want to add, but I could easily make a working pinball machine out of what I currently have working on the card.

The question boils down to where does the OPP go from here.  Joe (my head of marketing) says that people don’t want to populate their own boards.  I optimized the boards so they could easily be hand built by anybody with a very limited soldering ability.  The value proposition that I imagined was that you get a set of boards that you can run a pinball machine for half or a quarter of the cost of regular boards with a very little bit of work.  (It’s not like a person creating a pinball machine doesn’t need to know how to solder.)  I basically felt this was a fair trade.  At the most I would send blank boards out to people for whatever the boards cost me and be done with it.  (Blank boards cost about $15 for a whole late 80s/early 90s style pinball machine).

So my original idea was to start a Kickstarter project.  I would produce populated boards for two backers.  The first version of cards were originally sent to two people that were able to create flipping white woods, or flipping rewired existing playfields.  They were excited by their progress, but neither have produced a machine that interfaces to a controller for running all the pinball rules.  The only OPP machine that has a full set of rules is the SharpeShooter 3 machine that I created.  It doesn’t really tell me if there is enough documentation to create a full up pinball machine.  What would I really be getting out of Kickstarter except being able to maybe say that I had a successful campaign.  (I would fail to mention that the campaign was only for $100 dollars total).  The other downside is that Kickstarter takes 10% off the top, so I’d have to raise supporter rate accordingly.

After discussing Kickstarter with Joe, he mentioned I would need to do a video.  I don’t know any video editors (I guess that I would choose Blender which I have at least minimal experience), so that would be a problem.  Joe said he would help with the video editing, but that would be a huge burden on him.  He suggested instead to put something on Pinside.  In lieu of that, I’ll first put a note here.  If I can’t find the two people through the blog, I will consider adding a note to Pinside.  This gives the best chance to not bother Joe with doing any more free work for me.

So here is my email address if you are interested.  You can’t click it, you just have to type it into your email client.  I’m hoping this reduces the amount of spam that I get.

mail

If you become one of the two lucky chosen supporters, I’ll build your boards and talk you through all the wiring.  Just for reference SS3 required 3 solenoid wings (3 * 4.01), 4 input wings (4 * .27), and 6 incandescent wings (6 * 1.60), $15 for the PCBs, 4 PSoC 4200 (4 x (3.75 + .76)) = $55.75 total.  Note:  There is currently a bug in the incandescent boards that I have which I will spin a new version to fix the issue which will also allow the implementation of lamp matrices.

Oh yeah, happy new year.

12/29/2015, Initial code complete

Screwed up title of last post.  Oh well.  I’ll correct it after I finish this post so people don’t get spammed with two different updates on the same day.  Hopefully WordPress is smart enough not to do that, but who knows.

Joe keeps reminding me that I don’t need to support incandescent bulbs.  The incandescent boards simply support higher current outputs.  A typical incandescent bulb is about 200-350 mA of current (depending on if it is a 44 or a 47 bulb).  The Ablaze 4 SMD bulbs that I have tested are around 80 mA of current.  The incandescent board can support about 350 mA continuous current per bulb.  It can just as easily support LED bulbs, they just have less current per bulb.

So I had to send a bunch of Christmas presents out just before Christmas.  I ended up trying out Ship Gooder. In the US, these guys really rock for low cost shipping.  Even checked out prices of shipping stuff to New Zealand and they were way cheaper than anything else I could find.  Shipped a rather large 16 pound box (16.5″ x 16.5″ x 16.5″) to Pennsylvania for about $10.50.  Stuff got there in about 2 days which is fantastic.  The price was so cheap that I thought it might be a scam, but all packages were received successfully.

Spent far too much time creating a test fixture.  What I thought would take a day ended up taking about 3 days of my hobby time.  For some reason I love creating test fixtures and automated tests for that matter.  Maybe it is coming from a regulated environment and far too many startups.  It also partially comes from my first job of having manufacturing located in the same building as engineering.  When the people on the floor had problems they came up and asked you questions and then you had to go down and fix any issues.  It really taught you to make sure things were working/tested properly so you didn’t get to a situation where you had questionable software and hardware.  (For reference, it is almost always the software)  Of course, I’m not going to run the boards through a heat chamber for 3 days with cycling temperature to verify all the solder joints, but I do remember when that was the gold standard for board manufacturing.

Completed a significant update on the board interface document.  I removed the first generation card information to make it significantly more readable.  Added pictures showing the pinouts of the wing boards.  Added Python example code for generating the CRC8.  Here is a link to the brdIntf document in PDF format.

Still have issue with first neopixel, but haven’t actually taken time to look into it yet.

12/21/2015, Still on track

It’s been a busy couple of weeks with a lot of coding and a good amount of testing.  I’m still on track to finish the initial coding/testing of the Gen2 hardware by the end of the year.  It’s been a pretty big push towards the end of the year to get this to happen, but hey, soon it will be done.  The first version of the code will not support switch matrices or lamp matrices.  They should be trivial to implement in firmware, but making up a good test fixture to test those features will take a lot of time.  (I don’t have the necessary 64 switches or 128 diodes for that matter to test that functionality and it is feature creep compared to what is needed to run a pinball machine.  The only real reason to use those features is that you don’t want to distribute the control system, and want to pull all the cards back up into the backbox.  That would involve more wiring which is what I’ve been working to simplify).

While this has been happening the price of the PSoC 4200 has dropped to $3.75.  After I finish the code, I’ll work up another spreadsheet on the cost of getting a board set to run a pinball machine.  I did one before, but I’m not sure how up to date it is at this time.

So every once in a while a new processor comes out and I wonder to myself, why don’t I use that processor?  Well, this Thanksgiving the Rasp Pi zero came out (at a price of $5 which floors me) and of course I had to read up on all the specs because I’m a geek and absolutely fascinated at the price of cheap cool processors.  (One of the games that I love playing is how many processors in this room or house or car are bigger/more sophisticated than the processors that were used to send people to the moon and how much bigger.  Just so you don’t have to look it up on Wikipedia, it ran at 2 MHz, had 2K of RAM, and 36K of ROM and was a 15 bit processor.  The PSoC runs at 48 MHz, has 8K of RAM, only 32K of flash, and is a 32 bit processor so I’d say the PSoC is about 50 times faster/bigger.  Probably over sharing at this point, so let’s get back to pinball).

Some people might wonder why not use a Raspberry Pi zero for all the processors?  Hey it’s only $5.  Why not use that for the real time controller?

  1. First problem is that it isn’t 5V tolerant.  Some FETs have a threshold voltage of 3.3V, so it is easiest to use a 5V output to make sure that the FET is fully on.  If the FET is not fully on, it generates a ton of heat which can cause major issues (and eventually smells really bad because of the smoke coming off the chip.)
  2. 5V for inputs gives you more noise margin for noise generated by those pesky solenoids.  It shouldn’t be a problem but any extra margin is helpful.
  3. The Pi zero only has about 20 or 24 I/O.  I can’t remember off the top of my head.  The PSoC has 40 I/O which is a major sweet spot.  It has 4 full 8 bit ports for I/O, then the extra 4 pins can be used for communications (2 pins for tx and rx), and 1 pin can be used to synchonizing boards across multiple boards.  Very important if you want all your LEDs to blink at the same time since they are being driven by multiple boards.  It even has 1 extra pin that isn’t currently being used.
  4. More difficult to write a real time kernel.  I think somebody has taken the time to do this for the Pi, but the Pi contains a much more complicated and functional chip.  I can’t imagine going through the manual for that guy.  It must be absolutely immense.  The PSoC manual is about 1000 pages, maybe more if you are looking at the technical notes and that is enough to try and wade through.
  5. It has a ton of functionality that isn’t very useful in a real time environment (i.e. a GPU so it can drive a HDMI monitor).  That would not be useful at all on boards residing under the playfield.

The Rasp Pi zero might be perfect for the main controller, but as the real time controller, it just doesn’t make sense.  Even as the head end controller, the Rasp Pi A or B might make more sense because it has audio outputs and more USB ports to store extra data.

Finally got a chance to look at the incandescent wing card in more detail.  I was hoping to use the PSoc 4200 pins as open collectors and pulled to 12V so I didn’t need a second FET to step up the voltage.  Turns out that there is a clamping diode in the part (to protect against ESD/over voltage) so that won’t work.  The good part is that the incandescent wing boards will still work as low side switches as long as I swap the orientation of the FETs in the circuit.  The one down side of this is that it won’t work as a lamp matrix because you need to be able to support both high side and low side switching to turn on/off the individual columns.

To support the high side switch I would need a second set of FETs which will not fit on the boards using the TO-92 packages.  So one idea that I started toying with is switching to a SOT-23 package for this card.  This would break my rule of having no surface mount parts, but the SOT-23 package is large enough that it is rather easy to solder by hand.  (I think it might be easier to solder than the TO-92 packages since you need to bend the leads, solder, clip off the excess leads, etc).  I spent a little time this weekend and think that I can create a card that has the high side switches on the top of the card, and the low side switches on the bottom of the card.  Depending on what is needed, the user populates either the top or the bottom of the card.  That seems like the best of all worlds.  The layout is tight, but I’m pretty sure it will work.

Should I support incandescent bulbs fading slowly and quickly?  It would involve PWM’ing the incandescent bulbs, but it would probably be as simple as a little RAM based lookup table to support the multiple levels.  The neopixels support fading in and out, so it seems like the incandescent bulbs should also support that functionality.  I would probably not support that until a later version of code with a new command.

Depanelizing boards is rather difficult so I started to think about ways around that.  Turns out that ITeadstudio will do a groove for free as long as the boards are the same design.  That should be much more accurate than cutting them out with a tile saw.  Running the numbers (because now each of the boards would need to be separated on their own 5 cm x 5 cm design) it adds an extra $10 a set of boards.  Last time I depanellized a group of boards, it took me about 2 hours, so even I can’t seem to justify that doing that myself.

12/13/2015, Input/Solenoid Code Updated

Spent most of the last couple weeks rewriting the solenoid and input wing board code.  I originally simply ported the code to the new processor and thought I was done.  Then I quickly realized that it would work, but it wouldn’t work as efficiently as it could.  The problem was that the code used to run on two different processors.  Now the code would run on a single processor which is much more capable.  That means that I can use much of the same processing for both the input and the solenoid boards to trigger the solenoids.  Grrr, the only way to be happy with the code was to rewrite it.

The previous processor was an 8 bit processor, and the new one is a 32 bit processor.  That means some of the processing could be made more efficient because of the larger size of the registers.  I now also have registers.  Oh joy.  Before, everything was either on the stack or on the two registers.  (an 8 bit register, and a 16 bit index register)  No more.  The PSOC 4200 is much closer to a “real” processor which has the typical 16 registers for running code.

Next up it was time to start tackling the communications.  This involved rewriting the low level driver.  The code is now receiving commands and transmitting responses.  I’m still not 100% happy because the hardware comes with an 8 byte buffer in the transmit and receive directions.  At this point, it would be more efficient to keep the received data on the internal FIFO and have the higher level driver pull it off as needed.  That’s different than the previous processor which only contained a single byte buffer, and the data had to be pulled off very quickly.  Of course with the deeper FIFO, I’ve increased the speed of the UART to 115,200 bps so I should be able to send and receive a lot more data.  (The old driver ran at 19,200 bps which was plenty for a pinball machine, but more why not move to a 5 x faster rate).

Right now I think that I’m missing bytes every now and then because I’m moving the 8 byte FIFO through a single byte to hand it to the RS232 command processor.  It should be straight forward to leave the data in the receive FIFO and process it directly from the receive FIFO.

The nice part is now there are so many things working that it only takes a little bit of time to fix things.  I’m definitely over the learning curve hump on the processor.

I did find a weird issue with the processor.  In the SCB_INTR_RX register, there is an SCB_NOT_EMPTY bit that indicates that the receive FIFO is empty when set.  It is a write one to clear bit, so I was hitting the bit, and then reading it back immediately to see the status of the FIFO.  It turns out that the hardware doesn’t immediately write back the status of the FIFO, so it seems to confuse the processor.  I’ve changed the code to use the SCB_RX_STATUS register to get the number of bytes in the receive FIFO, but needs a couple more assembly instructions to have it work.

New commands to support the incandescent board have been added to the board interface document.

I’m still on target to have the fully running version of firmware by the end of the year.  I’m almost to the point where it is just testing/testing/testing.

11/17/2015, Chugging along

Boy, that is probably the dullest title of all the blog posts, so far, but, well, there it is.  Basically I’m converting the code from the old processor to code on the new processor, updating some of it since it is being changed from an 8 bit processor to a 32 bit processor.  The change in processor size makes some things easier and some things more difficult, but everything has to be revisited.  The ports are set up more coherently on this processor which simplifies a lot of the code.

I have currently converted the processing for the input boards and the solenoid drivers.  I had already done the Neopixel stuff, but need to use interrupts for the Neopixels so it doesn’t affect the switch/solenoid processing as much.  (It’s just the right thing to do).  I’ve added code to save the configuration off to flash memory, and retrieve it so the configuration is persistent.  Error checking has been added to the non-volatile configuration so that it is guaranteed to be valid.  Saving the configuration in flash allows the boards to be run without being tethered to a computer.  (AKA Joe/Cactus Jack configuration, and well the configuration I use most often when wiring up a playfield and batting around the ball).

The last big thing that I need to program before starting testing is the serial communication.  Should be a pretty simple module, just haven’t done it yet.  I need to merge all the commands from the input and solenoid driver into a single file.  At that point, after testing, it should be ready for the next step.

So I need some way to test the boards.  I’ll probably start out updating the PinBrdGUI python application to support Gen2 cards.  That will test the basic functionality.  Then I will probably hook the boards up to the old playfield that I used for original testing of the first generation of boards.  (I can’t even remember the name of that playfield at this moment)   It allows me to shoot the ball around, look at inputs, etc.  It’s not a very exciting playfield, but it is good enough for testing.

So the eventual destination of these boards is in a Dolly Parton machine that is going to be rethemed.  The goal is to not change any of the wiring harnesses and hook the boards directly up to the Dolly playfield.  (This excludes the sound and displays, because those are going to upgraded to an LCD display and full stereo music much like SS3.)  I’ll probably spin a board to do the interconnections and why not, it’s cheap.  It means that I will need to completely understand the wiring harness of the Dolly machine, but that shouldn’t be too bad.

The one thing that needs to be supported is a switch matrix and a lamp matrix.  I’ve already figured out how to do the switch matrix in the code.  It will need a couple more serial commands, but they are all based on the input board.  It will simply look like 8 output drivers to power the column strobe, and 8 input drivers to read each of the signals.  It will then send all 64 bits of data (8 bytes) back as a single command.

I haven’t figured out how I’m going to do the lamp matrix yet, but it may require another new board, or I might be able to combine two incandescent driver boards.  Just haven’t taken the time to do the design work yet.

That’s all I have for today.