Episodes
Tuesday Jun 19, 2018
Tuesday Jun 19, 2018
Bil Herd shares stories and design wisdom from years of experience as a hardware engineer, most famously at Commodore. Starting out self-taught, Bil found his way to working around brilliance and some of it rubbed off a little. Learn about his interesting journey from TV repair to Commodore, Hackaday and beyond. Today, Bil is self-employed and focused on networks, high-level architecture consulting and hardware projects.
Show Highlights:
- I never forgot how to do hardware design. It’s fun to be able to do that. I’m working on an Altium project right now.
- You get to be imaginative for a couple days, and then you spend the next couple months paying for it looking at every single line item, every footprint and trying to catch where your brain was wrong.
- Commodore Hardware lab, splitting bus for video and hired to lead the team shortly after.
- Going after a swag bag offered by Adafruit from an MIT Open hardware conference resulted in a video series with Hackaday.
- Almost all the errors I’ve made in CAD systems were related to parts I’ve made. For parts and footprints - you need to have someone check your work.
- To start a new CAD system - make a trash board, force yourself through.
- Process to start learning a new CAD system: Open CAD > Get Overwhelmed > DRINK
- Making a board on a new CAD tool. First I make a trash board knowing I won’t use it. Then make a real board, using all the rules.
Links and Resources:
See all show notes and video here.
Hey everyone, this is Judy Warner with Altium's OnTrack Podcast.
Welcome back.
Our audience continues to grow and we thank you for joining us again, and I want to give a shout out to Steven Newberry from LGS innovations who took away always marking his diodes with a K, and so many of you have chimed in and help driving actually who we have on the show and the topics we discuss. So thanks so much for joining. If you would please connect with me on LinkedIn or @AltiumJudy on Twitter and Altium is also on LinkedIn, Facebook and Twitter, and remember we're always on YouTube as well as on your favorite podcast apps.
So thanks again for joining and hold onto your hats because we're gonna have a little bit of a history lesson tied in with today's best practices. So today our guest is Bill Herd, who is actually a figure of history and he has a Wikipedia page that you will have to take a look at. So for those of you that are probably, I don't know 40s and above, might remember the Commodore personal computer. It was one of the first, I'll let Bill fill you in all the details, but I remember vividly when I was in my early 20s, my dad coming home with a Commodore 64 and it was all the rage and he thought the world is forever changed and I'll never ever use all the 64k that I possibly have.
So Bill, welcome and we're so glad to have you and can't wait. We're gonna tell some stories, you're gonna give us some design wisdom, so thanks so much for joining us.
My pleasure. Actually I do describe myself as a recovering Commodore Engineer the active recovery never stops you've just gotta keep trying to get better.
Well, I'm sorry but based on the background behind you I'm not sure about your recovery.
Okay - I've relapsed a little [laughter]
So, briefly tell us what you're - are you working now as a Consultant, you know like your own entity I forgot to clarify that with you?
Yeah actually I'm self-employed so to speak, I owned an ISP for about 15 years and had 16 people and we did all that and then it ran its course as ISPs do, and so I do a lot of networking and high-level architecture consultation, but I never forgot how to do hardware design. So actually you caught me in the middle of doing an Altium project right now, where we're going to a limited quantity but I just went through all the steps - all the dirty little details getting a PC board out, so it was kind of fun to still be able to do that.
I love the way you put it, 'the dirty little details' there's a lot of those right?
You get to be imaginative for a couple days and then you spend the next couple months paying for it by looking at each and every line item and every footprint and trying to catch where your brain was wrong you know, way back in the beginning.
Yeah well, so I also noticed you have - as I've gotten to get acquainted with you a little bit - back in the days of Commodore and the early days actually of the personal computer business the words nerds and hackers weren't really around but seeing you sit there in a Hackaday shirt with that lab behind you, I would say you are the quintessential original geek or nerd what do you have to say about that?
Well, one - we did call it home computers, back then the PC hadn't been invented yet, and I also mention I've never been to school for any of this. I was a - basically a high school dropout - and ended up in the service, and went back and almost got my degree. I own like a library book for the money $3.42; for a library book, in English class where me and the teacher just couldn't make it work.
So about three years later they sent home my diploma with my sister just going: here you'll need this someday. I used to say I was self-taught but what really happened, self-taught got me into a couple good places and then the education really started; working around really smart, really brilliant people, that's where I got the education that made it so I could do a product from beginning to end. So I was fixing TVs, got my TV Repairman License at the age of 17, in Indiana. You know and sometimes they'd answer the door and didn't want to let me in, because I got long hair and I'm carrying tube caddies and they're like: who are you? I'm like, van out front, TV repair, and people fed me cookies when they saw me fix their TV sets.
Right well we will share Bill's Wikipedia page and there are some awesome pictures of this long haired hippie, with this cut off denims...
Hey don't tease me about the shorts, it was 1980.
Hey I'm sorry, but I wore shorts just like that so yeah, so we will share that because there's a lot of history and fun and great pictures that I think you'll enjoy hearing. So tell us a little bit about how you got into the whole Commodore thing and then we're gonna dig in and give our listeners some really practical advice on those nitty gritty details you talked about, and then we'll wrap up with some more fun stories. So just briefly give us an overview of Commodore and Hackaday?
Okay if I back up just a little bit - I started at a digital scale company in Pennsylvania making instrumentation, so there I learned to do very accurate stuff with very good grounding. I understood analog and RF spectrum and all that, and it was all hand taped right. Well a guy named Terry Fisher who I just got through working with again, so after 35 years, we're still doing it and he was on Altium this time. So when I got to Commodore I had the background for how to make something expensive work. And then you just take that and you just shake it and it comes out of your head cuz now you've got to make it cheap, cheap, cheap, cheap.
Yeah.
And people are mad at me these days because they say: oh I have a 30 year old Commodore and it just failed! I'm like: it was designed to last five years. You should have put a switching supply... what? To put a dime more into it I'd have been fired if I did! So I got my job almost by accident at Commodore. I mean, there's a whole story here and I'm going to - let's just say I blew the interview like three times and still got hired. You know what, not even taking in my resume you know? But so I got there, and they didn't know what to do with me, and I read in a book that I was actually hired as a Technician. They just knew they could use people like me and then they sat me down, the guy was named Benny Prudent, and he said: well here, study all these software manuals. So now I was gonna be right for a programmer - I could do 6502 programming.
But for a disk drive - I'm like: that sounds like the most boring thing in the world, but sure. And then I walked into the hardware lab and I saw what they were doing - they were splitting the BUS for the video - which back then I was doing it at home - and that's why I said: I just built something at home where I actually don't wait till the vertical retrace time to ramp and two weeks later I was in charge of the project. The guy was leaving, they didn't have anybody else, so now I'm a Project Leader at Commodore within a couple weeks.
Oh my gosh, it was like the Wild West was it not?
Oh absolutely and I loved it and I brought a certain 'animal house' to that, because we had lost a lot of talent. I mean there was people like Chuck Peddle who designed the 6502, he's gone, but his cigar's burning in the ashtray. The chair's still warm right. So you knew that these people had been there, but they're gone - and there's these kind of older, stogier guys and me. And pretty soon it became an environment where shoes became optional, so we definitely made it into what we wanted to and you have to do that when you work 20 hour days.
That's crazy so you're sleeping in the office or not sleeping?
My record was 11 days without leaving. I had an air mattress, I would actually hot bunk with the technician so I would get something designed like 2:00 in the morning, check the air mattress out and they would build it for me and I'd go catch an hour to sleep and then they'd come back and kick the air mattress and say: it's built, and just taking showers out of the sink - things like that.
Well we'll talk more about some of your fun Commodore stories because I know we'll really want to dig into those a little bit more. But tell us also about your involvement with Hackaday?
Yeah it's actually interesting, that I used to watch Adafruit's Little Saturday Night Show right and they would do this thing where they'd give something away and usually it was a product and I didn't go for that as much because I could just - their products are so cheap I could just buy one. But one time they had been to the Open Hardware Venue - a conference - and it was actually at MIT I think - and they asked a question and I went right to a web page, found the answer because they were giving away the swag bag, so I said, that I'll go for! So in the swag bag was some cool things but one of them was a - it was like an Octopart - only it was somebody else's version of it. Well they're owned by the people that owned Pacada.
So I start talking with them, I ended up a Beta Tester, and the guy realized I just never shut up, that I'm always telling stories right? And so pretty soon he puts me in touch with Mike the head editor at Hackaday, and I'm doing the same to him only in emails, and finally he's like: all right that's it, write stuff or shut up, and so we came up with the video format because it just - it works for me - it works for my personality and I am a high school dropout which means my English ain't so good anyway, so the video works better for me.
Yeah well we will also share those for listeners here - I've seen a few of them and he is perfectly suited for that. So I'll share that as well for you wannabe hackers. So let's dig into some immediate content that I hope will help engineers and PCB designers that are listening to us. You have told me - how many EDA tools have you used over a year period?
Yeah I made it all up, hardly any at all [laughter]. No it had to be like seven, eight, or nine, depending on how you count them and to what degree. But going back to the 1980s when a workstation cost fifty thousand or a hundred thousand dollars and you couldn't get them as a home user or even as a small business and so, we started it. We started with hand tape and the cool thing with that is, if you can do good hand tape, you can use a tool like a CAD and do more. But you still have to be good to begin with right? You have to understand the principles and nowadays it's more common for engineers to do their own PCB layout but I'm still of that school that: do what you do really well, and use somebody when possible that does what he does as good as you, that's why I use a guy like I said, Terry Fisher.
He's as good and he knows when to ask me questions and I know when to shut up right so we have a good relationship for that kind of thing. And we started on Mentors, which actually we designed chips with, but he started on a system called a side card, and it was a card that plugged into the backs. Well when he'd start moving parts on the PC board everybody's computer slowed down right. These chip designers and stuff because it's on the VMBus, it's taking the cycles directly, so they give Terry his so - he actually he goes by Fish. They gave Fish his own VAX so now he's got a three hundred and fifty thousand dollar CAD system to lay out pc boards and so that's the 1980s, and in the Mentor, we - I hadn't really even seen a real mouse like we use until Sun's came out. It had a scratch pad so I actually grew my fingernail into a point so that I had a built-in stylus on my index finger - so yeah just genetically modified kind of you know... [laughter].
That's funny - so with all of those changing of tools which most people that I know, that are designers, once they get proficient on a tool they'd rather die than change tools because it can be such a painful process. So tell us about changing tools. If you have to do it, what is the least painful path?
Well management will always want you to do that right in the middle of a project right and that's - it's pretty key to not try and - we actually moved our hardware labs right in the middle of a project one time too. Just kind of in the same... But if you're going to change programs, realize that they're just tools, and after you've changed a couple times you start to go: okay I know how this play goes and and you do a couple of the same things and you sometimes learn and really appreciate your old tool and sometimes you learn that hey, the new tool's better. But they're no two the same, especially in CAD where there's so many complex things. So I think people picking up tools - I saw it a lot with EAGLE - what they did, and they did an amazing thing for the maker industry and the home users - even though I hate the program.
if you're a professional, you just go: what, I have to drag the trace off the screen to hit the menu? this is like somebody put a GUI on a command line program. Well guess what? EAGLE's were GUI on the command line program back in the old days. So you know the false attractiveness of something like EAGLE was, it did have huge libraries right, and especially for boards because I mean these, Arduino boards - I can't deal with the mechanics of them, they're not on the center's, I'm used to all that - but what you really have to come down to when you do a CAD system is, realize you've got to make your own parts at one time or another, so you might as well get proficient at it. And if you're using libraries you might just be using somebody else's problems.
So even if you do use somebody else's library - it's like you've gotta still vet the part. Right, so just realize that you're going to have to make your own parts. And then there's things like BSDL importing and stuff like, if you're doing a 250 pin FPGA, you don't want to hand-do that either, so there are tools to help you avoid the mistakes. But almost all the errors I've ever made in CAD systems are related to the parts I've made where - I actually have data books here not data sheets - a guy said: yeah you use the word book don't you? And while you're looking at the book, making the part, I've done simple things that I'll never catch myself - by having like D7 to D0, instead of D0 to D7. When I see what I think I want to see, and that's it, the mistake is in there until somebody else catches it. So we used to always have somebody else check our parts you know, in footprints or the same way I still think.
So that's the first thing, is realize you’ve got to make your parts and then I recommend you just sit down and trash a board - try not to ruin your library in the process - because you could screw up libraries right. But then throw that board away and start again. This time trying to obey every rule you know how, and actually even if you don't produce the board actually obey all the rules, look up every command you don't know that you actually need, and that's kind of how I started a new CAD system.
So for our audience, Bill sent me a few notes for the point of our conversation here, here is a note that he wrote: Starting a new CAD, do a couple of projects early on - sort of what he's talking about right now - his first line is, 'open CAD - get overwhelmed - drink' [laughter].
Yes, it can be overwhelming! It's like my drill sergeant said when I went through basics: 'we know it hurts gentlemen, you don't have to tell us' and we're like, oh I'll keep my pain to myself. It's the same thing: I'm supposed to be overwhelmed, okay let's you know. So you open it again, and you start looking for what you know. So there are some things you need to learn the quirks of upfront. Like how do you do a BUS? Everybody does it slightly different, that nomenclature, whether it's curly braces, brackets, whatever. And an 8 10 dot dot 8 zero - it might be low to high, it might be either way, but you got to learn those things. And interconnects, how to make sure that a part's really hooked up. One CAD system I was on, was called Ulti Board by National Instruments, and the DRC wasn't catching the fact that parts looked like they were hooked up, and they weren't. Well, how do you catch that?
Well yeah, how do you catch that?
Yeah so you've got to - you go around jiggling your parts and it's stupid you know, so do a good DRC and you know, Ben, when he looked over my shoulder to check my router - from Hackaday, Ben Jordan. He gave me an - actually a compliment that I took, which was: oh it's nice to see you have all your DRC errors fixed. Well I'm old enough, I don't remember fixing them, but I'm old enough that I know I would have fixed them. Cause that's it, that's your last chance to catch that you have a net floating, even though you don't know it. Whether it's a misspelling, even capitalization change, something like that. So yeah, you got to learn all those dirty deeds and details.
I was just talking to John Watson on this podcast about a week ago, we talked a lot about libraries and the same subject. It's like a theme that most headaches seem like they begin and end with the parts libraries and even having a data sheet that's correct or hasn't changed in the last five minutes. How do you address that?
I still have data books [laughter] - no, it's still like going over it, and over it, a couple of times and having somebody else look as well. I'll still take a highlighter to a schematic sometimes just if I feel I'm getting confused, out comes the highlighter to help me get more confused.
[laughter]
-at the end hopefully I get it.
That's funny! Okay (I keep bumping things sorry about that) so okay.
Let's talk about hidden nets...
Okay go!
I hate them [laughter] hidden nets are where somebody thought let's show up DIP package or something and we know we are hooking it up to +5 and grounds so there's no point in cluttering the schematic with it. Well my attitude is how do you know it hooked up to +5 and ground? Nowadays it's 9 +5 and ground is +3.3, 1.2, 1.0 - - so yeah whoever came up with that, they need to have something I don't want to say something bad happen... [laughter] They need to miss a CES deadline or something themself.
So it's the invitation for failure is what you're saying?
Yeah you can't check it, you make assumptions and that's where problems start so yeah.
Would you say that making assumptions is one of those easy pitfalls for designers to fall into?
Yeah, thinking SOIC is a size. It's not you know, there could be white body, skinny bodies, and it's like: oh but the picture looks like - no. You better learn to have - one thing is you have to learn with new CAD packages, is how to measure things. And you need to do that, and then look to see oh it's .43 inches or ... and I - one time I almost missed the fact that the the lead pitch was 0.5 instead of 0.75. That wouldn't have fit!
That would have meant instant failure. You made a comment about assembly drawings being readable what did you mean by that?
You know as parts got smaller the silkscreen no longer - it's not as important because of assembly techniques but if you still want to measure - you can't get that little silkscreen anywhere near the part sometimes, so you end up with an assembly drawing where you had to like put all these silk screens where you now want them inside the outlines and all that so it's like you can't use the silkscreen for an assembly drawing like the old days. You have to do a whole new one if you want to be able to find the part. But now these days what I do - but I'm working on a really dense... or troubleshooting, I actually keep the CAD open and I do the - jump to component - and find it that way it really is faster to use technology sometimes...
[laughter] sometimes.
Sometimes, at least I don't hand etch my boards anymore.
Yeah.
Remember that, the seventies?
I always say, because I was in the bare board industry for years, sales and marketing-wise and we would take people, walk through and do surveys, plus I actually worked on a shop floor for a short period of time like, I'm gonna die of heavy metal exposure man, the chemicals we had in there. I remember walking into a planing room at the first board shop I worked and my skin just burning, yes burning, just poor ventilation and there was sulphuric acid in there.
I'm told you can't have plating or PC board manufacture in New Jersey, that they've just kind of made it so you can't do that.
Well there's that - there's a little bit of toxicity going on in the chemicals.
Right, and at Commodore we made the ultimate printed wiring board printed circuit board right which is a chip - it's just really, really small, and we polluted the groundwater and you can look this up, but we had to buy dedicated lines for like 11 neighbors, and then we had those golf course sprinklers in the back aerating the ground water. Well my first day there I mean they're just literally spraying it in the air hoping the VOCs evaporate right.
Oh my gosh!
I parked too close my first day there and I come out and my car's covered with this sticky stuff right and not only that, I had parked under a tree so now the leaves are stuck to my windshield with this and to try and peel them off - they just break - and they're like: oh yeah dude, don't park there man, that's in the water.
Like I said, it was the Wild West days I mean.
Still a Superfund site I'm told.
I bet, like it's frightening - it's frightening and I'm glad we've gotten our act together a little bit environmentally oh my gosh because literally we could all die from those toxic...
Yeah I remember the day my dad brought home mercury to play with you know.
I remember my neighbor was an engineer - he brought home mercury to play with and we'd watch you know, roll it around on our hand or whatever, crazy! Forget about playing with it - you know putting it in your teeth we would like, oh here, pour it in my hand, let's roll it around, isn't that cool?
Yeah and you put it back in the jar and it's never quite as full as you started right because you're leaving a certain amount on the floor...
Good memories but we might die young, just saying...
[laughter] So when you start a new CAD program, do you just jump in and start designing? How do you take that on if you're gonna take on a new CAD, what's the way you approach it?
Well as I said, I kind of I go in knowing I'm going to do a trash board, it's all about just hooking some stuff up knowing that you're making mistakes and then I try and do something more real and try and really obey the rules and that's where it starts - that's how you're learning from page to page cuz every CAD system's slightly different, but it's kind of like how you think. You drop a part, you try and put a wire on it and the kind of mistakes you'll make is not having a clear way knowing how you want to do all the resistor values in the world right. Do you make a part for each resistor value, or do you use a generic part and assign the values? And those are things you just have to figure out yourself on each CAD system I think.
So I mean, I honestly don't know how it's done. I have lots of compassion for my engineering friends who are also laying out boards who really got no serious, formal training in PCB design, but alas they are laying out boards and then they get thrown a new tool like... So do you just hop on it and jump in and swim?
Remember, it's a tool too and they have some really great tools like things that'll help you plot RF noise on the ground plane or thermal or something but you know, at the end of the day that's not necessarily real life. It's a tool you know, so it's an opinion, and it might be a faster, better, more colorful opinion than we used to get with an old thermal probe. But you just got to kind of try it and if you work around people who can look over your shoulder they'll save you a lot of time - especially hot keys and stuff like that. And that's probably one of my pet peeves is I don't like having to rely on hot keys and that was even before I lost a finger, so now some of the hot key combinations are literally beyond this old man's ability to do without using my nose and stuff it's...
How did you lose a finger?
I tore it off!
Dare I ask?
I just caught my ring on something and I stepped eight inches off something - it stripped it off the bone, we have pictures on the web of that also. But I used to work at a trauma department and I've flown with a 103rd combat medics, I've been captain of a rescue squad. So I look down and I just go: I know where I'm going today - I didn't even tell my wife right. I figured she's away at a quilting bee, having a good day, the next day I was: Hi, uh, lost a finger and she got mad at me for not telling her.
I would get mad at you too... just saying like: oh Chee how's the quilt work? Good what did you do? I just lost my finger.
Yeah, yeah well my son actually looked at it and we took pictures - by the way I had to wait half an hour for an ambulance and being a former ambulance guy that was just like - that was an insult on top of injury literally. But I wanted him to think of it clinically and not be freaked out by it so we took pictures and stuff like that and then I told him, I said: well I'm going to - don't tell your mother - but I'm gonna leave with these ambulance people now and I'll be home probably tomorrow, because I know how things work, and he comes running to the door and he goes: dad, dad what's the key to unlock the Xbox? I'm like: okay you're gonna be fine by yourself. First time he was by himself, he's thinking about the Xbox so, all right!
Oh my gosh you crack me up. What else do you want to talk about relative to CAD tools? I'm looking at my notes here - you were talking about something - you talked about the buses, nomenclature and index based even that you'd said you hate those. So what else did we not cover?
I think the main thing is just how productive can you be? How well is it designed? And I was impressed by early CAD, which came out at like $4.99 in the 80s and we were like: whoa! I mean it's like that old monochrome purse, now there's PCs right - late 80s and the things you can do where if you copy a bunch of address lines, you can tell it when to paste it, auto-increment all those address lines as if I was continuing to do them. So if I grab a 0 to 7 and I paste it, now 8 to 15 is done for me. Well you can fly, when somebody has thought of things like that to do, you can go rogue. And it has to be controllable - sometimes you go, no I really wanted a zero to seven and - but there's tools like that, that can really make it. So, just the ability to double click and there's a new segment just like the one above it, tools like that are real important to me where I've just spent too many hours drawing in each line by hand.
Right, yeah I love when - well since I've been here at Altium, one of my fun parts of my job has been to help connect our developers with hard-working designers where they can say: do it this way, we don't work that way you know, it's really nice when CAD tools will actually get together with the guys that are watching and just watch 'em work and go: oh - because again it's easy for developers even if they've laid out boards - to make assumptions right. So I really love it when tool manufacturers actually take that into consideration and I love that we're doing that more and more these days. BOM distributor integration?
Uh it's probably the one thing we didn't have in the old days BOM integration where, and even picking the footprints, we had a three-ring binder of IPC footprints and that was always a step where errors could occur. I'm thinking this way, PCB designer's thinking that way, wrong footprint gets in there. But then even now, we can with Altium, for example, you see the part as it's a digi-key or arrow and you can make an attempt to select a part. Now; sounds great, but you end up getting into trouble when you go: oh wow, now I have to redo it for real, for the auto, for - I still call it auto insertion - for the pick and place. You know or, guess what? The stock status isn't quite what you thought it is, there's a delay in there and so now you're stalled, so you still have to, I think in my world, I still do a final BOM as a spreadsheet literally. But I get a lot closer in the tool. In the old days we were using microfiche if you didn't have the data book right? So nowadays it's integrated so it - again you have to be careful - it's a tool, it won't do your work for you and that's the thing.
I was just going to say - I've worked with also like hiCAD and now KiCAD... however they pronounce it.
I know, I never know how to say it either.
Yeah and it was good in that you could add modules to it. It was bad in that you could add modules to it. I kind of wanted already the 3d viewer working - ready to play with it and stuff like that. I'm really impressed with Proteus instead of EAGLE for that low-end market, not up here where Altium is, but that's when I was shooting little two inch by two inch boards for Hackaday and I'm doing a complete design every month and doing a video, and so I design it and it gets a minute of video time right. Then I throw it away to start on the next one. So it's called ARES and unfortunately the other one is ISIS, (nobody likes that name anymore), but that's the product name and they have an amazing auto router in there that'll get you a good completion, whereas if you've ever tried EAGLE it's like why do I even try the auto router you know? So that turns out to be in, and they singled out the maker market by including Arduino in issuing 80 mega parts in simulation and firmware simulation so now you can simulate it as if you've written the code. You don't even need to build the board to see if it works. And that's a cool feature. And we didn't talk about simulation - almost all CAD tools these days do include SPICE of some sort or a SPICE portal or something like that, and that's useful if you're down in the analog stuff especially. Again still just a tool. I've seen SPICE lie horribly to you, and you think it's going to work and it's really an artifact of zero volts or something like that.
Well there's a lot of talk these days about - because so many really capable designers like you, and like many people I know have learned this over a lifetime right - so if you're a new designer where are you gonna on board that outside of just one-on-one mentoring? Like any clues?
There's some good YouTubes out there, but I haven't found where you can - one, I don't have the attention span to watch somebody else work for five hours to pick up a couple tips right. So it's in the YouTubes showing you what they want to show you, but the best way is literally to be near somebody that's really good at it that's - unfortunately that's the best way - it's almost always like people almost pair off in engineering where one guy's learning from another even if they swap roles later that day because he's better at something else. That's just kind of the way it ends up going.
Yup, so I think what you're saying is find a mentor if you're not really good at it.
Right yeah and vice versa and mentor others. So I was talking about the wire, on each and every C128 board.
Okay, oh yeah actually Ben Jordan snuck that to me. So let's go into war stories a little bit and let's talk about 'the wire' also I'll get a screenshot of this I think Ben or do you have it? [Bill reaches over to show C128 board]
That's so cool okay for - oh my gosh okay, so for those of you that are listening to this on the straight-up podcast you need to go to this portion and look at the YouTube just to see this giant board that he's pulling out of the Commodore 128 and look at the keyboard.
This is what we call a 'barn door stop' it's too big to be a regular doorstop and that keyboard I designed by looking down at my BT 220 and I said, hey it works for me it'll work for future users too. I'll hold it up to the microphone for users at home right
Okay.
But there is a wire on each and every... we made 5.7 million of these.
Oh okay. Wait before you go into the wire story, give us the stats on Commodore 64 going towards it and compare that to the Apple because I thought it was really interesting.
Yeah the - and actually I narrated a video by a company called Junk Food about the - called the 8-Bit Generation, and I learned some things - our version of history wasn't quite as clean-cut as to who was the first and the best computer company out there so I'll give a little props there. But we often said, Apple's just using our parts, because we made the 6502. Well that's the processor they used, but we made the chip.
So in our minds Apple did come out and they were first to get a floppy drive and some color early on, but then we come whooshing by them with the Commodore 64 whereas they sold 5 million of the Apple 2 that you're always seeing on every show about the 80s right. You see a show about Silicon Valley: 'we created the home computer' I don't agree, sorry I'm from Commodore I am a competitor and we made 27 million Commodore 64's we had all 64K, we had these cool color chips and sound chips that they didn't have and we could do animation because we have these things called Sprites, except Sprite was trademarked by Texas Instruments so we had to call 'em movable object blocks, but everybody called them Sprites, so you could write a game right and the blocks are moving themselves around, you're not having to rewrite that whole screen and everything so it was an amazing computer and we called it the 'Apple killer' because we actually stopped talking about Apple.
Yeah then my boss wanted to kill Sinclair, remember the Timex Sinclairs?
I don't.
They're little tiny door stops now - I actually did use one of those for a doorstop and then the marketing department saw that and so suddenly every door in marketing has a Sinclair holding it open...
That's so funny I don't even remember that one which I'm kind of surprised. I was kind of tuned in at that time but not that tuned in I guess.
It was a $50 computer and actually, when the basic ran, the screen would go to crap because it couldn't share the BUS, remember I talked about that earlier, and then they came out with a color one and and it was cheap. I mean the Commodore 64 was $299 - by the way the Apple 2 was like $1500, $1700 and we're $299 - and then we did something like we lowered the price to $100 if you send us your old computer. So people were buying Sinclair's for $49, sending them to us to save $50 and that's of course 50 1980-dollars so this was - if you can see it through the microphone here - this was the one of the family that we called Ted and this was basically the Raspberry Pie of the day, it's all in there. The one chip does the video and the sound, and there's a processor.
Oh and the video sound chip runs all the D-RAM and does all the crazy interfaces to the keyboard. So it's literally like very close to a single chip board even though there was nine in the original - nine chips - yeah you cracked open an IBM PC and there was 280 something like that. That's crazy and even the 128, as big as that was, had a couple couple tens of chips in there. So and then Jack Turmel unfortunately left Commodore and this product I was showing, this Ted thing. Without him there to drive the vision, that product kind of failed and we even had a talking version. We had snagged the guys from TI Speak and Spell, which was a big thing in the 80s and we had them working at Commodore, so we had a talking version of a computer with a desktop that Apple tell you later they invented the desktop. Well no. The guys at PARC invented it but we had one, it was just our founder left and it floundered without the founder.
Crazy, okay show us the wire.
Okay, so then the 128. What happened was I had gone to a CES show and by the way CES shows drove everything for us, Consumer Electronics Shows, mostly cuz if you ask them if they'll move it a day so you can hit your schedule they'll say no, so the CES show is - this is a scheduled date you cannot miss - you can't miss it by five minutes, you can't miss it by a day and so we decided - and by we I mean the engineers, we didn't even really tell management about the C128 till it was too late and then we would do things to it. Like I added a z80 processor so it became - it's Commodore 64 compatible - so suddenly nobody's going to complain at me because there's no software, can run all the old software, but then turns out the z80 cartridge didn't work very well on the Commodore 64, so I just put the z80 right in the board and after the PC board Rev was done I said: oh by the way I added the z80, they knew they couldn't tell me to take it out now or we'll miss CES.
So then pretty soon the guy would be: I had a great idea to leave the z80 in there you're like, cool go tell marketing, take a doorstop with you right. So one of the things we did is, even as we're getting ready for the CES show - it was January 6th that year I think - we're already getting ready for FCC, so we're working on the final production and that's all in five months. I started this near August and we had six - five or six customized C's that needed to be done and so again that was our wheelhouse - this is custom, this is custom, this is custom, that - one of the other ones in here - and we're going like the wind right. Well right near the end, the z80 stopped working reliably. It wouldn't boot CPM 20% of the time, and me and my boss were fighting. It's bound to happen right, he'd already gotten his bonus I think to let me go around barefooted was like wearing thin right. But the - - oh, I lost my train of thought that almost never happens when you get old… [laughter]
You fought?
Oh I was fighting with the boss and he said: fine, I'll give it to somebody else to fix that problem! I said: fine, I'll take a shower and go home and get a nap! Right, so for a week, I mean I had a great week. I caught up on my hygiene, (I won't tell you some of the other things you do when you're full of testosterone when you're young). But he comes to my office Friday, and in my mind he puffed on his cigar (you could still smoke in the office back then). I don't know if he had a cigar that day, but that's my memory and he goes: fix it or you're fired. I'm, oh sure I can do that, you're ready now for me to rejoin the workforce?
Absolutely, I'm clean, I get along with people, and I just happen to luck out where I'm - the oscilloscopes of the day weren't like the Tektronix MSO scopes - like I got back there, I had to turn it up real bright, and then I would stare at it and then turn and look at a wall and I would see the reverse image and I go: there's a glitch right there - I'm pointing at it so someone can see it because he hasn't burned his retinas staring into the light - and they think I'm nuts, and I was right. There was a glitch on this A10 line, when the z80 was the processor, but when the 6502 was the processor there's no glitch. I mean it's right around when the D Rams were doing something and so it comes down to understanding how a signal propagates down and this is part of PC board layout right.
And I liken it to when the 6502 was driving the length of the line that drove it all the way to the end, like playing a flute correctly, but when the z80 drove it from an extension down the line it was like blowing into one of the holes on the flute and it's kind of not - and so I got a standing wave, where the wave’s going back and forth and bouncing into itself and it just happened to do it on A10 at the wrong time and I caught it on the scope in an hour. Of course nobody believes me right, so and the way I made it work, was I took that wire that I showed you that's redundant. There's already a trace on the PC board, I just soldered this again so now it's actually a loop right it can't bounce - -
Ah it had a return path, okay.
Yeah or propagates like this, but either way it's not a standing wave anymore at a certain spot, and it just happened to be that spot was the multiplexer for the D-RAMs and they think I'm nuts right because not only do I fix it an hour, I fixed it with a wire! So we ran 10,000 units to prove that Herd's gone off the deep end and we got a hundred percent pass rate on it. It actually fixed the problem. So now the wire drives me nuts because there's 5.7 million wires out there and people said: why didn't you just change the PC board? It's like: because actually I found it this time, if there were no tools to do anything, if I change the PC board I might have moved a glitch to somewhere I can't find right. So the devil you know - and that's how it ended up going out.
That's crazy - and from by the way - having a background in EMS. For an EMS provider, to have to put a wire on five million boards, that's crazy nobody would do that today but it's cool!
We called it post solder assembly and it's horribly expensive that's five point seven million dollars. It probably cost $1, the wire was a penny and 99 cents to put it on there, so we just did that. There was one other fun issue with the schedule of the 128.
Okay.
At one point - and see we didn't have real deep analyzers and stuff - so when the processor goes flying off the tracks because the memory is corrupted you'd go, well when in the last two minutes or two million cycles did the corruption occur? Because the analyzer's not going to catch it, unless you're so lucky right. So one of the things I noticed is, it would corrupt in the video memory and the video's memory is being scanned 15,000 times, 60 cycles a second and so I took a light pin and I put it on this spot on the screen right where the corruption would occur and I sent my analyzer, so soon as the spot on the screen occurred the light pen triggered my analyzer. And it's actually a commodore light pin - I still have it - was actually plugged into the joystick port of the system on troubleshooting and it turns out - it was called ground lift, and you're probably familiar with that. There was a stub of a little over an eighth of an inch on the ground pin of a DRAM multiplexer, and it's inductance mixed with the capacitance meant it would come off of ground when you went to switch a whole bunch of zeros to a one - except for one - that other one became a one also it just dragged everything with it.
Oh, got it.
Yeah and I also took - literally another little piece of wire - fixed it and then I yelled at Fish to fix. That one I made him fix but the only way we could catch it in that case, is I used a light pin to catch this little 1/8 inch piece of trace that was just playing with me.
Well you know what I love about these stories Bill, is that I think it's lovely to tell them and show people what a Wild West it was and how we solved, but people like you solved things really simply because now we sit on all these really complex tools and really we stand on the shoulders of people like you right, who were innovating back in the day where we did not have the complexity of tools or things and it's easy to take those things for granted now because so much can just run in the background and you so I think it's fascinating to hear these really - like these MacGyver ways that you figured out how to fix it - you're like the original PCB design MacGyver dude.
So one other quick story and it goes right to that - about the tools and the software simulations and things and it's the day I knew I was working in the right place. And this isn't my story - this actually is the chip designer stories for the Ted, for that thing I showed you. They had design roll checks when they laid out an ICs that told him if two things got too close to each other, but they didn't have an electrical rule check to tell 'em if it's supposed to be shorted together or not, so they turned a corner. They had like A7, A8 and A9 cut right across the other to address lines and it shorted 'em out, and they had no way to check that - unless they hand looked at every plot of every layer of what made up an integrated circuit. Well they - meanwhile cost a quarter million dollars to do another run. So what they did - I'm in the hardware lab, and the guy goes: okay turn on that - turn on the microscope light. Okay turn it off - good we're in NTSC mode. And I turn and I look and I'm like: did you just flip the status of a register with photons while looking at it under the microscope? And he goes: yeah uh huh, and I'm like: AH I'm in the right place, this is where I wanna be!
Yeah and they didn't have the tools that told them if what was on the schematics, what made it onto the chip. So yeah and they would spend five months, with a ruler actually called a scale, checking the plot. That's the only way they could do it.
That's amazing well thank you for sharing this - unfortunately we're running out of time. But thank you so much for sharing your history and your ingenuity and the stories of Commodore and giving our listeners really some practical ways of just jumping into a new tool, if they have to right, nobody likes change but I'm sure you would attest to that overall has probably helped you become a better designer to go ahead and jump in and you could probably jump into a new tool easily now it probably doesn't freak you out as much as it used to.
If you know you're going to be overwhelmed, then you're right on schedule when you get overwhelmed - and then you just go back into it and you know, how do you eat an elephant? A bite at a time - same thing.
Just acceptance that it's going to be frustrating and this is the cycle.
Yeah that you'll screw it up and then fix it, just don't ruin your libraries in the process.
Okay, well some good, good wisdom. So thank you again Bill for your time, it's been a delight to hear about everything and I just by the way - best background - those of you listening, you really need to go look at the YouTube version of this, because his lab looks like you'd all want to go live in it man it looks like there's everything in there it's awesome. What's up with the penguin by the way? There's a penguin, that looks like it's standing on your shoulders?
On the telly, it used to be on top of the oscilloscope but now it's just with you so that's that's a Monty Python penguin, that's from our era right?
Totally, that is so funny!
Okay, well thank you again for joining myself and Bill Herd today on Altium's OnTrack podcast. I'll make sure to share all of his colorful links and Wikipedia and videos from Hackaday and thanks for joining us again. We'll see you next time - until then remember to always stay on track.