Delicious Serial

NB Even by the standards of this part-time blog the following post is absurdly obscure.

My brother works for a large company who have been in business here since the 1950s. They have a lot of old equipment, and he asked my advice about an elderly PC. It interfaced with some industrial plant and displayed vital statistics about what the machinery was doing. The worry was it might fail one day, and they had no backup or replacement for it.

Intrigued, I paid a visit. It turned out to be a Sanyo PC, model number MBC-17LXS. It had a 5.25" floppy disk drive (empty) and interfaced with the plant via a serial cable. The software appeared to be running on DOS, but I couldn't confirm that as they were using the plant that day and didn't want me trying to quit the program.

Back home, I did a bit of digging. Not a lot of data on the Sanyo MBC-17LX online but I eventually found this advert in a scanned copy of the August 1990 Popular Electronics magazine:

Cool - so the machine was circa 1990, was probably running MS DOS 3.3, and was an AT-type design. Given it had been in near daily use, I considered it a miracle the unit was still working. I can't see today's PCs lasting so long.

The next step was how to get the software program that interfaced with the plant copied from the Sanyo. I thought if I could get the original executable out of it there was a fair chance it would run on a DOS emulator.

The easiest way would have been to use the floppy disk drive, but neither 5.25" disks, nor drives that can read them, are available to buy today. (No-one makes a USB 5.25" disk, although 3.5" drives are still relatively common.)

What about the serial port? I thought that might be an option. I'm just about old enough to remember sending files via a serial terminal.

Or (although the advert was a bit coy about this) I assumed the system had an internal hard disk. If it was an AT machine the chances are it would use an IDE connection and thus could be attached to a USB caddy. This would be the easiest solution in many ways, but had one drawback - it meant opening the system up, and I was worried about disturbing a 30-year old, mission-critical machine. No backup or replacement, remember.

It's not easy being green

I thought I would explore the serial-port route initially, as the lowest-risk option.

The first thing to do was get a test/staging environment set up. It turns out MS-DOS is available as a free download, so I bagged myself a copy of v6.2 (the last one before Windows took over) and set it up in VirtualBox. DOS runs very well as a guest OS, although you need to edit the CONFIG.SYS file and add the line 

DEVICE=C:\DOS\POWER.EXE

at the end of it, otherwise it will monopolise a CPU core at 100% utilisation.

In the VirtualBox settings, you can tell it to connect the guest's serial port to an existing pipe/socket, so I instructed it to connect COM1 (Guest DOS) to /tmp/myvbox (Host Linux). My intention was to use minicom on the linux side, but it wasn't immediately clear how you connected it to a pipe instead of to a physical device. A bit of googling later, I discovered socat, a swiss army knife of a networking tool:

socat UNIX-CONNECT:/tmp/myvbox PTY,link=/tmp/vbox-pty

This creates a link between the virtual serial port DOS is writing to (/tmp/myvbox) and a virtual device on the linux side. Running ls -l on /tmp/vbox-pty shows it is a symlink to /dev/pts/0 -- success, this is a device we can feed to minicom as a "real" port!

Now for the DOS side. Overwhelmingly, my google research pointed me towards a program called Kermit - a serial port transfer program for various operating systems. As a bonus, minicom can speak the kermit protocol.

It is necessary to get kermit onto the DOS vm before we can use it to transfer files. However, this is DOS, so no networking, and we can't use a disk image (as the Sanyo has no disk). This is known as the bootstrapping problem, and fortunately far cleverer minds than mine have come up with ingenious solutions to it.

Running a copy con

The DOS copy command has an obscure option to create a file and add text to it. We can use this to create an executable program (a .com file) -- but only if the executable code contains characters you can type. A contributor to the kermit mailing list called Brent L Carruth used this principal to create tcom.com:

  XPHPD[0GG0G,0G51G31GB'(G+(G:u'0g?(G>(GE1G@arwIV_F*=US@<1|_,5wXNg-7muTu(4
  1m0ss1k260s@3G1g360@3G0i7t2g3A1g350@3G2E1=0C1g350@3T2M0^\1g3>0@3T=1s2g0T
  1g3;0@3ToN2g391g0t@3G0^F1k0s2?0@3T4

So at your DOS prompt run COPY CON TCOM.COM and carefully type in the lines above, ending with a CTRL-Z. This will allow you to start listening on the DOS side for an ascii text file over the COM1 port. When you're ready to send, just type TCOM > FILE.EXT and it will hang, waiting, till you press escape. Like I said, very, very clever.

This really is minimalist, though, it can't handle binary code (at least, I couldn't get it to work with anything not ascii-encoded) and doesn't tell you anything about your transfer. Just wait a while and hope your file has come through. What will we transfer though?

Back to Basics

We can transfer a rather more capable bit of the kermit project as our first file, a BASIC program called MSBRCV.BAS written by F. da Cruz. (We could type this in directly, but it would be laborious and prone to errors, much easier to transfer it with TCOM.)

Older versions of DOS had GW-BASIC (gee-whizz basic as it used to be called), My version of DOS v6 came with Q-BASIC, a sort of early IDE.

So, DOS side we run TCOM > MS.BAS then on Linux side we run minicom (sudo minicom -s -c on) and make sure baud settings are right (300baud, no parity).

Then transfer the BASIC file by doing CTRL-A and choosing ASCII as our transfer type. All being well a few seconds later we have our code transferred and can run it up (QBASIC MS.BAS):

Microsoft's menus in evidence thirty-plus years ago...

In minicom we can up our baud rate to a giddy 1200 and send through a proper binary program at last. I chose to go with PKZIP.EXE, followed by the full version of Kermit (as a zip archive). (Note we send these using the kermit protocol, not ASCII.)

A little unzipping later and...

smile indeed!

Back to the real world

I was now reasonably certain I could bootstrap my way to transferring files from the Sanyo, if it was anything like my VirtualBox setup, so armed with a USB-to-Serial cable, and a null-modem cable (so retro!) I went back to my brother's workplace.

As soon as the computer booted up it went straight to the monitoring program, which had no exit option, so you had to do a quick CTRL-C before it started. This brought up the familiar "Terminate Batch Job Y/N" prompt. Answering "yes" brought me to a C> prompt (uh-oh, why no colon?).

"the only winning move is not to play"

I hit the first snag - this was a seriously cut down version of MS-DOS. It's often forgotten that a lot of the familiar DOS commands were, in effect, optional extras. All you really needed was COMMAND.COM and a few other bits and pieces. That's what had been done here... no DOS directory, and so no BASIC interpreter. This was a blow, as my bootstrapping plan required BASIC.

Then I noticed the free space on the disk was low. 130kb to be precise. Kermit.exe alone was 230kb! Time to think of a new strategy. 

No loose screws and no spinning plates

A bit fed up I decided to bite the bullet and go for the direct-disk-access via USB caddy method. 

A bit of screwdriver work later and I was looking at the machine's innards (where I saw "1987"  scrawled across a circuit board). But another snag... there was no hard disk visible. Floppy controller was there, and a slave connection for a second drive... but as far as I could see the internal hard disk was not a mechanical device at all. There was a circuit board with rows and rows of Sony black chips which I assume is some kind of early flash memory?

Nothing my USB caddy could interface with anyway, so back together the Sanyo went. And I went online to do a bit more research.

Zip-a-dee-BOO-dah

I found a less advanced (but much more svelte -- 18kb!) serial file transfer program called ZIP. (Despite the name it has nothing to do with PKZIP or zip compression.)

I had to be able to transfer this as binary though and TCOM could only do ASCII. The solution to this is another innovation from those clever kermit developers -- BOO. Basically it's a way of encoding a binary in ASCII form (a bit like Base64, but more compact). There is a bit of C source code for linux (ckbmkb.c) that boo-encodes binaries for you. You still need to de-boo them on the DOS side though, and fortunately another bright Kermit-contributor (Dan Norstedt) created a short ASCII COM program modelled on TCOM to do just that. It's a few lines longer, but still amazingly compact:

XPHPD[0GG0G,0G51G31GB'(G+(G:u'0g?(G>(GE1G@arwIV_F*=US@<1|_,5wXNg-7muTu(4
1m2?352t0osr2e3K1q2s0s3e0W1_F0:sss1@2G0t1k0s3p0@3T1m3>52f3>1k0t3<2C0@3T2
K1g2?0@3T1Fm3U51g3<1q0s3:0@3T1g3r1l0ts1>0I@3T1m3i52e0O2;h0L1_Eg352s0m3S2
j0W1g3of0<1;2?0r1m0s3:1>0m@3T2e0R1FH2E1m0s3:1>0B3^0=2g3=1g3s0@3T2e0@3^1t
2e0<1>0m1m0s361>0e1l0s371g3r1:0P@3T1:0P2e1hDk0s3q0V1F2M1_3_c2o3Z1=0Y1=0c
2s0o2Ag3H0CSCS1:0=F[1:0=2s0]352k0t1]2s0U390^3<1KL2D1Dc0sf1]2L0UE^1T2HfTZ
X3mS2@F5C6G3S2Y\_X3a25BB3W2HacTV^\aZ3S2gb3S2Y\_X3mSW28eebe3S2Whe\aZ3S2Y\
_X3S2<3b2B3W2Abg3S2XabhZ[3S2`X`bel3W4

(And of course we can use TCOM to transfer it, no need to type this.)

ZIP needs an instance of itself at both sides of the connection, so I had to run it inside VirtualBox on my own laptop rather than using minicom, but otherwise I thought I had the solution at last.

Seriously serially

Time to actually transfer some data! I laboriously typed in the TCOM source and ran it up. It seemed to work at first, but then I realised I was only getting the first few bytes of the files I sent.

On the point of giving up altogether, I took another look at the serial port protocols on the linux side and noticed hardware flow control was switched on. This hadn't affected the VirtualBox virtual serial port transfer but maybe running it on real hardware made a difference, so I turned it off and tried again.

And at last, data flowed. I transferred both ZIP.BOO and my de-BOOing COM file!

The deBOOing worked too and I still had a generous 80kb of disk space to play with.

The industrial control software had about 40 separate binaries in its directory, but fortunately ZIP has a batch transfer option. I ran it... and sent over about 15 files. Real progress at last. And then it crashed. I ran it again. Got a few more files, then another crash. Tried again. Instant crash - and now it was crashing on the same large(ish) file every time. Was I going to snatch defeat from the jaws of victory?

Went back in to the serial port settings in the ZIP config (you need to use the same options on both "sides") and dialed the bit rate right back to a lowly 9600baud. Things went sloooowly but -- they came. Still a few crashes, but ZIP was clever enough to re-download only what hadn't worked properly the last run. And about 20mins later I finally had all the data extracted from the Sanyo.

Absurd, I know, but got quite the sense of achievement. Kermit developers -- and ZIP developer -- I salute you.

Come on come on do the emulation with me

Time to test if running the software on something that wasn't a 1987 era PC with bespoke innards would work. Virtualbox was already set up with DOS, so just a case of rewiring the COM1 port to the host's /dev/tty and running the main .exe file.

And... up it came, exactly as it was on the old machine. It was a happier time, no installers, no DLLs, no hardware licence lock-in. 

There was only one problem, the old machine used to flag alarms by "boop"-ing the the PC speaker, but whilst VirtualBox could emulate a Soundblaster card it considered no-one would need a virtual internal speaker.

I looked at other options and came across DOSBox, a free emulator. Using the clear instructions and tweaking the config file to enable COM1 to host-device connection I soon had this working as well as VirtualBox, and the "boop"s were back too.

It was now possible to set up an "old" (ie 4-years-young Windows 10) laptop as a backup to the Sanyo and leave that with the company -- just in case the venerable warhorse finally decides to join the choir celestial.

One of these jobs where I had to educate myself on spectacularly obscure and well-nigh useless techniques, but a lot of fun nonetheless.