Building OW1.9 on Linux

[Philip Pemberton]

Hi,
Is there some secret to building OpenWatcom 1.9 on Linux/x86_64?

I'm running Ubuntu 10.10 beta on an AMD MV40 system (x86_64), with gcc in
'-m32' (32-bit) mode. Thus far, I've had to:

* replace '#!/bin/sh' with '#!/bin/bash' in build.sh
(to resolve the 'setvars.sh' issue)
* OW19/bld/wmake/gnumake line 4: add "CC := $(CC) -m32"
(forces GCC 32-bit mode)
* OW19/bld/build/local/local.mif
lines 100 to 106 inclusive: change 'gcc' to 'gcc -m32'
* OW19/bld/re2c/master.mif
line 24: change 'g++' to 'g++ -m32'

This is enough to get the compiler (more or less) bootstrapped, and
Builder working.

=============== 12:38:46 /home/philpem/psion/OW19/bld/cmdedit
===============
=============== 12:38:46 /home/philpem/psion/OW19/bld/cmdedit
===============
**** REL2 rule
=============== 12:38:46 /home/philpem/psion/OW19/bld/brinfo
================
**** REL2 rule
================ 12:38:46 /home/philpem/psion/OW19/bld/trap
=================
========= 12:38:46 /home/philpem/psion/OW19/bld/trap/lcl/linux/std
==========
========= 12:38:46 /home/philpem/psion/OW19/bld/trap/par/linux.srv
==========
========= 12:38:46 /home/philpem/psion/OW19/bld/trap/par/linux.trp
==========
========= 12:38:46 /home/philpem/psion/OW19/bld/trap/tcp/linux.srv
==========
========= 12:38:46 /home/philpem/psion/OW19/bld/trap/tcp/linux.trp
==========
================ 12:38:46 /home/philpem/psion/OW19/bld/trap
=================
================ 12:38:46 /home/philpem/psion/OW19/bld/trap
=================
**** REL2 rule
================= 12:38:46 /home/philpem/psion/OW19/bld/dip
=================
======== 12:38:46 /home/philpem/psion/OW19/bld/dip/codeview/linux386
========
wlink op q name codeview.dip debug dwarf option symfile sys pharlap rex
disable 1023,1014 op map file {dipimp.obj cvmisc.obj cvld.obj cvmod.obj
cvtype.obj cvsym.obj cvcue.obj cvvirt.obj cvloc.obj demangle.obj}
Warning! W1008: cannot open clib3r.lib : No such file or directory
Error! E2028: memset_ is an undefined reference
Error! E2028: memcmp_ is an undefined reference
Error! E2028: memicmp_ is an undefined reference
Error! E2028: memmove_ is an undefined reference
Error! E2028: itoa_ is an undefined reference
Error! E2028: toupper_ is an undefined reference
Error! E2028: __IsTable is an undefined reference
Error! E2028: tolower_ is an undefined reference
Error! E2028: strncmp_ is an undefined reference
file dipimp.obj(/home/philpem/psion/OW19/bld/dip/c/dipimp.c): undefined
symbol memset_
file cvld.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvld.c):
undefined symbol memset_
file cvtype.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvtype.c):
undefined symbol memcmp_
file cvtype.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvtype.c):
undefined symbol memicmp_
file cvsym.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvsym.c):
undefined symbol memicmp_
file cvsym.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvsym.c):
undefined symbol memset_
file cvvirt.obj(/home/philpem/psion/OW19/bld/dip/codeview/c/cvvirt.c):
undefined symbol memset_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol memmove_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol itoa_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol toupper_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol memicmp_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol __IsTable
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol tolower_
file demangle.obj(/home/philpem/psion/OW19/bld/lib_misc/c/demangle.c):
undefined symbol strncmp_
Error(E42): Last command making (codeview.dip) returned a bad status
Error(E02): Make execution terminated
<pmake -d build os_linux -h> => non-zero return: 512
Build failed


It looks to me like the bootstrapped compiler (the "first build" of
OW1.9) can't find the C library... but I can't see any obvious reason
why. Is there some environment variable I need to set before running
build.sh?

Thanks,
Phil.

[Rugxulo]

Hi,

Post by Philip Pemberton
Hi,
Is there some secret to building OpenWatcom 1.9 on Linux/x86_64?
I'm running Ubuntu 10.10 beta on an AMD MV40 system (x86_64), with gcc in
* replace '#!/bin/sh' with '#!/bin/bash' in build.sh
  (to resolve the 'setvars.sh' issue)

I'm a Linux noob, but even then, I heard that Ubuntu replaced Bash as
default with the smaller Dash (mostly POSIX compliant shell) a while
back. I guess they were trying to root out non-portable Bash-isms.
Yeah, a lot of people complained, but I think they were too strict to
care. At the risk of a bad suggestion, consider reading "man chsh".

As for the 32-bit stuff, that's probably par for the course. Since OW
doesn't support 64-bit, they pretty much assume you're only using 32-
bit anyways. (I guess 64-bit host isn't as popular among the main
developers.)

BTW, I recently read someone (Jiri?) say that bootstrapping from Linux
with GCC isn't tested as much, if at all, anymore. It's recommended to
use the previous OW build to compile the new one (in theory).

P.S. Say hi to Rod and Steven for me! (just kidding) ;-)

[Philip Pemberton]

Post by Rugxulo
I'm a Linux noob, but even then, I heard that Ubuntu replaced Bash as
default with the smaller Dash (mostly POSIX compliant shell) a while
back. I guess they were trying to root out non-portable Bash-isms. Yeah,
a lot of people complained, but I think they were too strict to care. At
the risk of a bad suggestion, consider reading "man chsh".

***@cheetah:~$ chsh
Password:
Changing the login shell for philpem
Enter the new value, or press ENTER for the default
Login Shell [/bin/bash]:

Mmmm, nope. But /bin/sh is a symlink to dash:

***@cheetah:~$ file /bin/sh
/bin/sh: symbolic link to `dash'

Strictly speaking though, those shell scripts should reference Bash in
the hashbang.

Post by Rugxulo
As for the 32-bit stuff, that's probably par for the course. Since OW
doesn't support 64-bit, they pretty much assume you're only using 32-
bit anyways. (I guess 64-bit host isn't as popular among the main
developers.)

Well, the problem is that I can't even get the binary installer to run...
I get a floating-point exception and the whole thing falls over.

All I want is a toolchain that I can run on a linux32 or linux64 system,
which can be made to produce 16-bit DOS EXE files... Watcom is the best
of the bunch in terms of feature set ('#pragma aux' is a BIG bonus), but
is a massive pain in the backside to build.

Post by Rugxulo
BTW, I recently read someone (Jiri?) say that bootstrapping from Linux
with GCC isn't tested as much, if at all, anymore. It's recommended to
use the previous OW build to compile the new one (in theory).

I can't even get the old OW to install, so I'm stuck in a chicken-and-egg
situation. Specifically, I need Watcom version [n-1] to build Watcom
version [n]...

Post by Rugxulo
P.S. Say hi to Rod and Steven for me! (just kidding) ;-)

Rod and Steven?

[Peter C. Chapin]

Post by Philip Pemberton
Well, the problem is that I can't even get the binary installer to run...
I get a floating-point exception and the whole thing falls over.

That's interesting. You say this is on Ubuntu 10.10 beta? The installer
has worked for me on several other Linux systems... but I'm not saying
it's perfect. You might try downloading the latest build of the
installer from Jiri's build server here:

http://owbuilder.malakovi.cz/

However when I just tried to access it, my browser timed out. Jiri... is
your build server running these days?

Peter

[Philip Pemberton]

Post by Peter C. Chapin
That's interesting. You say this is on Ubuntu 10.10 beta? The installer
has worked for me on several other Linux systems... but I'm not saying
it's perfect.

Yup, Ubuntu 10.10 Beta, x86_64 version.

Post by Peter C. Chapin
You might try downloading the latest build of the
http://owbuilder.malakovi.cz/
However when I just tried to access it, my browser timed out.

Same here -- the other buildserver isn't doing any better... :(

I've also tried building from the daily tarball, but got an altogether
different set of errors :-/

Thanks,

[Rugxulo]

Hi,

Post by Philip Pemberton
/bin/sh: symbolic link to `dash'
Strictly speaking though, those shell scripts should reference Bash in
the hashbang.

Then just change your symlink for the time being.

Post by Philip Pemberton
Well, the problem is that I can't even get the binary installer to run...
I get a floating-point exception and the whole thing falls over.

Can you post here the text screenshot of that?

Post by Philip Pemberton
All I want is a toolchain that I can run on a linux32 or linux64 system,
which can be made to produce 16-bit DOS EXE files... Watcom is the best
of the bunch in terms of feature set ('#pragma aux' is a BIG bonus), but
is a massive pain in the backside to build.

Dumb question, but what kind of 16-bit .EXEs are you trying to build
and why? I'm sure me and other FreeDOSers could help point you in the
right direction. But yeah, OpenWatcom is probably the best "open"
solution for 16-bit C or C++ apps (assuming you can live with the
OSI / Debian license feud).

Post by Philip Pemberton
I can't even get the old OW to install, so I'm stuck in a chicken-and-egg
situation. Specifically, I need Watcom version [n-1] to build Watcom
version [n]...

You could always run the DOS-hosted version in DOSEMU (or maybe
DOSBox, though that's slower, less convenient, and meant only for
games).

Post by Philip Pemberton

P.S. Say hi to Rod and Steven for me! (just kidding)   ;-)

Rod and Steven?

Just my little dumb joke. Rod Pemberton (coder extraordinaire)
frequents here sometimes. Steven Pemberton is the Pascal P4 book
documentation dude. ;-)

[Philip Pemberton]

Post by Rugxulo

Post by Philip Pemberton
Strictly speaking though, those shell scripts should reference Bash in
the hashbang.

Then just change your symlink for the time being.

... which has the potential to break my boot scripts. Unbootable system
or broken compiler... hmm, that's a tough one.

Post by Rugxulo

Post by Philip Pemberton
Well, the problem is that I can't even get the binary installer to
run... I get a floating-point exception and the whole thing falls over.

Can you post here the text screenshot of that?

***@ryoko:~/openwatcom$ wget http://openwatcom.mirror.fr/open-watcom-
c-linux-1.9
--2010-10-01 12:58:08-- http://openwatcom.mirror.fr/open-watcom-c-
linux-1.9
Resolving openwatcom.mirror.fr... 195.20.15.70
Connecting to openwatcom.mirror.fr|195.20.15.70|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83959748 (80M) [application/octet-stream]
Saving to: `open-watcom-c-linux-1.9'

100%[======================================>] 83,959,748 1.12M/s in
75s

2010-10-01 12:59:23 (1.07 MB/s) - `open-watcom-c-linux-1.9' saved
[83959748/83959748]

***@ryoko:~/openwatcom$ file open-watcom-c-linux-1.9
open-watcom-c-linux-1.9: ELF 32-bit LSB executable, Intel 80386, version
1 (GNU/Linux), statically linked, not stripped
***@ryoko:~/openwatcom$ md5sum open-watcom-c-linux-1.9
960fe6b5cf88769a42949f5fedf62827 open-watcom-c-linux-1.9
***@ryoko:~/openwatcom$ chmod +x open-watcom-c-linux-1.9
***@ryoko:~/openwatcom$ ./open-watcom-c-linux-1.9
Floating point exception (core dumped)

The MD5sum matches the one shown on mirror.fr and HEAnet...

Post by Rugxulo
Dumb question, but what kind of 16-bit .EXEs are you trying to build and
why?

Psion SIBO IMG executables. Small model (64k code, 64k data); you
basically create a .EXE and use a Psion tool to turn that into a .APP
file.

Post by Rugxulo
I'm sure me and other FreeDOSers could help point you in the right
direction. But yeah, OpenWatcom is probably the best "open" solution for
16-bit C or C++ apps (assuming you can live with the OSI / Debian
license feud).

Debian are always feuding with someone...

Post by Rugxulo
You could always run the DOS-hosted version in DOSEMU (or maybe DOSBox,
though that's slower, less convenient, and meant only for games).

That sounds like a plan.. will DOS/4GW apps like Watcom run under DOSEMU?

It would be even better if I could get my sticky mitts on a copy of the
Clarion TopSpeed C compiler, which is what Psion used -- unfortunately
trying to find a copy of that is like trying to find a bone needle in a
5.973x10^24kg haystack...

Thanks,
Phil.

[Philip Pemberton]

Post by Philip Pemberton
exception (core dumped)

Turns out this is because OW can't find the Terminfo files:

read(3, "\21table of contents\10ApplIcon\0\1 \6H"..., 74) = 74
lseek(3, 375058, SEEK_SET) = 375058
read(3, "\21table of contents\10ApplIcon\0\1 \6H"..., 74) = 74
open("/dev/tty", O_RDWR|O_CREAT|O_TRUNC, 0666) = 4
ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon
echo ...}) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
access("/home/philpem/.terminfo/x/xterm", R_OK) = -1 ENOENT (No such file
or di)
access("/usr/share/terminfo/x/xterm", R_OK) = -1 ENOENT (No such file or
direct)
--- SIGFPE (Floating point exception) @ 0 (0) ---
+++ killed by SIGFPE (core dumped) +++
Floating point exception (core dumped)

Linking /usr/share/terminfo/x to /lib/terminfo/x seems to fix it:
$ sudo ln -s /lib/terminfo/x /usr/share/terminfo
$ sudo ./open-watcom-c-linux-1.9

... and the installer works as expected.

Hmm.

[Rugxulo]

Hi,

Post by Philip Pemberton

Post by Rugxulo
Dumb question, but what kind of 16-bit .EXEs are you trying to build and
why?

Psion SIBO IMG executables. Small model (64k code, 64k data); you
basically create a .EXE and use a Psion tool to turn that into a .APP
file.

Even BCC/Dev86 can do small model, though you'd need COM to EXE, I
guess. But what is a Psion? Handheld?

http://en.wikipedia.org/wiki/Psion_Series_3

Is that it???

Post by Philip Pemberton

Post by Rugxulo
You could always run the DOS-hosted version in DOSEMU (or maybe DOSBox,
though that's slower, less convenient, and meant only for games).

That sounds like a plan.. will DOS/4GW apps like Watcom run under DOSEMU?

Yes. At risk of exaggerating, DOSEMU runs almost anything. Their site
has pre-built binaries (and FreeDOS). In Ubuntu (and Debian) it's
available only in multiverse, though. (Again, hate to complain, but
they take issue with OW's license, which they somehow hold against
FreeDOS proper, ugh. *shakes fist*)

Post by Philip Pemberton
It would be even better if I could get my sticky mitts on a copy of the
Clarion TopSpeed C compiler, which is what Psion used -- unfortunately
trying to find a copy of that is like trying to find a bone needle in a
5.973x10^24kg haystack...

The only real problem is what the converter accepts. Everything else
can be worked around easily.

[Philip Pemberton]

Post by Rugxulo
Even BCC/Dev86 can do small model, though you'd need COM to EXE, I
guess. But what is a Psion? Handheld?

The machine I'm working on is a Psion Workabout. Basically a Psion Series
3 machine with a barcode scanner attachment, backlit screen and a built-
in NiCad battery charger.

The problem with bcc / dev86 is that it's basically unmaintained (the
original author's site is long gone), and there's bugger all
documentation. It doesn't do register allocation, pragma-aux, or any of
the fun stuff Watcom can do.

Put it this way: to make bcc work, I'd need to write a full shim library
in as86 assembler. To do the same with Watcom, I "only" have to hack
Psion's header files about and add a bunch of #pragma-aux statements and
a few tiny slivers of ASM ("int XXh" a few dozen times).

Going from .com to .img is a pretty poor idea. You'd lose 256 bytes off
the start of the exec, and if memory serves everything's shoved in one
segment, so that's 64k total for code and data, not 64k for each. Not
good for apps which make use of large amounts of in-memory data.

Post by Rugxulo
The only real problem is what the converter accepts. Everything else can
be worked around easily.

There's an 'a.out'-to-IMG converter already; frankly I'd rather go EXE ->
IMG, it's not like MZ-EXEs are hard to read and decode.

[Rugxulo]

Hi,

Post by Philip Pemberton
The machine I'm working on is a Psion Workabout. Basically a Psion Series
3 machine with a barcode scanner attachment, backlit screen and a built-
in NiCad battery charger.
The problem with bcc / dev86 is that it's basically unmaintained (the
original author's site is long gone), and there's bugger all
documentation. It doesn't do register allocation, pragma-aux, or any of
the fun stuff Watcom can do.

It's not that bad, though, if you can get it working. Not saying it
matters here, though, and I'm honestly not sure what format it uses
(a.out or as86?).

Post by Philip Pemberton
Put it this way: to make bcc work, I'd need to write a full shim library
in as86 assembler. To do the same with Watcom, I "only" have to hack
Psion's header files about and add a bunch of #pragma-aux statements and
a few tiny slivers of ASM ("int XXh" a few dozen times).

Well, yeah, if you want C, go with OpenWatcom, assuming your converter
works.

Post by Philip Pemberton
Going from .com to .img is a pretty poor idea. You'd lose 256 bytes off
the start of the exec,

Not much of a loss. ;-)

Post by Philip Pemberton
and if memory serves everything's shoved in one
segment, so that's 64k total for code and data, not 64k for each.

BCC/Dev86 supports both tiny (64k) and small (64k + 64k) models but
nothing else, at least not directly. OpenWatcom of course also
supports compact, medium, and huge. You can of course do anything in
assembly, if you're keen on that.

Post by Philip Pemberton
Not good for apps which make use of large amounts of in-memory data.

"Large amounts", heh. Apparently your machine has 2 MB (wow!!).
^_^ In other words, it depends on what you're trying to actually
do! But you seem far more savvy than me, so I'll leave you to it. But
feel free to tell me more juicy details! ;-)

BTW, I guess (from reading your site) that this means you're more
comfortable in C than in OPL.

Post by Philip Pemberton

Post by Rugxulo
The only real problem is what the converter accepts. Everything else can
be worked around easily.

There's an 'a.out'-to-IMG converter already; frankly I'd rather go EXE ->
IMG, it's not like MZ-EXEs are hard to read and decode.

a.out is a very simple format. MZ can actually get pretty hairy (I
think??), but I'm far from experienced in either.

Good luck! (If only you'd gotten a PoqetPC instead, heh, at least then
it wouldn't be some oddball OS.)

[Captain Pugwash]

Post by Rugxulo
Just my little dumb joke. Rod Pemberton (coder extraordinaire)
frequents here sometimes. Steven Pemberton is the Pascal P4 book
documentation dude. ;-)

And who is Philip Pemberton. And are they really triplets? :-) :-) :-)