using the HMA under OS/2 - how to get more than 512MB of memory

[Elmar Stellnberger]

How can I make use of the HMA (High Memory Area) under OS/2 with the
Open Watcom C compiler? At the moment it allocates all segments in the
LMA (the memory region below 512MB): Heap, Stack, Code, Data. However a
DosQueryMem reveals that Open Watcom does an automatic allocation in the
HMA as soon as I use the standard C libraries.
* How can I do a hmalloc to get more than 100MB of heap memory
* How can I disable usage of the HMA by the standard C Library to make
use of the HMA on my own via DosAllocMem.

[Steven Levine]

On Sun, 29 Sep 2013 10:22:00 UTC, Elmar Stellnberger
<***@elstel.org> wrote:

Hi Elmar,

Post by Elmar Stellnberger
How can I make use of the HMA (High Memory Area) under OS/2 with the
Open Watcom C compiler?

One option, as I've mentioned elsewhere is for me to get sufficient
free time or sufficient incentive to finish the code for this.

Post by Elmar Stellnberger
At the moment it allocates all segments in the
LMA (the memory region below 512MB): Heap, Stack, Code, Data. However a
DosQueryMem reveals that Open Watcom does an automatic allocation in the
HMA as soon as I use the standard C libraries.

I see no evidence of this in the OpenWatcom code. that implements the
OpenWatcom standard C library. Search for OBJ_ANY in the sources.

Perhaps you mean some other standard C library. Which libary are you
referring to?

Post by Elmar Stellnberger
* How can I do a hmalloc to get more than 100MB of heap memory

What hmalloc do yo mean? AFAIK, OpenWatcom does not implement
hmalloc.

Steven

[Elmar Stellnberger]

Hi Steven,

I think I have erred in thinking that Open Watcom already does a
preallocation of the HMA. I have now tested to allocate 1GB high and
malloc at the same time with clib3r and it works in deed.

Post by Steven Levine
One option, as I've mentioned elsewhere is for me to get sufficient
free time or sufficient incentive to finish the code for this.

Well it will just fine for me to use DosSubSetMem on HMA and allocate
with DosSubAllocMem; I can write my own malloc with DosSubAllocMem
putting the length of the data area into the first word of the new
allocation item (i.e. with offset -4 for an ulong). Then I just will
have to achieve malloc of stdlib to be renamed somehow into lmalloc.
Well I could make a #define malloc(x) hmalloc(x) but that would hide the
clib3r malloc entirely. Or is there any other advantage of the C memory
allocation over OS/2s native Suballocation?

It would be more exciting to also put the Stack into the HMA as this
could resolve some of the memory contentions using to be discovered with
OS/2 programs. I think it should be no big deal for me to provide you
with such a program at least if it can do without Open Watcoms C stdlib
clib3r (Well, Steve if you could then approach clib3r integration in a
free minute that would be great!).

Post by Steven Levine
I see no evidence of this in the OpenWatcom code. that implements the
OpenWatcom standard C library. Search for OBJ_ANY in the sources.

The reason why I had been thinking that the HMA is already allocated was
that QueryMem an utility program of mine shows the HMA to be already
allocated compiling with Open Watcom.

total physical memory: 1.10G
residential memory (locked by device drivers): 54.3M
total memory available for all processes: 1.8G
maximum low private memory for this process: 303.5M
maximal available low shared memory: 240.0M
virtual address limit: 3.0G

lmaMax=28.3M, lmaSum=158.8M, lmaWhole=302.8M
start=00000000, end=20000000, pagesize=1000
whole=64.0K
privatearena=1.2M (free: 0%, commited:24%), freearea=64.0K
whole=302.7M
sharedarena=207.9M (free:17%, commited: 8%), freearea=128.0K

hmaMax=2.5G, hmaSum=2.5G, hmaWhole=0.0
start=20000000, end=c0000000, pagesize=1000
privatearena=2.5G (free: 0%, commited: 0%), freearea=2.0M

The program should among others be available soon for download under
http://www.elstel.org/OS2Warp/os2utils.html. However it could require
some more patching and testing ;) before.

Elmar

[Steve Fabian]

Elmar Stellnberger wrote:
...
| total physical memory: 1.10G
| residential memory (locked by device drivers): 54.3M
| total memory available for all processes: 1.8G
| maximum low private memory for this process: 303.5M
| maximal available low shared memory: 240.0M
| virtual address limit: 3.0G

The memory sizes seem strange. IF they are from a real system, the first
value seems to indicate a weird combination of multiple device sizes. Even
stranger is that with "1.1GB physical memory" one has an additional 700MB
more available for processes (1.8GB). Of course, I made the assumption that
the unreported measurement UNITS, modified by the reported SI unit PREFIXES,
are the same (B: byte) for each, a most likely asumption, but not one to
depend on.

If the report is not from actual execution of the program, please disregard
these comments...

[Elmar Stellnberger]

Post by Steve Fabian
...
| total physical memory: 1.10G
| residential memory (locked by device drivers): 54.3M
| total memory available for all processes: 1.8G
| maximum low private memory for this process: 303.5M
| maximal available low shared memory: 240.0M
| virtual address limit: 3.0G
The memory sizes seem strange. IF they are from a real system, the first
value seems to indicate a weird combination of multiple device sizes. Even
stranger is that with "1.1GB physical memory" one has an additional 700MB
more available for processes (1.8GB). Of course, I made the assumption that
the unreported measurement UNITS, modified by the reported SI unit PREFIXES,
are the same (B: byte) for each, a most likely asumption, but not one to
depend on.
If the report is not from actual execution of the program, please disregard
these comments...

Yes, the output is from QueryMem/DosQuerySysInfo and it shows a wrong
total physical memory with kernel IBM:14.104a#@_UNI. My physically
installed memory amounts to 2GB.
Also strange seem the results from QueryMem especially in the HMA: It
says 0%Free 100%allocated 0%commited in the HMA though I can happily do
there a DosAllocMem. I also wonder why it displays some pages as whole
i.e. returning an error on DosQueryMem rather than free. All other
parameters should be displayed right, I guess.

Steve, what does the program say at you?

[Steve Fabian]

Elmar Stellnberger wrote:
...
| Steve Fabian wrote:
|| Elmar Stellnberger wrote:
||| total physical memory: 1.10G
||| residential memory (locked by device drivers): 54.3M
||| total memory available for all processes: 1.8G
||| maximum low private memory for this process: 303.5M
||| maximal available low shared memory: 240.0M
||| virtual address limit: 3.0G
||
|| The memory sizes seem strange.
|| ...
|| If the report is not from actual execution of the program, please
|| disregard these comments...
|
| Yes, the output is from QueryMem/DosQuerySysInfo and it shows a
| wrong total physical memory with kernel IBM:14.104a#@_UNI. My
| physically installed memory amounts to 2GB.
| Also strange seem the results from QueryMem especially in the HMA:
| It says 0%Free 100%allocated 0%commited in the HMA though I can
| happily do there a DosAllocMem. I also wonder why it displays some
| pages as whole i.e. returning an error on DosQueryMem rather than
| free. All other parameters should be displayed right, I guess.
|
| Steve, what does the program say at you?

Sorry, I do not have OS/2. I use WinXP and Win7; my command processor is
TCC, the current Windows version from JP Software, Inc. ("JPsoft"). Its
command "memory" (probably available in 4OS2, which is now a freeware
replacement/alternative for OS2's built-in command processor CMD) reported
in another concurrent window as follows (I believe the terminology is from
MS):

-----------------------------------------------------------

37 % Memory load

2,138,427,392 bytes total physical RAM
1,338,388,480 bytes available physical RAM

4,123,287,552 bytes total page file
3,328,802,816 bytes available page file

2,147,352,576 bytes total virtual RAM
1,920,540,672 bytes available virtual RAM

262,144 characters total alias
192,552 characters free

131,072 characters total function
121,671 characters free

133,120 characters total history
-----------------------------------------------------------

Of course, for the WinNT line of OS-s (which require at least an 80386 or
higher processor, above the old 8086 with its 20-bit address bus, resulting
in the PC-DOS design with the old 640kB program size boundary and the 1MB
limit on directly addressable memory) the concept of HMA no longer exists.

[Elmar Stellnberger]

believe me, I have my reasons why I don`t use Windows or MacOS.

Post by Steve Fabian
...
||| total physical memory: 1.10G
||| residential memory (locked by device drivers): 54.3M
||| total memory available for all processes: 1.8G
||| maximum low private memory for this process: 303.5M
||| maximal available low shared memory: 240.0M
||| virtual address limit: 3.0G
||
|| The memory sizes seem strange.
|| ...
|| If the report is not from actual execution of the program, please
|| disregard these comments...
|
| Yes, the output is from QueryMem/DosQuerySysInfo and it shows a
| physically installed memory amounts to 2GB.
| It says 0%Free 100%allocated 0%commited in the HMA though I can
| happily do there a DosAllocMem. I also wonder why it displays some
| pages as whole i.e. returning an error on DosQueryMem rather than
| free. All other parameters should be displayed right, I guess.
|
| Steve, what does the program say at you?
Sorry, I do not have OS/2. I use WinXP and Win7; my command processor is
TCC, the current Windows version from JP Software, Inc. ("JPsoft"). Its
command "memory" (probably available in 4OS2, which is now a freeware
replacement/alternative for OS2's built-in command processor CMD) reported
in another concurrent window as follows (I believe the terminology is from
-----------------------------------------------------------
37 % Memory load
2,138,427,392 bytes total physical RAM
1,338,388,480 bytes available physical RAM
4,123,287,552 bytes total page file
3,328,802,816 bytes available page file
2,147,352,576 bytes total virtual RAM
1,920,540,672 bytes available virtual RAM
262,144 characters total alias
192,552 characters free
131,072 characters total function
121,671 characters free
133,120 characters total history
-----------------------------------------------------------
Of course, for the WinNT line of OS-s (which require at least an 80386 or
higher processor, above the old 8086 with its 20-bit address bus, resulting
in the PC-DOS design with the old 640kB program size boundary and the 1MB
limit on directly addressable memory) the concept of HMA no longer exists.

[Steve Fabian]

Elmar Stellnberger wrote:
| believe me, I have my reasons why I don`t use Windows or MacOS.

I have no issue with what you use regardless of your reasons, as I hope you
have no issue with what I use. As they said in ancient Rome: "De gustibus
non est disputandum" (tastes are not to be disputed).

I use MS Windows because it is bundled with the hardware (unless I build my
own system - good luck building a laptop from parts!), and most of my old
software and all of my data are compatible with it, and it does nearly
everything I want.