Discussion:
How do I explain REXX Storage Usage
(too old to reply)
Lizette Koehler
2016-06-06 12:58:43 UTC
Permalink
Raw Message
I have the need to provide a simple explanation as to why sometimes when running
a REXX with REGION=0M or REGION=128M in batch it fails. And other times it is
okay.
I am unable to post the code, but am willing to answer any questions about it.


The batch job runs IKJEFT01

The process reads an input file that could have from between 5 input lines and
100's or 1000's of input lines.

An EXECIO is used to read the entire Input file. Then the process does a DO i=1
to src.0 looking for a header record.

Once the header record is found they spin the data to an output file until the
next header record is found.

Occasionally they will get an error message stating: Machine storage is
exhausted.

There are no new stack/del stack process in this code. A lot of Signal
statements are used.


This is trying to take on large input file and split out all of the reports in
it into separate files. So long as there are not many input lines, it works.

I would like to provide a simple explanation as to why it does not work some of
the time.

I will also be working with my users and using a different process that will be
more successful. But for now, I need to explain why this is a coding technique
challenge.

Thanks.

Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Dyck, Lionel B. , TRA
2016-06-06 13:07:33 UTC
Permalink
Raw Message
This is basic but hopefully will help

1. Each line that is read in via execio into a stem takes storage. A few records takes less storage than a lot of records.
2. Assuming you are copying from the src.x stem into another stem for output then that consumes additional storage

Hope this helps


--------------------------------------------------------------------------
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10)
VA OI&T Service Delivery & Engineering

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
To: TSO-***@VM.MARIST.EDU
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage

I have the need to provide a simple explanation as to why sometimes when running a REXX with REGION=0M or REGION=128M in batch it fails. And other times it is okay.
I am unable to post the code, but am willing to answer any questions about it.


The batch job runs IKJEFT01

The process reads an input file that could have from between 5 input lines and 100's or 1000's of input lines.

An EXECIO is used to read the entire Input file. Then the process does a DO i=1 to src.0 looking for a header record.

Once the header record is found they spin the data to an output file until the next header record is found.

Occasionally they will get an error message stating: Machine storage is exhausted.

There are no new stack/del stack process in this code. A lot of Signal statements are used.


This is trying to take on large input file and split out all of the reports in it into separate files. So long as there are not many input lines, it works.

I would like to provide a simple explanation as to why it does not work some of the time.

I will also be working with my users and using a different process that will be more successful. But for now, I need to explain why this is a coding technique challenge.

Thanks.

Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Itschak Mugzach
2016-06-06 13:11:53 UTC
Permalink
Raw Message
Which command caused this msg? I usually see it at exec io. You may Use drop scr.i after processing the record to free up storage.

Best.
ITschak

נשלח מה-iPad שלי
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage. A few records takes less storage than a lot of records.
2. Assuming you are copying from the src.x stem into another stem for output then that consumes additional storage
Hope this helps
--------------------------------------------------------------------------
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10)
VA OI&T Service Delivery & Engineering
-----Original Message-----
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when running a REXX with REGION=0M or REGION=128M in batch it fails. And other times it is okay.
I am unable to post the code, but am willing to answer any questions about it.
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines and 100's or 1000's of input lines.
An EXECIO is used to read the entire Input file. Then the process does a DO i=1 to src.0 looking for a header record.
Once the header record is found they spin the data to an output file until the next header record is found.
Occasionally they will get an error message stating: Machine storage is exhausted.
There are no new stack/del stack process in this code. A lot of Signal statements are used.
This is trying to take on large input file and split out all of the reports in it into separate files. So long as there are not many input lines, it works.
I would like to provide a simple explanation as to why it does not work some of the time.
I will also be working with my users and using a different process that will be more successful. But for now, I need to explain why this is a coding technique challenge.
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately
----------------------------------------------------------------------
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
richard rozentals
2016-06-06 13:40:00 UTC
Permalink
Raw Message
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by using the STACK instead of a stem variable. My testing found that the STACK is stored in the spool. So you can read 100s of thousands of records . I suspect this may be the signal. I've written a lot of REXX code though the years and only had this problem when it was the execio to a stem variable. That problem was when I was reading a file with over 100,000 records. I do not use the signal as a GOTO. I assume they do. It may be that the signal is not internally written as GOTO. It may be that signal uses a lot of storage based on the number of times it's used.This is just a guess.

Can you reproduce this at will? Does just a simple rerun with the same data work? Does using different data clear the problem? Does the storage used on the batch job stats change based on the data read, not just number of records? More interested in the failure with REGION=128M.

Richard Rozentals


From: Itschak Mugzach <***@GMAIL.COM>
To: TSO-***@VM.MARIST.EDU
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage

Which command caused this msg? I usually see it at exec io. You may Use drop scr.i after processing the record to free up storage.

Best.
ITschak

נשלח מה-iPad שלי
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage.  A few records takes less storage than a lot of records.
2. Assuming you are copying from the src.x stem into another stem for output then that consumes additional storage
Hope this helps
--------------------------------------------------------------------------
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10)
VA OI&T Service Delivery & Engineering
-----Original Message-----
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when running a REXX with REGION=0M or REGION=128M in batch it fails.  And other times it is okay.
I am unable to post the code, but am willing to answer any questions about it.
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines and 100's or 1000's of input lines.
An EXECIO is used to read the entire Input file.  Then the process does a DO i=1 to src.0 looking for a header record.
Once the header record is found they spin the data to an output file until the next header record is found.
Occasionally they will get an error message stating:  Machine storage is exhausted.
There are no new stack/del stack process in this code.  A lot of Signal statements are used.
This is trying to take on large input file and split out all of the reports in it into separate files.  So long as there are not many input lines, it works.
I would like to provide a simple explanation as to why it does not work some of the time.
I will also be working with my users and using a different process that will be more successful.  But for now, I need to explain why this is a coding technique challenge.
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately
----------------------------------------------------------------------
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX



----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Hobart Spitz
2016-06-06 13:48:53 UTC
Permalink
Raw Message
If you have the PIPE command, use it.
Post by Itschak Mugzach
Which command caused this msg? I usually see it at exec io. You may Use
drop scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
כתב/ה:‬
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage. A
few records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem for
output then that consumes additional storage
Post by Dyck, Lionel B. , TRA
Hope this helps
--------------------------------------------------------------------------
Post by Dyck, Lionel B. , TRA
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10)
VA OI&T Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Post by Dyck, Lionel B. , TRA
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And other
times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions
about it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input
lines and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process does
a DO i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file
until the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the
reports in it into separate files. So long as there are not many input
lines, it works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work
some of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that
will be more successful. But for now, I need to explain why this is a
coding technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
Post by Dyck, Lionel B. , TRA
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
--
OREXXMan

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Paul Gilmartin
2016-06-06 15:00:37 UTC
Permalink
Raw Message
Post by Itschak Mugzach
Which command caused this msg? I usually see it at exec io. You may Use drop scr.i after processing the record to free up storage.
No. When last I researched it, DROPping individual members of
a compound symbol did not free the storage; only DROPping the
entire compound (DROP scr.) did that. I have a contrived test
in which DROPping individual members of a compound actually
increases the storage requirement. I don't know whether exiting
a PROCEDURE with a non-EXPOSEd compound is equivalent to DROP.
I should hope so.

Mark Zelden has a tool (thanks!) that nicely reports storage
usage.

I had a colleague who once naively misused SIGNAL repeatedly
to exit a nest of CALLs. Worked fine on simple test cases
With a larger data set it failed with call/return stack
exhausted. Rexx desperately needs longjump, or at least
longITERATE/longLEAVE.

-- gil

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Sevetson, Phil
2016-06-06 14:17:06 UTC
Permalink
Raw Message
Lizette,

I was told by my systems programmer that requests for memory are fulfilled on an as-available basis. The implication is that you might not get all the region size which you request. I'm unable to verify this independently -- I don't know how to read dumps and have no systems programming background.

That said, one of your options is to EXECIO 1 DISKW to write out your output as it occurs. It will definitely be quite a bit slower than putting all your output to stack-variables and writing it all when done... but it won't take a lot of memory, and won't blow up if your input suddenly decides to grow huge.

Or, if your developers are willing to write the rather-more-complex code, you could do a tradeoff and write a batch every hundredth record (EXECIO 100 DISKW), or even every "n-th" record (EXECIO chunksize DISKW), setting a variable. That would eliminate 99% of the duplicate EXECIO processing and still not take more than a few megabytes of storage.

--Phil Sevetson
***@fisa.nyc.gov
City of New York / Financial Info Svcs Agcy
New York, NY USA 10001


-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 8:59 AM
To: TSO-***@VM.MARIST.EDU
Subject: [TSO-REXX] How do I explain REXX Storage Usage

I have the need to provide a simple explanation as to why sometimes when running a REXX with REGION=0M or REGION=128M in batch it fails. And other times it is okay.
I am unable to post the code, but am willing to answer any questions about it.


The batch job runs IKJEFT01

The process reads an input file that could have from between 5 input lines and 100's or 1000's of input lines.

An EXECIO is used to read the entire Input file. Then the process does a DO i=1 to src.0 looking for a header record.

Once the header record is found they spin the data to an output file until the next header record is found.

Occasionally they will get an error message stating: Machine storage is exhausted.

There are no new stack/del stack process in this code. A lot of Signal statements are used.


This is trying to take on large input file and split out all of the reports in it into separate files. So long as there are not many input lines, it works.

I would like to provide a simple explanation as to why it does not work some of the time.

I will also be working with my users and using a different process that will be more successful. But for now, I need to explain why this is a coding technique challenge.

Thanks.

Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Lizette Koehler
2016-06-06 14:23:55 UTC
Permalink
Raw Message
So with regard to SIGNAL statements.

There are 7 SIGNAL ON coded in the REXX. Each is SIGNAL ON NAME routine, then some code, EXIT.

The use it to determine
O Did the dataset get allocated?
O Did the file get written to okay?
O Did the file close okay?
O Did the file delete okay?

So might this process be part of the issue?

I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...

Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX Storage
Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by using the
STACK instead of a stem variable. My testing found that the STACK is stored in
the spool. So you can read 100s of thousands of records . I suspect this may
be the signal. I've written a lot of REXX code though the years and only had
this problem when it was the execio to a stem variable. That problem was when
I was reading a file with over 100,000 records. I do not use the signal as a
GOTO. I assume they do. It may be that the signal is not internally written as
GOTO. It may be that signal uses a lot of storage based on the number of times
it's used.This is just a guess.
Can you reproduce this at will? Does just a simple rerun with the same data
work? Does using different data clear the problem? Does the storage used on
the batch job stats change based on the data read, not just number of records?
More interested in the failure with REGION=128M.
Richard Rozentals
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX Storage
Usage
Which command caused this msg? I usually see it at exec io. You may Use drop
scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage. A few
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem for
output then that consumes additional storage
Hope this helps
----------------------------------------------------------------------
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA OI&T
Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And other
times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions about
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file until
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines, it
works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work some
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a coding
technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Roach, Dennis
2016-06-06 14:40:42 UTC
Permalink
Raw Message
Here is what I use for looping through large files
Buffer_amt = # records to read at a time
"execio" buffer_amt "diskr input (stem in."
do while in.0 > 0
do i = 1 to in.0
<process record>
end
drop in.
"execio" buffer_amt "diskr input (stem in."
end

Dennis Roach, CISSP, PMP
AIG
IAM Access Administration – Consumer | Identy & Access Management

2929 Allen Parkway, America Building, 3rd Floor | Houston, TX 77019
Phone: 713-831-8799

***@AIG.com | www.aig.com

All opinions expressed by me are mine and may not agree with my employer or any person, company, or thing, living or dead, on or near this or any other planet, moon, asteroid, or other spatial object, natural or manufactured, since the beginning of time.

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 9:24 AM
To: TSO-***@VM.MARIST.EDU
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage

So with regard to SIGNAL statements.

There are 7 SIGNAL ON coded in the REXX. Each is SIGNAL ON NAME routine, then some code, EXIT.

The use it to determine
O Did the dataset get allocated?
O Did the file get written to okay?
O Did the file close okay?
O Did the file delete okay?

So might this process be part of the issue?

I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...

Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Behalf Of richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by
using the STACK instead of a stem variable. My testing found that the
STACK is stored in the spool. So you can read 100s of thousands of
records . I suspect this may be the signal. I've written a lot of REXX
code though the years and only had this problem when it was the execio
to a stem variable. That problem was when I was reading a file with
over 100,000 records. I do not use the signal as a GOTO. I assume they
do. It may be that the signal is not internally written as GOTO. It
may be that signal uses a lot of storage based on the number of times it's used.This is just a guess.
Can you reproduce this at will? Does just a simple rerun with the same
data work? Does using different data clear the problem? Does the
storage used on the batch job stats change based on the data read, not just number of records?
More interested in the failure with REGION=128M.
Richard Rozentals
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Which command caused this msg? I usually see it at exec io. You may
Use drop scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
‫ב-6 ביוני 2016, בשעה 16:07, ‏‏Dyck, Lionel B. (TRA)
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage.
A few
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem
for output then that consumes additional storage
Hope this helps
--------------------------------------------------------------------
--
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA
OI&T Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And
other times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions about
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file until
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines,
it works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work some
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a
coding technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Lizette Koehler
2016-06-06 14:46:54 UTC
Permalink
Raw Message
Dennis,

I have a similar snippet of REXX but it lacks the DROP Statement.

I will see if they will provide the file on the next failure so I can work with this concept.

Thanks everyone so far with these awesome suggestions.

Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Roach, Dennis
Sent: Monday, June 06, 2016 7:41 AM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage
Here is what I use for looping through large files Buffer_amt = # records to
read at a time
"execio" buffer_amt "diskr input (stem in."
do while in.0 > 0
do i = 1 to in.0
<process record>
end
drop in.
"execio" buffer_amt "diskr input (stem in."
end
Dennis Roach, CISSP, PMP
AIG
IAM Access Administration – Consumer | Identy & Access Management
2929 Allen Parkway, America Building, 3rd Floor | Houston, TX 77019
Phone: 713-831-8799
All opinions expressed by me are mine and may not agree with my employer or
any person, company, or thing, living or dead, on or near this or any other
planet, moon, asteroid, or other spatial object, natural or manufactured,
since the beginning of time.
-----Original Message-----
Sent: Monday, June 06, 2016 9:24 AM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage
So with regard to SIGNAL statements.
There are 7 SIGNAL ON coded in the REXX. Each is SIGNAL ON NAME routine, then
some code, EXIT.
The use it to determine
O Did the dataset get allocated?
O Did the file get written to okay?
O Did the file close okay?
O Did the file delete okay?
So might this process be part of the issue?
I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...
Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Behalf Of richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by
using the STACK instead of a stem variable. My testing found that the
STACK is stored in the spool. So you can read 100s of thousands of
records . I suspect this may be the signal. I've written a lot of REXX
code though the years and only had this problem when it was the execio
to a stem variable. That problem was when I was reading a file with
over 100,000 records. I do not use the signal as a GOTO. I assume they
do. It may be that the signal is not internally written as GOTO. It
may be that signal uses a lot of storage based on the number of times it's
used.This is just a guess.
Post by Dyck, Lionel B. , TRA
Can you reproduce this at will? Does just a simple rerun with the same
data work? Does using different data clear the problem? Does the
storage used on the batch job stats change based on the data read, not just
number of records?
Post by Dyck, Lionel B. , TRA
More interested in the failure with REGION=128M.
Richard Rozentals
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Which command caused this msg? I usually see it at exec io. You may
Use drop scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
‫ב-6 ביוני 2016, בשעה 16:07, ‏‏Dyck, Lionel B. (TRA)
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage.
A few
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem
for output then that consumes additional storage
Hope this helps
--------------------------------------------------------------------
--
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA
OI&T Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And
other times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions about
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file until
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines,
it works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work some
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a
coding technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Kopischke, David G.
2016-06-06 16:22:47 UTC
Permalink
Raw Message
I think the DROP is what's going to free memory.

If your EXECIO goes to a different stem for every report, it's going to use a lot of memory. Memory isn't freed. I believe the DROP addresses this to a certain extent, but I'm not sure it's fool proof.

Is there some reason the logic can't be streamed ??? You mentioned a header of some kind is a decision point, so can't you EXECIO 1 READ to a buffer and EXECIO 1 WRITE to a different file between headers ???




-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 8:47 AM
To: TSO-***@VM.MARIST.EDU
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage [ EXTERNAL ]

Dennis,

I have a similar snippet of REXX but it lacks the DROP Statement.

I will see if they will provide the file on the next failure so I can work with this concept.

Thanks everyone so far with these awesome suggestions.

Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Behalf Of Roach, Dennis
Sent: Monday, June 06, 2016 7:41 AM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage
Here is what I use for looping through large files Buffer_amt = #
records to read at a time "execio" buffer_amt "diskr input (stem in."
do while in.0 > 0
do i = 1 to in.0
<process record>
end
drop in.
"execio" buffer_amt "diskr input (stem in."
end
Dennis Roach, CISSP, PMP
AIG
IAM Access Administration – Consumer | Identy & Access Management
2929 Allen Parkway, America Building, 3rd Floor | Houston, TX 77019
Phone: 713-831-8799
All opinions expressed by me are mine and may not agree with my
employer or any person, company, or thing, living or dead, on or near
this or any other planet, moon, asteroid, or other spatial object,
natural or manufactured, since the beginning of time.
-----Original Message-----
Sent: Monday, June 06, 2016 9:24 AM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage
So with regard to SIGNAL statements.
There are 7 SIGNAL ON coded in the REXX. Each is SIGNAL ON NAME
routine, then some code, EXIT.
The use it to determine
O Did the dataset get allocated?
O Did the file get written to okay?
O Did the file close okay?
O Did the file delete okay?
So might this process be part of the issue?
I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...
Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Behalf Of richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by
using the STACK instead of a stem variable. My testing found that
the STACK is stored in the spool. So you can read 100s of thousands
of records . I suspect this may be the signal. I've written a lot of
REXX code though the years and only had this problem when it was the
execio to a stem variable. That problem was when I was reading a
file with over 100,000 records. I do not use the signal as a GOTO. I
assume they do. It may be that the signal is not internally written
as GOTO. It may be that signal uses a lot of storage based on the
number of times it's
used.This is just a guess.
Post by Dyck, Lionel B. , TRA
Can you reproduce this at will? Does just a simple rerun with the
same data work? Does using different data clear the problem? Does
the storage used on the batch job stats change based on the data
read, not just
number of records?
Post by Dyck, Lionel B. , TRA
More interested in the failure with REGION=128M.
Richard Rozentals
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage Usage
Which command caused this msg? I usually see it at exec io. You may
Use drop scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
‫ב-6 ביוני 2016, בשעה 16:07, ‏‏Dyck, Lionel B. (TRA)
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage.
A few
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem
for output then that consumes additional storage
Hope this helps
------------------------------------------------------------------
--
--
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA
OI&T Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why
sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And
other times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any
questions about
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file until
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input
lines, it works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work some
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a
coding technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX


This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
christel.u.w.pachl christel.u.w.pachl
2016-06-06 16:27:41 UTC
Permalink
Raw Message
BEWARE of EXECIO 1

It's slooooooooooooooooooooooooooooooooow

Walter

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
richard rozentals
2016-06-06 17:00:24 UTC
Permalink
Raw Message
Hi,
This is true. Each execio does a open,read and close. Vary slow reading a file.
That is another reason to use STACK for large files. 1 open and 1 close, full use of buffering. Then using the "get" for each record.
Richard Rozentals



From: christel.u.w.pachl christel.u.w.pachl <***@CHELLO.AT>
To: TSO-***@VM.MARIST.EDU
Sent: Monday, June 6, 2016 12:25 PM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage

BEWARE of EXECIO 1

It's slooooooooooooooooooooooooooooooooow

Walter

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX




----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Farley, Peter x23353
2016-06-06 16:57:59 UTC
Permalink
Raw Message
I use this technique all the time, with an argument to the EXEC to control how many to read at one time rather than a fixed amount I the EXEC, but with some reasonable default for the expected volume of records. For known small-volume files, I can then pass "*" as the argument to read all of them at once.

Also with appropriate validation of the argument value of course.

Peter

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of Roach, Dennis
Sent: Monday, June 06, 2016 10:41 AM
To: TSO-***@VM.MARIST.EDU
Subject: Re: How do I explain REXX Storage Usage

Here is what I use for looping through large files Buffer_amt = # records to read at a time
"execio" buffer_amt "diskr input (stem in."
do while in.0 > 0
do i = 1 to in.0
<process record>
end
drop in.
"execio" buffer_amt "diskr input (stem in."
end
--


This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
Anania, Philip D
2016-06-06 15:39:11 UTC
Permalink
Raw Message
If you write to the queue (stack) instead of a stem variable, then you will
free up storage for each pull. I've had to switch from one to the other on
occasion to get by this. If you are reading from an array and adding to an
array or the stack for your writes you'll end up with double the amount of
records read. My general rule of thumb - lots O' data - use the stack, not
lots O' data - use an array.
Post by Lizette Koehler
So with regard to SIGNAL statements.
There are 7 SIGNAL ON coded in the REXX. Each is SIGNAL ON NAME routine,
then some code, EXIT.
The use it to determine
O Did the dataset get allocated?
O Did the file get written to okay?
O Did the file close okay?
O Did the file delete okay?
So might this process be part of the issue?
I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...
Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Behalf Of
Post by Dyck, Lionel B. , TRA
richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage
Post by Dyck, Lionel B. , TRA
Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by using
the
Post by Dyck, Lionel B. , TRA
STACK instead of a stem variable. My testing found that the STACK is
stored in
Post by Dyck, Lionel B. , TRA
the spool. So you can read 100s of thousands of records . I suspect this
may
Post by Dyck, Lionel B. , TRA
be the signal. I've written a lot of REXX code though the years and only
had
Post by Dyck, Lionel B. , TRA
this problem when it was the execio to a stem variable. That problem was
when
Post by Dyck, Lionel B. , TRA
I was reading a file with over 100,000 records. I do not use the signal
as a
Post by Dyck, Lionel B. , TRA
GOTO. I assume they do. It may be that the signal is not internally
written as
Post by Dyck, Lionel B. , TRA
GOTO. It may be that signal uses a lot of storage based on the number of
times
Post by Dyck, Lionel B. , TRA
it's used.This is just a guess.
Can you reproduce this at will? Does just a simple rerun with the same
data
Post by Dyck, Lionel B. , TRA
work? Does using different data clear the problem? Does the storage used
on
Post by Dyck, Lionel B. , TRA
the batch job stats change based on the data read, not just number of
records?
Post by Dyck, Lionel B. , TRA
More interested in the failure with REGION=128M.
Richard Rozentals
Sent: Monday, June 6, 2016 9:11 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX
Storage
Post by Dyck, Lionel B. , TRA
Usage
Which command caused this msg? I usually see it at exec io. You may Use
drop
Post by Dyck, Lionel B. , TRA
scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
‫ב-6 ביוני 2016, בשעה 16:07, ‏‏Dyck, Lionel B. (TRA) ‏<
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage. A
few
Post by Dyck, Lionel B. , TRA
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem for
output then that consumes additional storage
Hope this helps
----------------------------------------------------------------------
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA OI&T
Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes
when
Post by Dyck, Lionel B. , TRA
running a REXX with REGION=0M or REGION=128M in batch it fails. And
other
Post by Dyck, Lionel B. , TRA
times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions
about
Post by Dyck, Lionel B. , TRA
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input
lines
Post by Dyck, Lionel B. , TRA
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file. Then the process
does a DO
Post by Dyck, Lionel B. , TRA
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file
until
Post by Dyck, Lionel B. , TRA
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating: Machine storage
is
Post by Dyck, Lionel B. , TRA
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the
reports
Post by Dyck, Lionel B. , TRA
in it into separate files. So long as there are not many input lines, it
works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not
work some
Post by Dyck, Lionel B. , TRA
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process
that will
Post by Dyck, Lionel B. , TRA
be more successful. But for now, I need to explain why this is a coding
technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
--
"Go about your Music citizen" - PdotDdot

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
richard rozentals
2016-06-06 15:41:47 UTC
Permalink
Raw Message
Hi Lizette,
Yes your approach is simpler and more efficient.  I was thinking if the signal was used for each record it would be a problem. I can't see 7 executions of signal being a issue.I can't see the execio being the cause. 1000 records is not that much.
I would ask your user for the listings of the failed and working runs of the rexx(I assume this is run in batch). Look for the pattern. If the file size or type of data does not increase the region used then it can't be that.

I also assume this is not a real big REXX EXEC. I found that if it get over 5 thousand lines it gets to be a problem.
You can also try compiling the REXX. Not to run it that way but to see how big it is compiled. 

I expect this is a issue of odd coding or environmental. Environmental issues may be harder to pinpoint. Make sure your user's CPU is not running close to 100% and has all the resources available. Could it be a limitation on the USERID?
RichardRozentals

From: Lizette Koehler <***@MINDSPRING.COM>
To: TSO-***@VM.MARIST.EDU
Sent: Monday, June 6, 2016 10:23 AM
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage

So with regard to SIGNAL statements.

There are 7 SIGNAL ON coded in the REXX.  Each is SIGNAL ON NAME routine, then some code,  EXIT.

The use it to determine
  O  Did the dataset get allocated?
  O  Did the file get written to okay?
  O  Did the file close okay?
  O  Did the file delete okay?

So might this process be part of the issue?

I would have used the RETURN CODES from EXECIO, ALLOC, DELETE, etc...

Lizette
Post by Dyck, Lionel B. , TRA
-----Original Message-----
richard rozentals
Sent: Monday, June 06, 2016 6:37 AM
Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX Storage
Usage
Hi Lizette,
I suspect that the EXECIO is not the cause. You can prove this by using the
STACK instead of a stem variable. My testing found that the STACK is stored in
the spool. So you can read 100s of thousands of records . I suspect this may
be the signal. I've written a lot of REXX code though the years and only had
this problem when it was the execio to a stem variable. That problem was when
I was reading a file with over 100,000 records. I do not use the signal as a
GOTO. I assume they do. It may be that the signal is not internally written as
GOTO. It may be that signal uses a lot of storage based on the number of times
it's used.This is just a guess.
Can you reproduce this at will? Does just a simple rerun with the same data
work? Does using different data clear the problem? Does the storage used on
the batch job stats change based on the data read, not just number of records?
More interested in the failure with REGION=128M.
Richard Rozentals
  Sent: Monday, June 6, 2016 9:11 AM
  Subject: Re: [TSO-REXX] [EXTERNAL] [TSO-REXX] How do I explain REXX Storage
Usage
Which command caused this msg? I usually see it at exec io. You may Use drop
scr.i after processing the record to free up storage.
Best.
ITschak
נשלח מה-iPad שלי
Post by Dyck, Lionel B. , TRA
This is basic but hopefully will help
1. Each line that is read in via execio into a stem takes storage.  A few
records takes less storage than a lot of records.
Post by Dyck, Lionel B. , TRA
2. Assuming you are copying from the src.x stem into another stem for
output then that consumes additional storage
Hope this helps
----------------------------------------------------------------------
----
Lionel B. Dyck (Contractor)
Mainframe Systems Programmer
Enterprise Infrastructure Support (Station 200) (005OP6.3.10) VA OI&T
Service Delivery & Engineering
-----Original Message-----
Behalf Of Lizette Koehler
Sent: Monday, June 06, 2016 7:59 AM
Subject: [EXTERNAL] [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails.  And other
times it is okay.
Post by Dyck, Lionel B. , TRA
I am unable to post the code, but am willing to answer any questions about
it.
Post by Dyck, Lionel B. , TRA
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
Post by Dyck, Lionel B. , TRA
An EXECIO is used to read the entire Input file.  Then the process does a DO
i=1 to src.0 looking for a header record.
Post by Dyck, Lionel B. , TRA
Once the header record is found they spin the data to an output file until
the next header record is found.
Post by Dyck, Lionel B. , TRA
Occasionally they will get an error message stating:  Machine storage is
exhausted.
Post by Dyck, Lionel B. , TRA
There are no new stack/del stack process in this code.  A lot of Signal
statements are used.
Post by Dyck, Lionel B. , TRA
This is trying to take on large input file and split out all of the reports
in it into separate files.  So long as there are not many input lines, it
works.
Post by Dyck, Lionel B. , TRA
I would like to provide a simple explanation as to why it does not work some
of the time.
Post by Dyck, Lionel B. , TRA
I will also be working with my users and using a different process that will
be more successful.  But for now, I need to explain why this is a coding
technique challenge.
Post by Dyck, Lionel B. , TRA
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX



----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
christel.u.w.pachl christel.u.w.pachl
2016-06-06 14:58:32 UTC
Permalink
Raw Message
Buffer the Input with something like this

Do While readrc=
'EXECIO DISKR 1000 (STEM X.
readrc=rc
Do i=1 To x.0
Call process x.i
End
End
process:
Parse Arg l
If ... Then Call o l
Return

o:
z=o.0+1
o.z=arg(1)
o.0=z
If o.0>=1000 Then Do
'EXECIO' DISKW o.0
o.0=0
End

To be completed and tested

Regards

Walter

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Robin
2016-06-06 17:11:21 UTC
Permalink
Raw Message
There is a trick to using EXECIO 1 that doesn't slow down processing too
severely.

Presuming the organization (RECFM, LRECL, etc) of input file is stable for
each execution of the REXX, you have the REXX program allocate to the file
as RECFM=U. Herein the EXECIO will read a physical block (with x number of
logical record) with 1 EXECIO. It is then the responsibility to 'deblock'
the physical file block of records. Handling 'short blocks' (e.g. last block
of the file)
Is also the responsibility of the REXX program. 'Deblocking' full physical
blocks is relatively simple for files where the true RECFM=FB.

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-***@VM.MARIST.EDU] On Behalf Of
Lizette Koehler
Sent: June 6, 2016 8:59 AM
To: TSO-***@VM.MARIST.EDU
Subject: [TSO-REXX] How do I explain REXX Storage Usage

I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And other
times it is okay.
I am unable to post the code, but am willing to answer any questions about
it.


The batch job runs IKJEFT01

The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.

An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.

Once the header record is found they spin the data to an output file until
the next header record is found.

Occasionally they will get an error message stating: Machine storage is
exhausted.

There are no new stack/del stack process in this code. A lot of Signal
statements are used.


This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines, it
works.

I would like to provide a simple explanation as to why it does not work some
of the time.

I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a coding
technique challenge.

Thanks.

Lizette Koehler
statistics: A precise and logical method for stating a half-truth
inaccurately

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email
to ***@VM.MARIST.EDU with the message: INFO TSO-REXX

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Hobart Spitz
2016-06-06 17:15:57 UTC
Permalink
Raw Message
Instead of EXECIO 1, you can use LINEIN().
Post by Robin
There is a trick to using EXECIO 1 that doesn't slow down processing too
severely.
Presuming the organization (RECFM, LRECL, etc) of input file is stable for
each execution of the REXX, you have the REXX program allocate to the file
as RECFM=U. Herein the EXECIO will read a physical block (with x number of
logical record) with 1 EXECIO. It is then the responsibility to 'deblock'
the physical file block of records. Handling 'short blocks' (e.g. last block
of the file)
Is also the responsibility of the REXX program. 'Deblocking' full physical
blocks is relatively simple for files where the true RECFM=FB.
-----Original Message-----
Lizette Koehler
Sent: June 6, 2016 8:59 AM
Subject: [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And other
times it is okay.
I am unable to post the code, but am willing to answer any questions about it.
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Once the header record is found they spin the data to an output file until
the next header record is found.
Occasionally they will get an error message stating: Machine storage is
exhausted.
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines, it
works.
I would like to provide a simple explanation as to why it does not work some
of the time.
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a coding
technique challenge.
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
--
OREXXMan

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Paul Gilmartin
2016-06-07 03:11:00 UTC
Permalink
Raw Message
Post by Hobart Spitz
Instead of EXECIO 1, you can use LINEIN().
I'd be surprised if that wasn't even slower.

But which LINEIN()? There's one with compiled Rexx that handles
only Classic data sets, and one with OMVS Rexx that handles only
UNIX files. They gotta fix that.
Post by Hobart Spitz
Post by Robin
Presuming the organization (RECFM, LRECL, etc) of input file is stable for
each execution of the REXX, you have the REXX program allocate to the file
as RECFM=U. Herein the EXECIO will read a physical block (with x number of
logical record) with 1 EXECIO. It is then the responsibility to 'deblock'
the physical file block of records. Handling 'short blocks' (e.g. last block
of the file)
Is also the responsibility of the REXX program. 'Deblocking' full physical
blocks is relatively simple for files where the true RECFM=FB.
Deblocking RECFM=VB is only a little harder, and VBS is only a little
more harder.

Or, if you're determined to RYO, you can just write an Assembler program.

-- gil

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Mickey Bee
2016-06-08 18:22:05 UTC
Permalink
Raw Message
You can also do this with any file type.

For example, I create a file 80 bytes, optimum half track blocking of 27920.

I do an execio 349 DISKR, and one EXCP gets me 349 records. Then it's just a
matter of interrogating the return code.

Mickey

-----Original Message-----
From: Hobart Spitz
Sent: Monday, June 06, 2016 1:15 PM
To: TSO-***@VM.MARIST.EDU
Subject: Re: [TSO-REXX] How do I explain REXX Storage Usage

Instead of EXECIO 1, you can use LINEIN().
Post by Robin
There is a trick to using EXECIO 1 that doesn't slow down processing too
severely.
Presuming the organization (RECFM, LRECL, etc) of input file is stable for
each execution of the REXX, you have the REXX program allocate to the file
as RECFM=U. Herein the EXECIO will read a physical block (with x number of
logical record) with 1 EXECIO. It is then the responsibility to 'deblock'
the physical file block of records. Handling 'short blocks' (e.g. last block
of the file)
Is also the responsibility of the REXX program. 'Deblocking' full physical
blocks is relatively simple for files where the true RECFM=FB.
-----Original Message-----
Lizette Koehler
Sent: June 6, 2016 8:59 AM
Subject: [TSO-REXX] How do I explain REXX Storage Usage
I have the need to provide a simple explanation as to why sometimes when
running a REXX with REGION=0M or REGION=128M in batch it fails. And other
times it is okay.
I am unable to post the code, but am willing to answer any questions about it.
The batch job runs IKJEFT01
The process reads an input file that could have from between 5 input lines
and 100's or 1000's of input lines.
An EXECIO is used to read the entire Input file. Then the process does a DO
i=1 to src.0 looking for a header record.
Once the header record is found they spin the data to an output file until
the next header record is found.
Occasionally they will get an error message stating: Machine storage is
exhausted.
There are no new stack/del stack process in this code. A lot of Signal
statements are used.
This is trying to take on large input file and split out all of the reports
in it into separate files. So long as there are not many input lines, it
works.
I would like to provide a simple explanation as to why it does not work some
of the time.
I will also be working with my users and using a different process that will
be more successful. But for now, I need to explain why this is a coding
technique challenge.
Thanks.
Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions, send email
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
--
OREXXMan

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
christel.u.w.pachl christel.u.w.pachl
2016-06-23 20:36:15 UTC
Permalink
Raw Message
One Location for A which gets reused

ten Locations for all the B.i's
----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
retired mainframer
2016-06-23 21:20:36 UTC
Permalink
Raw Message
Post by Dyck, Lionel B. , TRA
-----Original Message-----
Of Lizette Koehler
Sent: Thursday, June 23, 2016 12:15 PM
Subject: Re: How do I explain REXX Storage Usage
So I wanted a little clearer understanding at this point as to storage
allocation during REXX execution (I do not have the compiler so always
interpreted).
So if I have a simple REXX
A = myvar
B = myvar
Both of those take an individual storage location. Or does REXX point A
and B
Post by Dyck, Lionel B. , TRA
to the same storage location?
Can A and B point to the same location? Do interpreters do optimizations of
this sort? Would
A = B
cause A and B to point to the same location?

If you run this under TSO TEST or XDC, you should be able to determine where
A and B point.

How about
A = myvar
myvar = myvar+1
B = myvar

After the first assignment, does A point to the same place myvar points or
to a new area holding a copy of myvar's data? After the second, does myvar
point to a new area? If so and A pointed to the old area, REXX has to know
not to release that memory because A is still using it. Did the value in A
change? Is A equal to B?
Post by Dyck, Lionel B. , TRA
But if I now do
Do I = 10
Parse Var stem.i myvar .
A = myvar
B.i = myvar
End
I think on each iteration it is a new address for the storage location of
the
Post by Dyck, Lionel B. , TRA
vars A & B.i .
All REXX variables are stored as strings. Even the simple sequence
A = myvar
A = myvar
requires either new memory OR a determination that the new value can fit in
the space already allocated. Does the routine that performs assignments try
to determine if space can be reused? If new space is allocated, is the old
space released immediately or does some threshold trigger a garbage
collection algorithm?

If you test, does the address pointed to be A change after the second
assignment?
Post by Dyck, Lionel B. , TRA
Do I have that correct.
If so, then Dropping the stem would free up that storage and reduce what
storage
Post by Dyck, Lionel B. , TRA
is allocated for my variables.
Is the memory for dropped variables freed immediately or does garbage
collection occur later?
Post by Dyck, Lionel B. , TRA
So instead of having 10 storage locations for A and B.i - I would have
one for
Post by Dyck, Lionel B. , TRA
A and 10 for B.i ????? So I would need to drop B.i before finishing the
loop?

If there is a real reason for using B.i instead of B, then you probably
don't want to drop any B.i until well after the loop is finished. If you
drop B.i before the loop finishes, you cannot refer to B.(i-1) in the next
iteration so having a stem element seems a waste.

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Pedro Vera
2016-06-23 21:48:57 UTC
Permalink
Raw Message
My recollection from a similar discussion many years ago was that a DROP
marks the memory as unused, but does not necessarily result in a FREEMAIN
of the memory. It is still allocated and could be used for something
else. The FREEMAIN could be done sometime later, as determined by the rexx
memory management routines.

I think you should drop large stem variables when you are done using them,
but not necessarily drop each simple variable that you use.



Pedro Vera
DB2 Admin Tool
http://www-01.ibm.com/software/data/db2imstools/db2tools/db2admin/


----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to ***@VM.MARIST.EDU with the message: INFO TSO-REXX
Loading...