2.1.1 Introduction

The Error/Status Message (a.k.a. SMF, for "Status Message Facility") Tools are the lowest level of the Toolkit, since nearly all of the other Toolkit functions call these tools. Their purpose is to provide an error and status message handling mechanism for use in science software (and in Toolkit functions), and to provide means to send log files, informational messages and output data files to DAAC personnel or to remote users.

In this overview section, we walk you through the procedure of constructing your own error/status messages step-by-step, then show their application in log files, your own code, and in the Toolkit itself.

2.1.2 Constructing Your Own Error/Status Messages

This section explains how to use the Toolkit to construct files containing error and status messages, which your code can access at runtime.

The basic process of constructing these files consists of 2 steps: constructing the status message text file with an editor, then running the smfcompile utility provided in the Toolkit, before compiling and executing your code.

# Status Message Text File for Toolkit AVHRR/Land Pathfinder
# prototype
#
%INSTR	= AVHRR
%LABEL	= PATHFINDER
%SEED 	= 99
PATHFINDER_F_OPEN_BINARY_FILE   FATAL_ERROR...opening binary file
PATHFINDER_F_MEM_ALLOC_FAIL     FATAL ERROR...allocating memory %s
PATHFINDER_F_OPEN_ANC_FILE      FATAL_ERROR...%s
PATHFINDER_W_CLOSE_GAC_FILE     WARNING...could not close GAC file
PATHFINDER_W_OZONE_FILE_MISSING	Ozone file not found
                                      ::PATHFINDER_A_ALT_FILE_USED
PATHFINDER_A_ALT_FILE_USED      Alternate file used		
PATHFINDER_W_EPH_FILE_NOT_FOUND	Ephemeris file not found
                                      ::PATHFINDER_A_ALT_FILE_USED
PATHFINDER_W_NO_LOG_FILES       WARNING: Problem sending log files
PATHFINDER_N_PROCESSING_DONE    SUCCESS: processing complete at %s

The first 3 lines of this file are comments. The next 3 lines are required. They may appear only once per file, and must appear in this order.

%INSTR	= AVHRR

The "%INSTR" field is your instrument name.

%LABEL	= PATHFINDER

The label in the "%LABEL" field is arbitrary (see label below in this section) .

%SEED 	= 99

The "%SEED" field is a seed number assigned to you by ECS/SDPS. Most teams have been allocated 5,000 seed values in a specified range. The purpose of seed numbers is to ensure unique error messages for each instrument team or development group.

PATHFINDER_F_OPEN_BINARY_FILE   FATAL_ERROR...opening binary file

PATHFINDER_F_OPEN_BINARY_FILE FATAL_ERROR...opening binary file

PATHFINDER_F_OPEN_BINARY_FILE FATAL_ERROR...opening binary file

PATHFINDER_F_OPEN_BINARY_FILE  FATAL_ERROR...opening binary file

Table. Error/Status Message Levels. 
Level Name Description 
_S_ Success Normal return value 
_A_ Action For retrieving a string indicating action taken 
_M_ Message Message returned by Toolkit 
_U_ User information Informational message generated by user 
_N_ Notice E.g., for data availability notices 
_W_ Warning Possible problem in program 
_E_ Error Error in program 
_F_ Fatal error Fatal error in program

PATHFINDER_F_OPEN_BINARY_FILE  FATAL_ERROR...opening binary file

PATHFINDER_F_OPEN_BINARY_FILE  FATAL_ERROR...opening binary file

PATHFINDER_F_MEM_ALLOC_FAIL   FATAL ERROR...allocating memory %s

PATHFINDER_A_ALT_FILE_USED		Alternate file used		

PATHFINDER_W_OZONE_FILE_MISSING	Ozone file not found ::PATHFINDER_A_ALT_FILE_USED

PATHFINDER_A_ALT_FILE_USED

$PGSBIN/smfcompile -f AVHRR_99.t -r -i

$PGSBIN/smfcompile -f AVHRR_99.t -f77

$PGSBIN/smfcompile -f AVHRR_99.t -ada

2.1.3 Log files

11:PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_DATA_PRESENT:76807
The data requested is not in the line found.

11:PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_DATA_PRESENT:76807

11:PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_DATA_PRESENT:76807

11:PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_DATA_PRESENT:76807

11:PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_DATA_PRESENT:76807
The final entry on this line is the SMF error code, which is used internally by the Toolkit to identify the error or status.

The data requested is not in the line found.

11:():PATHFINDER_N_PROCESSING_DONE:813585
SUCCESS: AVHRR processing complete at Mon Sep 19 17:37:47 1994

2.1.4 Using error/status messages in your code

This section provides pointers to the major functions which you need to use to implement error/status messaging in your code. Only a brief summary is given in this section; the explanations of the individual Toolkit functions, along with detailed examples of usage, appear in the Tool Descriptions section.

2.1.5 How the Toolkit itself uses error/status messages

The Toolkit itself makes extensive use of PGS_SMF_* functions for error checking purposes. Much effort has gone into assuring that the maximum number of possible errors will be trapped, without sacrificing the speed and efficiency of the Toolkit code.

Nearly all Toolkit functions are of type PGSt_SMF_status in C, or INTEGER in FORTRAN, which means that they return a status or error value that may be checked and acted on using PGS_SMF_* functions.

Toolkit runtime ASCII message files have filenames of the form $PGSMSG/PGS_?, where ? = 1 to 13. They are derived from status message text files with filenames of the form $PGSMSG/PGS_grp_?.t, where grp = Toolkit group name (SMF, PC, IO, ...) and ? = 1 to 13. The corresponding include files have filenames of the form $PGSINC/PGS_grp_?.h (C), $PGSINC/PGS_grp_?.f (FORTRAN ), and $PGSINC/PGS_grp_?.ada (Ada).

The Toolkit bases no action on the severity of error levels; that task is left to the science software. In particular, the Toolkit never returns a fatal error, nor exits a program. In general, returned values from Toolkit functions are either of status levels "_S_", "_W_", or "_E_". The only time the Toolkit itself acts on the status level of a message is when it sends user-generated messages of status level "_N_" or "_U_" to the User log file, as explained in section 2.1.3, Log files.

Switching on the level of error is the province of the PGS_SMF_Test*Level set of tools. These tools are for use in the science software.

Since a message is written to the Status log file every time a PGS_SMF_Set*Msg function is called, many of these messages will be generated by Toolkit functions, in the event of warnings or errors. If a low-level Toolkit function detects an error or warning, it will write a message to the Status log file, then return the appropriate message to its calling function. That function also will write to the log file, if it is unable to handle the error, and return an appropriate error or warning message to its calling function. So a single error or warning can result in several messages in the log file; this enables traceability of the problem. The Status log file is in fact the only source of traceability for Toolkit errors.

There is a special case where warning messages are generated, when in fact there is no anomaly in processing. See the entry for PGS_IO_Gen_Open.

This concludes the "Overview" section of the error/status messaging tools.

2.2.1 PGS_SMF_CreateMsgTag

Short explanation of what it's for: Returns a string containing configuration information, for stamping such things as entries in your Report log file. Currently, configuration consists of Science Software Configuration ID and Production Run ID.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

For the examples, arbitrarily let the Science Software Configuration ID be "V3.0" and the Production Run ID be "SCF22" in the Process Control file (see Notes section).

C example:

#include <PGS_SMF.h>
char systemTag[PGSd_SMF_TAG_LENGTH_MAX];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
returnStatus = PGS_SMF_CreateMsgTag(systemTag);
/*
The resultant value of systemTag is "V3.0SCF22"
*/

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INTEGER pgs_smf_createmsgtag
      CHARACTER*60 systemtag
      INTEGER returnstatus
C
C Begin example
C
      returnstatus = pgs_smf_createmsgtag(systemtag)
C
C The resultant value of systemtag is "V3.0SCF22"

Notes:

Configuration information is read from the SYSTEM RUNTIME PARAMETERS section of the Process Control file $PGS_PC_INFO_FILE (see Process Control tool section), so this file must have been prepared first, and its environment variable set.

DAAC and hardware indentification are being considered as additions to the configuration data, for future deliveries of the Toolkit.

2.2.2 PGS_SMF_GenerateStatusReport

Short explanation of what it's for: Writes an arbitrary string to the Report log file. You may use this function as an alternative to preparing Toolkit SMF style error/status messages.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

Examples that follow show how to write an error message with configuration information to the Report log file. Example given is an error condition returned from Toolkit function PGS_IO_Gen_Open. Although this error message is automatically written to the Status log file by the Toolkit, you might want to write it to the User log file also, in order to more easily identify it as an important message.

Variable "systemTag" has been previously set to value "V3.0SCF22" (see PGS_SMF_CreateMsgTag ).

C example:

#include <stdio.h>
#include <PGS_SMF.h>
#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
char message[1024];
char systemTag[PGSd_SMF_TAG_LENGTH_MAX];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
returnStatus = PGS_IO_Gen_Open(	GOLDEN_BINARY, 				
                   PGSd_IO_Gen_Read, &processGolden, 1);
/*
   Test whether status level is "_E_" or worse
*/
if( PGS_SMF_TestStatusLevel(returnStatus) >= PGS_SMF_MASK_LEV_E )
{
/*
   If error, prepare msg, write to Report log file
*/
   sprintf( message, "%s : Error opening golden binary file\n", 		
			systemTag );
   returnStatus = PGS_SMF_GenerateStatusReport(message);
/*
   The string "V3.0SCF22 :  Error opening golden binary file"
   is written to the Report log file
*/
}

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_teststatuslevel
      INTEGER pgs_smf_generatestatusreport
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      CHARACTER*1024 message
      CHARACTER*60 systemtag
      INTEGER returnstatus
C
C Begin example
C
C Try to open a file
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
     .			PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
C
C Test whether status level is "_E_" or worse
      IF ( pgs_smf_teststatuslevel(returnstatus) .GE. 
     .			PGS_SMF_MASK_LEV_E ) THEN
C
C If error, prepare msg, write to Report log file
      message = systemtag // 'Error opening golden binary file'
      returnstatus = 
pgs_smf_generatestatusreport(message)

C
C The string "V3.0SCF22 :  Error opening golden binary file"
C  is written to the Report log file
C
      END IF

Notes:

Message passed to this function may be of arbitrary length.

Report log file name is read internally by Toolkit functions from the SUPPORT OUTPUT section of the Process Control file $PGS_PC_INFO_FILE (see the Process Control tool section ), the first time PGS_SMF_GenerateStatusReport is called. This file must have been prepared first, and its environment variable set. Normally you would not change this entry from the Process Control file template supplied with the Toolkit ($PGSRUN/PCF.v5).

The golden binary file of the example must have an entry in the Process Control file, i.e., integer 401 must be associated with a reference (physical filename) in that file.

Log files may be sent to a remote machine through use of either PGS_PC_Shell.sh, or PGS_PC_InitCom and PGS_PC_TermCom.

2.2.3 PGS_SMF_GetActionByCode

Short explanation of what it's for: Retrieves action string portion of an error/status action definition, given its mnemonic label code. You might use this to write the action string explicitly to the Report log file, for example.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

The examples assume that the following entries exist in the status message text file AVHRR_99.t :

PATHFINDER_A_ALT_FILE_USED      Alternate file used		
PATHFINDER_W_OZONE_FILE_MISSING	Ozone file not found
                                  ::PATHFINDER_A_ALT_FILE_USED

C example:

#include <PGS_SMF.h>
char actionString[PGSd_SMF_MAX_ACT_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
returnStatus = YourLowerLevelModule( arg1, arg2, ... );
if( returnStatus == PATHFINDER_W_OZONE_FILE_MISSING)
{
   returnStatus = PGS_SMF_GetActionByCode(
                      PATHFINDER_W_OZONE_FILE_MISSING, actionString );
/*
   actionString now contains the string "Alternate file used"
*/
   returnStatus = PGS_SMF_GenerateStatusReport(actionString);
/*
   The string "Alternate file used" is written to the
   Report log file
*/

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INTEGER pgs_smf_getactionbycode
      INTEGER yourlowerlevelmodule
      INTEGER pgs_smf_generatestatusreport
      CHARACTER*240 actionstring
      INTEGER returnstatus
C
C Begin example
C
      returnstatus = yourlowerlevelmodule( arg1, arg2, ... )
      IF( returnstatus .EQ. PATHFINDER_W_OZONE_FILE_MISSING) THEN
         returnstatus = pgs_smf_getactionbycode(
     .             PATHFINDER_W_OZONE_FILE_MISSING, actionstring )
C
C  actionstring now contains the string 'Alternate file used'
C
        returnstatus = pgs_smf_generatestatusreport(actionString)
C
C  The string 'Alternate file used' is written to the
C  Report log file
C
      END IF

Notes:

Messages must have been prepared in a status message text file first, and run through the smfcompile utility.

2.2.4 PGS_SMF_GetInstrName

Short explanation of what it's for: Returns the name of the instrument corresponding to a given mnemonic label code. This may be useful for determining which instrument generated an error/status message, in the case of code integrated between more than one instrument.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

Examples assume the first four lines of the status message text file AVHRR_99.t appear as follows:

%INSTR	= AVHRR
%LABEL	= PATHFINDER
%SEED 	= 99
PATHFINDER_F_OPEN_BINARY_FILE FATAL_ERROR...opening binary file
C example:

#include <PGS_SMF.h>
char instr[PGS_SMF_MAX_INSTR_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
returnStatus = PGS_SMF_GetInstrName(
                    PATHFINDER_F_OPEN_BINARY_FILE, instr );
/*
   instr now contains the string "AVHRR"
*/
FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INTEGER pgs_smf_getinstrname
      CHARACTER*10 instr
      INTEGER returnstatus
C
C Begin example
C
returnstatus = pgs_smf_getinstrname(
                       PATHFINDER_F_OPEN_BINARY_FILE, instr )
C
C  instr now contains the string 'AVHRR'
C
Notes:

Messages must have been prepared in a status message text file first, and run through the smfcompile utility.

2.2.5 PGS_SMF_GetMsg

Short explanation of what it's for: Retrieves the current error/status message from the message buffer. It is normally not necessary to call this function in production processing.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

Examples show how to use this function to print to the screen during development. Note that the recommended error handling for production is very different; see PGS_SMF_SetStaticMsg.

The examples contain two levels: (1) your main module, and (2) your lower level module, which attempts to open a file using PGS_IO_Gen_Open (C) or PGS_IO_Gen_OpenF (FORTRAN).

The following entry is assumed to appear in the status message text file AVHRR_99.t:

PATHFINDER_F_OPEN_BINARY_FILE FATAL_ERROR...opening binary file

The corresponding entry then must appear in the runtime ASCII message file PGS_99:

815107,PATHFINDER_F_OPEN_BINARY_FILE,NULL,FATAL_ERROR...error opening 
binary file
C example:

#include <PGS_SMF.h>
#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_SMF_code code;
char mnemonic[PGS_SMF_MAX_MNEMONIC_SIZE];
char message[PGS_SMF_MAX_MSGBUF_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Call low-level module, whose definition is given below
*/
returnStatus = LowLevelModule( &processGolden );
if( returnStatus != PGS_S_SUCCESS )
{
/*
   Get error message from buffer, print to screen
*/
   PGS_SMF_GetMsg( &code, mnemonic, message );
   printf("LowLevelModule: %s\n", message );
/*
   Values of the variables returned by PGS_SMF_GetMsg are
	code: 815107
	mnemonic: "PATHFINDER_F_OPEN_BINARY_FILE"
	message: "FATAL_ERROR...opening binary file"
   The string "LowLevelModule: FATAL_ERROR...opening binary file"
   is printed to the screen.
*/   
}
/*
End main module
*/
/*
Low level module definition
*/
PGSt_SMF_status LowLevelModule(
                PGSt_IO_Gen_FileHandle **processGoldenPtr )
{
/*
   Try to open a file
*/
returnStatus = PGS_IO_Gen_Open(	GOLDEN_BINARY, 				
                 PGSd_IO_Gen_Read, processGoldenPtr, 1);
/*
   Test whether status level is "_E_" or worse
*/
if( PGS_SMF_TestStatusLevel(returnStatus) >= PGS_SMF_MASK_LEV_E )
{
   PGS_SMF_SetStaticMsg( PATHFINDER_F_OPEN_BINARY_FILE, "" );
   return(PATHFINDER_F_OPEN_BINARY_FILE);
}
return(PGS_S_SUCCESS);
}
/*
End low level module definition
*/
FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER lowlevelmodule
      INTEGER pgs_smf_getmsg
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      INTEGER code
      CHARACTER*32 mnemonic
      CHARACTER*480 message
      INTEGER returnstatus
C
C Begin example
C
C Call low-level module, whose definition is given below
C
      returnstatus = lowlevelmodule( processgolden )
      IF( returnstatus != PGS_S_SUCCESS ) THEN
C
C  Get error message from buffer, print to screen
C
         call pgs_smf_getmsg( code, mnemonic, message )
         PRINT *, 'LowLevelModule: ', message
C
C  Values of the variables returned by PGS_SMF_GetMsg are
C	code: 815107
C	mnemonic: 'PATHFINDER_F_OPEN_BINARY_FILE'
C	message: 'FATAL_ERROR...opening binary file'
C  The string 'LowLevelModule: FATAL_ERROR...opening binary file'
C  is printed to the screen.
C   
      END IF
C
C End main module
C
C Low level module definition
C
      INTEGER FUNCTION LowLevelModule( processgolden )
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_teststatuslevel
      INTEGER processgolden
C
C  Try to open a file
C
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
     .              PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
C
C  Test whether status level is "_E_" or worse
C
      IF ( pgs_smf_teststatuslevel(returnstatus) .GE.
     .          PGS_SMF_MASK_LEV_E ) THEN
         pgs_smf_setstaticmsg( PATHFINDER_F_OPEN_BINARY_FILE, '' )
         RETURN(PATHFINDER_F_OPEN_BINARY_FILE)
      END IF
      RETURN(PGS_S_SUCCESS);
      END

C
C End low level module definition
C

Notes:

Message to be retrieved must have been previously written to the message buffer by one of the PGS_SMF_Set*Msg functions, e.g., PGS_SMF_SetStaticMsg .

This function retrieves what is currently in the message buffer.

The reason it is normally not necessary to call this function in production processing is that any message in the message buffer is automatically written to the Status log file internally by the Toolkit.

Messages must have been prepared in a status message text file first, and run through the smfcompile utility.

The golden binary file of the example must have an entry on the Process Control file, i.e., integer 401 must be associated with a reference (physical filename) in that file.

In the example, note that the values returned by the PGS_SMF_GetMsg call are independent of the return value of the LowLevelModule function, though they are associated with each other.

2.2.6 PGS_SMF_GetMsgByCode

Short explanation of what it's for: Returns the message definition string corresponding to a given mnemonic label code. Primary use is to enable creation of dynamic messages.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

This example first shows how to retrieve a message definition string for a given mnemonic label code using PGS_SMF_GetMsgByCode, then shows how the result can be used to create a dynamic message.

The following entry is assumed to appear in the status message text file, AVHRR_99.t:

C example:

PATHFINDER_F_OPEN_ANC_FILE		FATAL_ERROR...%s

FORTRAN example:

PATHFINDER_F_OPEN_ANC_FILE		('FATAL_ERROR...', A)
C example:

#include <PGS_IO.h> 
#include <PGS_SMF.h> 
#include <PGS_AVHRR_99.h> 

#define GOLDEN_BINARY 401
#define SUCCESS 0       /* GSFC AVHRR/Pathfinder error handling */
#define FATAL_ERROR -1  /* GSFC AVHRR/Pathfinder error handling */
char descrip[80];
char message[PGS_SMF_MAX_MSGBUF_SIZE];
char buf[PGS_SMF_MAX_MSGBUF_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
strcpy( descrip, "opening golden binary file" );
if (PGS_IO_Gen_Open( GOLDEN_BINARY, PGSd_IO_Gen_Read,
                     processGoldenPtr, 1)
       != PGS_S_SUCCESS) goto EXCEPTION;
.
.
.
return (SUCCESS);
/*
   Exception block
*/
EXCEPTION:
/*
   First retrieve the static message string into 'message'
*/
   PGS_SMF_GetMsgByCode( PATHFINDER_F_OPEN_ANC_FILE, message );
/*
   message now contains the string "FATAL_ERROR...%s"
*/
   sprintf( buf, message, descrip );
/*
   buf now contains the string
      "FATAL_ERROR...opening golden binary file"
*/
   PGS_SMF_SetDynamicMsg( PATHFINDER_F_OPEN_ANC_FILE, buf, "" );
/*
   Message buffer now contains the buf string, and it has been
      automatically written to the Status log file
*/
   return(FATAL_ERROR);
}
FORTRAN example:

      include 'PGS_IO.f' 
      include 'PGS_SMF.f'
      include 'PGS_AVHRR_99.f'

      character*80 descrip
      character*240 message
      character*240 buf
      character*20  func_name

      integer golden_binary
      integer success        ! GSFC AVHRR/Pathfinder error handling
      integer fatal_error    ! GSFC AVHRR/Pathfinder error handling
      integer golden_un
      integer version
      integer returnstatus

!  Begin example

!  Initialize variables

      func_name = 'avhrr_func()'  ! name of this function
      success = 0                 ! success return value
      fatal_error = -1            ! error return value
      golden_binaray = 401        ! file logical ID in PCF
      version = 1

!  Try to open a file

      descrip = 'opening golden binary file'
      returnstatus = pgs_io_gen_openf( golden_binary, 
     >                                 pgsd_io_gen_rseqfrm, 0,
     >                                 golden_un, version )

      if ( returnstatus .ne. pgs_s_success ) goto 999
.
      avhrr_func = success
      return

!  Exception block

 999  continue

!  First retrieve the static message string into 'message'

   pgs_smf_getmsgbycode( PATHFINDER_F_OPEN_ANC_FILE, message )

!  message now contains the string '('FATAL_ERROR...', A)'

      write(buf, ref=message) descrip

!  buf now contains the string:
!  'FATAL_ERROR...opening golden binary file

      pgs_smf_setdynamicmsg(PATHFINDER_F_OPEN_ANC_FILE, buf, func_name)

!  Message buffer now contains the buf string, and it has been
!  automatically written to the Status log file

      avhrr_func = fatal_error
      return
Notes:

The golden binary file of the example must have an entry on the Process Control file, i.e., integer 401 must be associated with a reference (physical filename) in that file.

Messages must have been prepared in a status message text file first, and run through the smfcompile utility.

2.2.7 PGS_SMF_SendRuntimeData

Short explanation of what it's for: Mark output files for sending to a designated machine. Email is also sent to designated recipient(s).

This function is in file: $PGSSRC/SMF/PGS_SMF_SendRuntimeData.c

Examples:

The examples assume the following exists in the Process Control File (PCF):

?   PRODUCT INPUT FILES 
#
201|87002002709.no9_gac||||gac_attributes|1
399|test10.hdf|||||3
399|test23.hdf|||||2
399|test06.hdf|||||1
C example:

#include <PGS_SMF.h>
#define GAC_FILE   201
#define HDF_INFILE 399
PGSt_integer sendFile[3];
PGSt_integer version[3];
PGSt_integer numFiles;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
sendFile[0] = GAC_FILE;
version[0] = 1;

sendFile[1] = HDF_INFILE;
version[1] = 1;

sendFile[2] = HDF_INFILE;
version[2] = 2;

numFiles = 3;

returnStatus = PGS_SMF_SendRuntimeData( 
                      numFiles, sendFile, version );

/* Files 87002002709.no9_gac, test10.hdf, and test23.hdf
   in default directory $PGS_PRODUCT_INPUT are now marked for 
   sending to a remote machine. */
FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INTEGER pgs_smf_sendruntimedata
      INTEGER GAC_FILE
      PARAMETER (GAC_FILE=201)
      INTEGER HDF_INFILE
      PARAMETER (HDF_INFILE=399)
      INTEGER sendfile(3)
      INTEGER version(3)
      INTEGER numfiles
      INTEGER returnstatus
C
C Begin example
C
      sendfile(1) = GAC_FILE
      version(1) = 1

      sendfile(2) = HDF_INFILE
      version(2) = 1

      sendfile(3) = HDF_INFILE
      version(3) = 2

      numfiles = 2
      returnstatus = pgs_smf_sendruntimedata( 
     .                   numfiles, sendfile, version )

C Files 87002002709.no9_gac, test10.hdf, and test23.hdf 
C in default directory $PGS_PRODUCT_INPUT are now marked for 
C sending to a remote machine.
Notes:

This function should only be called once in your program. Only the last call of this function is recognized by the Toolkit. This is to minimize overhead in file transfers.

The mechanism for doing the file transfers is currently Unix function ftp. This may change in the future, e.g., to use DCE or its equivalent, but the calling sequence will not change. In the production environment, files will be placed on an intermediate machine; then you can retrieve them at your convenience.

In the examples, of the 3 files with logical ID 399, files test10.hdf and test23.hdf are sent because they are listed #1 and #2 in order in the PCF.
For more information about version numbers vs. sequence numbers (sequence numbers are the ones listed in the last field of the PCF entries), see "sec. 4.1.2.2, Constructing your Process Control file, PRODUCT INPUT, Field 7, Sequence number".

In order for this function to work at the SCF, you must be using either PGS_PC_Shell.sh or Edit line 10106 to change "sandcrab" to the machine to which you want the files sent. In the production environment, this will be an intermediate machine, from which you will later retrieve the file.

Edit line 10107 to change "/usr/kwan/test/PC/data" to the fully qualified directory name to which you want the files written on that machine. The directory must exist, and have at least Unix user permission "w".

Edit line 10108 to change "kwan@eos.hitc.com" to the email address at which you want email notification that the files have been sent.

In your home directory $HOME on the machine on which you are executing your code at the SCF, the file ".netrc" must exist, with Unix permissions "-rw-------". This file must contain the line

machine sandcrab login kwan password kwan_password

where "sandcrab" is the machine of line 10106, "kwan" is a valid user of that machine, and "kwan_password" is his/her password.

Due to local security policy, the use of this mechanism of using '.netrc' files may be changed. The production environment may use a more secure mechanism.

2.2.8 PGS_SMF_SetArithmeticTrap

Short explanation of what it's for: Catching arithmetic errors in your program, in order to avoid core dumps.

Notes:

This tool was intended to be delivered with TK4. However, implementation has been problematic. At this writing delivery of the tool is uncertain. For details on the problems encountered, please read the signal handling investigation summary.

2.2.9 PGS_SMF_SetDynamicMsg

Short explanation of what it's for: Saves a runtime-defined error/status message to the message buffer. Every time this function is called, it writes an entry to the Status log file.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

There are at least two different ways to use this function. Example 1 shows C programmers how to construct dynamic messages through use of PGS_SMF_GetMsgByCode; Example 2 shows how to do it directly, in both C and FORTRAN.

Example 1: Using PGS_SMF_GetMsgByCode

This example first shows how to retrieve a message definition string for a given mnemonic label code using PGS_SMF_GetMsgByCode, then shows how the result can be used to create a dynamic message.

The following entry is assumed to appear in the status message text file , AVHRR_99.t:

PATHFINDER_F_OPEN_ANC_FILE   FATAL_ERROR...%s

The corresponding entry then must appear in the runtime ASCII message file PGS_99:

815104,PATHFINDER_F_OPEN_ANC_FILE,NULL,FATAL_ERROR...%s

C example 1:

#include <PGS_IO.h>
#include <PGS_SMF.h>
#define GOLDEN_BINARY 401
#define SUCCESS 0       /* GSFC AVHRR/Pathfinder error handling */
#define FATAL_ERROR -1  /* GSFC AVHRR/Pathfinder error handling */
char descrip[80];
char message[PGS_SMF_MAX_MSGBUF_SIZE];
char buf[PGS_SMF_MAX_MSGBUF_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
strcpy( descrip, "opening golden binary file" );
if (PGS_IO_Gen_Open( GOLDEN_BINARY, PGSd_IO_Gen_Read, 				
                         processGoldenPtr, 1)
         != PGS_S_SUCCESS) goto EXCEPTION;
.
.
.
return (SUCCESS);
/*
   Exception block
*/
EXCEPTION:
/*
   First retrieve the static message string into 'message'
*/
   returnStatus = PGS_SMF_GetMsgByCode(
                      PATHFINDER_F_OPEN_ANC_FILE, message );
/*
   message now contains the string "FATAL_ERROR...%s"
*/
   sprintf( buf, message, descrip );
/*
   buf now contains the string
      "FATAL_ERROR...opening golden binary file"
*/
   returnStatus = PGS_SMF_SetDynamicMsg(
                         PATHFINDER_F_OPEN_ANC_FILE, buf, "" );
/*
   Message buffer now contains the buf string, and the following
   entry appears in the Status log file:
11::PATHFINDER_F_OPEN_ANC_FILE:815104
FATAL_ERROR...opening golden binary file
*/
   return(FATAL_ERROR);
}

FORTRAN example 1:

Work is in progress on a FORTRAN function that is essentially a wrapper on C function "sprintf", which will enable the creation of dynamic messages in FORTRAN, similar to that given in the C example above. This method cannot be applied in FORTRAN until the "sprintf" wrapper is available. For now, FORTRAN users can only use the method of Example 2 below to construct dynamic messages.

Example 2: Direct Method

This example shows how to construct dynamic messages directly.

C example 2:

#include <PGS_IO.h>
#include <PGS_SMF.h>
#define GOLDEN_BINARY 401
#define SUCCESS 0       /* GSFC AVHRR/Pathfinder error handling */
#define FATAL_ERROR -1  /* GSFC AVHRR/Pathfinder error handling */
char descrip[80];
char message[PGS_SMF_MAX_MSGBUF_SIZE];
char buf[PGS_SMF_MAX_MSGBUF_SIZE];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
strcpy( descrip, "opening golden binary file" );
if (PGS_IO_Gen_Open( GOLDEN_BINARY, PGSd_IO_Gen_Read, 				
                      processGoldenPtr, 1)
         != PGS_S_SUCCESS) goto EXCEPTION;
.
.
.
return (SUCCESS);
/*
   Exception block
*/
EXCEPTION:
/*
   Construct error string manually
*/
   strcpy( buf, "FATAL_ERROR..." );
   strcat( buf, descrip );
/*
   buf now contains the string
      "FATAL_ERROR...opening golden binary file"
*/
   returnStatus = PGS_SMF_SetDynamicMsg( 
                       PATHFINDER_F_OPEN_ANC_FILE, buf, "" );
/*
   Message buffer now contains the buf string, and the following
   entry appears in the Status log file:
11::PATHFINDER_F_OPEN_ANC_FILE:815104
FATAL_ERROR...opening golden binary file
*/
   return(FATAL_ERROR);
}

FORTRAN example 2:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_setdynamicmsg
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      CHARACTER*32 mnemonic
      CHARACTER*480 buf
      CHARACTER*480 message
      INTEGER returnstatus
C

C Begin example
C
C  Try to open a file
C
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
     .			PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
      IF (returnstatus .NE. PGS_S_SUCCESS ) GO TO 999
.
.
.
      RETURN (SUCCESS)
C
C  Exception block
C
999   CONTINUE
C
C  Construct error string manually
C
      buf = 'FATAL_ERROR...' // descrip
C
C  buf now contains the string
C     'FATAL_ERROR...opening golden binary file'
C
      returnstatus =  pgs_smf_setdynamicmsg(
     .             PATHFINDER_F_OPEN_ANC_FILE, buf, '' )
C
C  Message buffer now contains the buf string, and the following
C  entry appears in the Status log file:
C 11::PATHFINDER_F_OPEN_ANC_FILE:815104
C FATAL_ERROR...opening golden binary file
C
      RETURN(FATAL_ERROR)
      END

Notes:

The value of the second argument of PGS_SMF_SetDynamicMsg buf is what is stored in the message buffer. The only effect of the first argument (the mnemonic label code) is that the mnemonic label code and string are written to the Status log file. The message that is permanently associated with the mnemonic label for this PGE run, as defined in the runtime ASCII message file PATHFINDER_99, does not change.

Both this method of using PGS_SMF_SetDynamicMsg (Example 2), and tool PGS_SMF_GenerateStatusReport, write a user-defined string to a log file. The differences between them are (1) the two methods write to different log files, (2) the log file entry from PGS_SMF_SetDynamicMsg contains an extra line with mnemonic label string, code, etc., and (3) PGS_SMF_SetDynamicMsg saves its message to the message buffer, for optional later retrieval.

Messages must have been prepared in a status message text file first, and run through the smfcompile utility.

2.2.10 PGS_SMF_SetStaticMsg

Short explanation of what it's for: Saves a pre-defined error/status message to the message buffer. This is the primary mechanism you use to handle errors that have static messages associated with them. Every time this function is called, it writes an entry to the Status log file.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

The example attempts to open a file using PGS_IO_Gen_Open.

The following entry is assumed to appear in the status message text file AVHRR_99.t:

PATHFINDER_F_OPEN_BINARY_FILE   FATAL_ERROR...opening binary file

The corresponding entry then must appear in the runtime ASCII message file PGS_99:

815107,PATHFINDER_F_OPEN_BINARY_FILE,NULL,FATAL_ERROR...error opening 
binary file

C example:

#include <PGS_SMF.h>

#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
returnStatus = PGS_IO_Gen_Open(	GOLDEN_BINARY, 				
                    PGSd_IO_Gen_Read, &processGolden, 1);
if( returnStatus != PGS_S_SUCCESS )
{
   PGS_SMF_SetStaticMsg( PATHFINDER_F_OPEN_BINARY_FILE, 		
                               "YourModuleNameHere" );
/*
   The following entry appears in the Status log file:
11:YourModuleNameHere:PATHFINDER_F_OPEN_BINARY_FILE:815107
FATAL_ERROR...error opening binary file
*/
   return( PATHFINDER_F_OPEN_BINARY_FILE );
}
return( PGS_S_SUCCESS );

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_setstaticmsg
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      INTEGER returnstatus
C
C Begin example
C
C  Try to open a file
C
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
                    PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
      IF ( returnstatus .NE. PGS_S_SUCCESS ) THEN
         pgs_smf_setstaticmsg( PATHFINDER_F_OPEN_BINARY_FILE,
     .                     'yourmodulenamehere' )
         RETURN(PATHFINDER_F_OPEN_BINARY_FILE)
C
C   The following entry appears in the Status log file:
C 11:yourmodulenamehere:PATHFINDER_F_OPEN_BINARY_FILE:815107
C FATAL_ERROR...error opening binary file
C
      END IF
      RETURN(PGS_S_SUCCESS);

Notes:

Ordinarily, you never need to retrieve the message saved to the buffer by this function, since it is written to the Status log file. If you want to retrieve it, e.g., for printing to the screen during SCF development, use PGS_SMF_GetMsg. For a detailed explanation of what is written to the Status log file, see section 3.1.3, "Log files".

2.2.11 PGS_SMF_SetUNIXMsg

Short explanation of what it's for: Saves a Unix-defined error/status message to the message buffer. Every time this function is called, it writes an entry to the Status log file.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Examples:

This hypothetical example shows how to trap the error if an unknown I/O error occurs while your program is using a native language I/O function to read from a file.

C example:

#include <PGS_SMF.h>
#define GOODESX 5004
#define GOODESY 2168
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_SMF_status returnStatus;
int jc,ret;
short shortvals[GOODESX];
/*
Begin example
*/
for (jc=0;jc<GOODESY;jc++)
{
/*
   Read from a previously opened file
*/
   ret=fread(shortvals,sizeof(short),GOODESX,processGolden);
/*
If error detected in "fread" or EOF, go to exception block
*/
   if (	(ret!=GOODESX) || 
                (feof(processGolden)!=0)||
                (ferror(processGolden)!=0)	)
      goto EXCEPTION;
.
.
.
}
.
.
.
return(PGS_S_SUCCESS);
/*
Exception block
*/
EXCEPTION:
/*
Save Unix error string in buffer, write to log file
Add string indicating error occurred while reading land/sea flags
Return error code for processing in calling module
*/
   returnStatus = PGS_SMF_SetUNIXMsg( errno,
      " -- error reading land/sea flags", "YourModuleNameHere" );
   return(PGS_E_UNIX);
/*
   The following entry appears in the Status log file:
11:YourModuleNameHere:PGS_E_UNIX:1798
UNIX: errno=5, I/O error -- error reading land/sea flags
("I/O error" is the Unix error string associated with
errno=5)
*/

FORTRAN example:

This example shows how to check for errors in obtaining the time of the system clock, i.e., the number of seconds elapsed since Jan. 1, 1970. (The example used is different from the C example because of the considerations explained in the Notes section below.)

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.h'
      INTEGER pgs_smf_setunixmsg
      INTEGER itime
      INTEGER ierror
C
C Begin example
C
C Call POSIX FORTRAN subroutine to get system time
C
      CALL PXFTIME( itime, ierror )
      IF( ierror .NE. 0) THEN
C
C Save Unix error string in buffer, write to log file
C
         pgs_smf_setunixmsg( ierror, '',
     .                       'YourModuleNameHere' );
C
C   The following entry appears in the Status log file:
C 11:YourModuleNameHere:PGS_E_UNIX:1798
C UNIX: errno=999, No system time
C [Note that this errno is fabricated]
C
      END IF

Notes:

FORTRAN users must have compiled their code with a FORTRAN POSIX compiler in order to use PGS_SMF_SetUNIXMessage; in addition, this function only traps errors occurring through use of FORTRAN POSIX functions. This is because the ierror parameter is only returned by FORTRAN POSIX functions. For reading and writing files, for example, this means that in order to use this tool, you must use the POSIX FORTRAN functions PXFREAD and PXFWRITE, and not the ANSI FORTRAN functions READ and WRITE. Unfortunately these POSIX functions only are for stream, or CHARACTER, I/O in FORTRAN, and hence are of limited use in the ECS environment; in addition, we are aware that few SCFs are likely to have FORTRAN POSIX compilers at this point. We recommend that in practice you stick with your current FORTRAN error handling for I/O functions. The fact that this function is of limited use to FORTRAN programmers is essentially a consequence of the fact that FORTRAN makes limited use of system calls.

Further explanation of the Status log file entry of the example is in order.

11:YourModuleNameHere:PGS_M_UNIX:1798
UNIX: errno=5, I/O error -- error reading land/sea flags

The first line of an error written by a call to PGS_SMF_SetUNIXMsg is always the same, except for that name of the module YourModuleNameHere (assuming you provided this in the call to the function). The second line gives the Unix errno, as defined in the system include file errno.h (usually in /usr/include or /usr/include/sys). The string "I/O error" is also from that system file. The example also shows how to add your own string, in this case " -- error reading land/sea flags", to the Unix system error string. The additional string is optional; set it to NULL (C) or '' (FORTRAN) if it is not needed.

2.2.12 PGS_SMF_Test*Level

Short explanation of what it's for: To test the status level severity of a returned value of a Toolkit function or your own lower level module.

This function is in file: $PGSSRC/SMF/PGS_SMF.c

Special Note: This section covers a whole group of functions, all of whose names are of the form PGS_SMF_Test*Level. Here "*" represents either Status, Fatal, Error, Warning, Message, UserInfo, or Success. For further explanation see the explanation of status level in section 3.1.2.3, "Constructing the status message text file".

There are two sub-groups of PGS_SMF_Test*Level functions: one, PGS_SMF_TestStatusLevel, and two, all the rest. PGS_SMF_TestErrorLevel is taken as a typical example of group two; the other functions all behave similarly.

Examples:

The first example is of the first sub-group; it shows how to branch if a Toolkit function returns an error of level "_E_" or "_F_".

The second example is of the second sub-group; it shows how to branch if a Toolkit function returns an error of level "_E_".

In both examples, the science software is interpreting a Toolkit error return code of level "_E_" to be a fatal error. (The Toolkit never returns an error code of level "_F_".)

The following entry is assumed to appear in the status message text file AVHRR_99.t:

PATHFINDER_F_OPEN_BINARY_FILE   FATAL_ERROR...opening binary file

Example 1: PGS_SMF_TestStatusLevel

This function takes as input a mnemonic label code, and returns one of the following values, corresponding to the status level of the mnemonic:

    PGS_SMF_MASK_LEV_S
    PGS_SMF_MASK_LEV_M
    PGS_SMF_MASK_LEV_U
    PGS_SMF_MASK_LEV_N
    PGS_SMF_MASK_LEV_W
    PGS_SMF_MASK_LEV_E
    PGS_SMF_MASK_LEV_F

The values of these mnemonics (which are hexadecimals) increase from top to bottom in the above list. That is, e.g., PGS_SMF_MASK_LEV_F is greater than PGS_SMF_MASK_LEV_E.

C example 1:

#include <PGS_SMF.h>
#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
returnStatus = PGS_IO_Gen_Open( GOLDEN_BINARY, PGSd_IO_Gen_Read,
                              &processGolden, 1 );
/*
   Test whether status level is "_E_" or worse
*/
if( PGS_SMF_TestStatusLevel(returnStatus) >= PGS_SMF_MASK_LEV_E )
{
   PGS_SMF_SetStaticMsg(PATHFINDER_F_OPEN_BINARY_FILE, 
                             "YourModuleNameHere");
/*
   The following entry appears in the Status log file:
11:YourModuleNameHere:PATHFINDER_F_OPEN_BINARY_FILE:815107
FATAL_ERROR...error opening binary file
*/
}

FORTRAN example 1:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_teststatuslevel
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      INTEGER returnstatus
C
C Begin example
C
C Try to open a file
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
     .               PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
C
C Test whether status level is "_E_" or worse ("_F_")
      IF ( pgs_smf_teststatuslevel(returnstatus) .GE. 
     .              PGS_SMF_MASK_LEV_E ) THEN
C
         pgs_smf_setstaticmsg( PATHFINDER_F_OPEN_BINARY_FILE,
     .             'yourmodulenamehere' )
C
C   The following entry appears in the Status log file:
C 11:yourmodulenamehere:PATHFINDER_F_OPEN_BINARY_FILE:815107
C FATAL_ERROR...error opening binary file
C
      END IF

Example 2: PGS_SMF_TestErrorLevel

This example also be applies to the Fatal, Warning, Message, UserInfo, and Success functions of class PGS_SMF_Test*Level. These functions return either PGS_TRUE or PGS_FALSE, depending on whether the mnemonic label error code is of the given status level or not. For example, PGS_SMF_TestFatalLevel(PATHFINDER_F_OPEN_BINARY_FILE) returns PGS_TRUE, while the rest of the functions of this sub-group return PGS_FALSE for this argument.

C example 2:

#include <PGS_SMF.h>
#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Try to open a file
*/
returnStatus = PGS_IO_Gen_Open(	GOLDEN_BINARY, 				
                     PGSd_IO_Gen_Read, &processGolden, 1);
/*
   Test whether status level is "_E_"
*/
if( PGS_SMF_TestErrorLevel(returnStatus) == PGS_TRUE )
{
   PGS_SMF_SetStaticMsg(PATHFINDER_F_OPEN_BINARY_FILE, 
                      "YourModuleNameHere");
/*
   The following entry appears in the Status log file:
11:YourModuleNameHere:PATHFINDER_F_OPEN_BINARY_FILE:815107
FATAL_ERROR...error opening binary file
*/
}

FORTRAN example 2:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_openf
      INTEGER pgs_smf_setstaticmsg
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER processgolden
      INTEGER returnstatus
C
C Begin example
C
C Try to open a file
      returnstatus = pgs_io_gen_openf(	GOLDEN_BINARY,
     .                 PGSd_IO_Gen_RSeqUnf, 0, processgolden, 1)
C
C Test whether status level is "_E_"
      IF ( pgs_smf_testerrorlevel(returnstatus) .EQ. 
     .              PGS_TRUE ) THEN
C
         pgs_smf_setstaticmsg( PATHFINDER_F_OPEN_BINARY_FILE,
     .                          'yourmodulenamehere' )
C
C   The following entry appears in the Status log file:
C 11:yourmodulenamehere:PATHFINDER_F_OPEN_BINARY_FILE:815107
C FATAL_ERROR...error opening binary file
C
      END IF

Notes:

Although the examples given are of Toolkit functions returning SMF error codes and then being tested, you can also use these functions to test your own modules, providing they are set up to return SMF error codes.

Please note that the Toolkit never returns a message of level "_F_" (fatal). Therefore the two examples given are actually equivalent ways of doing the same thing.

In reality, if your code detects a fatal error, you will want to branch use the exit() function (C) or STOP statement (FORTRAN). Codes to use for the argument of this function are not yet defined.

3.1.1 Introduction

The next highest level of the Toolkit above the SMF tools includes the Process Control (PC) tools. Their purpose is to provide a direct interface between the science software and the rest of the SDPS, including accessing file attributes (data about files), physical filenames (for use by HDF functions), and other functions. These tools are used internally by many Toolkit functions, such as Generic I/O, Ancillary Access, and other tools.

There are two sets of PC tools: the Command tools, which are callable from Unix shell scripts, and the API tools, callable from C and Fortran. Much of the functionality is duplicated between these two groups; many of the Command tools are simple wrappers on the C code of the API tools, with some exceptions.
For more information about the Command tools see below.
Note: Most of the information in this overview applies to both Command tools and API tools; in particular, both read from the same Process Control File.

The Process Control File (PCF) is central to the PC tools. At the SCF, you construct a PCF using a text editor, one for each PGE. These PCFs are part of the delivery of your software to the DAAC. Your software will access files by logical identifiers (essentially integers, defined by mnemonics). The PCF maps these logical identifiers to physical references (currently physical file names and directories). Each logical identifier corresponds to one or more physical references, or versions. At the SCF, you can use any physical reference you like. In the production environment, the physical reference is supplied by the DAAC. Details are given below.

In this overview section, we walk you through the procedure of constructing your own Process Control File step-by-step, then explain the workings of the pccheck utility, which checks the format of this file. The PCF is read by most of the PC tools (directly or indirectly), and is the current mechanism by which the Toolkit interfaces with the rest of the SDPS. The mechanism may change in the future, but the interface to your code will not.

3.1.2 Constructing your Process Control file

This section explains how to customize a Process Control File for use in your code.

A default Process Control File (PCF) is included in the TK5.1.1 delivery. It contains entries which are either required or optional for use of many Toolkit functions. This file is named $PGSRUN/PCF.relA.template. The particular example we use here is from the Pathfinder AVHRR/Land Toolkit Prototype study. The complete example file appears in Appendix B of this document.

It is recommended that you start with the same (customized) copy of the PCF each time you run at the SCF, especially if you are using temporary files in your processing. You don't want previous temporary file references in the PCF, since these files are deleted by the system (unless you are not using PGS_PC_Shell.sh or PGS_PC_TermCom).

The Unix environment variable $PGS_PC_INFO_FILE must point to your Process Control file in order for the Toolkit to work at all.

We go through the example file section-by-section. The sections of a Process Control file include:

• SYSTEM RUNTIME PARAMETERS
• PRODUCT INPUT
• PRODUCT OUTPUT
• SUPPORT INPUT
• SUPPORT OUTPUT
• USER-DEFINED RUNTIME PARAMETERS
• INTERMEDIATE INPUT
• INTERMEDIATE OUTPUT
• TEMPORARY IO

All sections of the PCF, except the SYSTEM RUNTIME PARAMETERS and USER- DEFINED RUNTIME PARAMETERS sections, consist of names, locations and other data about physical files. Each of these sections has a default file location, which is at the beginning of the section. The default file location is delimited by a '!' in column one of the PCF. This location points to the default directory in which these files are stored. This may be overridden for individual files, by inserting the fully-qualified physical directory path, as explained below. The PRODUCT INPUT section provides a detailed example of considerations that apply to all sections that involve files. Explanations of other sections provide only differences unique to those sections.

General considerations:

#   Process Control File: Pathfinder AVHRR/Land Toolkit
#      Prototype
#
#   Env variable PGS_PC_INFO_FILE must point to this file

Comments in a PCF are any lines that begin in the first column with "#".

?   SYSTEM RUNTIME PARAMETERS

The "?" symbol in the first column defines this line as the subject of the section. These nine subject names must not be changed nor deleted from the PCF.

Blank lines are not allowed.

Pipe character "|" must be used to delimit fields.

The exclamation point "!" must be used to designate the default file location. This must appear before any file entries in each section of the PCF.

The entire length of any line in the PCF may not exceed 1000 characters.

Different sections of the PCF have different numbers of required and optional fields for each entry. In the examples below, each entry is identified as required or optional.

?   SYSTEM RUNTIME PARAMETERS
# ----------------------------------------------------------
# Production Run ID - unique production run identifier
# ----------------------------------------------------------
1

# ----------------------------------------------------------
# Software ID - unique software configuration 
identifier
# ----------------------------------------------------------
1

?   PRODUCT INPUT FILES 
# [ next line is for default location ]
! ~/runtime

# ----------------------------------------------------------
# Pathfinder AVHRR/Land input files
# ----------------------------------------------------------
201|87002002709.no9_gac|||||1
401|goldtopolandsea8.bin|||||1
402|gridtoms_1987_sngl_ntwk|||||1
403|ephem8788.dat|||||1
404|timecorr8788.dat|||||1
405|SDSannotations.dat|||||1
406|HDFmetadata.dat|||||1
410|jan021987.proclog|||||1

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|||||1

#define GAC_FILE 201

      PARAMETER (GAC_FILE=201)

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|/fire2/toma/data||||1

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|||||1

201|87002002709.no9_gac|||||1

201|87002002710.no9_gac|||||2
201|87002002709.no9_gac|||||1

?   PRODUCT OUTPUT FILES 
# [ next line is for default location ]
! ~/runtime
#
# ----------------------------------------------------------
# Pathfinder AVHRR/Land main output file
# ----------------------------------------------------------
301|test11.hdf|||||1

#define HDF_FILE      301

      PARAMETER (HDF_FILE=301)

?   SUPPORT INPUT FILES 
# [ next line is for default location ]
! ~/runtime
#
# ----------------------------------------------------------
# Pathfinder AVHRR/Land support input files
# ----------------------------------------------------------

?   SUPPORT OUTPUT FILES 
# [ next line is for default location ]
! ~/runtime
#

?   USER DEFINED RUNTIME PARAMETERS
#
# ----------------------------------------------------------
# Pathfinder AVHRR/Land runtime parameters
# ----------------------------------------------------------
601|requested_size_x|409
602|requested_size_y|128
603|wait_time|3

601|requested_size_x|409

601|requested_size_x|409

601|requested_size_x|409

?   INTERMEDIATE INPUT
# [ next line is for default location]
! ~/runtime
#

?   INTERMEDIATE OUTPUT
# [next line is for default location]
! ~/runtime
#

?   TEMPORARY IO
# [ next line is for default location ]
! ~/runtime
#
# ----------------------------------------------------------
# Pathfinder AVHRR/Land temporary file
# ----------------------------------------------------------
901|pc1157318822894183312||0|0|0|0

#define BINARY_OUTPUT		901

      PARAMETER (BINARY_OUTPUT=901)

? END

3.1.3 Checking your Process Control File

Now that you have created your PCF, you can use it from your software through use of the Toolkit. However, you might want to check it to see if you have entered everything correctly. You can do this by using the pccheck utility, a Unix executable included with the Toolkit. This program is compiled at the time of Toolkit installation, and is located in directory $PGSBIN. You execute shell script pccheck.sh, which calls executable pctcheck; its source code is $PGSSRC/PC/PGS_PC_Check.c . To run it on your file mypcfile, on the Unix command line type

$PGSBIN/pccheck.sh -i mypcfile

If there are any errors in your file, you will see messages of the form

Error - problem with version number in Standard input file
Line number:  23
Line:  401|goldtopolandsea8.bin|||||

In this example the version number was omitted from the STANDARD INPUT file entry.

At the end, you will see a summary of the form

Check of mypcfile completed
Errors found:  7
Warnings found:  0

For this utility, a pccheck error is defined as a PCF entry that will cause a Toolkit PC function to return an error message. A pccheck warning is defined as an incorrect entry that will not cause the Toolkit trouble, but may cause the PGE to operate incorrectly. For example, a blank character in the file name field does not bother a Toolkit PC function, since it simply returns the string as is; pccheck will not return an error. But a blank character will certainly cause a Unix error, when the file open is attempted by a Toolkit function; pccheck will return a warning to this effect. Output is returned to stdout (usually the screen).

This is a simple explanation of how the pccheck utility works. For details, including a list of error messages, and information about other command line options, see "Validating Process Control Files", sec. C.2 of Appendix C, in the Toolkit Users Guide.

3.1.4 Metadata considerations

Protoype metadata (MET) tools, which format standard product metadata for ingest to the data server(SDS), will be available in the next Toolkit delivery.

For the present, the only Toolkit functions that deal with data about data are the tools PGS_PC_GetFileAttr, PGS_PC_GetFileAttrCom and PGS_PC_GetFileByAttr. These functions are involved in retrieving file "attribute" data from the system, via the Process Control file. Essentially, you can get character string metadata from a text file using these functions.

Since details about how the production system handles metadata are not yet available, this mechanism was determined to be the best that can be done about this issue at the moment. Every effort will be made to keep the calling sequence unchanged for these tools in the future. However, given the uncertainties about this issue, this cannot be guaranteed.

3.1.5 Command tools for use in shell scripts

This section briefly describes the usage of the set of Command tools, which are callable from Unix shell scripts. These tools are generally identified by the suffix "Com" in the function name.

Tool PGS_PC_Shell.sh is used to call your PGE. It is strongly recommended that you call your PGE from this function during testing at the SCF; among other things, it enables Toolkit (not user) shared memory, which speeds execution of certain Toolkit functions.
You must use either this tool or PGS_PC_InitCom if you want to enable the creation of Toolkit log files.
You must use either this tool or PGS_PC_TermCom if you want to send any files to a remote machine, through use of function PGS_SMF_SendRuntimeData.

Tools PGS_PC_InitCom and PGS_PC_TermCom are used to initialize and terminate your PGE respectively.
Ordinarily you would not use these, as they are called internally by PGS_PC_Shell.sh. They are included in the Toolkit documentation for reference in case you wish to customize PGS_PC_Shell.sh for some reason; however, please note that any such customization is not part of your delivery to the DAAC.

Note: The above three functions are used at the DAAC as well as at your SCF; however, it is not necessary to include them in scripts that you deliver to the DAAC for Integration and Test. They are included as part of the Toolkit delivery for your use in testing at the SCF.
The three functions are to be used outside of your PGE.

The rest of the PGS_PC_*Com tools are simply wrappers on Toolkit API tools. These functions are for use inside your PGE.

Further details are given in the Tool Descriptions.

3.2.1 PGS_PC_InitCom

Short explanation of what it's for: Command line function for initializing the Toolkit for use with your PGE.
Normally not used at the SCF, since its functionality is fully covered by PGS_PC_Shell.sh.

This function is in file: $PGSSRC/PC/PGS_PC_InitCom.c

Shell example:

# Execute a PGE, enabling Toolkit (not user) shared memory, 
#    and also initializing creation of Toolkit log files,
# and have an SMF Cache size of 50.

unix% PGS_PC_InitCom 1 1 50

Notes:

You might want to use this function if you decide to write a custom script to call your PGE, in lieu of using PGS_PC_Shell.sh; however, please note that any such customization is not part of your delivery to the DAAC.

This function enables your PGE to

• Use Toolkit (not user) shared memory, which speeds up Toolkit processing
• Automatically load the Process Control File into Toolkit shared memory
• Automatically create Toolkit log files

The first argument of this function is for turning on or off Toolkit (not user) shared memory, if available; the second argument is for turning on or off creation of Toolkit log files. The third argument is to specify the amount (in records) of SMF Cache memory to reserve for the storage of SMF messages.

In the example "unix%" is the Unix command line prompt.

For this particular function, the example is identical for any Unix shell.

3.2.2 PGS_PC_GenUniqueID

Short explanation of what it's for: Generates a string that uniquely identifies your standard product output file. May be used as file metadata.

This function is in file: $PGSSRC/PC/PGS_PC_GenUniqueID.c

Examples:

The examples assume the following exist in the Process Control File (PCF):

?   SYSTEM RUNTIME PARAMETERS
# ----------------------------------------------------------
# Production Run ID - unique production instance identifier
# ----------------------------------------------------------
1
# ----------------------------------------------------------
# Software ID - unique software configuration identifier
# ----------------------------------------------------------
1
C example:

#include <PGS_PC.h> 
#define HDF_FILE      301
char uniqueID[PGSd_PC_LABEL_SIZE_MAX];
PGSt_SMF_status returnStatus;
/*
Begin example
*/
returnStatus = PGS_PC_GenUniqueID(HDF_FILE,uniqueID);
/*
Variable uniqueID now contains the string
"PRID - 1 SID - 1 PRODID - 301"
*/
Fortran example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INTEGER pgs_pc_genuniqueid
      INTEGER HDF_FILE
      PARAMETER(HDF_FILE=301)
      CHARACTER*200 uniqueid
      INTEGER returnstatus
C
C Begin example
C
      returnstatus = pgs_pc_genuniqueid(HDF_FILE,uniqueid)
C
C Variable uniqueid now contains the string
C 'PRID - 1 SID - 1 PRODID - 301'
C
Notes:

The mechanism for using the output of this function as file metadata has not yet been defined.

4.2.1 PGS_IO_Gen_Close

Short explanation of what it's for: Close a file that was opened by PGS_IO_Gen_Open or PGS_IO_Gen_Temp_Open (C version).

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_Close.c

Examples:

C example:

#include <PGS_IO.h> 
PGSt_IO_Gen_FileHandle *handle;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
returnStatus = PGS_IO_Gen_Close( handle );

FORTRAN example:

This function is not callable from FORTRAN. See PGS_IO_Gen_CloseF .

Notes:

The Toolkit internally keeps track of which files have been opened and closed.

4.2.2 PGS_IO_Gen_CloseF

Short explanation of what it's for: Close a file that was opened by PGS_IO_Gen_OpenF or PGS_IO_Gen_Temp_OpenF (FORTRAN version).

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_CloseF.f

Examples:

C example:

This function is not callable from C. See PGS_IO_Gen_Close.

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER pgs_io_gen_closef
      INTEGER handle
      INTEGER returnstatus
C
C Begin example
C
      returnstatus = pgs_io_gen_closef(handle)

Notes:

The Toolkit internally keeps track of which files have been opened and closed.

4.2.3 PGS_IO_Gen_Open

Short explanation of what it's for: Open a generic file (C version). Intended for use in opening miscellaneous files in your software (see Notes).

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_Open.c

Examples:

The example assumes the following exists in the Process Control File (PCF):

?   PRODUCT INPUT FILES 
# ----------------------------------------------------------
# Pathfinder AVHRR/Land input files
# ----------------------------------------------------------
401|goldtopolandsea21.bin|||||2
401|goldtopolandsea33.bin|||||1

C example:

#include <stdio.h>
#include <PGS_IO.h>
#define GOLDEN_BINARY 401
PGSt_IO_Gen_FileHandle *processGolden;
PGSt_integer version;
long xScale;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
   Open a file for read
*/
version = 1;
returnStatus = PGS_IO_Gen_Open( GOLDEN_BINARY,
                  PGSd_IO_Gen_Read, &processGolden, version);
/*
   The file $PGS_PRODUCT_INPUT/goldtopolandsea21.bin is now open
      (see Notes)
*/
/*
    File handle variable processGolden may now be used as
    an argument to any native C I/O function that takes a
    variable of type FILE as input, e.g.,
*/
fscanf( processGolden, "%ld", &xScale );

FORTRAN example:

This function is not callable from FORTRAN. See PGS_IO_Gen_OpenF .

Notes:

Use NCSA HDF open function Hopen to open standard product files. Use other Toolkit functions to open Ancillary and Level 0 files. Use PGS_IO_Gen_Temp_Open to open temporary and intermediate files. This function is for opening any other files.

In the example, the user requested version 1 of the GOLDEN_BINARY file to be opened. This refers to the first entry in the PC file, i.e., file goldtopolandsea21.bin. The sequence numbers in the PC file are in reverse order from the version numbers used in arguments to Toolkit functions. So in the PC file, the entry for file goldtopolandsea21.bin has sequence number 2. For further explanation of sequence numbers in the PC file, see section 4.1.2.2 of the Process Control Overview, "PRODUCT INPUT".

The Toolkit internally keeps track of which files have been opened and closed.

A valid Process Control file (PCF) must have been constructed before using this tool. See section 4, "Process Control (PC) Tools".

The following is a complete listing of the access modes available (2nd argument in calling sequence):

PGS_IO_Gen_Open Access Modes

                 
    Toolkit                 C        Description


PGSd_IO_Gen_Read         "r"   Open file for reading
PGSd_IO_Gen_Write        "w"   Open file for writing, truncating 
                                existing file to 0 length, or 
                                creating a new file
PGSd_IO_Gen_Append       "a"   Open file for writing, appending 
                                to the end of existing file, or 
                                creating file
PGSd_IO_Gen_Update       "r+"  Open file for reading and writing
PGSd_IO_Gen_Trunc        "w+"  Open file for reading and writing,
                                truncating existing file to zero 
                                length, or creating new file
PGSd_IO_Gen_AppendUpdate "a+"  Open file for reading and writing,
                                appending to the end of existing 
                                file, or creating a new file; 
                                whole file can be read, but 
                                writing only appended

ToolkitMnemonic used as 2nd argument in calling sequenceCEquivalent access mode for native POSIX C function fopenDescriptionToolkit access mode description

!!!!!!!!!!! During testing of this tool, the mode AppendUpdate (a+)
!! ALERT !! was found to produce results that were not consistent
!!!!!!!!!!! with the documented POSIX standard.The sort of behavior
that was typically observed was for data, buffered during a read
operation, to be appended to the file along with other data that was
being written to the file. Note that this behavior could not be attributed
to the Toolkit since the same behavior was revealed when purely "POSIX"
calls were used.

If you are using the Toolkit LogStatus log file, you may see a sequence of messages like this:

PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

PGS_PC_GetPCSDataLocateEntry():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

PGS_PC_GetPCSData():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

The presence of these messages is an artifact of the way the Toolkit access the Process Control file. Up to 4 sets of these messages (12 total) are generated each time PGS_IO_Gen_Open is called.
A means of limiting this to one set of messages will be in place by the TK5 delivery of July 1995.

4.2.4 PGS_IO_Gen_OpenF

Short explanation of what it's for: Open a generic file (FORTRAN version). Intended for use in opening miscellaneous files in your software (see Notes).

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_OpenF.f (f77 version), $PGSSRC/IO/GEN/PGS_IO_Gen_OpenF90.f (F90 version).

Examples:

The example assumes the following exists in the Process Control File (PCF):

?   PRODUCT INPUT FILES 
# ----------------------------------------------------------
# Pathfinder AVHRR/Land input files
# ----------------------------------------------------------
401|goldtopolandsea21.bin|||||2
401|goldtopolandsea33.bin|||||1
C example:

This function is not callable from C. See PGS_IO_Gen_Open .

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER GOLDEN_BINARY
      PARAMETER (GOLDEN_BINARY=401)
      INTEGER pgs_io_gen_openf
      INTEGER processgolden
      INTEGER version
      INTEGER xscale
      INTEGER returnstatus
C

C Begin example
C
C Open a sequential unformatted file for read
C
      version = 1
      returnstatus = pgs_io_gen_openf( GOLDEN_BINARY,
     .         PGSd_IO_Gen_RSeqUnf, 0, processgolden, version)
C
C The file $PGS_PRODUCT_INPUT/goldtopolandsea21.bin is now open
C     (see Notes)
C
C File handle variable processGolden may now be used as
C    an argument to any native I/O FORTRAN function, e.g.,
C
      READ(processGolden) xscale

Notes:

Use NCSA HDF open function Hopen to open standard product files. Use other Toolkit functions to open Ancillary and Level 0 files. Use PGS_IO_Gen_Temp_OpenF to open temporary and intermediate files. This function is for opening any other files.

All FORTRAN access modes are supported. For an example of using direct access files, see the examples for function PGS_IO_Gen_Temp_Open.

The 3rd argument of PGS_IO_Gen_OpenF is for specifying record length.
In FORTRAN 77, this value must be at least 1 for direct access files; it is ignored for sequential files (as in the example).
In FORTRAN 90, this value must be at least 1 for direct access files. For sequential access, if this value is 0, the file is opened with a platform-dependent record length; otherwise, it is opened with the specified record length.

Files which are opened with one of the direct access modes must have been created by FORTRAN direct access writes. That is, you cannot expect to read a file with direct access reads if the file is sequential.

In the example, the user requested version 1 of the GOLDEN_BINARY file to be opened. This refers to the first entry in the PC file, i.e., file goldtopolandsea21.bin. The sequence numbers in the PC file are in reverse order from the version numbers used in arguments to Toolkit functions. So in the PC file, the entry for file goldtopolandsea21.bin has sequence number 2. For further explanation of sequence numbers in the PC file, see section 4.1.2.2 of the Process Control Overview, "PRODUCT INPUT".

The Toolkit internally keeps track of which files have been opened and closed.

A valid Process Control file (PCF) must have been constructed before using this tool. See section 4, "Process Control (PC) Tools".

The following is a complete listing of the access modes available (3rd argument in calling sequence):

               PGS_IO_Gen_OpenF Access Modes
                 

                                   ****FORTRAN****
    Toolkit             mode      'access='     'form='
PGSd_IO_Gen_RSeqFrm   Read    Sequential   Formatted
PGSd_IO_Gen_RSeqUnf   Read    Sequential   Unformatted
PGSd_IO_Gen_RDirFrm   Read    Direct       Formatted
PGSd_IO_Gen_RDirUnf   Read    Direct       Unformatted
       
PGSd_IO_Gen_WSeqFrm   Write   Sequential   Formatted
PGSd_IO_Gen_WSeqUnf   Write   Sequential   Unformatted
PGSd_IO_Gen_WDirFrm   Write   Direct       Formatted
PGSd_IO_Gen_WDirUnf   Write   Direct       Unformatted
       
PGSd_IO_Gen_USeqFrm   Update  Sequential   Formatted
PGSd_IO_Gen_USeqUnf   Update  Sequential   Unformatted
PGSd_IO_Gen_UDirFrm   Update  Direct       Formatted
PGSd_IO_Gen_UDirUnf   Update  Direct       Unformatted

The following modes are available in FORTRAN 90 only:

PGSd_IO_Gen_ASeqFrm   Append  Sequential   Formatted
PGSd_IO_Gen_ASeqUnf   Append  Sequential   Unformatted

ToolkitMnemonic used as 2nd argument in calling sequencemodeType of access allowed'access='Equivalent argument of ACCESS parameter in FORTRAN OPEN function'form='Equivalent argument of FORM parameter in FORTRAN OPEN function

If you are using the Toolkit LogStatus log file, you may see a sequence of messages like this:

PGS_PC_GetPCSDataGetIndex():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

PGS_PC_GetPCSDataLocateEntry():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

PGS_PC_GetPCSData():PGSPC_W_NO_FILES_EXIST:76802
No files exist for product group

The presence of these messages is an artifact of the way the Toolkit access the Process Control file. Up to 4 sets of these messages (12 total) are generated each time PGS_IO_Gen_OpenF is called.
A means of limiting this to one set of messages will be in place by the TK5 delivery of July 1995.

This function corresponds to two similar but separate source code files, one for FORTRAN 77, and one for FORTRAN 90. The FORTRAN 90 version has all the functionality of the FORTRAN 77 version, plus support for (1) Append mode and (2) specification of record length for sequential files.
The installation script compiles one of these versions, based on which flavor of FORTRAN you specified at the time of Toolkit installation.

4.2.5 PGS_IO_Gen_Temp_Delete

Short explanation of what it's for: Mark a temporary file for deletion. You would use this tool if you want to re-use a logical ID used by a temporary file you no longer need, to open a new temporary file.

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_Temp_Delete.c.

Examples:

C example:

#include <PGS_IO.h>
#define TEMP_BINARY 901
PGSt_SMF_status returnStatus;

returnStatus = PGS_IO_Gen_Temp_Delete( TEMP_BINARY );

/* The file corresponding to logical ID TEMP_BINARY is 
   now marked for deletion; the logical ID may be used to 
   open another file using PGS_IO_Gen_Temp_Open. */

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER TEMP_BINARY
      PARAMETER (TEMP_BINARY=901)
      INTEGER pgs_io_gen_temp_delete
      INTEGER returnstatus
C
C Begin example
C
      returnstatus = pgs_io_gen_temp_delete( TEMP_BINARY )
C The file corresponding to logical ID TEMP_BINARY is 
C  now marked for deletion; the logical ID may be used to 
C  open another file using PGS_IO_Gen_Temp_OpenF.

Notes:

This function is only for use with Temporary files, and not Intermediate files.

This function merely marks Temporary files for deletion, so you can re-use the same logical ID. It does not physically delete files.

If you are using PGS_PC_Shell.sh to wrap your PGE, then your Temporary files are automatically deleted at PGE termination.
If you are not wrapping your PGE with the shell, you should delete your Temporary files manually before each of your test runs. In addition, you need to start with a fresh Process Control file, i.e., the PCF at the beginning of any run should have no entries in the TEMPORARY I/O section.
In the production environment, Temporary files are always deleted automatically at the end of your PGE (since PGS_PC_Shell.sh is used there).

This is the only PGS_IO_Gen function that has no separate FORTRAN version. FORTRAN access is provided through bindings to the C code, as in the rest of the Toolkit.

4.2.6 PGS_IO_Gen_Temp_Open

Short explanation of what it's for: Open a temporary or intermediate file (C version). Temporary files exist for the duration of one PGE only; intermediate files may have a longer duration.

This function is in file: $PGSSRC/IO/GEN/PGS_IO_Gen_Temp_Open.c.

Examples:

The example assumes the following exists in the Process Control File (PCF):

?   INTERMEDIATE INPUT
# [set env var PGS_INTERMEDIATE_INPUT for default location]
701|pc1150283201028000395104034|||||
#
?   INTERMEDIATE OUTPUT
# [set env var PGS_INTERMEDIATE_OUTPUT for default location]
#
?   TEMPORARY IO
# [set env var PGS_TEMPORARY_IO for default location]
#
?   END

C example:

#include <PGS_IO.h>
#define INTERMEDIATE_IN 701
#define INTERMEDIATE_OUT 801
#define TEMP_BINARY 901
PGSt_IO_Gen_FileHandle *handle;
long xScale;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
Open the existing intermediate input file for read
Read a value
Close it
*/
returnStatus = PGS_IO_Gen_Temp_Open( PGSd_IO_Gen_Endurance,
                  INTERMEDIATE_IN, PGSd_IO_Gen_Read, &handle );
fscanf( handle, "%ld", &xScale );
returnStatus = PGS_IO_Gen_Close( handle );
/*
You have just read a value from file
$PGS_INTERMEDIATE_INPUT/pc1150283201028000395104034
*/
/*
Open a new intermediate output file for write
Write a value
Close it
*/
returnStatus = PGS_IO_Gen_Temp_Open( PGSd_IO_Gen_Endurance,
                 INTERMEDIATE_OUT, PGSd_IO_Gen_Write, &handle );
fprintf( handle, "%ld", xScale );
returnStatus = PGS_IO_Gen_Close( handle );
/*
You have just written a value to a new file in directory
$PGS_INTERMEDIATE_OUTPUT
*/
/*
Open a temporary file for write
Write a value
Close it
*/
returnStatus = PGS_IO_Gen_Temp_Open( PGSd_IO_Gen_NoEndurance,
                 TEMP_BINARY, PGSd_IO_Gen_Write, &handle );
fprintf( handle, "%ld", xScale );
returnStatus = PGS_IO_Gen_Close( handle );
/*
You have just written a value to a new file in directory
$PGS_TEMPORARY_IO
*/

FORTRAN example:

This function is not callable from FORTRAN. See PGS_IO_Gen_Temp_OpenF .

Notes:

The difference between this function and PGS_IO_Gen_Open is that this tool enables tracking of temporary and intermediate files in the production system, as well as providing for file name generation.

The following applies to Temporary, not Intermediate, files:

Process Control file entries for Temporary files are generated automatically by the Toolkit. You should never create a PCF entry for a Temporary file.

If you are using PGS_PC_Shell.sh to wrap your PGE, then your Temporary files are automatically deleted at PGE termination.
If you are not wrapping your PGE with the shell, you should delete your Temporary files manually before each of your test runs. In addition, you need to start with a fresh Process Control file, i.e., the PCF at the beginning of any run should have no entries in the TEMPORARY I/O section.
In the production environment, Temporary files are always deleted automatically at the end of your PGE (since PGS_PC_Shell.sh is used there).

The first argument of PGS_IO_Gen_Temp_Open specifies whether the file to open is Temporary or Intermediate. Currently this is a simple Boolean value. In the future this argument may be changed to specify the duration of an intermediate file. The calling sequence will not change.

After the Toolkit calls given in the example have been executed, the last 3 sections of the Process Control file will look like this:

?   INTERMEDIATE INPUT
# [set env var PGS_INTERMEDIATE_INPUT for default location]
701|pc1150283201028000395104034|||||
#
?   INTERMEDIATE OUTPUT
# [set env var PGS_INTERMEDIATE_OUTPUT for default location]
801|pc1150283201039509195162200|||||
#
?   TEMPORARY IO
# [set env var PGS_TEMPORARY_IO for default location]
901|pc1150283201039509195162229|||||
#
?   END

-- the toolkit has created file names for the new files and stored them in the PCF.

See the Notes section of tool PGS_PC_GetTempReferenceCom for an explanation of the form of the temporary file names.

A valid Process Control file (PCF) must have been constructed before using this tool. See section 4, "Process Control (PC) Tools".

The following is a complete listing of the access modes available (3rd argument in calling sequence):

PGS_IO_Gen_Temp_Open Access Modes

                 
    Toolkit                 C        Description


PGSd_IO_Gen_Read         "r"   Open file for reading
PGSd_IO_Gen_Write        "w"   Open file for writing, truncating 
                                existing file to 0 length, or 
                                creating a new file
PGSd_IO_Gen_Append       "a"   Open file for writing, appending 
                                to the end of existing file, or 
                                creating file
PGSd_IO_Gen_Update       "r+"  Open file for reading and writing
PGSd_IO_Gen_AppendUpdate "a+"  Open file for reading and writing,
                                appending to the end of existing 
                                file, or creating a new file; 
                                whole file can be read, but 
                                writing only appended

ToolkitMnemonic used as 3rd argument in calling sequenceCEquivalent access mode for native POSIX C function fopenDescriptionToolkit access mode description

!!!!!!!!!!! During testing of this tool, the mode AppendUpdate (a+)
!! ALERT !! was found to produce results that were not consistent
!!!!!!!!!!! with the documented POSIX standard.The sort of behavior
that was typically observed was for data, buffered during a read
operation, to be appended to the file along with other data that was
being written to the file. Note that this behavior could not be attributed
to the Toolkit since the same behavior was revealed when purely "POSIX"
calls were used.

4.2.7 PGS_IO_Gen_Temp_OpenF

Short explanation of what it's for: Open a temporary or intermediate file (FORTRAN version). Temporary files exist for the duration of one PGE only; intermediate files may exist for a longer duration.

This function is in file:
$PGSSRC/IO/GEN/PGS_IO_Gen_Temp_OpenF.f (F77 version),
$PGSSRC/IO/GEN/PGS_IO_Gen_Temp_OpenF90.f (F90 version).

Examples:

The example assumes the following exists in the Process Control File (PCF):

?   INTERMEDIATE INPUT
# [set env var PGS_INTERMEDIATE_INPUT for default location]
701|pc1150283201028000395104034|||||
#
?   INTERMEDIATE OUTPUT
# [set env var PGS_INTERMEDIATE_OUTPUT for default location]
#
?   TEMPORARY IO
# [set env var PGS_TEMPORARY_IO for default location]
#
?   END

C example:

This function is not callable from C. See PGS_IO_Gen_Temp_Open .

FORTRAN example:

      IMPLICIT NONE
      INCLUDE 'PGS_SMF.f'
      INCLUDE 'PGS_PC.f'
      INCLUDE 'PGS_PC_9.f'
      INCLUDE 'PGS_IO.f'
      INCLUDE 'PGS_IO_1.f'
      INTEGER INTERMEDIATE_IN
      PARAMETER (INTERMEDIATE_IN=701)
      INTEGER INTERMEDIATE_OUT
      PARAMETER (INTERMEDIATE_OUT=801)
      INTEGER TEMP_BINARY
      PARAMETER (TEMP_BINARY=901)
      INTEGER pgs_io_gen_temp_openf
      INTEGER pgs_io_gen_closef
      INTEGER handle
      INTEGER xscale
      INTEGER recordlength
      INTEGER returnstatus
C
C Begin example
C
C Open the existing intermediate sequential unformatted
C     input file for read
C Read a value
C Close it
C
      returnstatus = pgs_io_gen_temp_openf( PGSd_IO_Gen_Endurance,
     .   INTERMEDIATE_IN, PGSd_IO_Gen_RSeqUnf, 0, handle )
      READ(handle) xscale
      returnstatus = pgs_io_gen_closef( handle )
C
C You have just read a value from file
C $PGS_INTERMEDIATE_INPUT/pc1150283201028000395104034
C
C Open a new intermediate direct access unformatted
C     output file for write
C Write a value
C Close it
C
      recordlength = 4
      returnstatus = pgs_io_gen_temp_openf(
     .    PGSd_IO_Gen_Endurance, INTERMEDIATE_OUT,
     .    PGSd_IO_Gen_WDirUnf, recordlength, handle )
      WRITE( handle, REC=1 ) xscale
      returnstatus = pgs_io_gen_close( handle )
C
C You have just written a value to a new file in directory
C $PGS_INTERMEDIATE_OUTPUT
C
C Open a temporary sequential formatted file for write
C Write a value
C Close it
C
      returnstatus = pgs_io_gen_temp_openf(
     .    PGSd_IO_Gen_NoEndurance, TEMP_BINARY,
     .    PGSd_IO_Gen_WSeqFrm , 0, handle )
      WRITE( handle, 100 ) xscale
100   FORMAT(I6)
      returnstatus = pgs_io_gen_close( handle )
C
C You have just written a value to a new file in directory
C $PGS_TEMPORARY_IO
C

Notes:

The difference between this function and PGS_IO_Gen_Open is that this tool enables tracking of temporary and intermediate files in the production system, as well as providing for file name generation.

The following applies to Temporary, not Intermediate, files:

Process Control file entries for Temporary files are generated automatically by the Toolkit. You should never create a PCF entry for a Temporary file.

If you are using PGS_PC_Shell.sh to wrap your PGE, then your Temporary files are automatically deleted at PGE termination.
If you are not wrapping your PGE with the shell, you should delete your Temporary files manually before each of your test runs. In addition, you need to start with a fresh Process Control file, i.e., the PCF at the beginning of any run should have no entries in the TEMPORARY I/O section.
In the production environment, Temporary files are always deleted automatically at the end of your PGE (since PGS_PC_Shell.sh is used there).

The first argument of PGS_IO_Gen_Temp_Open specifies whether the file to open is Temporary or Intermediate. Currently this is a simple Boolean value. In the future this argument may be changed to specify the duration of an intermediate file. The calling sequence will not change.

After the Toolkit calls given in the example have been executed, the last 3 sections of the Process Control file will look like this:

?   INTERMEDIATE INPUT
# [set env var PGS_INTERMEDIATE_INPUT for default location]
701|pc1150283201028000395104034|||||
#
?   INTERMEDIATE OUTPUT
# [set env var PGS_INTERMEDIATE_OUTPUT for default location]
801|pc1150283201039509195162200|||||
#
?   TEMPORARY IO
# [set env var PGS_TEMPORARY_IO for default location]
901|pc1150283201039509195162229|||||
#
?   END

-- the toolkit has created file names for the new files and stored them in the PCF.

See the Notes section of tool PGS_PC_GetTempReferenceCom for an explanation of the form of the temporary file names.

All FORTRAN access modes are supported.

The 4th argument of PGS_IO_Gen_Temp_OpenF is for specifying record length.

In FORTRAN 77, this value must be at least 1 for direct access files; it is ignored for sequential files (as in the example).
In FORTRAN 90, this value must be at least 1 for direct access files. For sequential access, if this value is 0, the file is opened with a platform-dependent record length; otherwise, it is opened with the specified record length.

Files which are opened with one of the direct access modes must have been created by FORTRAN direct access writes. That is, you cannot expect to read a file with direct access reads if the file is sequential.

Temporary files are deleted automatically at PGE termination in the production environment. You may also delete them sooner by using tool PGS_IO_Gen_Temp_Delete.

A valid Process Control file (PCF) must have been constructed before using this tool. See section 4, "Process Control (PC) Tools".

The following is a complete listing of the access modes available (3rd argument in calling sequence):

PGS_IO_Gen_Temp_OpenF Access Modes

                 

                                   ****FORTRAN****
    Toolkit             mode      'access='     'form='
PGSd_IO_Gen_RSeqFrm   Read    Sequential   Formatted
PGSd_IO_Gen_RSeqUnf   Read    Sequential   Unformatted
PGSd_IO_Gen_RDirFrm   Read    Direct       Formatted
PGSd_IO_Gen_RDirUnf   Read    Direct       Unformatted
       
PGSd_IO_Gen_WSeqFrm   Write   Sequential   Formatted
PGSd_IO_Gen_WSeqUnf   Write   Sequential   Unformatted
PGSd_IO_Gen_WDirFrm   Write   Direct       Formatted
PGSd_IO_Gen_WDirUnf   Write   Direct       Unformatted
       
PGSd_IO_Gen_USeqFrm   Update  Sequential   Formatted
PGSd_IO_Gen_USeqUnf   Update  Sequential   Unformatted
PGSd_IO_Gen_UDirFrm   Update  Direct       Formatted
PGSd_IO_Gen_UDirUnf   Update  Direct       Unformatted

The following modes are available in FORTRAN 90 only:

PGSd_IO_Gen_ASeqFrm   Append  Sequential   Formatted
PGSd_IO_Gen_ASeqUnf   Append  Sequential   Unformatted

ToolkitMnemonic used as 2nd argument in calling sequencemodeType of access allowed'access='Equivalent argument of ACCESS parameter in FORTRAN OPEN function'form='Equivalent argument of FORM parameter in FORTRAN OPEN function

This function corresponds to two similar but separate source code files, one for FORTRAN 77, and one for FORTRAN 90. The FORTRAN 90 version has all the functionality of the FORTRAN 77 version, plus support for (1) Append mode and (2) specification of record length for sequential files.
The installation script compiles one of these versions, based on which flavor of FORTRAN you specified at the time of Toolkit installation.

5.1.1 Dynamic Memory Tools

These tools are essentially wrappers on the native memory allocation functions, with the addition of Toolkit error handling. In C these native functions include malloc, calloc, and free. The purpose of providing these wrappers is to enable the tracking of memory usage in the production environment. (Note that currently there is no SDPS mechanism external to the Toolkit which does this.) In contrast to the shared memory tools, these tools are for use within a single executable of your code. These functions use link-list utilities internally, in order to keep track of memory that has been allocated.

A brief description of each tool follows.

PGS_MEM_Malloc allocates an arbitrary number of bytes in memory.

PGS_MEM_Calloc allocates an arbitrary number of bytes in memory, and initializes them to zero.

PGS_MEM_Zero initializes an arbitrary block of memory to zero.

PGS_MEM_Realloc reallocates an arbitrary number of bytes to a variable which had previously been allocated memory by PGS_MEM_Malloc or PGS_MEM_Calloc.

PGS_MEM_Free deallocates a given block of memory that was previously allocated by PGS_MEM_Malloc, PGS_MEM_Calloc, or PGS_MEM_Realloc.

PGS_MEM_FreeAll deallocates all memory that was previously allocated by PGS_MEM_Malloc, PGS_MEM_Calloc, or PGS_MEM_Realloc within a given executable.

Most of these functions return a pointer ptr to memory, which looks like

(void **) &ptr

in your code, in the argument to the Toolkit function. This form is necessary because the type of the variable is not known to the Toolkit function.

Please note that all addresses passed to these tools must be initialized first, if they have previously held allocated memory and were subsequently freed. This is a general requirement on any re-use of pointers. If this is not done, very strange behavior may result. The tool examples explicitly indicate what needs to be done.

5.1.2 Shared Memory Tools

These tools are for sharing memory between executables, within a single PGE.

You might want to share data between executables this way, in order to save the processing time that would ordinarily go to file I/O, if you were writing to and then reading from a file.

# File /usr/test/sample_PGE
# Sample PGE shell script

level_1a.exe
level_1b.exe
level_2.exe

unix% PGS_PC_Shell.sh /usr/test/sample_PGE 1111

# File /usr/test/sample_SCF_script
# Sample shell script for encapsulating PGS at the SCF

PGS_PC_InitCom 1 1
/usr/test/sample_PGE
PGS_PC_TermCom 1 1

5.1.3 Fortran, Cray and COTS Considerations

ANSI Fortran 77 does not support manipulation of pointer variables, so no Fortran 77 Toolkit Memory Management functions are available. Tools that support memory management in Fortran 90 are currently under study.

The Cray YMPEL which we at ECS used to test tools on, does not support shared memory. Therefore the shared memory tools were not tested on the Cray.

There is a problem with COTS tools that allocate dynamic memory, specifically IMSL. The problem is that there appears to be no way for the Toolkit dynamic memory functions to track or free this memory. We are working on this.

5.2.1 PGS_MEM_Calloc

Short explanation of what it's for: Allocate memory in your process, initializing it to zero.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes the file with handle fileHandle has already been opened.

C example:

#include <PGS_MEM.h>
float *data;
int n_items = 10;
int fsize;
PGSt_IO_Gen_FileHandle *fileHandle;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
Initialize data pointer 
   (required if you previously freed memory associated with it;
     see Notes)
Allocate memory, initializing it to zero
Read 5 items of data from a file to partially fill array
*/
data = (float *)NULL;
fsize = sizeof(float);
returnStatus = 
PGS_MEM_Calloc( (void **) &data, n_items, fsize );
if( returnStatus == PGS_S_SUCCESS )
{
   fread( data, fsize, 5, fileHandle );
}
/*
array data now contains 5 floating point values, as read
from file fileHandle, plus 5 zero values
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

In the example, the line

data = (float *)NULL;

is required before any call to PGS_MEM_Calloc, if you have previously used the same pointer data in a call to PGS_MEM_Free or PGS_MEM_FreeAll. Behavior of your process is unpredictable if this line is not present. Effort will be made to check this at DAAC Integration and Test.

The only difference between PGS_MEM_Malloc and PGS_MEM_Calloc is that PGS_MEM_Calloc initializes the variable to zero.

5.2.2 PGS_MEM_Free

Short explanation of what it's for: Free memory for a single variable.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes the variable data has previously had memory allocated by either PGS_MEM_Malloc or PGS_MEM_Calloc.

C example:

#include <PGS_MEM.h>
float *data;
/*
Begin example
*/
/*
Free memory
*/
PGS_MEM_Free( data );
/*
Initialize data pointer 
   (required if you want to re-use this pointer in a later call
     to PGS_MEM_Malloc or PGS_MEM_Calloc; see Notes)
*/
data = (float *)NULL;

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

The line

data = (float *)NULL;

is required before any call to PGS_MEM_Malloc or PGS_MEM_Calloc, if you have previously used the same pointer variable data in a call to PGS_MEM_Free or PGS_MEM_FreeAll. Therefore it is prudent to do this right after you free the memory, just in case you forget later. Behavior of your process is unpredictable if this line is not present.

Use of this function is optional. All dynamically allocated memory is automatically freed to the system at the termination of the current executable.

5.2.3 PGS_MEM_FreeAll

Short explanation of what it's for: Free all memory that you previously allocated dynamically.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes that you have previously allocated some memory using PGS_MEM_Malloc and/or PGS_MEM_Calloc.

C example:

#include <PGS_MEM.h>
/*
Begin example
*/
/*
Free all dynamically allocated memory
*/
PGS_MEM_FreeAll();
/*
All dynamically allocated memory within this exectuable has 
now been freed
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

After you use this function, if you want to re-use any of the pointers to which you previously dynamically allocated memory, you must first re-initialize them to zero, before any call to PGS_MEM_Malloc or PGS_MEM_Calloc. Behavior of your process is unpredictable if this is not done.

Use of this function is optional. All dynamically allocated memory is automatically freed to the system at the termination of the current executable.

5.2.4 PGS_MEM_Malloc

Short explanation of what it's for: Allocate memory in your process.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes the file with handle fileHandle has already been opened.

C example:

#include <PGS_MEM.h>
float *data;
int n_items = 10;
int fsize;
PGSt_IO_Gen_FileHandle *fileHandle;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
/*
Initialize data pointer 
   (required if you previously freed memory associated with it;
     see Notes)
Allocate memory
Read 10 items of data from a file to fill array
*/
data = (float *)NULL;
fsize = sizeof(float);
returnStatus = 
PGS_MEM_Malloc( (void **) &data, n_items*fsize );
if( returnStatus == PGS_S_SUCCESS )
{
   fread( data, fsize, n_items, fileHandle );
}
/*
array data now contains 10 floating point values, as read
from file fileHandle
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

In the example, the line

data = (float *)NULL;

is required before any call to PGS_MEM_Malloc, if you have previously used the same pointer data in a call to PGS_MEM_Free or PGS_MEM_FreeAll. Behavior of your process is unpredictable if this line is not present. Effort will be made to check this at DAAC Integration and Test.

The only difference between PGS_MEM_Malloc and PGS_MEM_Calloc is that PGS_MEM_Calloc initializes the variable to zero.

5.2.5 PGS_MEM_Realloc

Short explanation of what it's for: Re-allocate memory in your process, to a variable to which memory has been allocated previously. Useful for extending arrays to a longer length than originally allocated.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes that array variable data of dimension 10 has had memory allocated previously by PGS_MEM_Malloc or PGS_MEM_Calloc, as in the examples given in those tool descriptions.

C example:

#include <PGS_MEM.h>
float *data;
int n_items = 20;
int fsize;
PGSt_SMF_status returnStatus;
/*
Begin example
*/
fsize = sizeof(float);
returnStatus = 
PGS_MEM_Realloc( (void **) &data, n_items*fsize );
/*
Array data now contains space for 20 floating point values 
The first 10 values are the same as before; the second 10 values 
   contain garbage
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

Only pointer variables which have been used in a previous call to PGS_MEM_Malloc or PGS_MEM_Calloc should be used in a call to PGS_MEM_Realloc. Do not use a pointer variable which has been freed and not re-initialized to zero; if you do, behavior of your process will be unpredictable.

5.2.6 PGS_MEM_ShmAttach

Short explanation of what it's for:

Make previously allocated block of shared memory available to your process, so you can put data to it or get data from it.

This function is in file: $PGSSRC/MEM/PGS_MEM1.c

Examples:

This example is a continuation of the one used in PGS_MEM_ShmCreate.

Two examples are presented: in the first, we show how the shared memory is attached in your first executable level1a.exe, then filled with data; In the second, we attach it again in your second executable level1b.exe, then access the shared memory data. (This is also valid for subsequent executables.)

The examples show one way of how you might share ephemeris data among your executables.

C example 1: First executable level1a.exe

Example 1 assumes
(1) you have already processed your data enough (via Toolkit calls to ephemeris tools) to know the total number of ephemeris points npts;
(2) you have already called PGS_MEM_ShmCreate.

#include <PGS_MEM1.h>

/*
Structure for storing ephemeris data
Memory assumed allocated previously for structure elements
*/
typdef struct
{
   long *clockTime;
   double *xPosition;
   double *yPosition;
   double *zPosition;
} ephemStruct;

ephemStruct *ephemeris;

int lsize;
int dsize;
long size1l;
long size1d;
long totsize;

/*
Previously determined total number of data points in structure
*/
long npts;

/*
Intermediate arrays previously allocated and filled with data
*/
long *clockTime;
double *xPosition;
double *yPosition;
double *zPosition;

PGSt_SMF_status returnStatus;

/*
Begin example
*/

/*
First make the shared memory available to your process
*/

returnStatus = PGS_MEM_ShmAttach( (void **) &ephemeris );

/*
Now copy data from intermediate arrays into shared memory
*/

lsize = sizeof(long);
dsize = sizeof(double);
size1l = npts * lsize;
size1d = npts * dsize;

memcpy( ephemeris->clockTime, clockTime, size1l );
memcpy( ephemeris->xPosition, xPosition, size1d );
memcpy( ephemeris->yPosition, yPosition, size1d );
memcpy( ephemeris->zPosition, zPosition, size1d );

/*
Ephemeris data is now saved in shared memory

To make the shared memory available to another executable 
later in the same PGE, call PGS_MEM_ShmAttach again
*/

C example 2: Second executable level1b.exe

Example 2 assumes that you have already filled the shared memory block with data, as in example 1.

#include <PGS_MEM1.h>
#include <math.h>
#define REARTH 6.4E6  /* crude earth radius (m) */
/*
Structure for storing ephemeris data
Memory assumed allocated previously for structure elements
*/
typdef struct
{
   long *clockTime;
   double *xPosition;
   double *yPosition;
   double *zPosition;
} ephemStruct;

ephemStruct *ephemeris;

double altitude;
PGSt_SMF_status returnStatus;

/*
Begin example
*/

/*
Make the shared memory available to your process
*/

returnStatus = PGS_MEM_ShmAttach( (void **) &ephemeris );

/*
Ephemeris data is now available to your process

For example, you might calculate the altitude of the spacecraft:
*/

altitude = sqrt( sqr(ephemeris->xPosition) +
     sqr(ephemeris->yPosition) + sqr(ephemeris->zPosition) )
      - REARTH;
 Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

This function must be the third (or later) Toolkit shared memory function called in your code. PGS_MEM_ShmSysInit and PGS_MEM_ShmCreate must have been called first, before calling this function.
You call PGS_MEM_ShmAttach once in each executable where you need to access the shared memory data.

It is preferred that you use one single structure for all of your shared memory. This is how these tools were tested. It is possible that spurious results may be obtained if you use more than one structure, or some other combination of data types.

Do not fill any variables in the shared memory block with data until after you call PGS_MEM_ShmAttach. Any data you put in these variables before the call may be overwritten.

The example used in no way reflects how the actual Toolkit ephemeris tools format this data. This is a contrived example for purposes of illustration only.

This function is not POSIX compliant, nor are any Toolkit shared memory functions.

5.2.7 PGS_MEM_ShmCreate

Short explanation of what it's for:

Used to Initiate the use of your shared memory block.

This function is in file: $PGSSRC/MEM/PGS_MEM1.c

Examples:

The example shows one way of how you might share ephemeris data among your executables. (Please note that this in no way reflects how the actual Toolkit ephemeris tools format this data.)

The example assumes
(1) you have already processed your data enough (via Toolkit calls to ephemeris tools) to know the total number of ephemeris points npts;
(2) you have already called PGS_MEM_ShmSysInit.

C example:

#include <PGS_MEM1.h>

/*
Structure for storing ephemeris data
*/
typdef struct
{
   long *clockTime;
   double *xPosition;
   double *yPosition;
   double *zPosition;
} ephemStruct;

ephemStruct *ephemeris;

int lsize;
int dsize;
long size1l;
long size1d;
long totsize;

/*
Previously determined total number of data points in structure
*/
long npts;

PGSt_SMF_status returnStatus;

/*
Begin example
*/

/*
First allocate memory for and copy over ephemeris data from 
   intermediate arrays
*/
lsize = sizeof(long);
dsize = sizeof(double);

ephemeris->clockTime = (long *)NULL;
ephemeris->xPosition = (double *)NULL;
ephemeris->yPosition = (double *)NULL;
ephemeris->zPosition = (double *)NULL;

returnStatus = 
PGS_MEM_Calloc( (void **) &(ephemeris->clockTime), npts,lsize);
returnStatus = 
PGS_MEM_Calloc( (void **) &(ephemeris->xPosition), npts,dsize);
returnStatus = 
PGS_MEM_Calloc( (void **) &(ephemeris->yPosition), npts,dsize);
returnStatus = 
PGS_MEM_Calloc( (void **) &(ephemeris->zPosition), npts,dsize);

/*
Calculate total size of shared memory block
Reserve space for your shared memory block
*/

size1l = npts * lsize;
size1d = npts * dsize;
totsize = sizeof( ephemStruct ) + size1l + 3*size1d;
returnStatus = PGS_MEM_ShmCreate( totsize );

/*
Space has now been reserved in shared memory for your structure
You are now ready to call PGS_MEM_ShmAttach to make the shared 
memory available to your process
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

This function must be the second Toolkit shared memory function called in your code. You only need to call it once per PGE; subsequent calls are ignored.

PGS_MEM_ShmSysInit must have been called first, before calling this function.

After you call this function, you must call PGS_MEM_ShmAttach to actually make the shared memory available in your code, in each executable where you want to use it.

What this function actually does is reserve a given amount of space in system memory for your shared memory block. You cannot make this amount bigger later; you must reserve it all at once by using this function.

It is preferred that you use one single structure for all of your shared memory. This is how these tools were tested. It is possible that spurious results may be obtained if you use more than one structure, or some other combination of data types.

Do not fill any variables in the shared memory block with data until after you call PGS_MEM_ShmAttach. Any data you put in these variables before the call may be overwritten.

This function initializes the use of your shared memory block, in contrast to PGS_MEM_ShmSysInit, which initializes the shared memory block that the Toolkit itself uses.

This function is not POSIX compliant, nor are any Toolkit shared memory functions.

5.2.8 PGS_MEM_ShmDetach

Short explanation of what it's for:

Optionally releases shared memory block, so the current executable can no longer access it.

This function is in file: $PGSSRC/MEM/PGS_MEM1.c

Examples:

C example

#include <PGS_MEM1.h>

PGS_MEM_ShmDetach();

/*
Shared memory data is no longer available to this executable

To make the shared memory available to another executable 
later in the same PGE, call PGS_MEM_ShmAttach again
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

The shared memory data is not erased by this function, it is simply no longer available to your current executable. To make it available again in this or a subsequent executable in this same PGE, call PGS_MEM_ShmAttach again.

Use of this function is optional. The system automatically detaches the shared memory block at the end of each executable.
However, if you are no longer using the shared memory block in the current executable, it is a good idea to detach it, so that the memory is available as dynamic memory to your process. You may detach and re-attach the shared memory block as much as you like.

This function is not POSIX compliant, nor are any Toolkit shared memory functions.

5.2.9 PGS_MEM_Zero

Short explanation of what it's for: Sets a given amount of memory to zero.

This function is in file: $PGSSRC/MEM/PGS_MEM.c

Examples:

Example assumes that variable data has had memory allocated previously by PGS_MEM_Malloc, PGS_MEM_Calloc, or PGS_MEM_Realloc, as in the examples given in those tool descriptions.

C example:

#include <PGS_MEM.h>
float *data;
int n_items = 10;
int fsize;
/*
Begin example
*/
fsize = sizeof(float);
PGS_MEM_Zero( data, n_items*fsize );
/*
array data now contains zero in all 10 positions 
*/

Fortran example:

Dynamic memory allocation is not allowed in Fortran 77. Fortran 90 tools are under study.

Notes:

You might use this function after using PGS_MEM_Realloc, to set the re-allocated memory to zero; or you just might want to reinitialize an array for re-use.

Warning: Be careful not to zero out past the ends of the bounds of an array. This would cause some other variable to be set to zero, giving unpredictable results in your program.

6.1.1 Introduction

This set of tools is designed to manage the metadata inserted into each EOS product; i.e. the per granule metadata. The user is the science PGE which initiates the tools in a specified sequence, to obtain and marshal metadata values.

Within ECS the term "Metadata" relates to all information of a descriptive nature which is associated with the product or dataset. Metadata of importance to production software developers are:

• Data elements commonly found in a product file header, such as temporal or spatial coverage
• documentation that accompanies the production algorithm software
• data origin information

In order to establish standards for the EOS project, a minimal set of parameters has been made mandatory to accompany standard products. This set is detailed in the ECS document DID 311.

Listed below are mandatory metadata parameters:

• Longname collection (dataset) name
• Spatial_coverage (group) one of spatial coverage options
• Temporal_coverage (group) one of temporal coverage options
• UR_OF_ECS_product_input ID of input product used to generate this product
• Quality_rating (group) all of group
• reprocessing_status (group) 2 attributes denoting status

Other Optional parameters:

• size_MB_ECS_data_granule size of product
• UR_of_ancillary_input_granules ID of ancillary product used to generate this product
• UR_of_Orbit_paramters_granule ID of orbit parameters file used to generate this product

Further information describing the attributes can be found in appendix J of the Toolkit Users Guide

The establishment of metadata values for ECS produced products will be important for the services which will be applied to the data upon request by users. This includes, for example, subsetting by geolocation. Note that the term 'users' could mean human or the production system itself.

6.1.2 Accessing Metadata

In order to manage the metadata in the ECS, and to avoid future changes in the toolkit software interface affecting user code, we have designed the metadata access to be file driven. The Toolkit metadata tools will rely on a Metadata Configuration File (MCF). The purpose of the MCF is to provide a structured medium which acts as a repository for the attributes which will be attached to data products. A template for constructing an MCF is provided with Toolkit 5 (Aug. 95), to be edited by the instrument software development teams. The template, as presented in the example consists of two sections or GROUPS, namely granule and product specific. The attributes in the granule GROUP, represent core metadata. The attributes in the product specific GROUP are to be established at the discretion of each instrument team. It is expected that one MCF per data product will be specified.

Note: Any new additions to the MCF, must also be mirrored in the data dictionary.

An example of an MCF is presented below. The MCF is provided to each Instrument Team as a template. The MCF has been designed around the Object Development Language (ODL) libraries, which access data in a Group, Object and Attribute context. Each meaningful collection of data, described by a name is known as an OBJECT. Individual pieces of information about the form and content of the object are called ATTRIBUTES. When it is convenient to group together a number of objects under one label, a GROUP is constructed. For more information on ODL and documentation, please press icon.

ODL enables the metadata tools to access data held within the MCF, and to output values to the MCF. Data are held in a PARAMETER = VALUE (PVL) format. For information on PVL please perform a search using the keyword PVL at the following site, please press icon.

Below is an example of PVL syntax from the Metadata Configuration File (MCF).

GROUP = GRANULEDATAOBJECT = LongNameData_Location = "MCF"Mandatory = "TRUE"Value = "MODIS_SST"END_OBJECT = LongNameEND_GROUP = GRANULEDATA

The parameter, Data Location may be to any of the following:

• Data Location = PCF. Values are obtained automatically from the Process Control File
• Data Location = MCF. Values are preset during the creation of the MCF from the template by the Instrument Team.
• Data Location = PGE. Values are set by the PGE.

If the Data_Location field is filled in with the value "MCF", there will always be a Value = field. With Data_Location = PGE or PCF , there is no value field; this is added when the MCF is written to the product. The Mandatory field denotes whether or not the attribute is mandatory. If it is mandatory and it is not set by the PGE a warning message is returned. Strict adherence to the format and structure of the MCF is advisable, to prevent any needless or spurious errors.

For more information and a copy of the MCF template, please refer to appendix J of the Toolkit Users Guide, or click on

Values written to the MCF while it is held in memory after initialisation are checked against a data dictionary. The data dictionary is supplied with the toolkit routines at present. The data dictionary provided with the toolkit is in PVL/ODL format. It contains data descriptions relevant to all attribute to be found in the granule specific group. Any additional information pertinent to product specific Metadata must be added by the Instrument team. At present, there is no "keeper" of the data dictionary. In the future, the data dictionary as well as the MCF will be prepared by the Data Server.

For more information on the data dictionary, please refer to appendix J of the Toolkit Users Guide, or click on

A scenario for Tool usage can be seen below:

STEP 1 - Initialize MCF

PGS_MET_Init(filelogical, metadata handles)

STEP 2 - Extract Value from file in PCF

PGS_MET_GetPCAttr(product file id, product version number, name of hdf attribute containing metadata, metadata parameter, returned metadata parameter value)

The output will be the value of the metadata attribute from the HDF metadata attribute or header. e.g. Obtain the QA_%_of_MissingData from an input product.

STEP 3 - Write the value extracted to the MCF in memory

PGS_MET_SetAttr(metadata group name, name.class of parameter, value to be inserted)

This will locate the group in the MCF, then the object name, and class if this is specified, and attach a new attribute to the object, which will hold the value to be associated with that attribute. The value will also be checked against the data dictionary; if the value is within the specified range, and of the correct type, it will be associated. (i.e. held in memory location)

STEP 4 - A value already held in the MCF in memory is needed to calculate a new value for a product specific object.

PGS_MET_GetSetAttr(metadata group name, name.class of parameter, value to be passed back)

STEP 5 - In order to calculate this new value, information is also needed from the Configuration parameters set up in the Process Control File.

PGS_MET_GetConfigData(name of parameter, value to be passed back)

This will search the Process control file, and return the value back to the algorithm.

STEP 6 - The PGE has used the two inputs to calculate a new value for one of the MCF objects, and wants to write it to the MCF held in memory.

PGS_MET_SetAttr(metadata group name, name.class of parameter, value to be inserted)

STEP 7 - The PGE has finished setting all the values which are mandatory in the MCF, but there is still some relevant granule information which the PGE wants to add to the MCF. The PGE accomplishes this by adding this information to the PRODUCT_SPECIFIC_METADATA group. Located within this group lie the object names the instrument team has already specified as being product specific.

PGS_MET_SetAttr(product specific metadata group name, name.class of parameter, value to be inserted)

STEP 8 - After multiple calls to PGS_MET_SetAttr the MCF in memory is now complete, all the granule specific metadata have been set, and the relevant product specific metadata have been set. The PGE now writes the metadata out as an HDF attribute attached to the product.

PGS_MET_Write(metadata group to be written out, HDF file attribute name, HDF file ID)

6.2.1 PGS_MET_Init

The first step in reading from or writing to the MCF is with initialization. The contents of the MCF are read into memory and any values which are to be set automatically from the PCF (i.e. where location = PCF), are located and inserted. Any values preset in the MCF(Data_Location = MCF) are checked against the data dictionary. The MCF is also checked to see if it is in the correct ODL syntactical format.

For calling sequences go to PGS_MET_Init.

6.2.2 PGS_MET_SetAttr

The PGE can use the PGS_MET_SetAttr tool to set values already known to the algorithm, or to set those values which are available from the Process Control File (PCF). This function also acts to check out the validity of the value being set. The value is checked against the data dictionary for type and whether it falls within a predefined range.

For calling sequences go to PGS_MET_SetAttr.

6.2.3 PGS_MET_GetSetAttr

When the PGE needs to find and use a value from the MCF after it has been intialized, PGS_MET_GetSetAttr is used. This tool is used to get values pre - set by the Instrument Team, i.e. where data_Location is set to MCF.

For calling sequences go to PGS_MET_GetSetAttr.

6.2.4 PGS_MET_GetPCAttr

NOTE: These ASCII files must be in flat ODL format. The HDF attributes are guaranteed to be in this format if they have been written out to the file using the PGS_MET_Write function.

For calling sequences go to PGS_MET_GetPCAttr.

6.2.5 PGS_MET_GetConfig

For calling sequences go to PGS_MET_GetConfig.

6.2.6 PGS_MET_Write

NOTE: To see examples of the format of the resulting metadata written to the product, consult Appendix J of the Toolkit Users Guide.

For calling sequences go to PGS_MET_Write.

6.2.7 PGS_MET_Remove

For calling sequences go to PGS_MET_Remove.