Thursday, March 28, 2019

File Transfer Utility - Simplicity of Design

XXFDXFR.SQR - Generic Multi-Purpose File Transfer Utility

The secret to effective program design is isolating exactly what the requirements are (this includes anticipating future needs), streamlining the process by eliminating unnecessary complexity & delivering a solid product that is 100% reliable & very simple to use.

Here's a basic example. Our organization has a need to transfer different files via email from our file server to users/groups. They can be external files from vendors such as invoice information, security permissions for the administrator distribution group, output from other processes, monthly data extracts, etc. Regardless, the task at hand is to identify file X at folder location Y and email it to distribution group Z.

For any circumstance I'll need - a Transfer Definition, Email Distribution, Run Control & File Watcher/Interface Component - and finally, a generic transfer program (XXFDXFR) to utilize all of these components to fulfill the requirement.

Note - The File Watcher/Interface Component is our custom 24/7 Interface/Process Request Loader. It's extremely powerful - in general, within 5 minutes (or less) it links a file with a process & launches it. It can handle hundreds of files in multiple folder locations in an instant (part of our pre-Go-Live stress test). I developed the PeopleSoft/Oracle/API Components in 2004 (with Chuck Hamilton writing the underlying Unix shell scripts & cron job). Still running strong...

File Transfer Definition - Defines Key/Email Information:

Xfer Key: SPO_ERR
 Subject: ADP/SharePoint Security *ERRORS* <YMD> - [<DBASE>]
    Text: Please review the attached ADP/SharePoint ERROR file for <MDY>^^File: <FFULL>^^

Email Distribution - Directs output to PeopleSoft Oprid OR Email Address:

 Pgm Key: XXFDXFR     XXFDXFR
Xfer Key: SPO_ERR     SPO_ERR
    Type: P           E
   Value: TDELIA      user@email.com


Process Run Control - Passes Transfer Key to Process:

   Oprid: PSBATCH
Run Cntl: FD_XFR_SPOERR
Xfer Key: SPO_ERR

File Watcher Configuration - Detects File & Launches Process/Run Control:

    File: FDXSPO*.*
 Process: XXFDXFR
   Oprid: PSBATCH
Run Cntl: FD_XFR_SPOERR
  Folder: /app/iface/<DBASE>/ALL/AP/inbound
Day/Time: 24/7

XXFDXFR.SQR - File Transfer Program:

...
do IDB_Init('I', $RC_entity, 'AP', $RC_file)

do Process_Main

do IDB_Term

...

#include 'xx_tdf.sqc'    !TD Custom SQR Function Library
#include 'xx_idb.sqc'    !TD Interface Dashboard API
 


The XXFDXFR program (above) is launched via the File Watcher/Interface mechanism... i.e. minutes after the file is dropped in the folder. The core of the program is as follows:

IDB_Init() - Accepts the file that was detected via the File Watcher definition.
Process_Main - Reads the run control, assigns the Xfer Key & emails to the distribution.
IDB_Term - Automatically archives the input file.

File Watcher Configuration Page:


Above is the actual File Watcher Configuration page. It is also known as the IDB Scheduler (Interface Dashboard Scheduler) due to it's extensive interface audit capabilities.


Above is a sample screenshot of XXFDXFR on Process Monitor. Process instance 5106203 at 6:40AM contains the SPO_ERR Transfer we're using as an example.


The email with the file attachment has been successfully sent.


The subject & email body have been derived from the initial File Transfer Defintion for the key SPO_ERR. Notice the embedded tokens have been resolved in the Subject & Email Body:

<YMD> set to 2019-03-28
<DBASE> set to NSFSPRD
<MDY> set to 03-28-2019
<FFULL> set to Full Path/Filename of the attached file.
"^" Character set to CRLF.

Any additional file transfers that may be required are simply a matter of configuration without unnecessary development to engage in.