File Watcher - Introduction
I've been fortunate to have worked on many interesting custom projects over the years. Designing & developing robust solutions for a variety of unique clients has been my forte. One of my favorites has been the implementation of a highly efficient Interface Methodology - known as File Watcher - for PeopleSoft Financials, HR & Payroll. Back in 2004 the organization was very large & distributed among multiple independent systems. The goal was to develop an interface system to automatically manage a sometimes chaotic amount of files. 19 years later it still runs as designed. Now in 2023 the company is much smaller & has shifted to a leaner, more economically viable NetSuite ERP - mostly vanilla with little customization.
Looking back over my career I feel a sense of melancholy as the systems I created & nurtured are inevitably put to rest. I'm adding this blog post to keep the memory of File Watcher alive - and perhaps bring it back to life at another organization.
From my LinkedIn Projects List:
Recruited by NSS/ACSI to Design/Develop a comprehensive 24/7 automated interface file system. Custom components include Interface Dashboard (Audit/Archive), File Watcher (Detection/Queueing) & File Depot (Inbound/Outbound Server) to effectively manage all inbound/outbound interfaces. The heart of the ACSI Interface System – each business unit has inbound/outbound/archive folders, File Watcher configuration defines processing windows, file prefixes & run controls – when file detected it submits appropriate request via custom Process Request adapter. Interface Dashboard is the custom API that communicates with File Depot & Process Scheduler & regulates the processing, audit & archiving of all files. Custom Email Notifications sent to all users on Distribution List containing Report Manager HyperLink & detailed statistics. Examples include JPMC ACH payments/returns, Epiq Check Files, Hyperion Budget loads, HR/Benefit Vendor files, Debit Card processing, Concur Invoice data, PCard payments, ADP information, Payroll to GL/AP Feeds, AP Expenses to Payroll, Mactive Advertising Refunds, Circulation Carrier Payments/Subscriber Refunds, Custom AP XLS Invoice Uploads, etc... API Framework for Custom AP Voucher Edit/Build Process. Extremely Powerful One-Of-A-Kind Creation.
Files, Files and More Files
The little Blue Guy below has some real problems. He has to facilitate the automatic handling of hundreds of inbound & outbound files. ACH transactions, checks, reconciliation, invoices, benefits, vendor creation, IRS transmissions, security maintenance and much, much more... When a file is sent it must be detected, linked to a process & executed via Process Scheduler. Furthermore, each file instance - incoming & outgoing - must be archived & registered for audit & due diligence.
He definitely needs some help. The blue guy is getting buried with a daily barrage of critical files all hours of the day - and night. He's falling behind very rapidly. Just look at that mess...
Hang in there little guy - help is here!
Simple, Elegant & Effective
No worries. The developer side API I created is very simple & easy to use. At a minimum, just include the 'xx_idb.sqc' library and call two routines - IDB_Init() and IDB_Term.
All his team needs to do is A) incorporate this standard API in all interface processes - and B) update the custom configuration table with the process name, file pattern & run control information. Run times can be defined for specific days & time windows - or 24/7 - configure as needed (More on that later).
No mess - no confusion - no chaos.
Mission Accomplished
Now that the Interface Methodology has been adopted & implemented all files are processed automatically & archived in an orderly fashion. File handling has never been easier.
The results have exceeded his expectations. The processes for the appropriate files are queued & successfully completed.
Mr. Blue Guy loves his new Interface Audit reports. He can produce detail level information for any file - dates, statistics, ownerids - or create variations of summary level data. Here he's reviewing a summary of all interface processes for the Financials Database. From Go-Live (2005) to the Current Year (2023). All the data required was maintained by the simple API calls.
Audit Summary Sample - Financial Interface Processes through the years.
This consolidated approach is used to regulate processing of all files relative to each database instance (Financials or HR/Payroll) which includes Production, Test, Development, Support, etc...
More to come...
