wes/Python_Examples
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Added some more documentation to ETL scripts

Browse Source
This commit is contained in:
wes
2026-02-11 07:21:01 -05:00
parent c4b970cbde
commit 24b2eca388
2 changed files with 107 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
stm_etl/Finished_Goods_Interface.py
+70 -16
View File
@@ -1,22 +1,76 @@
'''
Finished Goods Interface - AS400 to Jomar Inventory Synchronization
Created on Jun 24, 2015
If no arguments supplied script looks for CSV's in the folder specified by PATH.
Can be run for single file by supplying file path as arg. This is the preferred way of doing things
if the filewatcher powershell script is enabled to avoid collisions.
Creates various inventory transactions needed to keep Jomar inventory up to date with what is entering
shipping dept from the AS400
Transaction data is written to TXRYBW10 and processed immediatley.
See 'Inventory Control' -> 'Visibility' -> 'Inventory Errors' for transactions that failed to process
TEST_MODE, when True:
* Prevents files being moved to the PROCESSED folder.
* Will _NOT_ e-mail Aaron
* Does not commit any transactions to the database
@author: Wes
PURPOSE:
Reads CSV files generated by the AS400 when inventory is scanned in shipping and creates
corresponding Jomar ERP inventory transactions to keep the two systems in sync.
INPUT:
CSV files in D:\INTERFACE\ named as400_jsi_#####.csv (where ##### is a sequential transaction number).
Each row represents a roll/piece with columns:
[0] ORDER - AS400 order number (e.g. U17435)
[1] LINE - AS400 line number
[2] TICKET - Roll/piece ticket number (unique identifier)
[3] FINISHED_YARDS
[4] INVOICED_YARDS
[5] BIN - Physical bin location
[6] WAREHOUSE - Warehouse code (SH, BG, B8, etc.)
[7] WEIGHT - Roll weight in lbs
[8] SPAT - Pattern code
[9] SCOL - Color code
Can be run two ways:
- No arguments: processes all CSV files in PATH
- Single file path argument: processes just that file (preferred when using the PowerShell
file watcher to avoid collisions between concurrent runs)
PROCESSING LOGIC (per row):
1. Validation checks (skip row to error list if any fail):
- Duplicate ticket within same transmission
- Invalid bin (HLD, or blank bin in SH warehouse)
- Finished yards <= 0
- Invoiced yards <= 0
2. Order/line lookup in Jomar:
- Query TXUYUV00 (order header) to map AS400 order -> Jomar order number
- Query TXUYUF01 (order detail) + TXMYAT03 (item info) to find Jomar line number
- Verify pattern and color match between AS400 and Jomar
3. Transaction generation based on roll state in Jomar (TXRYRT00 roll/tag master):
a. Roll does NOT exist in Jomar:
-> Create Production Receipt (08) transaction to add the roll
-> Sets warehouse/bin, yardage, weight, lot, NAFTA qualification
-> Pull orders (PULL_ORDERS list) get no order/line linkage
b. Roll ALREADY exists in Jomar:
- If shipped yards > 0: error (roll already shipped, can't modify)
- If yardage or weight differs from current values:
-> Create adjustment Production Receipt (08) with the delta
- If bin or warehouse differs:
-> Create Movement (06) transaction to relocate the roll
- If nothing differs: skip (log and ignore)
All transactions are written to TXRYBW10 (the finished goods interface table) and
processed immediately by Jomar.
ERROR HANDLING:
- Rows that fail validation or lookup are collected in eRows
- Error rows are written to D:\INTERFACE\ERRORS\ERRORS_{lot}.csv
- Error CSV is emailed to ROLL_ERROR_RECIPIENTS
- If a file with a lower transaction number is still unprocessed when a newer file arrives,
the script alerts TX_ERROR_RECIPIENTS and exits (prevents out-of-order processing)
OUTPUT:
- Processed CSV files are moved to D:\INTERFACE\PROCESSED\
- Log file written to D:\INTERFACE\Finished_Goods_Interface.log
TEST_MODE (when True):
- Prevents files being moved to the PROCESSED folder
- Suppresses email notifications
- Rolls back all database transactions instead of committing
'''
import ConfigParser
import csv, os, smtplib