Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NJE Python Library

What the hell is NJE?

NJE is known as 'Network Job Entry.' It is used by mainframes all over the world to communicate seamlessly with one another. It allows for the sending on files, jobs, system control, etc.

The easiest way to explain NJE is using an example. Let's say I'm a super huge mega corporation. I have offices in Washington, DC and New York NY and each have their own mainframe (DC is where we process payroll, New York is where we process insurance claims). Our IT center is headquartered in New York and we don't want to hire more people than we need in DC so we setup NJE with the following in a config (JES2 Parmlib for you pedantics) file:

 NJEDEF   NODENUM=2,
          OWNNODE=1,
          LINENUM=1

 NODE(1)  NAME=NEWYORK
 NODE(2)  NAME=WASHDC

 NETSRV(1) SOCKET=LOCAL
 LINE(1)  UNIT=TCPIP
 SOCKET(WASHDC) NODE=2,
         IPADDR=33.1.3.37

Then, from NEWYORK, we enable it and connect with these commands:

$S NETSERV1
$S LINE1
$S N,LINE1,WASHDC

Now the two mainframes can talk to one another over TCPIP.

NOTE: The above isn't secure. It isn't using SSL and there's no password required to connect.

With that setup you can now send commands to one another through whats called 'NMRs.' For example, if we wanted to display the current NJE setup at NEWYORK (from WASHDC) i would issue the $D NJEDEF command and get the reply:

$HASP831 NJEDEF
$HASP831 NJEDEF  OWNNAME=NEWYORK,OWNNODE=1,CONNECT=(YES,10),
$HASP831         DELAY=120,HDRBUF=(LIMIT=10,WARN=80,FREE=10),
$HASP831         JRNUM=1,JTNUM=1,SRNUM=1,STNUM=1,LINENUM=1,
$HASP831         MAILMSG=NO,MAXHOP=0,NODENUM=2,PATH=1,
$HASP831         RESTMAX=262136000,RESTNODE=100,RESTTOL=0,
$HASP831         TIMETOL=1440

But being able to issue commands isn't all NJE can do. The most important part is being able to submit jobs and transfer files. Jobs on the mainframe are scripts with input and output directives. To send a job from Washington DC (WASHDC) to New York (NEWYORK) we place an execution line (e.g. /*XEQ NEWYORK) in the job.

 //H4CKRNJE JOB (1234567),'ABC 123',CLASS=A,
 //             MSGLEVEL=(0,0),MSGCLASS=K,NOTIFY=&SYSUID
 /*XEQ    NEWYORK
 //TSOCMD   EXEC  PGM=IKJEFT01
 //SYSTSPRT DD    SYSOUT=*
 //SYSOUT   DD    SYSOUT=*
 //SYSTSIN  DD    *
   TIME
/*

When we run this job (aka submit it to be processed) JES2 will read the /*XEQ line and send it to be processed at NEWYORK instead of locally. You can determine (if you have the rights) what nodes exist through the SDSF command NODE or through JES2 $D NODE or by reading the JES2 config (parmlib) file or by reading JCL we find on the system, sharepoint, wherever.

Finally, using XMIT you can also transfer files between two systems. So if you have a library on the development system you want to move to production you could use XMIT in stead of FTP. Keep in mind, however, that if SSL isn't turned on you'll be sending this data in clear text (hint hint PCI assessors).

Local Nodes

Connecting two NJE nodes does not by itself authorize jobs or propagated user identities. On a RACF-protected receiver, NODES class profiles determine whether an incoming user and group are accepted, translated, or rejected. The security headers produced by makeSYSIN_header() identify the remote submitting user and group as assertions; they do not prove that the individual user authenticated. Configure node trust and identity propagation according to your site's security policy. NJE secure signon, described below, authenticates the peer node with an APPCLU session key.

Using this Library

This library connects to a mainframe serving up NJE and pretends to be a mainframe. NJE generally runs over TCP/IP; port 175 is commonly used for cleartext NJE, while a site-defined port such as 2252 may be protected by TLS. To use NJE you need the OHOST and RHOST names.

  • OHOST: Target System node name (could be hostname but not always). In our examples, NEWYORK is the OHOST.
  • RHOST: The system we're pretending to be. In these examples WASHDC is the RHOST.

First we create an NJE object:

import njelib
nje = njelib.NJE("WASHDC", "NEWYORK")

Now we need to connect to a mainframe:

connected = nje.session(host="3.1.33.7", port=175)

if not connected:
    raise SystemExit("That didn't work")

The session() function returns True if the connection and NJE signon succeed, and False otherwise. A session is cleartext unless setTLS() is called before session(); TLS failure does not silently fall back to cleartext.

Notice that this library is silent unless you turn on debugging with:

nje.set_debuglevel(1)

Once we're connected we can issue commands, send messages and/or submit JCL:

# Send a command. Replies may contain multiple lines.
reply = nje.sendCommand("$D NJEDEF", wait=5.0)
print(reply)

# Send a message to someone.
nje.sendMessage("MESS WITH THE BEST DIE LIKE THE REST", "plague")

# Send a message to the master console.
nje.sendMessage("ARF ARF")

# Send a JCL file as a specific user and wait for its complete SYSOUT stream.
nje.sendJCL("cookie.jcl", userid="plague")

# Send an NJE type-B signoff and close TLS/TCP cleanly.
nje.disconnect()

sendCommand() keeps the session open, clears old NMR replies by default, and collects multi-packet replies for up to wait seconds. Pass clear=False to retain earlier NMRs. Long-idle sessions get an NJE heartbeat before the next command; reconnect if sendCommand() returns False because the peer has already dropped the link.

When you submit JCL/commands you'll get messages (aka NMR) and/or SYSOUT (job output) back. To access that information you can access dictionaries which collect all the headers, footers etc as described in the NJE documentation through a handful of functions:

  • getNMR() - returns a list of dictionaries with message headers and message contents
  • getSYSIN() - returns a list of dictionaries with job/dataset headers/footers and dataset contents
  • getSYSOUT() - returns a list of dictionaries with job/dataset headers/footers and dataset contents
#send JCL
nje.sendJCL("cookie.jcl", "plague")
#Print any messages
for record in nje.getNMR():
   if 'NMRUSER' in record:
      print("[+] User Message")
      print("[+] To User:", record['NMRUSER'])
      print("[+] Message:", record['NMRMSG'])
   elif 'NMRMSG' in record :
      print("[+] Message:", record['NMRMSG'])

#Prints any data we've received
print("[+] Records in SYSOUT:")
for record in nje.getSYSOUT():
    if 'Record' in record:
        print(record['Record'])

Offline Analysis

Using Wireshark you can easily capture NJE records flying across the network. Unfortunately there's currently no formatting available for NJE (future project perhaps). Using this library however, and the raw data extracted from Wireshark you can assess what was sent across the wires. You can use the set_offline()

import njelib
nje = njelib.NJE()
nje.set_debuglevel(1)
nje.set_offline()
nje.analyze('./wireshark/nje.packet')

TLS and AT-TLS

TLS is enabled explicitly with setTLS() before session(). The initial OPEN SSL exchange is cleartext, as required by application-controlled AT-TLS; the library upgrades the socket only after JES2 acknowledges the open. The negotiated TLS version comes from the Python/OpenSSL runtime and the peer, so TLS 1.3 is used when both support it and can otherwise negotiate a supported TLS 1.2 configuration.

For normal server-authenticated TLS, provide the CA that issued the z/OS server certificate. The hostname used for the connection must match the certificate. Use server_hostname when connecting by IP address or by a name different from the certificate's DNS name:

import njelib

nje = njelib.NJE("N50", "S0W1")
nje.setTLS(
    cafile="certificates/ca.pem",
    server_hostname="zos.example.com",
)

connected = nje.session(host="10.1.1.2", port=2252, timeout=10)

If the AT-TLS policy requires client authentication, also supply the client certificate and key:

nje.setTLS(
    cafile="certificates/ca.pem",
    certfile="certificates/client.pem",
    keyfile="certificates/client-key.pem",
    password="private-key-password",
    server_hostname="zos.example.com",
)

Verification and hostname checks are on by default. verify=False is available for isolated troubleshooting but should not be used in production. If an older TLS 1.2 peer has no cipher in common with the OpenSSL defaults, call nje.addTLSCiphers() before session(), or pass a colon-separated OpenSSL cipher string to that method. TLS 1.3 cipher selection is handled by OpenSSL and is not changed by addTLSCiphers().

The included pagttls.conf is a sample Policy Agent configuration for inbound and outbound JES2 traffic with application-controlled AT-TLS and TLS 1.2/1.3 enabled. Before installing it, update the keyring, ports, addresses, cipher suites, and trace settings for your system. The inbound rule must treat JES2/AT-TLS as the TLS server (HandshakeRole Server) and must have ApplicationControlled On.

RACF secure signon

TLS protects the network connection. NJE SIGNON=SECURE separately authenticates the node with a RACF APPCLU session key. Enable it by passing sesskey to the constructor:

import njelib

nje = njelib.NJE("WASHDC", "NEWYORK", sesskey="PASSWORD")
nje.setTLS(cafile="certificates/ca.pem", server_hostname="zos.example.com")
connected = nje.session(host="10.1.1.2", port=2252)

The session key may be up to eight ASCII characters, exactly 16 hexadecimal digits (optionally prefixed with 0x), or exactly eight raw bytes. Short text keys are uppercased, converted to EBCDIC, and padded with binary zero bytes to match JES2. Secure signon requires either pycryptodome or cryptography to provide DES.

Configure the same key on z/OS, substituting the local and remote node names and protecting the profile according to your site's policy:

RDEFINE APPCLU NJE.LNODE.RNODE SESSION(SESSKEY(PASSWORD)) UACC(NONE)
SETROPTS CLASSACT(APPCLU)

The node must also be configured with SIGNON=SECURE. A plain NJE password (password=) and a secure-signon session key (sesskey=) are different settings.

Submitting identity

sendJCL() accepts userid and an optional group. Both are normalized to uppercase and must be valid 1-8 character RACF names. Leaving group=None allows RACF to select or map the propagated user's group rather than asserting the old hard-coded SYS1 default.

nje.sendJCL("JCL/id.jcl", userid="PLAGUE", group=None)

These fields assert the submitting identity; they do not authenticate that user or bypass RACF. Configure the receiving node's RACF NODES profiles to permit, translate, or reject propagated identities as appropriate.

Uploading text data sets

upload_text() creates and submits a temporary IEBGENER job. It can write an ASCII text file to an existing sequential data set or PDS/PDSE member:

result = nje.upload_text(
    "local/config.txt",
    "PLAGUE.CONFIG(MEMBER)",
    userid="PLAGUE",
    long_lines="wrap",
)
print(result)

Set create=True to allocate a new sequential data set:

result = nje.upload_text(
    "local/report.txt",
    "PLAGUE.REPORT.TEXT",
    userid="PLAGUE",
    create=True,
    recfm="FB",
    lrecl=80,
    blksize=0,
    primary=5,
    secondary=5,
    unit="SYSDA",
)

Only ASCII input and fixed record formats F and FB are supported, with LRECL from 1 through 80. Lines longer than LRECL raise an error by default; choose long_lines="wrap" or long_lines="truncate" to change that behavior. create=True supports sequential data sets only. By default the call waits for the complete SYSOUT stream; set wait_for_sysout=False to return after submission.

What's missing?

There is still no native XMIT or binary data-set transfer support. upload_text() is specifically an ASCII-text upload implemented by submitting an IEBGENER job.

Credits/Sources:

To get a LOT more information about NJE than you ever wanted to know you can check out the documentation about the protocol in IBM book HAS2A620: Network Job Entry: Formats and Protocols. Available Here: http://publibz.boulder.ibm.com/epubs/pdf/has2a620.pdf. I also used the online documentation frequently and on top of that sometimes the z/VM documentation was a little clearer (for example this entry on NMR headers and contents).

Some notes/thoughts about the documentation:

  • IBM did a great job documenting everything (this is not sarcasm, nor was that)
  • TCP is a Non-SNA Buffer Format (labelled BCS sometimes)
  • The sections are described in alphabetical (not always!) order, not in the order within the packet being sent/received
  • My SCB compression algorithm beats IBMs by 2 bytes!
  • Not everything is documented well or completely (but I'm just grateful the documentation was available) for example accounting headers

Included Files:

There's a bunch of files included with this library to provide examples on usage:

  • iNJEctor.py: A script created for DEFCON 23 to send messages and commands to a target node.
  • analyze.py: Example script to conduct offline analysis of NJE packets.
  • client.py: a dummy NJE client to connect and receive any outstanding messages or heartbeats until timeout.
  • jcl.py: Example python script to send JCL to a target system. Take two arguments: JCL to send and a userID.
  • JCL Folder: Example JCL files for testing:
    • id.jcl: Executes the UNIX commands 'sh id;who;uname -a' on the NEWYORK node.
    • nop.jcl: Executes the 'does nothing' program IEFBR14 on the NEWYORK node.
    • tso.jcl: Executes the the TSO command 'TIME' on the NEWYORK node.

About

z/OS (mainframe) Network Job Entry (NJE) python library and example scripts.

Resources

Stars

28 stars

Watchers

7 watching

Forks

Releases

Packages

Contributors

Languages