hpoj reference: Device protocols

This document briefly summarizes the various device protocols used by the hpoj software and the peripherals that it supports, and where possible gives pointers to additional information and/or official specifications. Some of the standards mentioned below have additional features not discussed here which are not used by the hpoj software or the devices it supports.

IEEE standards documents may be ordered in hard-copy form from the IEEE Standards Association.

Certain HP standards and protocols described below are available from the HP Developer Solutions website. Others are "available on request" from the hpoj project maintainer for use in conjunction with the hpoj project. In both cases, the documents may not be redistributed verbatim (for example, posted on a web or FTP site), but it is permissible to use them to develop Free/open-source software, such as the hpoj project.

Parallel: IEEE 1284

IEEE 1284 specifies a "Standard Signaling Method for a Bidirectional Parallel Peripheral Interface for Personal Computers".

It specifies the following physical signal lines:

Host-to-peripheral "control" lines: nSelectIn, nAutoFd, nStrobe, and nInit
Peripheral-to-host "status" lines: PError, nAck, Busy, nFault, and Select
8 bidirectional data lines

Depending on the mode, data may be transferred in one direction:

Forward -- host to peripheral
Reverse -- peripheral to host

It specifies the following basic modes (among others):

Compatibility mode -- Forward data only, 8 bits at a time.
Nibble mode -- Reverse data only, 4 bits at a time, using only the peripheral-controlled status lines, for the sake of unidirectional parallel ports.
Device ID mode -- A variant on nibble mode, where a special string is passed back to the host identifying the manufacturer, model, supported command sets, etc.
ECP mode -- Forward or reverse data, 8 bits at a time, with a lightweight handshake to switch the direction of the data lines. It also allows special "command" bytes to be transmitted, which allow up to 128 different "channels" to be used, but these are a different kind of channel than that used in MLC and 1284.4 (see below).

It specifies the following signalling sequences:

Sending a byte in compatibility mode.
Negotiating out of compatibility mode into another mode.
Terminating back to compatiblity mode from another mode.
Receiving a byte in nibble mode.
Sending a byte in ECP mode, and a "host transfer recovery" handshake to cancel the byte if the peripheral fails to respond because it needs to send reverse data.
Switching the ECP bus direction from forward to reverse.
Receiving a byte in ECP mode.
Switching the ECP bus direction from reverse to forward.

Depending on the particular mode in use, the signal lines may have different names in the standard than the compatibility mode names listed above.

At the physical level, IEEE 1284 defines:

the 25-pin "A" connector typically used on the host side
the larger 36-pin "B" connector traditionally used on the peripheral side
a smaller 36-pin "C" connector used on some newer peripherals
wiring between the connectors in a cable

Parallel: Microsoft ECP register interface

The Microsoft Extended Capabilities Port Protocol and ISA Interface Standard defines the register set used in PC parallel ports, specifically ones with ECP capabilities. This register set has its roots in that of the unidirectional parallel ports in the original IBM PC; it therefore carries forward some rather confusing inverted register bits relative to the value on the wire, which is hidden deep within ptal-mlcd's ParPort class. This standard provides for hardware-assisted compatibility-mode and ECP-mode transfers, although this capability is not currently used in the hpoj software.

Parallel: IEEE 1284.3

IEEE 1284.3 specifies "Standard Interface and Protocol Extensions to IEEE Std 1284 Compliant Peripherals and Host Adapters".

Among other things, it introduces a "bounded ECP" mode with added restrictions over regular IEEE 1284 ECP mode, to address design problems with the Microsoft ECP register interface specification.

Parallel: meta-modes

HP peripherals typically support communication in some or all of the following basic combinations of IEEE 1284 modes:

Unidirectional ("Centronics") -- Compability-mode forward only.
Bidirectional ("Bitronics") -- Compatibility-mode forward and nibble-mode reverse.
ECP, which again has several possibilities:
- Bidirectional ECP (forward and reverse).
- ECP forward and nibble reverse (sometimes referred to as "slippy").
- Bounded ECP forward and reverse.

The following kinds of data may be transferred over the above modes:

Raw print data in the forward direction for "Centronics," "Bitronics," and ECP channel 0. The device may or may not generate reverse data.
MLC or 1284.4 packets over ECP channel 77.
A null byte sent over ECP channel 78 (or on some peripherals, ECP channel 0 or compatibility mode) to reset the peripheral's MLC/1284.4 protocol driver.

USB -- Universal Serial Bus

USB (Universal Serial Bus) provides an easier and more generalized way to connect a wider variety of peripherals to your computer, compared to traditional serial and parallel ports. By using hubs in a star-like configuration, you can theoretically connect up to 127 devices to a single bus.

The hardware, signalling, and protocol specification may be downloaded from the USB Implementor's Forum. Since the protocol is largely implemented in hardware and in the kernel, we won't go into much detail here. Suffice it to say that devices may advertise various kinds of interfaces, some of which may be used concurrently and others of which are "alternate settings" of the same interface. Interface descriptors advertise a "Class/Subclass/Protocol" code, such as "7/1/3". There are also "endpoints" of various types (control and bulk for our purposes, and also sometimes interrupt and isochronous), which are basically the channels over which data is transferred. Endpoints may be "OUT" for forward data or "IN" for reverse data.

USB printer class

For our purposes we are concerned with the USB printer class. It specifies three possible interfaces, which are usually mutually-exclusive alternate settings when a device supports more than one of them:

7/1/1 (unidirectional) -- has a single bulk-OUT endpoint.
7/1/2 (bidirectional) -- has a bulk-OUT and a bulk-IN endpoint.
7/1/3 (MLC/1284.4) -- has a bulk-OUT and a bulk-IN endpoint, and sometimes an interrupt-IN endpoint, which isn't used by the hpoj software.

Normally, 7/1/1 and 7/1/2 are for raw print data, and 7/1/3 is for MLC and 1284.4 packets. Some HP peripherals support a vendor-specific command that simulates switching ECP channels as described above, so the result of switching a 7/1/2 interface to channel 77 would be the same as if you had a 7/1/3 interface.

The printer class also supports reading the peripheral's device ID string, just like IEEE 1284 (described above).

HP MLC and IEEE 1284.4

HP's MLC (Multiple Logical Channels) transport protocol and the IEEE 1284.4 protocol based on MLC both allow multiple data channels to be multiplexed over a single bidirectional link, such as ECP channel 77 or a USB 7/1/3 interface.

There is a command channel that is open all the time, and other channels may be opened and closed over time using request/reply transactions over the command channel. A channel is defined as a pair of socket IDs, one for each side (referred to as "host"/"peripheral" socket IDs in MLC and "primary"/"secondary" socket IDs in 1284.4). Each data packet is prepended with a 6-byte header, which among other things contains the two socket IDs so it can be routed to the correct channel, and the packet length including the header.

Data flow control for a given channel is accomplished where each side issues "credit" to the other side, which is the number of packets (of a previously negotiated maximum size) that are allowed to be transmitted before more credit is given. Most MLC/1284.4 hosts and peripherals are considered "credit gushers", where they automatically supply credit to the other side. However, a few older MLC peripherals are "credit misers," where the host must always request credit whenever it needs some.

MLC and 1284.4 support the following request/reply transactions over the command channel (among others that are generally are not used in practice):

Init -- Starts the MLC/1284.4 session and negotiates which protocol revision (1284.4 or MLC) to use.
Error -- Abruptly stops a MLC/1284.4 session due to an error.
GetSocketID and GetServiceName (1284.4 only) -- Converts between socket IDs and service names. For example, this is how a host wanting to open the ECHO service on the peripheral would know to open socket 6 on the peripheral.
ConfigSocket (MLC only) -- One-time negotiation of maximum forward and reverse packet sizes for a given peripheral socket ID, in preparation for opening it.
Open -- Opens a data channel.
Close -- Closes a data channel.
Credit -- Issues credit for a data channel.
CreditRequest -- Requests credit for a data channel.

In addition to granting or requesting credit over the command channel, there's a "piggyback credit" field in the header that may be used as a lower-overhead way to grant more credit to the other side at the same time when sending a data packet.

The HP MLC specification is "available on request" (see above). The IEEE 1284.4 standard document may be ordered in hard-copy form from the IEEE Standards Association. The final IEEE 1284.4 draft specification may also still be downloadable.

PML -- Peripheral Management Language

PML is a request/reply protocol which provides control and status of a device. Each setting or piece of status information in a device is encapsulated in an "object", and all of the objects are grouped into a tree-like hierarchy called a MIB (Management Information Base). Each object has a dotted-decimal "object identifier" or OID, such as "1.2.2.1.12".

PML supports the following operations:

Get -- get the value of an object
GetNext -- get the ID and value of the next object after the specified object (not yet supported by the hpoj software)
Set -- set the value of an object
Trap -- an asynchronous notification from the peripheral that it changed the value of an object (not yet supported by the hpoj software)
EnableTrap, DisableTrap -- enable or disable traps on an object

Control objects are typically read/write, and status objects are typically read-only. Depending on the object, writing to some control objects changes a setting in the device, and writing to others causes an action to be taken, such as starting a scan operation.

The PML protocol specification is available from the HP Developer Solutions web site (see above). A partial list of PML objects that may or may not be supported by HP all-in-ones is available. In this table, the OID_ "object name" is provided only as a hint of the meaning, and you must enter the numerical object ID on the command line. Some more specific information about the meaning or valid values of certain objects is "available on request" (see above), or you can just look at the source code to ptal-hp.

HP JetDirect

HP JetDirect "external" (parallel-port or USB) print servers use TCP/IP ports to access most stream services that would be accessible as MLC/1284.4 channels in a local-connect case. In the following list, the standard port number is the first number in the set of three, and the other two numbers are for the second and third ports on a multi-port JetDirect, such as the 500X:

9100/9101/9102 -- Print service. Buffered or pass-through to the peripheral upon connection.
9290/9291/9292 -- Scan service. Sends two status bytes back to indicate connection success/failure before becoming pass-through.
9220/9221/9222 ("generic gateway") -- Other services. A simple SMTP-like command-line conversation is used for 1284.4 service name lookup, to open a socket ID on the peripheral, and to become pass-through after a successful open.

JetDirect encapsulates PML within SNMP (Simple Network Management Protocol), with one of several prefixes before the PML OID (object ID).

See the file ptal/ptal-hpjd.c in the hpoj source-code package for more information.

Printing protocols

Most HP printers support some form of PCL, HP's Printer Control Language. PCL can be as simple as some escape sequences added to an otherwise ASCII print job, or it can be as complex as an entire job pre-rendered as a raster image. There are various levels of PCL, such as PCL3, PCL3GUI, PCL5, PCLXL, etc., but of course, not all devices support all levels.

There is also PJL, HP's Printer Job Language, which depending on the level of device support, consists of job control and status commands sent between jobs. Some commands on some devices can even result in data coming back from the device. PJL is not currently used by the hpoj software, other than the UEL (universal escape language) sequence sent by ptal-printd.

More information on PCL and PJL may be found on the HP Developer Solutions web site (see above).

Some HP printers also support Adobe's PostScript language, which is a (mostly) device-independent page description language (PDL). Most Unix applications generate PostScript data when asked to print, which would then be converted into PCL or another PDL for non-PostScript printers. HP's Enhanced Print System can be used in conjunction with a device-specific "PPD" file to access special features of a PostScript printer, such as selecting paper trays, duplex printing, and stapling. PostScript is well-documented in books from Adobe and others.

Scanning protocols

Scanning protocols for supported HP all-in-one peripherals generally fall into two categories:

SCL -- HP's Scanner Control Language, used on the flatbed OfficeJets and newer scrollfed OfficeJets. Escape sequences are sent over the scan channel to set up scan parameters and start the scan, and scan data is returned over the same channel. SCL is documented on the HP Developer Solutions web site (see above).
PML/MFPDTF -- The older scrollfed OfficeJets and supported LaserJet all-in-ones use PML to set scan parameters and start the scan, and scan data is returned over the scan channel, wrapped in MFPDTF (Multi-Function Peripheral Data Transfer Format) packets (which also can be but currently isn't used on some models of SCL-based OfficeJets). Some information on these protocols is "available on request" (see above), or you can just look at the source code to ptal-hp.

Some models are capable of returning JPEG-compressed grayscale and color scan data, and some models can only return grayscale and color as JPEG. The book JPEG Still Image Data Compression Standard by Pennebaker and Mitchell provides a thorough explanation and even includes the official specification as an appendix.

Most JPEG files you encounter are stored in the JPEG File Interchange Format (JFIF), which is described in a separate document you can find in various formats on many websites (just do a Google search on JFIF). This document assumes familiarity with the base JPEG specification, however.