EBCDIC

Jump to navigation Jump to search

EBCDIC encoding family
Classification8-bit basic Latin encodings (non‑ASCII)
Preceded byBCD

Extended Binary Coded Decimal Interchange Code[1] (EBCDIC;[1] /ˈɛbsɪdɪk/) is an eight-bit character encoding used mainly on IBM mainframe and IBM midrange computer operating systems. It descended from the code used with punched cards and the corresponding six-bit binary-coded decimal code used with most of IBM's computer peripherals of the late 1950s and early 1960s.[2] It is supported by various non-IBM platforms, such as Fujitsu-Siemens' BS2000/OSD, OS-IV, MSP, and MSP-EX, the SDS Sigma series, Unisys VS/9, Burroughs MCP and ICL VME.

History[edit]

Punched card with the Hollerith encoding of the 1964 EBCDIC character set. Contrast at top enhanced to show the printed characters.

EBCDIC was devised in 1963 and 1964 by IBM and was announced with the release of the IBM System/360 line of mainframe computers. It is an eight-bit character encoding, developed separately from the seven-bit ASCII encoding scheme. It was created to extend the existing Binary-Coded Decimal (BCD) Interchange Code, or BCDIC, which itself was devised as an efficient means of encoding the two zone and number punches on punched cards into six bits. The distinct encoding of 's' and 'S' (using position 2 instead of 1) was maintained from punched cards where it was desirable not to have hole punches too close to each other to ensure the integrity of the physical card.

While IBM was a chief proponent of the ASCII standardization committee,[3] the company did not have time to prepare ASCII peripherals (such as card punch machines) to ship with its System/360 computers, so the company settled on EBCDIC.[2] The System/360 became wildly successful, together with clones such as RCA Spectra 70, ICL System 4, and Fujitsu FACOM, thus so did EBCDIC.

All IBM mainframe and midrange peripherals and operating systems use EBCDIC as their inherent encoding[4] (with toleration for ASCII, for example, ISPF in z/OS can browse and edit both EBCDIC and ASCII encoded files). Software and many hardware peripherals can translate to and from encodings, and modern mainframes (such as IBM Z) include processor instructions, at the hardware level, to accelerate translation between character sets.

There is an EBCDIC-oriented Unicode Transformation Format called UTF-EBCDIC proposed by the Unicode consortium, designed to allow easy updating of EBCDIC software to handle Unicode, but not intended to be used in open interchange environments. Even on systems with extensive EBCDIC support, it has not been popular. For example, z/OS supports Unicode (preferring UTF-16 specifically), but z/OS only has limited support for UTF-EBCDIC.

IBM AIX running on the RS/6000 and its descendants including the IBM Power Systems, Linux running on IBM Z, and operating systems running on the IBM PC and its descendants use ASCII, as did AIX/370 and AIX/390 running on System/370 and System/390 mainframes.

Compatibility with ASCII[edit]

The fact that all the code points were different was less of a problem for inter-operating with ASCII than the fact that sorting EBCDIC put lowercase letters before uppercase letters and letters before numbers, exactly the opposite of ASCII.

Programming languages and file formats and network protocols designed for ASCII quickly made use of available punctuation marks (such as the curly braces '{ ' and  ' }' ) that did not exist in EBCDIC, making translation to EBCDIC ambiguous (this also prevented various attempts to make internationalized versions of ASCII which also replaced these punctuation marks with letters).

The gaps between letters made simple code that worked in ASCII fail on EBCDIC. For example, "for (c='A';c<='Z';++c)" would set c to the 26 letters in the ASCII alphabet, but 41 characters including a number of unassigned ones in EBCDIC. Fixing this required complicating the code with function calls which was greatly resisted by programmers.

By using all eight bits EBCDIC may have encouraged the use of the eight-bit byte by IBM, while ASCII was more likely to be adopted by systems with 36 bits (as five seven-bit ASCII characters fit into one word).

As eight-bit bytes became widespread, ASCII systems sometimes used the "unused" bit for other purposes, such as metacharacters to mark the borders of records or words. This made it difficult to change the code to work with EBCDIC. On the PDP-11 bytes with the high bit set were treated as negative numbers, behavior that was copied to C, causing unexpected problems with EBCDIC.

Code page layout[edit]

The table below shows the "invariant subset"[5] of EBCDIC, which are characters that should have the same assignments on all EBCDIC code pages. It also shows (in gray) missing ASCII and EBCDIC punctuation, located where they are in CCSID 037 (one of the code page variants of EBCDIC). Unassigned codes are typically filled with international or region-specific characters in the various EBCDIC code page variants, but the characters in gray are often moved around or swapped as well.

In each table cell below, the first row is an abbreviation for a control code or (for printable characters) the character itself; and the second row is the Unicode code (blank for controls that don't exist in Unicode).

EBCDIC
_0 _1 _2 _3 _4 _5 _6 _7 _8 _9 _A _B _C _D _E _F
0_ NUL
0000
SOH
0001
STX
0002
ETX
0003
SEL
 
HT
0009
RNL
 
DEL
007F
GE
 
SPS
 
RPT
 
VT
000B
FF
000C
CR
000D
SO
000E
SI
000F
1_ DLE
0010
DC1
0011
DC2
0012
DC3
0013
res/enp
 
NL
0085
BS
0008
POC
 
CAN
0018
EM
0019
UBS
 
CU1
 
IFS
001C
IGS
001D
IRS
001E
ius/itb
001F
2_ DS
 
SOS
 
FS
 
WUS
 
byp/inp
 
LF
000A
ETB
0017
ESC
001B
SA
 
SFE
 
sm/sw
 
CSP
 
MFA
 
ENQ
0005
ACK
0006
BEL
0007
3_ SYN
0016
IR
 
PP
 
TRN
 
NBS
 
EOT
0004
SBS
 
IT
 
RFF
 
CU3
 
DC4
0014
NAK
0015
SUB
001A
4_ SP
0020
¢
00A2
.
002E
<
003C
(
0028
+
002B
|
007C
5_ &
0026
!
0021
$
0024
*
002A
)
0029
;
003B
¬
00AC
6_ -
002D
/
002F
¦
00A6
,
002C
%
0025
_
005F
>
003E
?
003F
7_ `
0060
:
003A
#
0023
@
0040
'
0027
=
003D
"
0022
8_ a
0061
b
0062
c
0063
d
0064
e
0065
f
0066
g
0067
h
0068
i
0069
±
00B1
9_ j
006A
k
006B
l
006C
m
006D
n
006E
o
006F
p
0070
q
0071
r
0072
A_ ~
007E
s
0073
t
0074
u
0075
v
0076
w
0077
x
0078
y
0079
z
007A
B_ ^
005E
[
005B
]
005D
C_ {
007B
A
0041
B
0042
C
0043
D
0044
E
0045
F
0046
G
0047
H
0048
I
0049
D_ }
007D
J
004A
K
004B
L
004C
M
004D
N
004E
O
004F
P
0050
Q
0051
R
0052
E_ \
005C
S
0053
T
0054
U
0055
V
0056
W
0057
X
0058
Y
0059
Z
005A
F_ 0
0030
1
0031
2
0032
3
0033
4
0034
5
0035
6
0036
7
0037
8
0038
9
0039
EO
 

Definitions of non-ASCII EBCDIC controls[edit]

Following are the definitions of EBCDIC control characters which either don't map onto the ASCII control characters, or have additional uses. When mapped to Unicode, these are mostly mapped to C1 control character codepoints in a manner specified by IBM's Character Data Representation Architecture (CDRA).[6][7]

Although the default mapping of New Line (NL) corresponds to the ISO/IEC 6429 Next Line (NEL) character (the behaviour of which is also specified, but not required, in Unicode Annex 14),[8] most of these C1-mapped controls match neither those in the ISO/IEC 6429 C1 set, nor those in other registered C1 control sets such as ISO 6630.[9] Although this effectively makes the non-ASCII EBCDIC controls a unique C1 control set, they are not among the C1 control sets registered in the ISO-IR registry,[10] meaning that they do not have an assigned control set designation sequence (as specified by ISO/IEC 2022, and optionally permitted in ISO/IEC 10646 (Unicode)).[11]

Besides U+0085 (Next Line), the Unicode Standard does not prescribe an interpretation of C1 control characters, leaving their interpretation to higher level protocols (it suggests, but does not require, their ISO/IEC 6429 interpretations in the absence of use for other purposes),[12] so this mapping is permissible in, but not specified by, Unicode.

Mnemonic EBCDIC CDRA pairing[6][7] Name Description[13]
SEL 04 009C Select Device control character taking a single-byte parameter.
RNL 06 0086 Required New Line Line-break resetting Indent Tab mode
GE 08 0097 Graphic Escape Non-locking shift that changes the interpretation of the following character (see e.g. Code page 310). Compare ISO/IEC 6429's SS2 (008E).
SPS 09 008D Superscript Begin superscript or undo subscript. Compare ISO/IEC 6429's PLU (008C).
RPT 0A 008E Repeat Switch to an operation mode repeating a print buffer
RES/ENP 14 009D Restore, Enable Presentation Resume output (after BYP/INP)
NL 15 0085 (000A) New Line Line break. Default mapping (0085) matches ISO/IEC 6429's NEL. Mappings sometimes swapped with Line Feed (EBCDIC 0x25) in accordance with UNIX line breaking convention.[6]
POC 17 0087 Program Operator Communication Followed by two one-byte operators that identify the specific function, for example a light or function key. Contrast with ISO/IEC 6429's CSI (009B), OSC (009D) and APC (009F).
UBS 1A 0092 Unit Backspace A fractional backspace.
CU1 1B 008F Customer Use One Not used by IBM; for customer use.
IUS/ITB 1F 001F Interchange Unit Separator, Intermediate Transmission Block Either used as an information separator to terminate a block called a "unit" (as in ASCII; see also IR), or used as a transmission control code to delimit the end of an intermediate block.
DS 20 0080 Digit Select Used by S/360 CPU edit (ED) instruction
SOS 21 0081 Start of Significance Used by S/360 CPU edit (ED) instruction. (Note: different from ISO/IEC 6429's SOS.)
FS 22 0082 Field Separator Used by S/360 CPU edit (ED) instruction. (Note: (Interchange) File Separator, as abbreviated FS in ASCII, is at 0x1C and abbreviated IFS.)[13]
WUS 23 0083 Word Underscore Underscores the immediately preceding word. Contrast with ISO/IEC 6429's SGR.
BYP/INP 24 0084 Bypass, Inhibit Presentation De-activates output, i.e. ignores all graphical characters and control characters besides transmission control codes and RES/ENP, until the next RES/ENP.
SA 28 0088 Set Attribute Marks the beginning of a fixed-length device specific control sequence. Deprecated in favour of CSP.
SFE 29 0089 Start Field Extended Marks the beginning of a variable-length device specific control sequence. Deprecated in favour of CSP.
SM/SW 2A 008A Set Mode, Switch Device specific control that sets a mode of operation, such as a buffer switch.
CSP 2B 008B Control Sequence Prefix Marks the beginning of a variable-length device specific control sequence. Followed by a class byte specifying a category of control function, a count byte giving the sequence length (including count and type bytes, but not the class byte or initial CSP), a type byte identifying a control function within that category, and zero or more parameter bytes. Contrast with ISO/IEC 6429's DCS (0090) and CSI (009B).
MFA 2C 008C Modify Field Attribute Marks the beginning of a variable-length device specific control sequence. Deprecated in favour of CSP.
30 0090 (reserved) Reserved for future use by IBM
31 0091 (reserved) Reserved for future use by IBM
IR 33 0093 Index Return Either move to start of next line (see also NL), or terminate an information unit (see also IUS/ITB).
PP 34 0094 Presentation Position Followed by two one-byte parameters (firstly function, secondly number of either column or line) to set the current position. Contrast with ISO/IEC 6429's CUP and HVP.
TRN 35 0095 Transparent Followed by one byte parameter that indicates the number of bytes of transparent data that follow.
NBS 36 0096 Numeric Backspace Move backwards the width of one digit.
SBS 38 0098 Subscript Begin subscript or undo superscript. Compare ISO/IEC 6429's PLD (008B).
IT 39 0099 Indent Tab Indents the current and all following lines, until RNL or RFF is encountered.
RFF 3A 009A Required Form Feed Page-break resetting Indent Tab mode.
CU3 3B 009B Customer Use Two Not used by IBM; for customer use.
3E 009E (reserved) Reserved for future use by IBM
EO FF 009F Eight Ones All ones character used as filler

Criticism and humor[edit]

Open-source software advocate and software developer Eric S. Raymond writes in his Jargon File that EBCDIC was loathed by hackers, by which he meant[14] members of a subculture of enthusiastic programmers. The Jargon File 4.4.7 gives the following definition:[15]

EBCDIC: /eb´[email protected]·dik/, /eb´see`dik/, /eb´[email protected]·dik/, n. [abbreviation, Extended Binary Coded Decimal Interchange Code] An alleged character set used on IBM dinosaurs. It exists in at least six mutually incompatible versions, all featuring such delights as non-contiguous letter sequences and the absence of several ASCII punctuation characters fairly important for modern computer languages (exactly which characters are absent varies according to which version of EBCDIC you're looking at). IBM adapted EBCDIC from punched card code in the early 1960s and promulgated it as a customer-control tactic (see connector conspiracy), spurning the already established ASCII standard. Today, IBM claims to be an open-systems company, but IBM's own description of the EBCDIC variants and how to convert between them is still internally classified top-secret, burn-before-reading. Hackers blanch at the very name of EBCDIC and consider it a manifestation of purest evil.

— The Jargon file 4.4.7

EBCDIC design was also the source of many jokes. One such joke[citation needed] went:

Professor: "So the American government went to IBM to come up with an encryption standard, and they came up with—"
Student: "EBCDIC!"

References to the EBCDIC character set are made in the classic Infocom adventure game series Zork. In the "Machine Room" in Zork II, EBCDIC is used to imply an incomprehensible language:

This is a large room full of assorted heavy machinery, whirring noisily. The room smells of burned resistors. Along one wall are three buttons which are, respectively, round, triangular, and square. Naturally, above these buttons are instructions written in EBCDIC...

See also[edit]

References[edit]

  1. ^ a b Mackenzie, Charles E. (1980). Coded Character Sets, History and Development. The Systems Programming Series (1 ed.). Addison-Wesley Publishing Company, Inc. ISBN 0-201-14460-3. LCCN 77-90165. ISBN 978-0-201-14460-4. Retrieved 2016-05-22. [1]
  2. ^ a b Bemer, Bob. "EBCDIC and the P-Bit (The Biggest Computer Goof Ever) - Computer History Vignettes". Archived from the original on 2018-05-13. Retrieved 2013-07-02. […] but their printers and punches were not ready to handle ASCII, and IBM just HAD to announce.
  3. ^ "X3.4-1963". 1963. p. 4. Archived from the original on 2016-08-12. (NB. IBM had four staff members on the final 21-member ASA X3.2 sub-committee.)
  4. ^ IBMnt (2008). "IBM confirms the use of EBCDIC in their mainframes as a default practice". Archived from the original on 2013-01-03. Retrieved 2008-06-16.
  5. ^ IBM Knowledge Center Invariant character set
  6. ^ a b c Umamaheswaran, V.S. (1999-11-08). "3.3 Step 2: Byte Conversion". UTF-EBCDIC. Unicode Consortium. Unicode Technical Report #16. The 64 control characters […], the ASCII DELETE character (U+007F)[…] are mapped respecting EBCDIC conventions, as defined in IBM Character Data Representation Architecture, CDRA, with one exception -- the pairing of EBCDIC Line Feed and New Line control characters are swapped from their CDRA default pairings to ISO/IEC 6429 Line Feed (U+000A) and Next Line (U+0085) control characters
  7. ^ a b Steele, Shawn (1996-04-24). cp037_IBMUSCanada to Unicode table. Microsoft/Unicode Consortium.
  8. ^ Heninger, Andy (2019-02-15). "NL: Next Line (A) (Non-tailorable)". Unicode Line Breaking Algorithm. Revision 43. Unicode Consortium. Unicode Standard Annex #14.
  9. ^ ISO/TC 46 (1986-02-01). Additional Control Functions for Bibliographic Use according to International Standard ISO 6630 (PDF). ITSCJ/IPSJ. ISO-IR-124.
  10. ^ ISO/IEC International Register of Coded Character Sets To Be Used With Escape Sequences (PDF), ITSCJ/IPSJ, ISO-IR
  11. ^ ISO/IEC JTC 1/SC 2 (2017). "12.4: Identification of control function set". Information technology — Universal Coded Character Set (UCS) (5th ed.). ISO. pp. 19–20. ISO/IEC 10646. For other C0 or C1 sets, the final octet F shall be obtained from the International Register of Coded Character Sets. […] If such an escape sequence appears within a code unit sequence conforming to this International Standard, it shall be padded in accordance with Clause 11.
  12. ^ Unicode Consortium (2019). "23.1: Control Codes" (PDF). The Unicode Standard (12.0.0 ed.). pp. 868–870. ISBN 978-1-936213-22-1.
  13. ^ a b "Appendix G-1. EBCDIC control character definitions". Character Data Representation Architecture. IBM Corporation. Archived from the original on 2018-09-11.
  14. ^ Raymond, Eric S. (1997). "The New Hacker's Dictionary". p. 310.
  15. ^ "EBCDIC". Jargon File. Archived from the original on 2018-05-13. Retrieved 2018-05-13.

External links[edit]